CN114298061B - Machine translation and model training quality evaluation method, electronic device and storage medium - Google Patents
Machine translation and model training quality evaluation method, electronic device and storage medium Download PDFInfo
- Publication number
- CN114298061B CN114298061B CN202210217054.8A CN202210217054A CN114298061B CN 114298061 B CN114298061 B CN 114298061B CN 202210217054 A CN202210217054 A CN 202210217054A CN 114298061 B CN114298061 B CN 114298061B
- Authority
- CN
- China
- Prior art keywords
- data
- source language
- branch
- translation
- quality
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The embodiment of the application provides a machine translation quality evaluation and model training quality evaluation method, electronic equipment and a computer storage medium, wherein the machine translation quality evaluation method comprises the following steps: encoding source language data to be translated to obtain source language encoded data; translating and decoding the source language coded data to obtain target language data; reconstructing and decoding the target language data again to obtain newly generated source language reconstruction data; determining translation quality for the source language data from the source language data and the source language reconstructed data. According to the method and the device, the translation accuracy of the target language data can be determined through the data corresponding to the two source languages, the translation effect is evaluated, and the self-quality evaluation function of the machine translation model is realized.
Description
Technical Field
The embodiment of the application relates to the technical field of machine learning, in particular to a machine translation quality evaluation method, a machine translation model training quality evaluation method, corresponding electronic equipment and a computer storage medium.
Background
Machine translation, also known as automatic translation, is the process of converting one natural language (source language) to another (target language) using a computer. Currently, machine translation is mostly implemented by a machine translation model based on machine learning.
At present, the evaluation of the translation quality of the model is implemented by means of manual or additional translation quality evaluation models in an inference stage or a training stage of the machine translation model. This results in a higher cost for translation quality assessment.
Disclosure of Invention
In view of the above, embodiments of the present application provide a machine translation quality assessment and model training quality assessment scheme thereof to at least partially solve the above problems.
According to a first aspect of embodiments of the present application, a method for evaluating machine translation quality is provided, including: encoding source language data to be translated to obtain source language encoded data; translating and decoding the source language coded data to obtain target language data; reconstructing and decoding the target language data again to obtain newly generated source language reconstruction data; determining translation quality for the source language data from the source language data and the source language reconstructed data.
According to a second aspect of the embodiments of the present application, there is provided a method for evaluating training quality of a machine translation model, including: obtaining source language training sample data without labeled data; inputting the source language training sample data into an encoder in a machine translation model for encoding to obtain sample encoding data; inputting the sample coded data into a first decoder in the machine translation model for translation decoding to obtain target language sample data; inputting the target language sample data into a second decoder in the machine translation model for reconstruction decoding again to obtain newly generated source language reconstruction sample data; and performing quality evaluation on the training of the machine translation model according to the source language training sample data and the source language reconstruction sample data.
According to a third aspect of embodiments of the present application, there is provided an electronic apparatus, including: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface are communicated with each other through the communication bus; the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the corresponding operation of the method of the first aspect or the second aspect.
According to a fourth aspect of embodiments of the present application, there is provided a computer storage medium having stored thereon a computer program which, when executed by a processor, implements a method according to the first or second aspect.
According to the machine translation quality evaluation scheme provided by the embodiment of the application, after the target language data is obtained, the target language data is reconstructed and decoded again to realize reverse translation of the target language data, and then the target language data is translated back to the source language to obtain corresponding reconstructed source language data. And further, determining the translation quality of the translation based on the source language reconstruction data and the original source language data. If the target language data obtained by translation is more accurate, the translation quality determined according to the source language reconstruction data obtained by reverse translation and the original source language data is higher. Therefore, the translation accuracy of the target language data can be determined through the data corresponding to the two source languages, and the translation effect quality evaluation is carried out. Therefore, the self-evaluation function of the machine translation model for evaluating the translation quality by the machine translation model can be realized without the help of manpower or an additional translation quality evaluation model, and the realization cost of evaluating the translation quality is reduced.
According to the training quality evaluation scheme of the machine translation model provided by the embodiment of the application, the result of the reverse translation of the target language sample data (namely the result of the reconstruction decoding) by the second decoder and the original source language training sample data can be used for determining the effect of the current iterative training of the machine translation model, namely the translation quality of the current training. Therefore, in the scheme, the labeling data of the training sample is not required, the source area of the training sample is greatly expanded, special or additional processing on the source language training sample data is not needed, the model can output the translation quality evaluation result, and therefore the cost of performing quality evaluation on model training is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the embodiments of the present application, and other drawings can be obtained by those skilled in the art according to the drawings.
FIG. 1 is a schematic diagram of an exemplary system to which the method of machine translation quality assessment and model training quality assessment of embodiments of the present application may be applied;
FIG. 2A is a flowchart illustrating steps of a method for evaluating quality of machine translation according to an embodiment of the present disclosure;
FIG. 2B is a diagram illustrating an example of a scenario in the embodiment shown in FIG. 2A;
FIG. 2C is a diagram illustrating another exemplary scenario in the embodiment shown in FIG. 2A;
FIG. 3A is a flowchart illustrating steps of a method for evaluating training quality of a machine translation model according to a second embodiment of the present disclosure;
FIG. 3B is a block diagram of a machine translation model according to the embodiment shown in FIG. 3A;
fig. 4 is a schematic structural diagram of an electronic device according to a third embodiment of the present application.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application shall fall within the scope of protection of the embodiments in the present application.
The following further describes specific implementations of embodiments of the present application with reference to the drawings of the embodiments of the present application.
Fig. 1 illustrates an exemplary system to which the machine translation quality assessment and model training quality assessment method according to the embodiment of the present application are applicable. As shown in fig. 1, the system 100 may include a server 102, a communication network 104, and/or one or more user devices 106, illustrated in fig. 1 as a plurality of user devices.
In some embodiments, the communication network 104 may be any suitable combination of one or more wired and/or wireless networks. For example, the communication network 104 can include any one or more of the following: the network may include, but is not limited to, the internet, an intranet, a Wide Area Network (WAN), a Local Area Network (LAN), a wireless network, a Digital Subscriber Line (DSL) network, a frame relay network, an Asynchronous Transfer Mode (ATM) network, a Virtual Private Network (VPN), and/or any other suitable communication network. The user device 106 can be connected to the communication network 104 by one or more communication links (e.g., communication link 112), and the communication network 104 can be linked to the server 102 via one or more communication links (e.g., communication link 114). The communication link may be any communication link suitable for communicating data between the user device 106 and the server 102, such as a network link, dial-up link, wireless link, hardwired link, any other suitable communication link, or any suitable combination of such links.
Although the server 102 is illustrated as one device, in some embodiments any suitable number of devices may be used to perform the functions performed by the server 102. For example, in some embodiments, multiple devices may be used to implement the functions performed by the server 102. Alternatively, the functionality of the server 102 may be implemented using a cloud service.
Based on the system, the embodiment of the application provides a machine translation quality assessment and model training quality assessment scheme, and the following description is provided through a plurality of embodiments.
Example one
Referring to fig. 2A, a flowchart illustrating steps of a method for evaluating quality of machine translation according to an embodiment of the present application is shown.
The machine translation quality evaluation method of the embodiment comprises the following steps:
step S202: and coding the source language data to be translated to obtain source language coded data.
In this embodiment, the source languages corresponding to different machine translation models may be different, but no matter what source language is, after the language of the source language is determined, the machine translation model corresponding to the language is found. That is, the source language in the embodiment of the present application may be a language of any language. In addition, some machine translation models can also realize multi-language translation, and all the machine translation models can be applied to the scheme of the embodiment of the application and are also in the protection scope of the application.
After the source language data to be translated is determined, the source language data can be encoded to obtain source language encoded data. Wherein, the coding can be realized by adopting a conventional coding mode. In an advantageous scheme, the method can be implemented by using an attention-based encoding mode, such as an encoder based on a Transformer structure.
Step S204: translating and decoding the source language coded data to obtain target language data; and reconstructing and decoding the target language data again to obtain newly generated source language reconstruction data.
In this step, on one hand, the target language data is obtained by decoding the encoded source language data, and this process may be implemented in a conventional manner, or in an advantageous scheme, may be implemented in a decoding manner based on attention calculation, such as in a decoder based on a transform structure. On the other hand, unlike the conventional loss calculation of tag data corresponding to the target language data and the source language data, in the embodiment of the present application, the target language data obtained by translation is decoded again, and this decoding is used to perform reverse translation on the target language data and then translate the target language data back to the source language data, which is referred to as source language reconstructed data in the embodiment of the present application. Theoretically, if the target language data is a more accurate translation, the reverse translated source language reconstructed data should be the same as or slightly different from the original source language data. Based on this, the following step S206 may be performed.
Step S206: from the source language data and the source language reconstructed data, translation quality for the source language data is determined.
As previously described, if the difference between the source language data and the source language reconstructed data is small, then the characterized target language data is more accurate, and vice versa, the accuracy is less.
In an alternative implementation, source language reconstructed data may be input to an encoder to obtain reconstructed encoded data; from the reconstructed encoded data and the source language encoded data, a translation quality for the source language data is determined. For example, pairwise cosine similarity calculation is performed on the reconstructed encoded data and the source language encoded data, and the similarity between the reconstructed encoded data and the source language encoded data is obtained according to the calculation result; from the similarity, translation quality for the source language data is determined. Or performing cross entropy loss calculation on the reconstructed encoding data and the source language encoding data, and determining translation quality aiming at the source language data according to the calculation result.
On this basis, in a feasible manner, the method for evaluating machine translation quality according to this embodiment may further include: if the translation quality is greater than the preset quality threshold value, determining that the translation quality is higher, and determining the target language data as the translation result of the source language data; and if the translation quality is not greater than the preset quality threshold, the translation quality does not meet the quality requirement, further, the target language data can be corrected subsequently, and the translation result of the source language data is determined according to the correction result. The preset quality threshold may be flexibly set by a person skilled in the art according to actual requirements, such as 0.8-0.9, and the like, which is not limited in the embodiment of the present application.
If the translation quality determined from the source language data and the source language reconstructed data is not greater than the preset quality threshold, the target language data may be corrected, for example, manually, or by a correction model, which may take the source language data and the target language data as input and output the corrected target language data. Of course, other correction methods are also applicable to the scheme of the embodiment of the present application.
As described above, the machine translation quality evaluation in the embodiment of the present application may be implemented by a machine translation model, and the following takes the machine translation model to perform translation and quality evaluation as an example, and the implementation of the above process is exemplarily described as shown in fig. 2B and fig. 2C, respectively.
The machine translation models shown in fig. 2B and 2C each include an encoder (schematically shown as Enc), a first decoder (schematically shown as Dec) 1 ) And a second decoder (shown schematically as Dec in the figure) 2 ) Thus forming an "encoder-decoder" structure.
The encoder is used for encoding input source language data and outputting encoded source language encoding vectors; the first decoder is used for translating and decoding the source language coding vector output by the encoder as input and outputting target language data; and the second decoder is used for outputting source language reconstruction data after reconstructing and decoding the target language data output by the first decoder.
As can be seen from fig. 2B and 2C, the source language data X to be translated is first translated into target language data Y' by processing of the encoder and the first decoder; then, firstThe output of the decoder is input to a second decoder to reconstruct the new data, i.e., the source language reconstructed data X' (FIG. 2B), shown schematically as X in FIG. 2C, that is still in the source language shift 。
Furthermore, in a possible manner, the source language reconstruction data X 'may be input to the encoder to obtain reconstruction encoded data Enc (X'); the translation quality for the source language data X is then determined based on the reconstructed encoded data Enc (X') and the source language encoded data Enc (X). By the method, the characteristic that the encoder of the machine translation model can output the encoding vector is fully utilized, and the encoding vector can carry effective semantic information, so that quality evaluation can be carried out on translation quality based on reconstructed encoding data and source language encoding data output by the encoder, and resources of the machine translation model are effectively utilized.
In a specific implementation scheme for determining translation quality, pairwise cosine similarity (cosine similarity) calculation may be performed on reconstructed encoded data and source language encoded data, and the similarity between the reconstructed encoded data and the source language encoded data is obtained according to a calculation result; from the similarity, translation quality for the source language data is determined. The method is simple to implement, and the speed of determining the translation quality is high. This approach is illustrated in fig. 2B.
In specific calculation, whether the source language data or the source language reconstruction data, the identifier expression of each word generates a corresponding vector sequence after being processed by an encoder. For example, for source language data X, its corresponding original character sequence is
Its corresponding vector sequence can be expressed as:
(ii) a For the source language reconstructed data X ', its corresponding original character sequence is X' = n<x 1 ’,…，x k ’>The corresponding vector sequence can be expressed as:
. The formula for performing the pairwise cosine similarity calculation on the two can be expressed as:
wherein Q P Indicates a recall score, Q R Represents the accuracy score, Q F Represents the F1 metric; i is a count value ranging from 1 to
The number of all elements of (a); j is also a count value ranging from 1 to
The number of all elements of (a);
a scalar quantity representing x is then calculated,
a scalar representing x'.
In another specific implementation of determining translation quality, cross entropy loss calculations may be performed on the reconstructed encoded data and the source language encoded data, and translation quality for the source language data may be determined based on the calculations. This approach, while somewhat more complex, is computationally accurate and can more accurately determine translation quality based on the reconstructed encoded data and the source language encoded data. This approach is illustrated in fig. 2C.
In the specific calculation, the cross entropy loss can be calculated by using the following cross entropy loss function:
wherein Y' represents source language reconstructed data; x represents source language data;
representing the length of the source language data;
representing a quality assessment score;
parameters representing an encoder;
a parameter representing a first decoder;
a parameter representing a second decoder; x represents a character in an original character sequence corresponding to the source language data X; i is a count value that ranges from 1 to the number of all elements in x.
It should be noted that, in the manner shown in fig. 2C, the decoding of the first decoder may adopt a free decoding (including but not limited to the beam search algorithm), and the decoding of the second decoder may adopt a forced decoding (performing the translation of the subsequent words based on the translation result of the previous word generated by the model itself).
In addition, in a possible scheme, when the encoding of the encoder, the decoding of the first decoder and the decoding of the second decoder are implemented specifically, a beam search algorithm may be used to implement the encoding, the decoding of the first decoder and the decoding of the second decoder, so as to improve the translation efficiency.
As described above, if the translation quality is greater than the preset quality threshold, the target language data may be determined as the translation result of the source language data; and if the translation quality is not greater than the preset quality threshold, correcting the target language data, and determining the translation result of the source language data according to the correction result.
Therefore, in the scheme of evaluating the machine translation quality based on the machine translation model, the strategy of 'translation-reconstruction-evaluation' is adopted, so that the machine translation model can evaluate the translation quality by itself without depending on extra data, and the realization of evaluating the translation quality of the machine translation model is simplified.
Through the embodiment, after the target language data is obtained, the target language data is reconstructed and decoded again to realize reverse translation of the target language data, and then the target language data is translated back to the source language to obtain corresponding source language reconstructed data. And further, determining the translation quality of the translation based on the source language reconstruction data and the original source language data. If the target language data obtained by translation is more accurate, the translation quality determined according to the source language reconstruction data obtained by reverse translation and the original source language data will be higher. Therefore, the translation accuracy of the target language data can be determined through the data corresponding to the two source languages, and the translation effect evaluation is carried out. Therefore, the self-quality evaluation function of the machine translation model for evaluating the translation quality by the self can be realized without the help of manpower or an additional translation quality evaluation model, and the realization cost of the translation quality evaluation is reduced.
Example two
Referring to fig. 3A, a flowchart illustrating steps of a method for evaluating training quality of a machine translation model according to a second embodiment of the present application is shown.
The embodiment focuses on the training quality evaluation of the machine translation model in the above embodiment, and the method for evaluating the training quality of the machine translation model includes the following steps:
step S302: and obtaining source language training sample data without labeled data.
The training sample data in the embodiment of the application does not need corresponding marking data, so that the training sample does not need to be additionally processed, and the available range of the training sample data is greatly expanded. For example, training of an english-to-chinese machine translation model may be performed based on only data of a chinese-english parallel corpus without labeling translation accuracy or translation quality of the parallel corpus.
In this embodiment, the specific language of the source language is not limited.
Step S304: inputting the source language training sample data into an encoder in a machine translation model for encoding to obtain sample encoding data.
For convenience of explanation, the architecture of the machine translation model in the present embodiment is first described below with reference to fig. 3B.
In FIG. 3B, the encoder Enc and the first decoder Dec are included 1 And a second decoder Dec 2 。
As can be seen in the figure, the encoder Enc is based on a transform structure and comprises N1 coding modules, each of which comprises at least a Self-Attention computing layer (Self-Attention) and a Feed-Forward layer (Feed Forward). N1 is a positive integer
First decoder Dec 1 Comprises N2 decoding modules, each of which comprises a first branch (Dec in the figure) 1 Shown in solid line portion) and a second branch (Dec in the figure) 1 Shown in dashed lines), the first branch and the second branch share Parameters (Parameters Shared) to enable parameter sharing of both ends (target language end and source language reconstruction end). N2 is a positive integer, and the specific number of N2 can be the same as or different from N1.
Wherein, the first branch is mainly used for Translation (Translation), and at least comprises: the mask is from the Attention computing layer (Masked Self Attention), the Joint Attention computing layer (Joint Attention), and the Feed Forward layer (Feed Forward). Wherein the input of the joint attention computation layer is from the output of the feedforward layer of the encoder. In addition, the first branch also has a translation output layer Softmax to output a translation result, i.e., target language sample data. Therefore, through the first branch, mask self-attention calculation, joint attention calculation and forward propagation can be sequentially performed on the sample coded data, and then the target language sample data of the first branch is obtained.
The second branch is mainly used for language Reconstruction (reconstraction), and at least comprises the following components: a Self-Attention computation layer (Self Attention), and a Feed Forward layer (Feed Forward). Through the second branch, the sample coding data can be sequentially subjected to self-attention calculation and forward propagation to obtain target language sample data of the second branch, and the target language sample data of the second branch is transmitted to a second decoder Dec 2 . And first divisionThe difference is that, firstly, the second branch does not have a Joint Attention computing layer (Joint Attention), that is, it skips the encoding output of the encoder to the source language data, thereby avoiding that the second decoder cannot be trained better due to information leakage; secondly, the second branch adopts a Self-Attention computing layer (Self-Attention) instead of a mask Self-Attention computing layer (Masked Self-Attention), so that more comprehensive characteristics can be learned, and a better training effect is achieved. As can be seen from the figure, the output of the Feed Forward layer (Feed Forward) of the second branch is transmitted out of the second decoder through the joint attention computing layer of the second decoder, where it plays a role.
Wherein the second decoder Dec 2 The decoding device comprises N3 decoding modules, wherein N3 is a positive integer, and the specific number of N3 can be the same as or different from N1 or N2. For each decoding module, it comprises at least: the mask is from the Attention computing layer (Masked Self Attention), the Joint Attention computing layer (Joint Attention), and the Feed Forward layer (Feed Forward). Wherein the joint attention computation layer receives the output of the second branch of the first decoder, thereby embedding the second decoder in the "encoder-decoder" structure originally formed by the encoder and the first decoder.
As can be seen from the above, the encoder of the machine translation model of the present embodiment is an attention-based encoder, and both the first decoder and the second decoder are attention-based decoders.
In this step, the source language training sample data may be input to the encoder in the machine translation model shown in fig. 3B, and encoded by the encoder to obtain sample encoded data.
Step S306: and inputting the sample coded data into a first decoder in a machine translation model for translation decoding to obtain target language sample data.
As described above, in this embodiment, the sample encoding data output by the encoder may be input to the first decoder, and on one hand, the target language sample data may be obtained; on the other hand, the target language sample data may be further input into the second decoder as described in step S308.
Step S308: and inputting the target language sample data into a second decoder in the machine translation model for reconstruction decoding again to obtain newly generated source language reconstruction sample data.
After the target language data is obtained, the target language data is input into a second decoder again for reverse translation, namely reconstruction decoding, and new source language reconstruction sample data is obtained. The language of the source language reconstruction sample data is the same as that of the source language sample data, but is translated by the target language sample data.
Step S310: and performing quality evaluation on the training of the machine translation model according to the source language training sample data and the source language reconstruction sample data.
In a feasible mode, the pairwise cosine similarity calculation can be carried out on the coded data corresponding to the sample coded data and the source language reconstruction sample data, and the similarity between the sample coded data and the coded data is obtained according to the calculation result; and evaluating the quality of the training of the machine translation model according to the similarity. In the method, source language reconstruction sample data is input into an encoder of a machine translation model to obtain encoded data corresponding to the source language reconstruction sample data, then pairwise cosine similarity calculation is carried out on the encoded data and the encoded data corresponding to the source language training sample data, namely the sample encoded data, so that the similarity between the encoded data and the source language training sample data is obtained, training quality evaluation is carried out on the basis of the similarity, and translation quality is determined. For a specific calculation manner, reference may be made to the description of the relevant parts in the foregoing first embodiment, which is not described herein again. After the translation quality is determined, the translation quality condition of the training can be evaluated. In one possible approach, the translation quality obtained may be used only for reference or information presentation; in another possible way, parameters of the machine translation model may also be adjusted based on the translation quality, and training may be continued until a training termination condition is reached, such as a preset number of times of training.
In another feasible mode, cross entropy loss calculation can be performed on the sample encoding data and the encoding data corresponding to the source language reconstruction sample data, and quality evaluation can be performed on training of the machine translation model according to the calculation result. In the method, source language reconstruction sample data is input into an encoder of a machine translation model to obtain encoded data corresponding to the source language reconstruction sample data, then cross entropy loss calculation is carried out on the encoded data and the encoded data corresponding to the source language training sample data, namely the sample encoded data, to obtain a calculation result, and the translation quality of the training is obtained based on the calculation result. For a specific calculation manner of the cross entropy loss calculation, reference may be made to the description of the relevant parts in the foregoing first embodiment, which is not described herein again. After the translation quality is determined, parameters of the machine translation model can be adjusted based on the translation quality, and training is continued until a training termination condition is reached, such as a preset training frequency is reached.
By the training quality evaluation scheme of the machine translation model of this embodiment, the result of the reverse translation of the target language sample data (i.e., the result of the reconstruction decoding) by the second decoder and the original source language training sample data can be used to determine the effect of the current iterative training of the machine translation model, i.e., the translation quality of the current training. Therefore, in the scheme, the labeling data of the training sample is not required, the source face of the training sample is greatly expanded, special or additional processing on the source language training sample data is not needed, the model can output the translation quality evaluation result, and therefore the cost of quality evaluation on model training is reduced.
EXAMPLE III
Referring to fig. 4, a schematic structural diagram of an electronic device according to a third embodiment of the present application is shown, and the specific embodiment of the present application does not limit a specific implementation of the electronic device.
As shown in fig. 4, the electronic device may include: a processor (processor) 402, a Communications Interface 404, a memory 406, and a Communications bus 408.
Wherein:
the processor 402, communication interface 404, and memory 406 communicate with each other via a communication bus 408.
A communication interface 404 for communicating with other electronic devices or servers.
The processor 402 is configured to execute the program 410, and may specifically perform the relevant steps in the above method embodiments.
In particular, program 410 may include program code comprising computer operating instructions.
The processor 402 may be a CPU, or an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement embodiments of the present Application. The intelligent device comprises one or more processors which can be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.
And a memory 406 for storing a program 410. Memory 406 may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
The program 410 may be specifically configured to enable the processor 402 to execute operations corresponding to the method described in any of the method embodiments.
For specific implementation of each step in the program 410, reference may be made to corresponding steps and corresponding descriptions in units in the foregoing method embodiments, and corresponding beneficial effects are provided, which are not described herein again. It can be clearly understood by those skilled in the art that, for convenience and simplicity of description, the specific working processes of the above-described devices and modules may refer to the corresponding process descriptions in the foregoing method embodiments, and are not described herein again.
Embodiments of the present application further provide a computer program product, which includes computer instructions that instruct a computing device to perform operations corresponding to any one of the methods in the foregoing method embodiments.
It should be noted that, according to the implementation requirement, each component/step described in the embodiment of the present application may be divided into more components/steps, and two or more components/steps or partial operations of the components/steps may also be combined into a new component/step to achieve the purpose of the embodiment of the present application.
The above-described methods according to the embodiments of the present application may be implemented in hardware, firmware, or as software or computer code that may be stored in a recording medium such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk, or as computer code downloaded through a network, originally stored in a remote recording medium or a non-transitory machine-readable medium, and to be stored in a local recording medium, so that the methods described herein may be stored in such software processes on a recording medium using a general purpose computer, a dedicated processor, or programmable or dedicated hardware such as an ASIC or FPGA. It will be appreciated that a computer, processor, microprocessor controller, or programmable hardware includes memory components (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code that, when accessed and executed by a computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code for implementing the methods illustrated herein, execution of the code transforms the general-purpose computer into a special-purpose computer for performing the methods illustrated herein.
Those of ordinary skill in the art will appreciate that the various illustrative elements and method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the embodiments of the present application.
The above embodiments are only used for illustrating the embodiments of the present application, and not for limiting the embodiments of the present application, and those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application, so that all equivalent technical solutions also belong to the scope of the embodiments of the present application, and the scope of the patent protection of the embodiments of the present application should be defined by the claims.
Claims (12)
1. A machine translation quality assessment method, the method being performed based on a machine translation model, the machine translation model comprising an encoder, a first decoder and a second decoder; the method comprises the following steps:
encoding source language data to be translated through the encoder to obtain source language encoded data;
translating and decoding the source language coded data through the first decoder to obtain target language data; reconstructing and decoding the target language data again through the second decoder to obtain newly generated source language reconstruction data; wherein the first decoder comprises a first branch and a second branch, the first branch and the second branch sharing a parameter; the first branch comprises a mask self-attention calculation layer, a joint attention calculation layer, a forward feedback layer and a translation output layer, wherein the input of the joint attention calculation layer is from the output of the forward feedback layer of the encoder, and the first branch is used for sequentially carrying out mask self-attention calculation, joint attention calculation and forward propagation on the source language encoding data to obtain target language data of the first branch; the second branch comprises a self-attention layer and a forward feedback layer and does not have the joint attention calculation layer, and the second branch is used for carrying out self-attention calculation and forward propagation on the source language encoded data in sequence to obtain target language data of the second branch and transmitting the target language data to the second decoder;
determining translation quality for the source language data from the source language data and the source language reconstructed data.
2. The method of claim 1, wherein the determining translation quality for the source language data from the source language data and the source language reconstructed data comprises:
inputting the source language reconstruction data into the encoder to obtain reconstruction encoding data;
determining translation quality for the source language data from the reconstructed encoded data and the source language encoded data.
3. The method of claim 2, wherein said determining translation quality for the source language data from the reconstructed encoded data and the source language encoded data comprises:
performing pairwise cosine similarity calculation on the reconstructed coded data and the source language coded data, and obtaining the similarity between the reconstructed coded data and the source language coded data according to the calculation result;
and determining translation quality aiming at the source language data according to the similarity.
4. The method of claim 2, wherein said determining a translation quality for the source language data from the reconstructed encoded data and the source language encoded data comprises:
and performing cross entropy loss calculation on the reconstructed coded data and the source language coded data, and determining translation quality aiming at the source language data according to a calculation result.
5. The method of any of claims 1-4, wherein the method further comprises:
if the translation quality is larger than a preset quality threshold value, determining the target language data as a translation result of the source language data;
and if the translation quality is not greater than the preset quality threshold, correcting the target language data, and determining the translation result of the source language data according to the correction result.
6. A machine translation model training quality assessment method comprises the following steps:
obtaining source language training sample data without labeled data;
inputting the source language training sample data into an encoder in a machine translation model for encoding to obtain sample encoding data;
inputting the sample coded data into a first decoder in the machine translation model for translation decoding to obtain target language sample data, wherein the first decoder comprises a first branch and a second branch, and the first branch and the second branch share parameters; the first branch comprises a mask self-attention calculation layer, a joint attention calculation layer, a forward feedback layer and a translation output layer, wherein the input of the joint attention calculation layer is the output of the forward feedback layer of the encoder, and the first branch is used for sequentially carrying out mask self-attention calculation, joint attention calculation and forward propagation on the sample coded data to obtain target language sample data of the first branch; the second branch comprises a self-attention layer and a forward feedback layer and does not have the joint attention calculation layer, and is used for sequentially carrying out self-attention calculation and forward propagation on the sample coded data to obtain target language sample data of the second branch and transmitting the target language sample data of the second branch to a second decoder;
inputting the target language sample data of the second branch into a second decoder in the machine translation model for reconstruction decoding again to obtain newly generated source language reconstruction sample data;
and performing quality evaluation on the training of the machine translation model according to the source language training sample data and the source language reconstruction sample data.
7. The method of claim 6, wherein the encoder is an attention-based encoder, and the first decoder and the second decoder are both attention-based decoders.
8. The method of claim 6 or 7, wherein the quality assessment of the training of the machine translation model from the source language training sample data and the source language reconstruction sample data comprises:
performing pairwise cosine similarity calculation on the sample coded data and coded data corresponding to the source language reconstruction sample data, and obtaining the similarity between the sample coded data and the coded data according to the calculation result;
and according to the similarity, carrying out quality evaluation on the training of the machine translation model.
9. The method of claim 6 or 7, wherein the quality assessment of the training of the machine translation model from the source language training sample data and the source language reconstruction sample data comprises:
and performing cross entropy loss calculation on the sample encoding data and the encoding data corresponding to the source language reconstruction sample data, and performing quality evaluation on the training of the machine translation model according to a calculation result.
10. The method of claim 6 or 7, wherein the method further comprises:
and adjusting the model parameters of the machine translation model in the training process according to the quality evaluation result.
11. An electronic device, comprising: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface are communicated with each other through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the corresponding operation of the method according to any one of claims 1-10.
12. A computer storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210217054.8A CN114298061B (en) | 2022-03-07 | 2022-03-07 | Machine translation and model training quality evaluation method, electronic device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210217054.8A CN114298061B (en) | 2022-03-07 | 2022-03-07 | Machine translation and model training quality evaluation method, electronic device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114298061A CN114298061A (en) | 2022-04-08 |
| CN114298061B true CN114298061B (en) | 2022-12-06 |
Family
ID=80978623
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210217054.8A Active CN114298061B (en) | 2022-03-07 | 2022-03-07 | Machine translation and model training quality evaluation method, electronic device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114298061B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115268868B (en) * | 2022-07-26 | 2023-06-27 | 中国海洋大学 | Intelligent source code conversion method based on supervised learning |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102043774A (en) * | 2011-01-13 | 2011-05-04 | 北京交通大学 | Machine translation evaluation device and method |
| CN108829685A (en) * | 2018-05-07 | 2018-11-16 | 内蒙古工业大学 | A kind of illiteracy Chinese inter-translation method based on single language training |
| WO2019107625A1 (en) * | 2017-11-30 | 2019-06-06 | 주식회사 시스트란인터내셔널 | Machine translation method and apparatus therefor |
| CN112464676A (en) * | 2020-12-02 | 2021-03-09 | 北京捷通华声科技股份有限公司 | Machine translation result scoring method and device |
| CN113673262A (en) * | 2014-12-29 | 2021-11-19 | 贝宝公司 | Machine translation between different languages using statistical streaming data |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20200075615A (en) * | 2018-12-18 | 2020-06-26 | 삼성전자주식회사 | Method and apparatus for machine translation |
| CN109829172B (en) * | 2019-01-04 | 2023-07-04 | 北京先声智能科技有限公司 | Automatic grammar correction model of bi-directional decoding based on nerve translation |
| CN112446221B (en) * | 2019-08-14 | 2023-12-15 | 阿里巴巴集团控股有限公司 | Translation evaluation method, device, system and computer storage medium |
| CN112800785B (en) * | 2021-04-13 | 2021-07-27 | 中国科学院自动化研究所 | Multimodal machine translation method, device, electronic equipment and storage medium |
| CN113901847A (en) * | 2021-09-16 | 2022-01-07 | 昆明理工大学 | Neural machine translation method based on source language syntax enhanced decoding |
-
2022
- 2022-03-07 CN CN202210217054.8A patent/CN114298061B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102043774A (en) * | 2011-01-13 | 2011-05-04 | 北京交通大学 | Machine translation evaluation device and method |
| CN113673262A (en) * | 2014-12-29 | 2021-11-19 | 贝宝公司 | Machine translation between different languages using statistical streaming data |
| WO2019107625A1 (en) * | 2017-11-30 | 2019-06-06 | 주식회사 시스트란인터내셔널 | Machine translation method and apparatus therefor |
| CN108829685A (en) * | 2018-05-07 | 2018-11-16 | 内蒙古工业大学 | A kind of illiteracy Chinese inter-translation method based on single language training |
| CN112464676A (en) * | 2020-12-02 | 2021-03-09 | 北京捷通华声科技股份有限公司 | Machine translation result scoring method and device |
Non-Patent Citations (1)
| Title |
|---|
| 《基于编码器-解码器重构框架的蒙汉神经机器翻译》;孙晓骞等;《计算机应用与软件》;20200424;第150-155,163页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114298061A (en) | 2022-04-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Blattmann et al. | Align your latents: High-resolution video synthesis with latent diffusion models | |
| CN110612538B (en) | Generating discrete potential representations of input data items | |
| CN108170686B (en) | Text translation method and device | |
| US20240054767A1 (en) | Multi-modal Model Training Method, Apparatus and Device, and Storage Medium | |
| CN111241789A (en) | Text generation method and device | |
| CN107766319B (en) | Sequence conversion method and device | |
| CN114298061B (en) | Machine translation and model training quality evaluation method, electronic device and storage medium | |
| CN111401037B (en) | Natural language generation method and device, electronic equipment and storage medium | |
| CN110619124A (en) | Named entity identification method and system combining attention mechanism and bidirectional LSTM | |
| JP2022552382A (en) | Audio transmission method and its system, apparatus, computer program, and computer equipment | |
| CN107832300A (en) | Towards minimally invasive medical field text snippet generation method and device | |
| CN111539199A (en) | Text error correction method, device, terminal and storage medium | |
| CN112446221B (en) | Translation evaluation method, device, system and computer storage medium | |
| US11694041B2 (en) | Chapter-level text translation method and device | |
| CN111738435A (en) | Online sparse training method and system based on mobile equipment | |
| CN111723194A (en) | Abstract generation method, device and equipment | |
| CN116208772A (en) | Data processing method, device, electronic equipment and computer readable storage medium | |
| CN116434752A (en) | Speech recognition error correction method and device | |
| CN116958738A (en) | Training method and device of picture recognition model, storage medium and electronic equipment | |
| CN116070591A (en) | Natural text coding method based on contrast learning | |
| CN115167863A (en) | Code completion method and device based on code sequence and code graph fusion | |
| US20220171926A1 (en) | Information processing method, storage medium, and information processing device | |
| CN115941950A (en) | Encoding method, decoding method, encoding device, and decoding device | |
| CN115496134A (en) | Traffic scene video description generation method and device based on multi-modal feature fusion | |
| CN114492457B (en) | Semantic recognition method, semantic recognition device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Room 554, 5 / F, building 3, 969 Wenyi West Road, Wuchang Street, Yuhang District, Hangzhou City, Zhejiang Province Applicant after: Alibaba (China) Co.,Ltd. Address before: Room 508, 5 / F, building 4, No. 699, Wangshang Road, Changhe street, Binjiang District, Hangzhou City, Zhejiang Province Applicant before: Alibaba (China) Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |