CN109635269A - A kind of post-editing method and device of machine translation text - Google Patents
A kind of post-editing method and device of machine translation text Download PDFInfo
- Publication number
- CN109635269A CN109635269A CN201910079518.1A CN201910079518A CN109635269A CN 109635269 A CN109635269 A CN 109635269A CN 201910079518 A CN201910079518 A CN 201910079518A CN 109635269 A CN109635269 A CN 109635269A
- Authority
- CN
- China
- Prior art keywords
- text
- post
- editing
- machine translation
- vector
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/40—Processing or translation of natural language
- G06F40/58—Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Machine Translation (AREA)
Abstract
The invention discloses a kind of post-editing methods of machine translation text, comprising: obtains source text and machine translation text；By extracting the first text feature of source text from attention mechanism, and the first text feature is handled using feedforward neural network, obtains the primary vector for indicating source text；By the second text feature from attention mechanism extraction machine cypher text, by using the second text feature of attention mechanism optimization to primary vector；The second text feature after optimization is handled using feedforward neural network, obtains the secondary vector for indicating machine translation text；Generate the post-editing text of machine translation text by word from left to right according to primary vector and secondary vector.This method can be improved the treatment effeciency and accuracy rate of post-editing, so that the accuracy for the post-editing text that processing obtains is more preferably.Post-editing device, equipment and the readable storage medium storing program for executing of a kind of machine translation text disclosed by the invention, similarly have above-mentioned technique effect.
Description
Technical field
The present invention relates to text automatic translation technical field, more specifically to after the translating of a kind of machine translation text
Edit methods, device, equipment and readable storage medium storing program for executing.
Background technique
Machine translation is also known as automatic translation, be using computer a kind of natural source language shift is another nature mesh
The process of poster speech, refers generally to the translation of sentence and full text between natural language.Correspondingly, machine translation text, which refers to, utilizes calculating
Machine translates a kind of language text, obtained another language text.Post-editing refers to the cypher text generated to machine
Perfect process is carried out, so that machine translation text is more in line with human language style.
In the prior art, it is generally basede on Recognition with Recurrent Neural Network and realizes automatically processing for post-editing.It should be noted that
The feature for the language text that Recognition with Recurrent Neural Network extracts is not fine enough, wherein using it is logarithmic linear combination come handle source text and
Machine translation text, the feature that can not be also associated between source text and machine translation text, causes to source text and machine translation
The characterization scarce capacity of text, so that the accuracy rate of post-editing is reduced, so that the post-editing text that post-editing obtains
Accuracy decrease.Wherein, post-editing text is after carrying out post-editing processing to machine translation text, to obtain
Text.
Therefore, the accuracy rate for how improving post-editing is those skilled in the art's problem to be solved.
Summary of the invention
The purpose of the present invention is to provide a kind of post-editing method, apparatus of machine translation text, equipment and readable deposit
Storage media, to improve the accuracy rate of post-editing.
To achieve the above object, the embodiment of the invention provides following technical solutions:
A kind of post-editing method of machine translation text, comprising:
Obtain the machine translation text of source text and the source text；
By extracting the first text feature of the source text from attention mechanism, and using feedforward neural network to described
First text feature is handled, and the primary vector for indicating the source text is obtained；
By extracting the second text feature of the machine translation text from attention mechanism, by the primary vector
Use the second text feature described in attention mechanism optimization；Using feedforward neural network to second text feature after optimization
It is handled, obtains the secondary vector for indicating the machine translation text；
After generating the translating of the machine translation text by word from left to right according to the primary vector and the secondary vector
Edit text.
Wherein, first text feature by extracting the source text from attention mechanism, and utilize feed forward neural
Network handles first text feature, obtains the primary vector for indicating the source text, comprising:
By source text described in residual error Processing with Neural Network, the primary vector is obtained；
Wherein, each network layer in the residual error neural network is by from attention mechanism sublayer and Feedforward Neural Networks string bag
Layer is constituted.
It is wherein, described by using the second text feature described in attention mechanism optimization to the primary vector, comprising:
According to the second text feature described in attention mechanism processing formula optimization, the attention mechanism handles formula are as follows:
Wherein, Q indicates the query term in second text feature；K, V indicate a pair of of key assignments.
Wherein, described that the machine translation text is generated by word from left to right according to the primary vector and the secondary vector
This post-editing text, comprising:
The post-editing text, the text generation formula are generated according to text generation formula are as follows:
Wherein, the x expression primary vector, the m expression secondary vector, the y expression post-editing text, P (y | m,
X) conditional probability of the generation post-editing text is indicated；The item that any one word in the post-editing text generates
Part probability are as follows: P (yt|y< t, m, x) and=Softmax (Wo·zt+bo), ytIndicate the word that t moment generates, WoAnd boTo generate ginseng
Number, ZtIndicate the output result after network layer.
Wherein, described that the machine translation text is generated by word from left to right according to the primary vector and the secondary vector
After this post-editing text, further includes:
Calculate the cross entropy loss function value of the standard translation text of the post-editing text and the source text；
Judge whether the cross entropy loss function value is less than preset threshold value；
If it is not, then carrying updated generation parameter according to the more newly-generated parameter of cross entropy loss function value and executing
The post-editing for generating the machine translation text by word from left to right according to the primary vector and the secondary vector
The step of text.
Wherein, it is described calculate the post-editing text and the standard translation text of the source text intersect entropy loss letter
Numerical value, comprising:
The standard translation text is obtained, by extracting the of the standard translation text from attention mechanism with mask
Three text features；
By using third text feature described in attention mechanism optimization to the primary vector, and by described second
Vector uses third text feature described in the second suboptimization of attention mechanism；
The third text feature after the second suboptimization is handled using feedforward neural network, is obtained described in expression
The third vector of standard translation text；
The post-editing text vector is turned into the 4th vector, and calculates the 4th vector and the third vector
Cross entropy loss function value.
A kind of post-editing device of machine translation text, comprising:
Module is obtained, for obtaining the machine translation text of source text and the source text；
First processing module for the first text feature by extracting the source text from attention mechanism, and utilizes
Feedforward neural network handles first text feature, obtains the primary vector for indicating the source text；
Second processing module, for the second text feature by extracting the machine translation text from attention mechanism,
By using the second text feature described in attention mechanism optimization to the primary vector；Using feedforward neural network to optimization after
Second text feature handled, obtain the secondary vector for indicating the machine translation text；
Generation module is turned over for generating the machine by word from left to right according to the primary vector and the secondary vector
The post-editing text of translation sheet.
Wherein, further includes:
Computing module is damaged for calculating the cross entropy of standard translation text of the post-editing text and the source text
Lose functional value；
Judgment module, for judging whether the cross entropy loss function value is less than preset threshold value；
Execution module is used for when the probability value is not less than preset threshold value, according to the cross entropy loss function value
More newly-generated parameter carries updated generation parameter and executes the step in the generation module.
A kind of post-editing equipment of machine translation text, comprising:
Memory, for storing computer program；
Processor realizes translating for machine translation text described in above-mentioned any one when for executing the computer program
The step of postedit method.
A kind of readable storage medium storing program for executing is stored with computer program, the computer program quilt on the readable storage medium storing program for executing
The step of processor realizes the post-editing method of machine translation text described in above-mentioned any one when executing.
By above scheme it is found that a kind of post-editing method of machine translation text provided in an embodiment of the present invention, packet
It includes: obtaining the machine translation text of source text and the source text；By extracting the first of the source text from attention mechanism
Text feature, and first text feature is handled using feedforward neural network, obtain indicate the source text
One vector；By extracting the second text feature of the machine translation text from attention mechanism, by the primary vector
Use the second text feature described in attention mechanism optimization；Using feedforward neural network to second text feature after optimization
It is handled, obtains the secondary vector for indicating the machine translation text；According to the primary vector and the secondary vector from
Left-to-right generates the post-editing text of the machine translation text by word.
As it can be seen that the method by from attention mechanism extract source text and machine translation text text feature, can
The internal structure of source text and machine translation text is captured, so that the text feature extracted is specifically and finely, so as to
Improve the accuracy rate of the post-editing of machine translation text；Meanwhile attention mechanism is used by the primary vector to source text
Second text feature of optimization machine translation text can mention to be associated with the feature between source text and machine translation text
The generalization ability of high post-editing；Feedforward neural network can be further increased in conjunction with the characterization information of different location for sentence
The information representation ability of son.Therefore this method can be improved the treatment effeciency and accuracy rate of post-editing, so that processing obtained
The accuracy of post-editing text is more preferably.
Correspondingly, it a kind of post-editing device of machine translation text provided in an embodiment of the present invention, equipment and readable deposits
Storage media similarly has above-mentioned technique effect.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
It obtains other drawings based on these drawings.
Fig. 1 is a kind of post-editing method flow diagram of machine translation text disclosed by the embodiments of the present invention；
Fig. 2 is the post-editing method flow diagram of another machine translation text disclosed by the embodiments of the present invention；
Fig. 3 is a kind of post-editing schematic device of machine translation text disclosed by the embodiments of the present invention；
Fig. 4 is a kind of post-editing equipment schematic diagram of machine translation text disclosed by the embodiments of the present invention；
Fig. 5 is a kind of post-editing network model framework schematic diagram disclosed by the embodiments of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
The embodiment of the invention discloses post-editing method, apparatus, equipment and the readable storages of a kind of machine translation text
Medium, to improve the accuracy rate of post-editing.
Referring to Fig. 1, a kind of post-editing method of machine translation text provided in an embodiment of the present invention, comprising:
S101, the machine translation text for obtaining source text and source text；
Specifically, the machine translation text of source text is to the text after source text progress machine translation, obtained.
S102, the first text feature by extracting source text from attention mechanism, and using feedforward neural network to the
One text feature is handled, and the primary vector for indicating source text is obtained；
S103, by the second text feature from attention mechanism extraction machine cypher text, by making to primary vector
With the second text feature of attention mechanism optimization；The second text feature after optimization is handled using feedforward neural network,
Obtain indicating the secondary vector of machine translation text；
S104, the post-editing text for generating machine translation text by word from left to right according to primary vector and secondary vector
This.
It should be noted that attention mechanism has imitated the internal procedure of biological observation behavior, i.e., it is a kind of by internal experience
With external sensation alignment to the mechanism of the subregional observation fineness of increased portion.Attention mechanism can be with the sparse number of rapidly extracting
According to important feature, thus be widely used in natural language processing task.It can learn sentence therein from attention mechanism
Dependence between different location.
Attention mechanism is generally used for handling machine translation duties, and in this application, attention mechanism is used to handle
Post-editing handles task, and combines the text feature from attention mechanism crawl source text and machine translation text, can not only
Specific and fine text feature is enough extracted, and the treatment effeciency of post-editing can be improved.
As it can be seen that present embodiments providing a kind of post-editing method of machine translation text, the method passes through from attention
Power mechanism extracts the text feature of source text and machine translation text, can capture the internal junction of source text and machine translation text
Structure, so that the text feature extracted is specifically and finely, so that the accurate of the post-editing of machine translation text can be improved
Rate；Meanwhile the second text feature of attention mechanism optimization machine translation text is used by the primary vector to source text, from
And it is associated with the feature between source text and machine translation text, the generalization ability of post-editing can be improved；Feedforward neural network
The information representation ability for sentence can be further increased in conjunction with the characterization information of different location.Therefore this method can mention
The treatment effeciency and accuracy rate of high post-editing, so that the accuracy for the post-editing text that processing obtains is more preferably.
The embodiment of the invention discloses the post-editing method of another machine translation text, relative to a upper embodiment,
The present embodiment has made further instruction and optimization to technical solution.
Referring to fig. 2, the post-editing method of another machine translation text provided in an embodiment of the present invention, comprising:
S201, the machine translation text for obtaining source text and source text；
S202, the first text feature by extracting source text from attention mechanism, and using feedforward neural network to the
One text feature is handled, and the primary vector for indicating source text is obtained；
S203, by the second text feature from attention mechanism extraction machine cypher text, by making to primary vector
With the second text feature of attention mechanism optimization；The second text feature after optimization is handled using feedforward neural network,
Obtain indicating the secondary vector of machine translation text；
S204, the post-editing text for generating machine translation text by word from left to right according to primary vector and secondary vector
This；
205, the cross entropy loss function value of the standard translation text of post-editing text and source text is calculated；
Specifically, the standard translation text of source text are as follows: carry out the machine translation text that machine translation obtains to source text
After carrying out post-editing, what is obtained meets the final text of human language style.Calculate post-editing text and standard translation text
This cross entropy loss function value is it is to be understood that judge the similarity of post-editing text Yu standard translation text.
When the cross entropy loss function value of post-editing text and standard translation text is larger, show post-editing text
It is smaller with the similarity of standard translation text, it is believed that the two is not identical, and post-editing text also needs to be optimized and locates
Reason；When the cross entropy loss function value of post-editing text and standard translation text is smaller, show post-editing text and mark
The similarity of quasi- cypher text is bigger, to a certain extent it is considered that the two is identical.
The present embodiment takes into account the loss function of sentence level, can provide more preferably for the generation of post-editing text
Optimization foundation.
206, judge whether cross entropy loss function value is less than preset threshold value；If so, executing S208；If it is not, then holding
Row S207；
S207, according to the more newly-generated parameter of cross entropy loss function value, carry updated generations parameter execution
S204；
Specifically, the loss of post-editing text may be considered the difference of post-editing text Yu standard translation text,
Generally indicated with the editing distance of two texts.If the editing distance of two texts is smaller, show that the two texts get over phase
Seemingly.
S208, the standard translation result that the post-editing text of generation is determined as to machine translation text.
Wherein, it is described calculate the post-editing text and the standard translation text of the source text intersect entropy loss letter
Numerical value, comprising:
The standard translation text is obtained, by extracting the of the standard translation text from attention mechanism with mask
Three text features；
By using third text feature described in attention mechanism optimization to the primary vector, and by described second
Vector uses third text feature described in the second suboptimization of attention mechanism；
The third text feature after the second suboptimization is handled using feedforward neural network, is obtained described in expression
The third vector of standard translation text；
The post-editing text vector is turned into the 4th vector, and calculates the 4th vector and the third vector
Cross entropy loss function value.
As it can be seen that present embodiments providing the post-editing method of another machine translation text, the method passes through from note
Power mechanism of anticipating extracts the text feature of source text and machine translation text, can capture the inside of source text and machine translation text
Structure, so that the text feature extracted is specifically and finely, so that the standard of the post-editing of machine translation text can be improved
True rate；Meanwhile the second text feature of attention mechanism optimization machine translation text is used by the primary vector to source text,
To be associated with the feature between source text and machine translation text, the generalization ability of post-editing can be improved；Feedforward Neural Networks
Network can further increase the information representation ability for sentence in conjunction with the characterization information of different location.Therefore this method can
The treatment effeciency and accuracy rate of post-editing are improved, so that the accuracy for the post-editing text that processing obtains is more preferably.
Based on above-mentioned any embodiment, it should be noted that described by extracting the source text from attention mechanism
First text feature, and first text feature is handled using feedforward neural network, it obtains indicating the source text
Primary vector, comprising:
By source text described in residual error Processing with Neural Network, the primary vector is obtained；
Wherein, each network layer in the residual error neural network is by from attention mechanism sublayer and Feedforward Neural Networks string bag
Layer is constituted.
Based on above-mentioned any embodiment, it should be noted that described by using attention mechanism to the primary vector
Optimize second text feature, comprising:
According to the second text feature described in attention mechanism processing formula optimization, the attention mechanism handles formula are as follows:
Wherein, Q indicates the query term in second text feature；K, V indicate a pair of of key assignments.
Based on above-mentioned any embodiment, it should be noted that it is described according to the primary vector and the secondary vector from
Left-to-right generates the post-editing text of the machine translation text by word, comprising:
The post-editing text, the text generation formula are generated according to text generation formula are as follows:
Wherein, the x expression primary vector, the m expression secondary vector, the y expression post-editing text, P (y | m,
X) conditional probability of the generation post-editing text is indicated；The item that any one word in the post-editing text generates
Part probability are as follows: P (yt|y< t, m, x) and=Softmax (Wo·zt+bo), ytIndicate the word that t moment generates, WoAnd boTo generate ginseng
Number, ZtIndicate the output result after network layer.
Wherein, if handling model, the network layer according to post-editing method provided by the invention building post-editing
The final layer of as entire post-editing processing model.
A kind of post-editing device of machine translation text provided in an embodiment of the present invention is introduced below, is hereafter retouched
A kind of post-editing device for the machine translation text stated and a kind of above-described post-editing method of machine translation text
It can be cross-referenced.
Referring to Fig. 3, a kind of post-editing device of machine translation text provided in an embodiment of the present invention, comprising:
Module 301 is obtained, for obtaining the machine translation text of source text and the source text；
First processing module 302, for the first text feature by extracting the source text from attention mechanism, and benefit
First text feature is handled with feedforward neural network, obtains the primary vector for indicating the source text；
Second processing module 303, it is special for the second text by extracting the machine translation text from attention mechanism
Sign, by using the second text feature described in attention mechanism optimization to the primary vector；Using feedforward neural network to excellent
Second text feature after change is handled, and the secondary vector for indicating the machine translation text is obtained；
Generation module 304, for generating the machine by word from left to right according to the primary vector and the secondary vector
The post-editing text of device cypher text.
Wherein, further includes:
Computing module is damaged for calculating the cross entropy of standard translation text of the post-editing text and the source text
Lose functional value；
Judgment module, for judging whether the cross entropy loss function value is less than preset threshold value；
Execution module is used for when the probability value is not less than preset threshold value, according to the cross entropy loss function value
More newly-generated parameter carries updated generation parameter and executes the step in the generation module.
Wherein, the computing module includes:
Acquiring unit, for obtaining the standard translation text, by extracting the mark from attention mechanism with mask
The third text feature of quasi- cypher text；
First optimization unit, for by special using third text described in attention mechanism optimization to the primary vector
Sign, and by using third text feature described in the second suboptimization of attention mechanism to the secondary vector；
Second optimization unit, for being carried out using feedforward neural network to the third text feature after the second suboptimization
Processing, obtains the third vector for indicating the standard translation text；
Computing unit, for the post-editing text vector to be turned to the 4th vector, and calculate the 4th vector with
The cross entropy loss function value of the third vector.
Wherein, the first processing module is specifically used for:
By source text described in residual error Processing with Neural Network, the primary vector is obtained；
Wherein, each network layer in the residual error neural network is by from attention mechanism sublayer and Feedforward Neural Networks string bag
Layer is constituted.
Wherein, the Second processing module is specifically used for:
According to the second text feature described in attention mechanism processing formula optimization, the attention mechanism handles formula are as follows:
Wherein, Q indicates the query term in second text feature；K, V indicate a pair of of key assignments.
Wherein, the generation module is specifically used for:
The post-editing text, the text generation formula are generated according to text generation formula are as follows:
Wherein, the x expression primary vector, the m expression secondary vector, the y expression post-editing text, P (y | m,
X) conditional probability of the generation post-editing text is indicated；The item that any one word in the post-editing text generates
Part probability are as follows: P (yt|y< t, m, x) and=Softmax (Wo·zt+bo), ytIndicate the word that t moment generates, WoAnd boTo generate ginseng
Number, ZtIndicate the output result after network layer.
As it can be seen that present embodiments providing a kind of post-editing device of machine translation text, comprising: obtain module, first
Processing module, Second processing module and generation module.The machine of source text and the source text is obtained by acquisition module first
Cypher text；Then first processing module is by extracting the first text feature of the source text from attention mechanism, and utilizes
Feedforward neural network handles first text feature, obtains the primary vector for indicating the source text；And then second
Processing module is by extracting the second text feature of the machine translation text from attention mechanism, by the primary vector
Use the second text feature described in attention mechanism optimization；Using feedforward neural network to second text feature after optimization
It is handled, obtains the secondary vector for indicating the machine translation text；Module is ultimately produced according to the primary vector and institute
State the post-editing text that secondary vector generates the machine translation text by word from left to right.Divide the work between such modules
Cooperation, Each performs its own functions, so that the treatment effeciency and accuracy rate of post-editing are improved, so that the post-editing text that processing obtains
Accuracy more preferably.
A kind of post-editing equipment of machine translation text provided in an embodiment of the present invention is introduced below, is hereafter retouched
A kind of post-editing equipment for the machine translation text stated and a kind of above-described post-editing method of machine translation text
And device can be cross-referenced.
Referring to fig. 4, the post-editing equipment of a kind of machine translation text provided in an embodiment of the present invention, comprising:
Memory 401, for storing computer program；
Processor 402 realizes the text of machine translation described in above-mentioned any embodiment when for executing the computer program
The step of this post-editing method.
Wherein, processor can be central processing unit (CPU) or graphics processor (GPU).GPU is in processing large-scale data
When, there is good advantage.
A kind of readable storage medium storing program for executing provided in an embodiment of the present invention is introduced below, one kind described below is readable to deposit
Storage media can be cross-referenced with a kind of above-described post-editing method, device and equipment of machine translation text.
A kind of readable storage medium storing program for executing is stored with computer program, the computer program quilt on the readable storage medium storing program for executing
The step of post-editing method of the machine translation text as described in above-mentioned any embodiment is realized when processor executes.
Post-editing network model as shown in Figure 5, the model can be constructed according to post-editing method provided by the invention
Including source document present treatment network, machine translation text processing network and standard translation text processing network, wherein source document present treatment
Network, machine translation text processing network and standard translation text processing network are residual error network.
Source document present treatment network is N layers, and each layer from attention mechanism sublayer and feedforward neural network sublayer by constituting.Machine
It is N layers that device cypher text, which handles network, and each layer is by from attention mechanism sublayer, attention mechanism sublayer and feedforward neural network
Sublayer is constituted, and the attention mechanism sublayer in machine translation text processing network indicates to use attention mechanism to source text.Mark
Quasi- cypher text processing network is N layers, each layer by with mask from attention mechanism sublayer, attention mechanism sublayer and feedforward
Neural network sublayer is constituted, and the attention mechanism sublayer in standard translation text processing network includes: to use attention to source text
Power mechanism and to machine translation text use attention mechanism.
Wherein, attention mechanism is calculated and is looked by a map locating (Query) and one group of key-value pair (key-values)
After item Q (query) is ask to the dot product of all key K (key) values, divided byIt zooms in and out, finally uses a softmax function
Obtain the weight distribution of key assignments K (values).It can specifically be described with following formula:
Bull attention mechanism allows model to combine attention from different characterization subspaces in the information of different location, can use
Following formula indicates:
MultiHead (Q, K, V)=Concat (head1..., headh)Wo
whereheadi=Attention (QWi Q, KWi K, VWi V)
Wherein,
Feedforward neural network includes two linear changes, and ReLu activation primitive is used between linear transformation, can be with following
Formula indicates:
FFN (x)=max (0, xW1+b1)W2+b2
Wherein, W1、W2、b1、b2Being can training parameter.
Discriminator in Fig. 5 is the arbiter in post-editing network model, and the arbiter is using adopting
With Recognition with Recurrent Neural Network, selection characterizes sentence using two-way (Gated Recurrent Unit, abbreviation GRU) structure.Differentiate
Device reads in post-editing text and standard translation text, obtains content after the word insertion of two sentences is characterized with two-way GRU
Vector gives loss function, and loss function target is differentiated between generation text and referenced text, so that is differentiated gets over
Come more quasi-.
Arbiter differentiates the calculation formula of the intersection entropy loss of post-editing text and standard translation text are as follows:
P (y, r)=sigmoid (Wd·||Hy-Hr||+bd)
The loss function of arbiter is indicated with following formula:
L(Hy, Hr)=- log (sigmoid (Wd·||Hy-Hr||+bd))
Wherein, | | Hy-Hr| | indicate the Euclidean distance between post-editing text and the content vector of standard translation text,
WdAnd bdBeing can training parameter.
When the differentiation result of arbiter output is unsatisfactory for preset output condition, the loss of post-editing text is calculated,
And feed back the loss, to optimize the network parameter of post-editing network model, make to generate more accurately post-editing text
This.
Wherein, the maximization desired value setting of the objective function of post-editing text is generated are as follows:
The post-editing text of generation is sampled, and calculates gradient
ThereforeThe parameter renewal function of generator are as follows:
When the arbiter in training post-editing network model, freeze generator parameter, and minimize the damage of arbiter
Lose function.Specifically, every training generator for carrying out 4 epoch, reuses an epoch training arbiter, successively iteration is instructed
Practice, the deconditioning after the generator and arbiter of model are restrained.
Each embodiment in this specification is described in a progressive manner, the highlights of each of the examples are with other
The difference of embodiment, the same or similar parts in each embodiment may refer to each other.
The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present invention.
Various modifications to these embodiments will be readily apparent to those skilled in the art, as defined herein
General Principle can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, of the invention
It is not intended to be limited to the embodiments shown herein, and is to fit to and the principles and novel features disclosed herein phase one
The widest scope of cause.
Claims (10)
1. a kind of post-editing method of machine translation text characterized by comprising
Obtain the machine translation text of source text and the source text；
By extracting the first text feature of the source text from attention mechanism, and using feedforward neural network to described first
Text feature is handled, and the primary vector for indicating the source text is obtained；
By extracting the second text feature of the machine translation text from attention mechanism, by using the primary vector
Second text feature described in attention mechanism optimization；Second text feature after optimization is carried out using feedforward neural network
Processing, obtains the secondary vector for indicating the machine translation text；
Generate the post-editing of the machine translation text by word from left to right according to the primary vector and the secondary vector
Text.
2. the post-editing method of machine translation text according to claim 1, which is characterized in that described by paying attention to certainly
Power mechanism extracts the first text feature of the source text, and using feedforward neural network to first text feature at
Reason, obtains the primary vector for indicating the source text, comprising:
By source text described in residual error Processing with Neural Network, the primary vector is obtained；
Wherein, each network layer in the residual error neural network is by from attention mechanism sublayer and feedforward neural network sublayer structure
At.
3. the post-editing method of machine translation text according to claim 2, which is characterized in that described by described
Primary vector uses the second text feature described in attention mechanism optimization, comprising:
According to the second text feature described in attention mechanism processing formula optimization, the attention mechanism handles formula are as follows:
Wherein, Q indicates the query term in second text feature；K, V indicate a pair of of key assignments.
4. the post-editing method of machine translation text according to claim 3, which is characterized in that described according to described
Secondary vector described in one vector sum generates the post-editing text of the machine translation text by word from left to right, comprising:
The post-editing text, the text generation formula are generated according to text generation formula are as follows:
Wherein, the x expression primary vector, the m expression secondary vector, the y expression post-editing text, P (y | m, x) table
Show the conditional probability for generating the post-editing text；The condition that any one word in the post-editing text generates is general
Rate are as follows: P (yt|y< t, m, x) and=Softmax (Wo·zt+bo), ytIndicate the word that t moment generates, WoAnd boTo generate parameter, Zt
Indicate the output result after network layer.
5. the post-editing method of machine translation text according to any one of claims 1-4, which is characterized in that described
Generate the post-editing text of the machine translation text by word from left to right according to the primary vector and the secondary vector
Later, further includes:
Calculate the cross entropy loss function value of the standard translation text of the post-editing text and the source text；
Judge whether the cross entropy loss function value is less than preset threshold value；
If it is not, then being carried described in updated generation parameter execution according to the more newly-generated parameter of cross entropy loss function value
Generate the post-editing text of the machine translation text by word from left to right according to the primary vector and the secondary vector
The step of.
6. the post-editing method of machine translation text according to claim 5, which is characterized in that translated described in the calculating
The cross entropy loss function value of the standard translation text of postedit text and the source text, comprising:
The standard translation text is obtained, the third text for extracting the standard translation text from attention mechanism with mask is passed through
Eigen；
By using third text feature described in attention mechanism optimization to the primary vector, and by the secondary vector
Use third text feature described in the second suboptimization of attention mechanism；
The third text feature after the second suboptimization is handled using feedforward neural network, obtains indicating the standard
The third vector of cypher text；
The post-editing text vector is turned into the 4th vector, and calculates intersecting for the 4th vector and the third vector
Entropy loss functional value.
7. a kind of post-editing device of machine translation text characterized by comprising
Module is obtained, for obtaining the machine translation text of source text and the source text；
First processing module for the first text feature by extracting the source text from attention mechanism, and utilizes feedforward
Neural network handles first text feature, obtains the primary vector for indicating the source text；
Second processing module passes through for the second text feature by extracting the machine translation text from attention mechanism
Second text feature described in attention mechanism optimization is used to the primary vector；Using feedforward neural network to the institute after optimization
It states the second text feature to be handled, obtains the secondary vector for indicating the machine translation text；
Generation module, for generating the machine translation text by word from left to right according to the primary vector and the secondary vector
This post-editing text.
8. the post-editing device of machine translation text according to claim 7, which is characterized in that further include:
Computing module intersects entropy loss letter for calculate the post-editing text and the standard translation text of the source text
Numerical value；
Judgment module, for judging whether the cross entropy loss function value is less than preset threshold value；
Execution module, for being updated according to the cross entropy loss function value when the probability value is not less than preset threshold value
Parameter is generated, updated generation parameter is carried and executes the step in the generation module.
9. a kind of post-editing equipment of machine translation text characterized by comprising
Memory, for storing computer program；
Processor realizes machine translation text as claimed in any one of claims 1 to 6 when for executing the computer program
The step of this post-editing method.
10. a kind of readable storage medium storing program for executing, which is characterized in that be stored with computer program, the meter on the readable storage medium storing program for executing
Calculation machine program realizes the post-editing side of machine translation text as claimed in any one of claims 1 to 6 when being executed by processor
The step of method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910079518.1A CN109635269B (en) | 2019-01-31 | 2019-01-31 | Post-translation editing method and device for machine translation text |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910079518.1A CN109635269B (en) | 2019-01-31 | 2019-01-31 | Post-translation editing method and device for machine translation text |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109635269A true CN109635269A (en) | 2019-04-16 |
| CN109635269B CN109635269B (en) | 2023-06-16 |
Family
ID=66062387
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910079518.1A Active CN109635269B (en) | 2019-01-31 | 2019-01-31 | Post-translation editing method and device for machine translation text |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109635269B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110765791A (en) * | 2019-11-01 | 2020-02-07 | 清华大学 | Automatic post-editing method and device for machine translation |
| CN110909527A (en) * | 2019-12-03 | 2020-03-24 | 北京字节跳动网络技术有限公司 | Text processing model operation method and device, electronic equipment and storage medium |
| CN116069901A (en) * | 2023-02-03 | 2023-05-05 | 上海一者信息科技有限公司 | Non-translated element identification method based on editing behavior and rule |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107301173A (en) * | 2017-06-22 | 2017-10-27 | 北京理工大学 | A kind of automatic post-editing system and method for multi-source neutral net that mode is remixed based on splicing |
| CN107967262A (en) * | 2017-11-02 | 2018-04-27 | 内蒙古工业大学 | A kind of neutral net covers Chinese machine translation method |
| CN108563640A (en) * | 2018-04-24 | 2018-09-21 | 中译语通科技股份有限公司 | A kind of multilingual pair of neural network machine interpretation method and system |
| CN109241536A (en) * | 2018-09-21 | 2019-01-18 | 浙江大学 | It is a kind of based on deep learning from the sentence sort method of attention mechanism |
| CN109271646A (en) * | 2018-09-04 | 2019-01-25 | 腾讯科技（深圳）有限公司 | Text interpretation method, device, readable storage medium storing program for executing and computer equipment |
-
2019
- 2019-01-31 CN CN201910079518.1A patent/CN109635269B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107301173A (en) * | 2017-06-22 | 2017-10-27 | 北京理工大学 | A kind of automatic post-editing system and method for multi-source neutral net that mode is remixed based on splicing |
| CN107967262A (en) * | 2017-11-02 | 2018-04-27 | 内蒙古工业大学 | A kind of neutral net covers Chinese machine translation method |
| CN108563640A (en) * | 2018-04-24 | 2018-09-21 | 中译语通科技股份有限公司 | A kind of multilingual pair of neural network machine interpretation method and system |
| CN109271646A (en) * | 2018-09-04 | 2019-01-25 | 腾讯科技（深圳）有限公司 | Text interpretation method, device, readable storage medium storing program for executing and computer equipment |
| CN109241536A (en) * | 2018-09-21 | 2019-01-18 | 浙江大学 | It is a kind of based on deep learning from the sentence sort method of attention mechanism |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110765791A (en) * | 2019-11-01 | 2020-02-07 | 清华大学 | Automatic post-editing method and device for machine translation |
| CN110765791B (en) * | 2019-11-01 | 2021-04-06 | 清华大学 | Automatic post-editing method and device for machine translation |
| CN110909527A (en) * | 2019-12-03 | 2020-03-24 | 北京字节跳动网络技术有限公司 | Text processing model operation method and device, electronic equipment and storage medium |
| CN110909527B (en) * | 2019-12-03 | 2023-12-08 | 北京字节跳动网络技术有限公司 | Text processing model running method and device, electronic equipment and storage medium |
| CN116069901A (en) * | 2023-02-03 | 2023-05-05 | 上海一者信息科技有限公司 | Non-translated element identification method based on editing behavior and rule |
| CN116069901B (en) * | 2023-02-03 | 2023-08-11 | 上海一者信息科技有限公司 | Non-translated element identification method based on editing behavior and rule |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109635269B (en) | 2023-06-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110334219B (en) | Knowledge graph representation learning method based on attention mechanism integrated with text semantic features | |
| Xiang et al. | A convolutional neural network-based linguistic steganalysis for synonym substitution steganography | |
| Lee et al. | Transfer learning for deep learning on graph-structured data | |
| CN109635269A (en) | A kind of post-editing method and device of machine translation text | |
| CN107480132A (en) | A kind of classic poetry generation method of image content-based | |
| CN111966812B (en) | Automatic question answering method based on dynamic word vector and storage medium | |
| CN108845990A (en) | Answer selection method, device and electronic equipment based on two-way attention mechanism | |
| CN107220220A (en) | Electronic equipment and method for text-processing | |
| CN110334357A (en) | A kind of method, apparatus, storage medium and electronic equipment for naming Entity recognition | |
| CN109977861A (en) | Offline handwritten form method for identifying mathematical formula | |
| CN108920446A (en) | A kind of processing method of Engineering document | |
| CN113094596A (en) | Multitask rumor detection method based on bidirectional propagation diagram | |
| Zhang et al. | Bayesian attention belief networks | |
| CN108052683A (en) | A kind of knowledge mapping based on cosine measurement rule represents learning method | |
| Cui et al. | Learning topic representation for smt with neural networks | |
| Ye et al. | Multi-level composite neural networks for medical question answer matching | |
| Fei et al. | Uncertainty-aware image captioning | |
| CN109740672A (en) | Multi-streaming feature is apart from emerging system and fusion method | |
| Yang et al. | CLIP-KD: An Empirical Study of Distilling CLIP Models | |
| Tang et al. | Point-PEFT: Parameter-efficient fine-tuning for 3D pre-trained models | |
| Gao et al. | Chinese fingerspelling sign language recognition using a nine-layer convolutional neural network | |
| Oh et al. | BERTAC: Enhancing transformer-based language models with adversarially pretrained convolutional neural networks | |
| He et al. | Image captioning algorithm based on multi-branch cnn and bi-lstm | |
| Tang et al. | A pruning based method to learn both weights and connections for LSTM | |
| CN116258147A (en) | Multimode comment emotion analysis method and system based on heterogram convolution |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |