WO2020192523A1 - 译文质量检测方法、装置、机器翻译系统和存储介质 - Google Patents

译文质量检测方法、装置、机器翻译系统和存储介质 Download PDF

Info

Publication number
WO2020192523A1
WO2020192523A1 PCT/CN2020/079964 CN2020079964W WO2020192523A1 WO 2020192523 A1 WO2020192523 A1 WO 2020192523A1 CN 2020079964 W CN2020079964 W CN 2020079964W WO 2020192523 A1 WO2020192523 A1 WO 2020192523A1
Authority
WO
WIPO (PCT)
Prior art keywords
translation
machine translation
training
original text
machine
Prior art date
Application number
PCT/CN2020/079964
Other languages
English (en)
French (fr)
Inventor
张檬
刘群
蒋欣
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2020192523A1 publication Critical patent/WO2020192523A1/zh
Priority to US17/479,420 priority Critical patent/US20220004721A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/40—Processing or translation of natural language
    • G06F40/58—Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/40—Processing or translation of natural language
    • G06F40/51—Translation evaluation

Definitions

  • This application relates to the field of machine translation technology, and more specifically, to a translation quality detection method, device, machine translation system, and storage medium.
  • Artificial intelligence is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results.
  • artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a similar way to human intelligence.
  • Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making.
  • the traditional solution is to manually edit the machine translation to obtain the manually edited machine translation (equivalent to a more accurate translation), and then determine the translation quality of the machine translation by analyzing the difference between the machine translation and the manually edited translation.
  • the traditional method of determining the quality of translation by simply comparing the difference between machine translation and manual editing is relatively simple, and the efficiency of detecting translation quality is not high.
  • This application provides a translation quality detection method, device, machine translation system, and storage medium, so as to perform machine translation quality detection more targeted, thereby improving the detection efficiency of translation quality.
  • a translation quality detection method includes: obtaining the original text and the machine translation; and determining the translation quality of the machine translation according to the original text, the machine translation, and the application scenarios of the machine translation.
  • the above-mentioned machine translation is the translation obtained by the original text through the machine translation system, that is, the above-mentioned machine translation is the translation obtained after the machine translation system performs the machine translation on the original text.
  • the execution subject of the translation quality detection method of this application may be a translation quality detection device or a translation quality detection system.
  • Natural language usually refers to a language that naturally evolves with culture.
  • the original text belongs to a first natural language
  • the machine translation belongs to a second natural language
  • the first language and the second language are different types of natural languages.
  • the original text belonging to the first natural language may mean that the original text is a paragraph of text expressed in a first natural language, and the machine translation belongs to a second natural language, and it may mean that the machine translation is a paragraph of text expressed in a second natural language.
  • the above-mentioned original text and machine translation can belong to any two different kinds of natural languages.
  • the application scenario of the aforementioned machine translation may be an application scenario of a downstream system that subsequently processes the machine translation, and the application scenario of the downstream system may specifically refer to a processing task of the downstream system.
  • processing tasks of the downstream system are one or more of sentiment classification, spam detection, intent recognition, and named entity recognition.
  • the translation quality of the machine translation when determining the translation quality of the machine translation, by combining the application scenarios of the machine translation, the translation quality of the machine translation can be detected more targeted, and the detection efficiency of the machine translation quality can be improved.
  • the above determination of the translation quality of the machine translation based on the original text, the machine translation, and the application scenarios of the machine translation includes: using the translation quality detection model corresponding to the downstream system to compare the original and The machine translation is processed to obtain the translation quality of the machine translation.
  • the above-mentioned downstream system is used for subsequent processing of machine translations, and the translation quality detection model corresponding to the downstream system is obtained by training based on training samples and training targets.
  • the training samples include training original texts and training machine translations.
  • the training targets include training machines.
  • the translation quality of the translation is obtained by training based on training samples and training targets.
  • the above-mentioned training original text may be a text specially used for training a translation quality detection model, and the training original text may include multiple paragraphs of text.
  • the above-mentioned training machine translation is the translation obtained by the machine translation system of the training original text (the training machine translation is the translation obtained by the machine translation system of the training original text), and the translation quality of the training machine translation is based on the first processing result and the second processing The difference between the results is determined, where the first processing result is the processing result of the downstream system processing the training machine translation, and the second processing result is the processing result of the downstream system processing the reference translation of the training original text.
  • the translation quality detection model corresponding to the aforementioned downstream system may be a pre-trained model. When testing the translation quality, it is necessary to use the translation quality detection model corresponding to the downstream system (the subsequent processing of the machine translation) to detect the translation quality.
  • training data The above-mentioned training source text, training machine translation, and training machine translation quality can be collectively referred to as training data.
  • the model parameters of the translation quality detection model can be obtained, and then the trained translation quality detection can be obtained Model, the translation quality detection model obtained after training can process the original text and the machine translation to obtain the translation quality of the machine translation.
  • the training original text and the reference translation of the training original text come from known bilingual parallel corpus.
  • the above-mentioned known bilingual parallel corpus can come from a bilingual parallel corpus stored locally or a bilingual parallel corpus stored in the cloud.
  • the reference translation of the aforementioned training original text is a manually edited translation.
  • the translation quality of the above-mentioned machine translation is the acceptability information of the machine translation.
  • the acceptability information of the above machine translation can indicate the acceptability of the machine translation. If the acceptability of the machine translation is greater, the quality of the machine translation is higher. On the contrary, the acceptability of the machine translation The poorer the translation quality.
  • the translation quality of the aforementioned machine translation is the difference value between the machine translation and the standard reference translation.
  • the acceptability information of the machine translation described above is used to indicate whether the machine translation is acceptable.
  • the acceptability information of the machine translation can include a flag, and the value of the flag is used to indicate whether the machine translation is acceptable.
  • the value of the flag is 1, which means that the machine translation can be accepted ( The machine translation can be sent to the downstream system for processing).
  • the value of the flag is 0, it means that the machine translation cannot be accepted or the machine translation is not accepted (the machine translation cannot be sent to the downstream system for processing).
  • the acceptability information of the machine translation described above is used to indicate the probability of the machine translation being accepted or not.
  • the above-mentioned acceptability information of the machine translation may specifically be the probability value of the probability that the machine translation is accepted or the probability value of the probability that the machine translation is not accepted.
  • the above-mentioned translation quality detection model is a neural network model.
  • the translation quality detection model is a neural network model
  • the effect of processing the original text and the machine translation to obtain the translation quality is better.
  • the above-mentioned translation quality detection model is a model based on a support vector machine.
  • a translation quality detection device which includes various modules for executing the method in the first aspect.
  • a machine translation system in a third aspect, includes a machine translation device and the translation quality detection device in the second aspect, wherein the machine translation device is used to obtain the original text and translate the original text to obtain For machine translation, the translation quality detection device is used to detect the machine translation to obtain the translation quality of the machine translation.
  • the translation quality detection device is used to detect the machine translation to obtain the translation quality of the machine translation, including: the translation quality detection device is used to detect the translation quality of the machine translation based on the original text, the machine translation, and the machine translation.
  • the application scenario determines the translation quality of the machine translation.
  • a cross-language processing system includes: a machine translation device for obtaining the original text and translating the original text to obtain a machine translation; a translation quality detection device for obtaining a machine translation based on the original text, The machine translation and the application scenario of the machine translation determine the translation quality of the machine translation; the downstream system is used to process the machine translation when the translation quality of the machine translation meets a preset requirement.
  • the aforementioned translation quality detection device, machine translation system, and cross-language processing system may be an electronic device (or a module located in an electronic device), and the electronic device may specifically be a mobile terminal (for example, a smart phone), a computer, or a personal digital device. Assistants, wearable devices, in-vehicle devices, IoT devices, or other devices capable of natural language processing.
  • a computer-readable medium stores program code for device execution, and the program code includes the method for executing the method in the first aspect.
  • a computer program product containing instructions is provided, when the computer program product is run on a computer, the computer is caused to execute the method in the first aspect.
  • a chip in a seventh aspect, includes a processor and a data interface, and the processor reads instructions stored on a memory through the data interface, and executes the method in the first aspect.
  • the chip may further include a memory in which instructions are stored, and the processor is configured to execute instructions stored on the memory.
  • the processor is used to execute the method in the first aspect.
  • an electronic device which includes the translation quality detection device in the second aspect, the machine translation system in the third aspect, or the cross-language processing system in the fourth aspect.
  • FIG. 1 is a schematic diagram of an application scenario of natural language processing provided by an embodiment of the present application
  • FIG. 2 is a schematic diagram of another application scenario of natural language processing provided by an embodiment of the present application.
  • FIG. 3 is a schematic diagram of related equipment for natural language processing provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a system architecture provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of translation quality detection based on a CNN model provided by an embodiment of the present application.
  • FIG. 6 is another schematic diagram of performing translation quality detection based on a CNN model according to an embodiment of the present application.
  • FIG. 7 is a schematic diagram of the hardware structure of a chip provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of an application scenario provided by an embodiment of the present application.
  • FIG. 9 is a schematic diagram of a translation quality detection process provided by an embodiment of the present application.
  • FIG. 10 is a schematic flowchart of a translation quality detection method provided by an embodiment of the present application.
  • FIG. 11 is a schematic diagram of a process of obtaining training data provided by an embodiment of the present application.
  • FIG. 12 is a schematic diagram of a translation quality detection process provided by an embodiment of the present application.
  • FIG. 13 is a schematic diagram of a translation quality detection process provided by an embodiment of the present application.
  • FIG. 14 is a schematic diagram of a translation quality detection process provided by an embodiment of the present application.
  • 15 is a schematic block diagram of a translation quality detection device provided by an embodiment of the present application.
  • Fig. 16 is a schematic block diagram of a machine translation system provided by an embodiment of the present application.
  • Figure 1 shows a natural language processing system, which includes user equipment and data processing equipment.
  • user equipment includes smart terminals such as mobile phones, personal computers, or information processing centers.
  • the user equipment is the initiator of natural language data processing, and as the initiator of requests such as language question and answer or query, usually the user initiates the request through the user equipment.
  • the aforementioned data processing device may be a device or server with data processing functions such as a cloud server, a network server, an application server, and a management server.
  • the data processing equipment receives the query sentence/voice/text question sentence from the smart terminal through the interactive interface, and then performs machine learning, deep learning, search, reasoning, decision-making, etc. through the memory of the data storage and the processor of the data processing. data processing.
  • the memory in a data processing device can be a general term, including a database for local storage and storing historical data.
  • the database can be on the data processing device or on other network servers.
  • the user equipment can receive instructions from the user, perform machine translation on the original text (for example, the original text may be a piece of English input by the user) to obtain the machine translation, and then initiate a request to the data processing equipment.
  • the data processing equipment can detect the translation quality of the machine translation obtained by the user equipment translation, thereby obtaining the translation quality of the machine translation.
  • the data processing device can execute the translation quality detection method of the embodiment of the present application.
  • Figure 2 shows another natural language processing system.
  • the user equipment is directly used as a data processing device.
  • the user equipment can directly receive input from the user and process it directly by the hardware of the user equipment itself.
  • Figure 1 is similar, and you can refer to the above description, which will not be repeated here.
  • the user equipment can receive the user's instructions, perform machine translation on the original text to obtain the machine translation, and then the user equipment itself performs translation quality detection on the machine translation obtained by the machine translation, thereby obtaining the machine The translation quality of the translation.
  • the user equipment itself can execute the translation quality detection method of the embodiment of the present application.
  • Fig. 3 is a schematic diagram of a natural language processing related device provided by an embodiment of the present application.
  • the user equipment in FIG. 1 and FIG. 2 may specifically be the local device 301 or the local device 302 in FIG. 3, and the data processing device in FIG. 1 may specifically be the execution device 210 in FIG. 3, where the data storage system 250 may be To store the to-be-processed data of the execution device 210, the data storage system 250 may be integrated on the execution device 210, or may be set on the cloud or other network servers.
  • the processors in Figures 1 and 2 can perform data training/machine learning/deep learning through neural network models or other models (for example, support vector machine-based models), and use the data to finally train or learn the model to translate the machine
  • the translation quality of the machine translation is tested to obtain the translation quality of the machine translation.
  • a neural network can be composed of neural units.
  • a neural unit can refer to an arithmetic unit that takes x s and intercept 1 as inputs.
  • the output of the arithmetic unit can be:
  • s 1, 2,...n, n is a natural number greater than 1
  • W s is the weight of x s
  • b is the bias of the neural unit.
  • f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into an output signal.
  • the output signal of the activation function can be used as the input of the next convolutional layer, and the activation function can be a sigmoid function.
  • a neural network is a network formed by connecting multiple above-mentioned single neural units together, that is, the output of one neural unit can be the input of another neural unit.
  • the input of each neural unit can be connected with the local receptive field of the previous layer to extract the characteristics of the local receptive field.
  • the local receptive field can be a region composed of several neural units.
  • Deep neural network also called multi-layer neural network
  • DNN can be understood as a neural network with multiple hidden layers.
  • DNN is divided according to the positions of different layers.
  • the neural network inside the DNN can be divided into three categories: input layer, hidden layer, and output layer.
  • the first layer is the input layer
  • the last layer is the output layer
  • the number of layers in the middle are all hidden layers.
  • the layers are fully connected, that is to say, any neuron in the i-th layer must be connected to any neuron in the i+1th layer.
  • DNN looks complicated, it is not complicated in terms of the work of each layer. Simply put, it is the following linear relationship expression: among them, Is the input vector, Is the output vector, Is the offset vector, W is the weight matrix (also called coefficient), and ⁇ () is the activation function.
  • Each layer is just the input vector After such a simple operation, the output vector is obtained Due to the large number of DNN layers, the coefficient W and the offset vector The number is also relatively large.
  • the definition of these parameters in the DNN is as follows: Take the coefficient W as an example: Suppose that in a three-layer DNN, the linear coefficients from the fourth neuron in the second layer to the second neuron in the third layer are defined as The superscript 3 represents the number of layers where the coefficient W is located, and the subscript corresponds to the output third layer index 2 and the input second layer index 4.
  • the coefficient from the kth neuron in the L-1th layer to the jth neuron in the Lth layer is defined as
  • the input layer has no W parameter.
  • more hidden layers make the network more capable of portraying complex situations in the real world. Theoretically speaking, a model with more parameters is more complex and has a greater "capacity", which means it can complete more complex learning tasks.
  • Training a deep neural network is also a process of learning a weight matrix, and its ultimate goal is to obtain the weight matrix of all layers of the trained deep neural network (a weight matrix formed by vectors W of many layers).
  • Convolutional neural network (convolutional neuron network, CNN) is a deep neural network with convolutional structure.
  • the convolutional neural network contains a feature extractor composed of a convolution layer and a sub-sampling layer.
  • the feature extractor can be regarded as a filter.
  • the convolutional layer refers to the neuron layer that performs convolution processing on the input signal in the convolutional neural network.
  • a neuron can be connected to only part of the neighboring neurons.
  • a convolutional layer usually contains several feature planes, and each feature plane can be composed of some rectangularly arranged neural units. Neural units in the same feature plane share weights, and the shared weights here are the convolution kernels.
  • Sharing weight can be understood as the way to extract image information has nothing to do with location.
  • the convolution kernel can be initialized in the form of a matrix of random size. During the training of the convolutional neural network, the convolution kernel can obtain reasonable weights through learning. In addition, the direct benefit of sharing weights is to reduce the connections between the layers of the convolutional neural network, while reducing the risk of overfitting.
  • RNN Recurrent Neural Networks
  • RNN can process sequence data of any length.
  • the training of RNN is the same as the training of traditional CNN or DNN.
  • the neural network can use an error back propagation (BP) algorithm to modify the size of the parameters in the initial neural network model during the training process, so that the reconstruction error loss of the neural network model becomes smaller and smaller. Specifically, forwarding the input signal to the output will cause error loss, and the parameters in the initial neural network model are updated by backpropagating the error loss information, so that the error loss is converged.
  • the backpropagation algorithm is a backpropagation motion dominated by error loss, and aims to obtain the optimal neural network model parameters, such as the weight matrix.
  • an embodiment of the present application provides a system architecture 100.
  • the data collection device 160 is used to collect training data.
  • the training data includes training original text, training machine translation (translated by the training original text by a machine translation system), and the translation quality of the training machine translation.
  • the translation quality of the training machine translation can be determined by the difference between the processing result of the training machine translation by the downstream system to which the machine translation is applied and the processing result of the training reference translation by the downstream system.
  • the downstream system here may be located inside the client device or in other devices other than the user device.
  • the data collection device 160 stores the training data in the database 130, and the training device 120 trains to obtain the target model/rule 101 based on the training data maintained in the database 130.
  • the training device 120 processes the input training text and the training machine translation, and compares the output translation quality with the translation quality of the training machine translation until the training The difference between the output translation quality of the device 120 and the translation quality of the training machine translation is less than a certain threshold, thereby completing the training of the target model/rule 101.
  • the above-mentioned target model/rule 101 can be used to implement the translation quality detection method of the embodiment of the present application, that is, the original text and machine translation images are processed by relevant preprocessing (the preprocessing module 113 and/or the preprocessing module 114 can be used for processing) and then input the Target model/rule 101, you can get the translation quality of the machine translation.
  • the target model/rule 101 in the embodiment of the present application may specifically be a neural network. It should be noted that in actual applications, the training data maintained in the database 130 may not all come from the collection of the data collection device 160, and may also be received from other devices.
  • the training device 120 does not necessarily perform the training of the target model/rule 101 completely based on the training data maintained by the database 130. It may also obtain training data from the cloud or other places for model training. The above description should not be used as a reference to this application. Limitations of Examples.
  • the target model/rule 101 trained according to the training device 120 can be applied to different systems or devices, such as the execution device 110 shown in FIG. 4, which can be a terminal, such as a mobile phone terminal, a tablet computer, notebook computers, augmented reality (AR)/virtual reality (VR), vehicle-mounted terminals, etc., can also be servers or clouds.
  • the execution device 110 is configured with an input/output (input/output, I/O) interface 112 for data interaction with external devices.
  • the user can input data to the I/O interface 112 through the client device 140.
  • the input data in this embodiment of the application may include: the original text and machine translation input by the client device.
  • the preprocessing module 113 and the preprocessing module 114 are used to preprocess the input data (such as the original text and the machine translation) received by the I/O interface 112 (specifically, the original text and the machine translation can be processed to obtain the word vector).
  • the preprocessing module 113 and the preprocessing module 114 may not be provided, and the calculation module 111 is directly used to process the input data.
  • the execution device 110 may call data, codes, etc. in the data storage system 150 for corresponding processing .
  • the data, instructions, etc. obtained by corresponding processing may also be stored in the data storage system 150.
  • the I/O interface 112 feeds back the processing result, for example, the translation quality of the machine translation, to the client device 140.
  • the training device 120 can generate a target model/rule 101 corresponding to the downstream system for different downstream systems, and the corresponding target model/rule 101 can be used to achieve the above goals or complete the above tasks, thereby providing users Provide the desired result.
  • the user can manually set input data (for example, input a paragraph of text), and the manual setting can be operated through the interface provided by the I/O interface 112.
  • the client device 140 can automatically send input data (for example, input a paragraph of text) to the I/O interface 112. If the client device 140 is required to automatically send the input data and the user's authorization is required, the user can log in to the client device Set corresponding permissions in 140. The user can view the result output by the execution device 110 in the client device 140, and the specific presentation form may be display, sound, action and other specific methods (for example, the output result may be whether the machine translation is acceptable).
  • the client device 140 can also be used as a data collection terminal to collect the input data of the input I/O interface 112 and the output result of the output I/O interface 112 as new sample data and store it in the database 130 as shown in the figure.
  • the I/O interface 112 directly uses the input data input to the I/O interface 112 and the output result of the output I/O interface 112 as a new sample as shown in the figure.
  • the data is stored in the database 130.
  • FIG. 4 is only a schematic diagram of a system architecture provided by an embodiment of the present application, and the positional relationship among the devices, devices, modules, etc. shown in the figure does not constitute any limitation.
  • the data storage system 150 is an external memory relative to the execution device 110. In other cases, the data storage system 150 may also be placed in the execution device 110.
  • the target model/rule 101 is obtained by training according to the training device 120.
  • the target model/rule 101 may be the translation quality detection model in the embodiment of the present application.
  • the neural network provided in the embodiment of the present application may be CNN, deep convolutional neural network (deep convolutional neural network, DCNN), recurrent neural network (recurrent neural network, RNN), etc.
  • CNN is a very common neural network
  • the structure of CNN will be introduced in detail below in conjunction with Figure 5.
  • a convolutional neural network is a deep neural network with a convolutional structure. It is a deep learning architecture.
  • a deep learning architecture refers to a machine learning algorithm. Multi-level learning is carried out on the abstract level of
  • CNN is a feed-forward artificial neural network. Each neuron in the feed-forward artificial neural network can respond to the input image.
  • a convolutional neural network (CNN) 200 may include an input layer 110, a convolutional layer/pooling layer 120 (the pooling layer is optional), and a neural network layer 130.
  • CNN convolutional neural network
  • the convolutional layer/pooling layer 120 may include layers 121-126 as in the examples.
  • layer 121 is a convolutional layer
  • layer 122 is a pooling layer
  • layer 123 is a convolutional layer
  • 124 is a pooling layer
  • 121 and 122 are convolutional layers
  • 123 is a pooling layer
  • 124 and 125 are convolutional layers
  • 126 is a convolutional layer.
  • Pooling layer That is, the output of the convolutional layer can be used as the input of the subsequent pooling layer, or as the input of another convolutional layer to continue the convolution operation.
  • the convolutional layer 121 can include many convolution operators.
  • the convolution operator is also called a kernel. Its role in natural language processing is equivalent to an extraction from input speech or semantic information.
  • the filter for specific information, the convolution operator can essentially be a weight matrix, which is usually predefined.
  • weight values in these weight matrices need to be obtained through a lot of training in practical applications, and each weight matrix formed by the weight values obtained through training can extract information from the input image, thereby helping the convolutional neural network 100 to make correct predictions.
  • the initial convolutional layer (such as 121) often extracts more general features, which can also be called low-level features; with the convolutional neural network
  • the features extracted by the subsequent convolutional layers (for example, 126) become more and more complex, such as features such as high-level semantics, and features with higher semantics are more suitable for the problem to be solved.
  • pooling layer after the convolutional layer, that is, the 121-126 layers as illustrated by 120 in Figure 5, which can be a convolutional layer followed by a layer
  • the pooling layer can also be a multi-layer convolutional layer followed by one or more pooling layers.
  • the only purpose of the pooling layer is to reduce the size of the data space.
  • the convolutional neural network 100 After processing by the convolutional layer/pooling layer 120, the convolutional neural network 100 is not enough to output the required output information. Because as mentioned above, the convolutional layer/pooling layer 120 only extracts features and reduces the parameters brought by the input data. However, in order to generate the final output information (required class information or other related information), the convolutional neural network 100 needs to use the neural network layer 130 to generate one or a group of required classes of output. Therefore, the neural network layer 130 may include multiple hidden layers (131, 132 to 13n as shown in FIG. 5) and an output layer 140. The parameters contained in the multiple hidden layers can be based on specific task types. Relevant training data of is obtained through pre-training. For example, the task type may include speech or semantic recognition, classification or generation, etc.
  • the output layer 140 After the multiple hidden layers in the neural network layer 130, that is, the final layer of the entire convolutional neural network 100 is the output layer 140.
  • the output layer 140 has a loss function similar to the classification cross entropy, which is specifically used to calculate the prediction error.
  • the convolutional neural network 100 shown in FIG. 5 is only used as an example of a convolutional neural network. In specific applications, the convolutional neural network may also exist in the form of other network models.
  • a convolutional neural network (CNN) 200 may include an input layer 110, a convolutional layer/pooling layer 120 (the pooling layer is optional), and a neural network layer 130.
  • CNN convolutional neural network
  • Multiple convolutional layers/pooling layers in the convolutional layer/pooling layer 120 are in parallel, and the respectively extracted features are input to the full neural network layer 130 for processing.
  • FIG. 7 is a schematic diagram of the hardware structure of a chip provided by an embodiment of the application.
  • the chip includes a neural network processor (neural processing unit, NPU) 50.
  • the chip can be set in the execution device 110 as shown in FIG. 4 to complete the calculation work of the calculation module 111.
  • the chip can also be set in the training device 120 as shown in FIG. 4 to complete the training work of the training device 120 and output the target model/rule 101.
  • the algorithms of each layer in the convolutional neural network as shown in FIG. 5 and FIG. 6 can be implemented in the chip as shown in FIG. 7.
  • the translation quality detection method of the embodiment of the present application can be specifically executed in the arithmetic circuit 503 and/or the vector calculation unit 507 in the NPU 50, so as to obtain the translation quality of the machine translation.
  • the NPU 50 can be mounted on the host CPU as a coprocessor, and the host CPU distributes tasks.
  • the core part of the NPU 50 is the arithmetic circuit 503.
  • the controller 504 in the NPU 50 can control the arithmetic circuit 503 to extract data in the memory (weight memory or input memory) and perform operations.
  • the arithmetic circuit 503 includes multiple processing units (process engines, PE). In some implementations, the arithmetic circuit 503 is a two-dimensional systolic array. The arithmetic circuit 503 may also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 503 is a general-purpose matrix processor.
  • the arithmetic circuit fetches the corresponding data of matrix B from the weight memory 502 and buffers it on each PE in the arithmetic circuit.
  • the arithmetic circuit takes the matrix A data and matrix B from the input memory 501 to perform matrix operations, and the partial or final result of the obtained matrix is stored in an accumulator 508.
  • the vector calculation unit 507 can perform further processing on the output of the arithmetic circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison and so on.
  • the vector calculation unit 507 can be used for network calculations in the fully connected layers (FC) of the neural network, such as pooling, batch normalization, and partial response. Normalization (local response normalization), etc.
  • the vector calculation unit 507 can store the processed output vector in the unified buffer 506.
  • the vector calculation unit 507 may apply a nonlinear function to the output of the arithmetic circuit 503, such as a vector of accumulated values, to generate the activation value.
  • the vector calculation unit 507 generates a normalized value, a combined value, or both.
  • the processed output vector can be used as an activation input to the arithmetic circuit 503, for example for use in subsequent layers in a neural network.
  • the unified memory 506 is used to store input data and output data.
  • the weight data directly transfers the input data in the external memory to the input memory 501 and/or the unified memory 506 through the storage unit access controller 505 (direct memory access controller, DMAC), and stores the weight data in the external memory into the weight memory 502, And the data in the unified memory 506 is stored in the external memory.
  • DMAC direct memory access controller
  • the bus interface unit (BIU) 510 is used to implement interaction between the main CPU, the DMAC, and the fetch memory 509 through the bus.
  • An instruction fetch buffer 509 connected to the controller 504 is used to store instructions used by the controller 504;
  • the controller 504 is configured to call the instructions cached in the memory 509 to control the working process of the computing accelerator.
  • the unified memory 506, the input memory 501, the weight memory 502, and the fetch memory 509 can all be on-chip memories.
  • the external memory of the NPU may be a memory external to the NPU, and the external memory may be a double data rate synchronous dynamic random access memory (double data rate synchronous dynamic random access memory, DDR SDRAM), high bandwidth memory (high bandwidth memory, HBM), or Other readable and writable memory.
  • the translation quality detection method of the embodiment of the present application will be described in detail below with reference to the accompanying drawings.
  • the translation quality detection method of the embodiment of the present application may be executed by the data processing device in FIG. 1, the user equipment in FIG. 2, the execution device 210 in FIG. 3, and the execution device 110 in FIG. 4.
  • Fig. 8 is a schematic diagram of an application scenario provided by an embodiment of the present application.
  • the machine translation can be obtained by translating the original text through the machine translation system, and the machine translation can be sent to the downstream system for processing.
  • the translation quality detection method of the embodiment of the present application can detect or evaluate the translation quality of the machine translation (before the machine translation is sent to the downstream system for processing), so as to obtain the translation quality of the machine translation, which is convenient to determine the translation quality of the machine translation. Determine whether to send the machine translation to the downstream system for processing.
  • Fig. 9 is a schematic diagram of a translation quality detection process provided by an embodiment of the present application.
  • the machine translation system performs machine translation on the original text to obtain the machine translation.
  • the machine translation and the original text are sent to the translation quality inspection system for quality inspection to determine the translation quality of the machine translation.
  • the translation quality of the machine translation meets the requirements (for example, the translation quality of the machine translation is better)
  • the machine translation can be sent to the downstream system for processing, and when the translation quality of the machine translation cannot meet the requirements (for example, the translation quality of the machine translation is better) Poor)
  • the machine translation can be sent to other processing modules for other processing (for example, the machine translation can be discarded without processing).
  • the translation quality of a machine translation is generally related to the specific application scenarios of the machine translation.
  • the downstream task when facing different downstream tasks (the task of processing the machine translation by the downstream system can be called the downstream task) , The acceptability of the machine translation may not be the same. That is to say, the translation quality standard of the machine translation is related to the specific application scenario of the machine translation. Therefore, in the embodiment of the present application, by combining the application scenario of the machine translation, the translation quality of the machine translation can be determined more specifically.
  • FIG. 10 is a schematic flowchart of a translation quality detection method provided by an embodiment of the present application.
  • the method shown in FIG. 10 can be executed by a translation quality detection system (specifically, the translation quality detection system shown in FIG. 9).
  • the translation quality detection system may specifically be the data processing device in FIG. 1, or a diagram.
  • the user equipment in 2 may also be the execution device 210 in FIG. 3 or the execution device 110 in FIG. 4.
  • the method shown in FIG. 10 includes step 1001 and step 1002, and these two steps are respectively described in detail below.
  • the above-mentioned machine translation is a translation obtained by the original text through a machine translation system, that is, the above-mentioned machine translation is a translation obtained by a machine translation system on the original text.
  • the machine translation in the above step 1001 may be a translation obtained by machine translation performed by the machine translation system in FIG. 9.
  • the foregoing original text may be a paragraph of text written in a certain natural language, for example, the foregoing original text may be a paragraph of English or Chinese.
  • the original text belongs to a first natural language
  • the machine translation belongs to a second natural language
  • the first language and the second language are different natural languages.
  • the above-mentioned original text belongs to the first natural language may mean that the above-mentioned original text is a paragraph of text expressed in the first natural language
  • the above-mentioned machine translation belongs to the second natural language which may mean that the above-mentioned machine translation is expressed in the second natural language.
  • Text The above-mentioned original text and machine translation can belong to any two different natural languages.
  • the above-mentioned original text may be a piece of Chinese
  • the machine translation of the original text may be a piece of English (this piece of Chinese is translated by a machine translation system to obtain a piece of English).
  • the above-mentioned original text may be a piece of Chinese
  • the machine translation of the Chinese may be a piece of Japanese (translate this piece of Chinese through a machine translation system to obtain a piece of Japanese).
  • 1002 Determine the translation quality of the machine translation according to the original text, the machine translation and the application scenarios of the machine translation.
  • the downstream system can reflect the application scenario of the machine translation, and the application scenario of the machine translation in step 1002 can be the downstream system that subsequently processes the machine translation
  • the application scenario of the downstream system may specifically refer to the processing task of the downstream system.
  • the processing tasks of the downstream system can be one or more of sentiment classification, spam detection, intent recognition, and named entity recognition.
  • NER named entity recognition
  • proprietary name recognition specifically refers to identifying entities with specific meanings in the text.
  • the entities mainly include person names, place names, organization names, proper nouns, etc. .
  • the translation quality of the machine translation when determining the translation quality of the machine translation, by combining the application scenarios of the machine translation, the translation quality of the machine translation can be detected more targeted, and the detection efficiency of the machine translation quality can be improved.
  • determining the translation quality of the machine translation according to the original text, the machine translation, and the application scenarios of the machine translation in the above step 1002 includes: using the translation quality detection model corresponding to the downstream system to process the original text and the machine translation to obtain the machine translation The quality of the translation.
  • the above-mentioned downstream system is used for subsequent processing of the machine translation, and the downstream system can reflect the application scenario of the machine translation. Therefore, when processing the original text and the machine translation according to the translation quality detection model corresponding to the downstream system, it is equivalent to considering the machine translation.
  • the application scenario of the translation is equivalent to considering the machine translation.
  • the translation quality detection model corresponding to the downstream system may be obtained by training based on training samples and training targets, where the training samples include training original texts and training machine translations, and the training targets include the translation quality of the training machine translations.
  • the above-mentioned training machine translation may be the translation obtained by the machine translation system of the training original text.
  • the above-mentioned training machine translation is the translation obtained after the machine translation system performs machine translation on the training original text.
  • the translation quality of the above-mentioned training machine translation may be based on The downstream system determines the processing result of the training machine translation and the downstream system the processing result of the reference translation of the training original text.
  • the translation quality of the above-mentioned training machine translation may be determined based on the difference between the processing result of the downstream system on the training machine translation and the processing result of the downstream system on the reference translation of the training original text. That is to say, the translation quality of the training machine translation can be determined according to the difference between the processing result of the downstream system on the training machine translation and the processing result of the downstream system on the reference translation of the training original text.
  • the translation quality of the above-mentioned training machine translation may be determined according to the difference between the first processing result and the second processing result.
  • the translation quality of the training machine translation can be determined according to the difference between the first processing result and the second processing result, where the first processing result is the processing result of the downstream system processing the training machine translation, and the second processing The result is the processing result of the downstream system processing the reference translation of the training original.
  • the translation quality of the training machine translation is better.
  • the training machine The translation quality of the translation is relatively poor.
  • the translation quality detection model corresponding to the aforementioned downstream system may be a pre-trained model. When testing the translation quality, it is necessary to use the translation quality detection model corresponding to the downstream system (the subsequent processing of the machine translation) to detect the translation quality.
  • training data The above-mentioned training source text, training machine translation, and training machine translation quality can be collectively referred to as training data.
  • the model parameters of the translation quality detection model can be obtained, and then the trained translation quality detection can be obtained Model, the translation quality detection model obtained after the training can process the original text and the machine translation to obtain the translation quality of the machine translation.
  • the above-mentioned training source text and training machine translation can be called training input data.
  • the output of the translation quality detection model is as close as possible to the training target, and
  • the model parameter when the difference between the output of the translation quality detection model and the training target meets a preset requirement is determined as the final model parameter of the translation quality detection model.
  • the training original text and the reference translation of the training original text come from known bilingual parallel corpus.
  • the above-mentioned known bilingual parallel corpus can come from a bilingual parallel corpus stored locally or a bilingual parallel corpus stored in the cloud.
  • the difficulty of obtaining training data can be reduced, and the process of obtaining the translation quality detection model based on the training data can be simplified.
  • the reference translation of the aforementioned training original text is a manually edited translation.
  • the translation quality detection model can be obtained by training based on training data.
  • the training data here includes the training original text, the training machine translation and the translation quality of the training machine translation. The following describes the process of obtaining training data in conjunction with the attached drawings. brief introduction.
  • FIG. 11 is a schematic diagram of obtaining training data provided by an embodiment of the present application.
  • the machine translation system performs machine translation on the training original text to obtain the training machine translation, and the downstream system processes the training machine translation to obtain the first processing result.
  • the downstream system also needs to process the reference translation of the training original text , The second processing result is obtained; then, the translation quality of the training machine translation can be determined according to the difference between the first processing result and the second processing result.
  • determining the translation quality of the training machine translation according to the difference between the first processing result and the second processing result can be specifically executed by the downstream system itself, or by other equipment or devices other than the downstream system.
  • the training original text and the reference translation of the training original text in Figure 11 may come from known bilingual parallel corpus, and the training original text may contain multiple paragraphs of text (at this time, the reference translation of the training original text is the reference translation corresponding to the multiple paragraphs of text) .
  • the training data can be expressed as ( ⁇ training the original training machine translation>, the translation quality of the training machine translation), these The training data can be used to subsequently train the translation quality detection model.
  • the translation quality detection model can be trained, and then the translation quality detection model can be used to detect the translation quality to determine the translation quality of the machine translation.
  • Example 1 The following is a combination of Example 1 and Example 2 to illustrate how to determine the translation quality of the training translation, and then obtain the training data.
  • Example 1 The downstream system performs a subjective classification task.
  • Example 1 the original text and the reference translation of the original text are as follows:
  • the downstream system processes the reference translation of original 1 as follows:
  • the machine translation system After the machine translation system performs machine translation on the original text 1, the translation obtained is the machine translation 1a or the machine translation 1b.
  • the two machine translations are as follows:
  • Machine translation 1a I thought it rained yesterday.
  • Machine translation 1b It rained last night.
  • the downstream system is used to process machine translation 1a and machine translation 1b respectively, and the specific processing results and acceptability labeling information obtained are as follows:
  • Machine translation 1a I thought it rained yesterday. (Classification result: subjective; acceptability mark: acceptable);
  • Machine translation 1b It rained last night. (Classification result: objective; acceptability mark: unacceptable).
  • the downstream system is used to process the machine translation 1a, and the processing result obtained is subjective, which is the same as the processing result of the original text 1. It can be considered that the translation quality of the machine translation 1a meets the requirements, and the translation quality of the machine translation 1a is acceptable.
  • the downstream system is used to process the machine translation 1b, and the processing result obtained is objective, which is different from the processing result of the original text 1. It can be considered that the translation quality of the machine translation 1b cannot meet the requirements, and the translation quality of the machine translation 1b is unacceptable.
  • the "night" in the machine translation 1a is not translated, but it does not affect the subsequent classification results.
  • the downstream system's classification result of the machine translation 1a is still subjective, and the classification result of the machine translation 1a is the same as that of the original 1.
  • the classification results of the reference translations are the same, so for the classification tasks of the downstream system, the machine translation 1a is acceptable.
  • Example 1 the process of obtaining data is explained by taking the example of the subjective classification task performed by the downstream system.
  • the process of obtaining data will be explained below in conjunction with the example of the recognition of named entities performed by the downstream system in Example 2.
  • Example 2 The downstream system performs the recognition of named entities.
  • Example 2 the original text and the reference translation of the original text are as follows:
  • the translation obtained is any one of machine translation 2a, machine translation 2b, and machine translation 2c.
  • machine translations are as follows:
  • Machine translation 2a Bush and Sharon in New York.
  • Machine translation 2b Bush gave a talk at a salon in New York.
  • Machine translation 2c Bush held a talk with Shalong in New York.
  • Machine translation 2a Bush and Sharon in New York. (Named entity: Bush, Sharon, New York; acceptability mark: acceptable);
  • Machine translation 2b Bush gave a talk at a salon in New York. (Named entity: Bush, New York; acceptability mark: unacceptable);
  • Machine translation 2c Bush held a talk with Shalong in New York. (Named entity: Bush, Shalong, New York; acceptability mark: unacceptable).
  • judging whether the machine translation 2a, the machine translation 2b, and the machine translation 2c are acceptable are all based on the acceptance criteria that the reference translation and the machine translation obtain the same result through the downstream system.
  • the acceptable criteria for machine translation can also be determined according to the actual situation. For example, if you are very sensitive to the omission of the named entity in the translation process, but can tolerate the error of the named entity, you can judge whether it is acceptable or not based on the number of elements in the named entity result set. In this case, all three named entities obtained by the named entity recognition in the machine translation 2a and the machine translation 2c mentioned above. Therefore, both machine translations can be marked as acceptable. In this case, the three training data obtained after labeling are as follows:
  • the translation quality of the machine translation is the acceptability of the machine translation as an example.
  • the translation quality of the machine translation can also have many manifestations.
  • FIG. 12 is a schematic diagram of a translation quality detection process provided by an embodiment of the present application.
  • the process shown in Figure 12 can be divided into two stages, one is the training stage and the other is the detection stage.
  • the training phase when the training data ( ⁇ training the original training machine translation>, the translation quality of the training machine translation) is obtained, the translation quality detection model in the translation quality detection system can be trained based on the training data, and the translation After the quality detection model is trained, the translation quality detection system can then be used for translation quality detection.
  • the training data used in the training phase shown in FIG. 12 corresponds to the downstream system in the following detection phase, that is, the translation quality of the training machine translation in the training data is based on the downstream system's translation of the training machine translation. The difference between the processing result and the processing result of the reference translation of the training original text by the downstream system is determined.
  • the machine translation system performs machine translation on the original text to obtain the machine translation.
  • the translation quality detection system determines the translation quality of the machine translation based on the original text and the machine translation.
  • the machine translation can be sent to the downstream system for processing.
  • the translation quality of the machine translation cannot meet the requirements (for example, the translation quality of the machine translation is unacceptable), then the machine translation can be sent to Processing in other processing modules.
  • the translation quality of the machine translation can have multiple manifestations.
  • the following describes several possible manifestations of the translation quality of the machine translation.
  • the translation quality of the machine translation can be any of the following information.
  • the acceptability information of the machine translation can be used to indicate whether the machine translation is acceptable or not, and can also indicate the probability that the machine translation is accepted or not.
  • the acceptability information of the machine translation may include a flag, and the value of the flag is used to indicate whether the machine translation is acceptable, for example, When the value of the flag is 1, it means that the machine translation is acceptable (it can be used for downstream system processing), and when the value of the flag is 0, it means that the machine translation is not acceptable (or called unacceptable, In this case, the machine translation cannot be used for downstream system processing).
  • the acceptability information of the machine translation can specifically be the probability value of the probability that the machine translation is accepted or the machine translation is not accepted. The probability value of the probability.
  • the acceptability information of the above machine translation is used to indicate the probability of the machine translation being accepted, if the probability of the machine translation being accepted is greater than a certain preset value, then the machine translation can be sent to the downstream system for subsequent processing. Otherwise, the machine translation cannot be sent to the downstream system for subsequent processing.
  • the probability that the machine translation is accepted is 85%, 85% is greater than 80% (the preset value can also be other values, here is just an example), the machine translation can be sent to the downstream system for subsequent processing.
  • the above-mentioned translation quality detection model can be either a neural network model or a model based on a support vector machine.
  • the above-mentioned translation quality detection model is a neural network model
  • the neural network model since the neural network model has a stronger ability to process data, it can perform better processing effects on the original text and machine translation.
  • Figure 13 shows the processing process of the original text and machine translation by the translation quality detection model.
  • each word in the original text and the machine translation can be mapped to a word embedding, and then passed through a two-way recurrent neural network (recurrent neural network, RNN).
  • the word vector of the original text and the word vector of the machine translation are processed to obtain the intermediate vector of the original text and the intermediate vector of the machine translation respectively.
  • the attention mechanism is used for the intermediate vector of the original text and the intermediate vector of the machine translation.
  • a target vector is obtained, and finally the target vector is processed by a feed-forward neural network to obtain an estimate of whether the machine translation is acceptable relative to the original text.
  • the network unit in the RNN may be a gated recurrent unit (GRU).
  • the above calculation process can be simplified as substituting the original text and machine translation into the function f(x,z; ⁇ ) to obtain the translation quality of the machine translation, where x represents the original text, z represents the machine translation, and ⁇ is the translation quality detection model
  • the parameters include word vector parameters, cyclic neural network parameters, attention mechanism parameters, and feedforward neural network parameters.
  • the acceptability can be marked with the corpus
  • a backpropagation algorithm is used for training to obtain a translation quality detection model.
  • m represents the number of the acceptability of an entry annotated data
  • x i is the i-th original
  • z i is the i-th machine translation
  • Z i is x i for machine translation machine translation obtained
  • y i to Z i Acceptability annotation at this time, the translation quality of the machine translation in the training data set is represented by acceptability annotation.
  • using the translation quality detection model to process the original text and machine translation specifically includes the following steps:
  • formula (1) may be used to obtain the word vector of each word in the original text s
  • formula (2) may be used to obtain the word vector of each word in the machine translation t.
  • si represents the vector represented by the one-hot form of the i-th word in the original s (this vector has only one feature that is not 0, and the others are 0)
  • t i represents the vector of the i-th word in the machine translation t in one-hot form.
  • the RNN is used to process the word vector of each word in the original s and the word vector of each word in the machine translation t respectively to obtain the first intermediate vector of the original s and the second intermediate vector of the machine translation t.
  • step 2002 formula (3) can be used to process the word vector of each word in the original text s, and formula (4) can be used to process the word vector of each word in the machine translation t.
  • BiGRU S represents the operation performed by the source language bidirectional GRU network on the original word vector
  • BiGRU T represents the operation performed by the target language bidirectional GRU network on the machine translation word vector
  • the non-linear layer is used to perform non-linear processing on the first intermediate vector of the original text s processed in step 2002 to obtain a third intermediate vector.
  • formula (5) may be used to perform nonlinear processing on the first intermediate vector to obtain the third intermediate vector.
  • h i S represents the third intermediate vector
  • ReLU is a nonlinear activation function
  • W g S represents a parameter matrix
  • b g S represents a parameter vector
  • step 2004 Use the nonlinear layer to perform nonlinear processing on the second intermediate vector of the machine translation t obtained in step 2002 to obtain a fourth intermediate vector.
  • h i T represents the fourth intermediate vector
  • ReLU is a nonlinear activation function
  • W g T represents a parameter matrix
  • b g T represents a parameter vector
  • the attention mechanism is used to process the two sets of sentence vectors obtained in the above steps 2003 and 2004 to obtain a target vector.
  • w represents a parameter vector.
  • the target vector is processed through the nonlinear layer and the linear layer to obtain an estimate of whether the machine translation is acceptable relative to the original text.
  • step 2006 an estimate of whether the machine translation is acceptable relative to the original text can be calculated according to formulas (9) and (10).
  • W u and W v represent two parameter matrices
  • b u and b v represent two parameter vectors
  • the translation quality detection method of the embodiment of the present application is described in detail above with reference to the accompanying drawings.
  • the translation quality detection device of the embodiment of the application is described below with reference to FIG. 15. It should be understood that the translation quality detection device 2000 described in FIG. 15 can Each step of the translation detection method in the embodiment of the present application is executed, and repeated descriptions are appropriately omitted when introducing the translation detection apparatus in the embodiment of the present application.
  • the translation detection device 3000 shown in FIG. 15 includes:
  • the memory 3001 is used to store programs
  • the processor 3002 is configured to execute a program stored in the memory. When the program stored in the memory is executed by the processor 3002, the processor 3002 is configured to:
  • the translation detection device 3000 may also include an input and output interface 3003, through which the original text and machine translation can be obtained from other devices (for example, a machine translation system). After obtaining the original text and the machine translation, the processor 3002 can be used to process the original text and the machine translation to obtain the translation quality of the machine translation. Further, after obtaining the translation quality of the machine translation, the translation device 3000 may also transmit the translation quality of the machine translation to other devices (for example, downstream systems) through the input/output interface 3003.
  • other devices for example, downstream systems
  • the translation detection device 3000 described above is equivalent to the translation quality detection system shown in FIGS. 9 and 12, and can be used to detect the translation quality of the machine translation obtained by the machine translation system.
  • the aforementioned translation detection device 3000 may be equivalent to the data processing device shown in FIG. 1 or the user equipment shown in FIG. 2.
  • the translation detection device 3000 may be equivalent to the execution device 210 shown in FIG. 3 and the execution device 110 shown in FIG. 4.
  • Fig. 16 is a schematic block diagram of a machine translation system provided by an embodiment of the present application.
  • the machine translation system 4000 shown in FIG. 16 includes:
  • the memory 4001 is used to store programs
  • the processor 4002 is configured to execute a program stored in the memory.
  • the processor 4002 is configured to:
  • the above-mentioned machine translation system 4000 may also include an input and output interface 4003.
  • the machine translation system 4000 can obtain the original text through the input and output interface 4003. Specifically, the original text can be obtained from other devices (for example, a terminal device) through the input and output interface 4003. After the original text is processed by the processor 4002, the translation quality of the machine translation can be finally obtained.
  • the machine translation system 4000 can transmit the quality of the machine translation to other devices (for example, downstream systems) through the input and output interface 4003.
  • the machine translation system 4000 can not only implement the translation of the original text, but also detect the machine translation obtained by its own translation, so as to obtain the translation quality of the machine translation.
  • the machine translation system 4000 here is equivalent to the combination of the machine translation system and the translation quality detection system shown in FIGS. 9 and 12.
  • the machine translation system 4000 may be equivalent to the data processing device shown in FIG. 1 or the user device shown in FIG. 2.
  • the machine translation system 4000 may be equivalent to the execution device 210 shown in FIG. 3 and the execution device 110 shown in FIG. 4.
  • the disclosed system, device, and method may be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components can be combined or It can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • each unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of this application essentially or the part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disk or optical disk and other media that can store program code .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Machine Translation (AREA)

Abstract

本申请公开了人工智能领域中的一种译文质量检测方法、装置、机器翻译系统和存储介质。该方法包括：获取原文和原文的机器译文，并综合考虑原文、机器译文以及机器译文的应用场景来确定机器译文的译文质量。本申请在确定译文质量时，通过结合机器译文的应用场景，能够更有针对性地进行译文质量的检测，可以提高译文质量的检测效率。

Description

译文质量检测方法、装置、机器翻译系统和存储介质
本申请要求于2019年03月27日提交中国专利局、申请号为201910237160.0、申请名称为“译文质量检测方法、装置、机器翻译系统和存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及机器翻译技术领域，并且更具体地，涉及一种译文质量检测方法、装置、机器翻译系统和存储介质。
背景技术
人工智能(artificial intelligence,AI)是利用数字计算机或者数字计算机控制的机器模拟、延伸和扩展人的智能，感知环境、获取知识并使用知识获得最佳结果的理论、方法、技术及应用系统。换句话说，人工智能是计算机科学的一个分支，它企图了解智能的实质，并生产出一种新的能以人类智能相似的方式作出反应的智能机器。人工智能也就是研究各种智能机器的设计原理与实现方法，使机器具有感知、推理与决策的功能。
随着人工智能技术的不断发展，让人机之间能够通过自然语言进行交互的自然语言人机交互系统变的越来越重要。人机之间能够通过自然语言进行交互，就需要系统能够识别出人类自然语言的具体含义。通常，系统通过采用对自然语言的句子进行关键信息提取来识别句子的具体含义。
随着机器翻译技术的快速发展，机器翻译得到了越来越广泛的应用。但是，目前通过机器翻译得到的机器译文仍然存在着译文质量较差、译文不可靠等问题。因此，如何检测机器译文的译文质量是一个需要解决的问题。
传统方案主要是通过对机器译文进行人工编辑，得到人工编辑后的机器译文(相当于是比较准确的译文)，然后通过分析机器译文与人工编辑后的译文的差异来确定机器译文的译文质量。传统方案这种通过简单的比对机器译文与人工编辑后的差异来确定译文质量的方式比较单一，检测译文质量的效率不高。
发明内容
本申请提供一种译文质量检测方法、装置、机器翻译系统和存储介质，以更有针对性地进行机器译文质量的检测，进而提高译文质量的检测效率。
第一方面，提供了一种译文质量检测方法，该方法包括：获取原文和机器译文；根据原文、机器译文以及机器译文的应用场景，确定机器译文的译文质量。
其中，上述机器译文是原文经过机器翻译系统翻译得到的译文，也就是说，上述机器译文是机器翻译系统对原文进行机器翻译后得到的译文。
应理解，本申请的译文质量检测方法的执行主体可以是译文质量检测装置或者译文质 量的检测系统。
上述原文和机器译文均可以属于自然语言，自然语言通常是指一种自然地随文化演化的语言。
可选地，上述原文属于第一自然语言，上述机器译文属于第二自然语言，第一语言和第二语言为不同种类的自然语言。
上述原文属于第一自然语言可以是指上述原文是采用第一自然语言表达的一段文字，上述机器译文属于第二自然语言，可以是指上述机器译文是采用第二自然语言表达的一段文字。上述原文和机器译文可以属于任意两种不同种类的自然语言。
上述机器译文的应用场景可以是后续对机器译文进行处理的下游系统的应用场景，该下游系统的应用场景具体可以是指下游系统的处理任务。
可选地，下游系统的处理任务为情感分类、垃圾邮件检测、意图识别以及命名实体识别等用途中的一种或者多种。
本申请中，在确定机器译文的译文质量时，通过结合机器译文的应用场景，能够更有针对性地对机器译文的译文质量进行检测，可以提高机器译文质量的检测效率。
结合第一方面，在第一方面的某些实现方式中，上述根据原文、机器译文以及机器译文的应用场景，确定机器译文的译文质量，包括：采用下游系统对应的译文质量检测模型对原文和机器译文进行处理，得到机器译文的译文质量。
其中，上述下游系统用于对机器译文进行后续处理，下游系统对应的译文质量检测模型是根据训练样本和训练目标训练得到的，该训练样本包括训练原文和训练机器译文，该训练目标包括训练机器译文的译文质量。
上述训练原文可以是专门用于训练译文质量检测模型的文字，该训练原文可以包含多段文字。上述训练机器译文是训练原文经过机器翻译系统翻译得到的译文(训练机器译文是机器翻译系统对训练原文进行机器翻译得到的译文)，训练机器译文的译文质量是根据第一处理结果与第二处理结果之间的差异确定的，其中，第一处理结果为下游系统对训练机器译文进行处理的处理结果，第二处理结果为下游系统对训练原文的参考译文进行处理的处理结果。
应理解，本申请中，当采用下游系统对应的译文质量检测模型对原文和机器译文进行处理时相当于考虑到了机器译文的应用场景，能够更有针对性地对机器译文的译文质量进行检测，可以提高机器译文质量的检测效率。
上述下游系统对应的译文质量检测模型可以是预先训练好的模型。在对译文质量进行检测时，需要采用(后续对机器译文进行处理的)下游系统对应的译文质量检测模型对译文质量进行检测。
上述训练原文、训练机器译文和训练机器译文的译文质量可以统称为训练数据，通过该训练数据对译文质量检测模型进行训练，能够得到译文质量检测模型的模型参数，进而得到训练后的译文质量检测模型，训练后得到的译文质量检测模型可以对原文和机器译文进行处理，从而得到机器译文的译文质量。
本申请中，通过采用训练好的译文质量检测模型对原文和机器译文进行处理，能够提高译文检测的效率。
结合第一方面，在第一方面的某些实现方式中，上述训练原文和训练原文的参考译文 来自于已知的双语平行语料。
上述已知的双语平行语料可以来自于本地存储的双语平行语料库，也可以来自于云端存储的双语平行语料库。
通过从已知的双语平行语料库中获取原文和原文的参考译文，能够获取大量的训练数据，减少训练数据的获取难度，进而能够简化根据训练数据得到译文质量检测模型的过程。
可选地，上述训练原文的参考译文为人工编辑后的译文。
一般来说，由于人工编辑者的水平较高，能够确保参考译文的准确性，在参考译文数目相当的情况下，更准确的参考译文才能使译文质量检测模型更精准。因此，在获取到训练原文之后，通过人工编辑获得训练原文的参考译文，能够获取到准确的参考译文，便于后续得到更精准的译文质量检测模型。
结合第一方面，在第一方面的某些实现方式中，上述机器译文的译文质量为机器译文的可接受性信息。
上述机器译文的可接受性信息可以表示机器译文的可接受程度，如果机器译文的可接受程度越大，机器译文的译文质量就越高，反之，机器译文的可接受程度越小，机器译文的译文质量就越差。
可选地，上述机器译文的译文质量为机器译文与标准参考译文之间的差异值。
当机器译文与标准参考译文之间的差异值越大时，机器译文的译文质量越差；当机器译文与标准参考译文之间的差异值越小时，机器译文的译文质量越好。
结合第一方面，在第一方面的某些实现方式中，上述机器译文的可接受性信息用于指示所述机器译文是否可以接受。
此时，机器译文的可接受性信息可以包含一个标识位，该标识位的取值用于表示机器译文是否可以接受，例如，该标识位的取值为1时表示该机器译文可以被接受(机器译文可以送入到下游系统进行处理)，该标识位的取值为0时表示该机器译文不可以被接受或者机器译文不被接受(机器译文不能够送入下游系统进行处理)。
结合第一方面，在第一方面的某些实现方式中，上述机器译文的可接受性信息用于指示机器译文被接受的概率或者不被接受的概率。
上述机器译文的可接受性信息具体可以是机器译文被接受的概率的概率值或者机器译文不被接受的概率的概率值。
结合第一方面，在第一方面的某些实现方式中，上述译文质量检测模型为神经网络模型。
本申请中，当译文质量检测模型为神经网络模型时，在对原文和机器译文进行处理得到译文质量的效果更好。
可选地，上述译文质量检测模型为基于支持向量机的模型。
第二方面，提供了一种译文质量检测装置，该装置包括用于执行上述第一方面中的方法中的各个模块。
第三方面，提供了一种机器翻译系统，该机器翻译系统包括机器翻译装置以及第二方面中的译文质量检测装置，其中，机器翻译装置用于对获取原文并对所述原文进行翻译，得到机器译文，译文质量检测装置用于对所述机器译文进行检测，得到所述机器译文的译文质量。
可选地，译文质量检测装置用于对所述机器译文进行检测，得到所述机器译文的译文质量，包括：译文质量检测装置用于根据所述原文、所述机器译文以及所述机器译文的应用场景，确定所述机器译文的译文质量。
第四方面，提供了一种跨语言处理系统，该系统包括：机器翻译装置，用于获取原文，并对所述原文进行翻译，得到机器译文；译文质量检测装置，用于根据所述原文、所述机器译文以及所述机器译文的应用场景，确定所述机器译文的译文质量；下游系统，用于在所述机器译文的译文质量满足预设要求的情况下对所述机器译文进行处理。
上述译文质量检测装置、机器翻译系统和跨语言处理系统可以是一种电子设备(或者是位于电子设备中的模块)，该电子设备具体可以是移动终端(例如，智能手机)，电脑，个人数字助理，可穿戴设备，车载设备，物联网设备或者其他能够进行自然语言处理的设备。
第五方面，提供一种计算机可读介质，该计算机可读介质存储用于设备执行的程序代码，该程序代码包括用于执行第一方面中的方法。
第六方面，提供一种包含指令的计算机程序产品，当该计算机程序产品在计算机上运行时，使得计算机执行上述第一方面中的方法。
第七方面，提供一种芯片，所述芯片包括处理器与数据接口，所述处理器通过所述数据接口读取存储器上存储的指令，执行第一方面中的方法。
可选地，作为一种实现方式，所述芯片还可以包括存储器，所述存储器中存储有指令，所述处理器用于执行所述存储器上存储的指令，当所述指令被执行时，所述处理器用于执行第一方面中的方法。
第八方面，提供一种电子设备，该电子设备包括上述第二方面中的译文质量检测装置或者上述第三方面中的机器翻译系统或者上述第四方面中的跨语言处理系统。
附图说明
图1是本申请实施例提供的一种自然语言处理的应用场景示意图；
图2是本申请实施例提供的另一种自然语言处理的应用场景示意图；
图3是本申请实施例提供的自然语言处理的相关设备的示意图；
图4是本申请实施例提供的一种系统架构的示意图；
图5是本申请实施例提供的一种根据CNN模型进行译文质量检测的示意图；
图6本申请实施例提供的另一种根据CNN模型进行译文质量检测的示意图；
图7是本申请实施例提供的一种芯片的硬件结构的示意图；
图8是本申请实施例提供的一种应用场景的示意图；
图9是本申请实施例提供的译文质量检测过程的示意图；
图10是本申请实施例提供的译文质量检测方法的示意性流程图；
图11是本申请实施例提供的获取训练数据的过程的示意图；
图12是本申请实施例提供的译文质量检测过程的示意图；
图13是本申请实施例提供的译文质量检测过程的示意图；
图14是本申请实施例提供的译文质量检测过程的示意图；
图15是本申请实施例提供的译文质量检测装置的示意性框图；
图16是本申请实施例提供的机器翻译系统的示意性框图。
具体实施方式
下面将结合附图，对本申请中的技术方案进行描述。
为了更好地理解本申请实施例的方案，下面先结合图1至图3对本申请实施例可能的应用场景进行简单的介绍。
图1示出了一种自然语言处理系统，该自然语言处理系统包括用户设备以及数据处理设备。其中，用户设备包括手机、个人电脑或者信息处理中心等智能终端。用户设备为自然语言数据处理的发起端，作为语言问答或者查询等请求的发起方，通常用户通过用户设备发起请求。
上述数据处理设备可以是云服务器、网络服务器、应用服务器以及管理服务器等具有数据处理功能的设备或服务器。数据处理设备通过交互接口接收来自智能终端的查询语句/语音/文本等问句，再通过存储数据的存储器以及数据处理的处理器环节进行机器学习，深度学习，搜索，推理，决策等方式的语言数据处理。数据处理设备中的存储器可以是一个统称，包括本地存储以及存储历史数据的数据库，数据库可以再数据处理设备上，也可以在其它网络服务器上。
在图1所示的自然语言处理系统中，用户设备可以接收用户的指令，对原文(例如，该原文可以是用户输入的一段英文)进行机器翻译得到机器译文，然后向数据处理设备发起请求，使得数据处理设备对用户设备翻译得到的机器译文进行译文质量检测，从而得到机器译文的译文质量。
在图1中，数据处理设备可以执行本申请实施例的译文质量检测方法。
图2示出了另一种自然语言处理系统，在图2中，用户设备直接作为数据处理设备，该用户设备能够直接接收来自用户的输入并直接由用户设备本身的硬件进行处理，具体过程与图1相似，可参考上面的描述，在此不再赘述。
在图2所示的自然语言处理系统中，用户设备可以接收用户的指令，对原文进行机器翻译得到机器译文，然后再由用户设备自身对机器翻译得到的机器译文进行译文质量检测，从而得到机器译文的译文质量。
在图2中，用户设备自身就可以执行本申请实施例的译文质量检测方法。
图3是本申请实施例提供的自然语言处理的相关设备的示意图。
上述图1和图2中的用户设备具体可以是图3中的本地设备301或者本地设备302，图1中的数据处理设备具体可以是图3中的执行设备210，其中，数据存储系统250可以存储执行设备210的待处理数据，数据存储系统250可以集成在执行设备210上，也可以设置在云上或其它网络服务器上。
图1和图2中的处理器可以通过神经网络模型或者其它模型(例如，基于支持向量机的模型)进行数据训练/机器学习/深度学习，并利用数据最终训练或者学习得到的模型对机器译文的译文质量进行检测，从而得到机器译文的译文质量。
由于本申请实施例涉及大量神经网络的应用，为了便于理解，下面先对本申请实施例可能涉及的神经网络的相关术语和概念进行介绍。
(1)神经网络
神经网络可以是由神经单元组成的，神经单元可以是指以x s和截距1为输入的运算单元，该运算单元的输出可以为：
Figure PCTCN2020079964-appb-000001
其中，s＝1、2、……n，n为大于1的自然数，W s为x s的权重，b为神经单元的偏置。f为神经单元的激活函数(activation functions)，用于将非线性特性引入神经网络中，来将神经单元中的输入信号转换为输出信号。该激活函数的输出信号可以作为下一层卷积层的输入，激活函数可以是sigmoid函数。神经网络是将多个上述单一的神经单元联结在一起形成的网络，即一个神经单元的输出可以是另一个神经单元的输入。每个神经单元的输入可以与前一层的局部接受域相连，来提取局部接受域的特征，局部接受域可以是由若干个神经单元组成的区域。
(2)深度神经网络
深度神经网络(deep neural network，DNN)，也称多层神经网络，可以理解为具有多层隐含层的神经网络。按照不同层的位置对DNN进行划分，DNN内部的神经网络可以分为三类：输入层，隐含层，输出层。一般来说第一层是输入层，最后一层是输出层，中间的层数都是隐含层。层与层之间是全连接的，也就是说，第i层的任意一个神经元一定与第i+1层的任意一个神经元相连。
虽然DNN看起来很复杂，但是就每一层的工作来说，其实并不复杂，简单来说就是如下线性关系表达式：
Figure PCTCN2020079964-appb-000002
其中，
Figure PCTCN2020079964-appb-000003
是输入向量，
Figure PCTCN2020079964-appb-000004
是输出向量，
Figure PCTCN2020079964-appb-000005
是偏移向量，W是权重矩阵(也称系数)，α()是激活函数。每一层仅仅是对输入向量
Figure PCTCN2020079964-appb-000006
经过如此简单的操作得到输出向量
Figure PCTCN2020079964-appb-000007
由于DNN层数多，系数W和偏移向量
Figure PCTCN2020079964-appb-000008
的数量也比较多。这些参数在DNN中的定义如下所述：以系数W为例：假设在一个三层的DNN中，第二层的第4个神经元到第三层的第2个神经元的线性系数定义为
Figure PCTCN2020079964-appb-000009
上标3代表系数W所在的层数，而下标对应的是输出的第三层索引2和输入的第二层索引4。
综上，第L-1层的第k个神经元到第L层的第j个神经元的系数定义为
Figure PCTCN2020079964-appb-000010
需要注意的是，输入层是没有W参数的。在深度神经网络中，更多的隐含层让网络更能够刻画现实世界中的复杂情形。理论上而言，参数越多的模型复杂度越高，“容量”也就越大，也就意味着它能完成更复杂的学习任务。训练深度神经网络的也就是学习权重矩阵的过程，其最终目的是得到训练好的深度神经网络的所有层的权重矩阵(由很多层的向量W形成的权重矩阵)。
(3)卷积神经网络
卷积神经网络(convolutional neuron network，CNN)是一种带有卷积结构的深度神经网络。卷积神经网络包含了一个由卷积层和子采样层构成的特征抽取器，该特征抽取器可以看作是滤波器。卷积层是指卷积神经网络中对输入信号进行卷积处理的神经元层。在卷积神经网络的卷积层中，一个神经元可以只与部分邻层神经元连接。一个卷积层中，通常包含若干个特征平面，每个特征平面可以由一些矩形排列的神经单元组成。同一特征平面的神经单元共享权重，这里共享的权重就是卷积核。共享权重可以理解为提取图像信息的方式与位置无关。卷积核可以以随机大小的矩阵的形式初始化，在卷积神经网络的训练过程中卷积核可以通过学习得到合理的权重。另外，共享权重带来的直接好处是减少卷积神经网络各层之间的连接，同时又降低了过拟合的风险。
(4)循环神经网络(recurrent neural networks，RNN)是用来处理序列数据的。在传统的神经网络模型中，是从输入层到隐含层再到输出层，层与层之间是全连接的，而对于每一层层内之间的各个节点是无连接的。这种普通的神经网络虽然解决了很多难题，但是却仍然对很多问题无能无力。例如，你要预测句子的下一个单词是什么，一般需要用到前面的单词，因为一个句子中前后单词并不是独立的。RNN之所以称为循环神经网路，即一个序列当前的输出与前面的输出也有关。具体的表现形式为网络会对前面的信息进行记忆并应用于当前输出的计算中，即隐含层本层之间的节点不再无连接而是有连接的，并且隐含层的输入不仅包括输入层的输出还包括上一时刻隐含层的输出。理论上，RNN能够对任何长度的序列数据进行处理。对于RNN的训练和对传统的CNN或DNN的训练一样。
既然已经有了卷积神经网络，为什么还要循环神经网络？原因很简单，在卷积神经网络中，有一个前提假设是：元素之间是相互独立的，输入与输出也是独立的，比如猫和狗。但现实世界中，很多元素都是相互连接的，比如股票随时间的变化，再比如一个人说了：我喜欢旅游，其中最喜欢的地方是云南，以后有机会一定要去。这里填空，人类应该都知道是填“云南”。因为人类会根据上下文的内容进行推断，但如何让机器做到这一步？RNN就应运而生了。RNN旨在让机器像人一样拥有记忆的能力。因此，RNN的输出就需要依赖当前的输入信息和历史的记忆信息。
(5)损失函数
在训练深度神经网络的过程中，因为希望深度神经网络的输出尽可能的接近真正想要预测的值，所以可以通过比较当前网络的预测值和真正想要的目标值，再根据两者之间的差异情况来更新每一层神经网络的权重向量(当然，在第一次更新之前通常会有初始化的过程，即为深度神经网络中的各层预先配置参数)，比如，如果网络的预测值高了，就调整权重向量让它预测低一些，不断地调整，直到深度神经网络能够预测出真正想要的目标值或与真正想要的目标值非常接近的值。因此，就需要预先定义“如何比较预测值和目标值之间的差异”，这便是损失函数(loss function)或目标函数(objective function)，它们是用于衡量预测值和目标值的差异的重要方程。其中，以损失函数举例，损失函数的输出值(loss)越高表示差异越大，那么深度神经网络的训练就变成了尽可能缩小这个loss的过程。
(6)反向传播算法
神经网络可以采用误差反向传播(back propagation，BP)算法在训练过程中修正初始的神经网络模型中参数的大小，使得神经网络模型的重建误差损失越来越小。具体地，前向传递输入信号直至输出会产生误差损失，通过反向传播误差损失信息来更新初始的神经网络模型中参数，从而使误差损失收敛。反向传播算法是以误差损失为主导的反向传播运动，旨在得到最优的神经网络模型的参数，例如权重矩阵。
如图4所示，本申请实施例提供了一种系统架构100。在图4中，数据采集设备160用于采集训练数据，本申请实施例中训练数据包括训练原文、训练机器译文(训练原文经过机器翻译系统翻译得到的译文)以及训练机器译文的译文质量，其中，训练机器译文的译文质量可以是机器译文所应用的下游系统对训练机器译文的处理结果与该下游系统对训练参考译文的处理结果之间的差异确定的。这里的下游系统可以位于客户设备内部或者位于用户设备之外的其他设备中。
在采集到训练数据之后，数据采集设备160将这些训练数据存入数据库130，训练设备120基于数据库130中维护的训练数据训练得到目标模型/规则101。
下面对训练设备120基于训练数据得到目标模型/规则101进行描述，训练设备120对输入的训练原文和训练机器译文进行处理，将输出的译文质量与训练机器译文的译文质量进行对比，直到训练设备120输出译文质量与训练机器译文的译文质量的差值小于一定的阈值，从而完成目标模型/规则101的训练。
上述目标模型/规则101能够用于实现本申请实施例的译文质量检测方法，即将原文和机器译文像通过相关预处理(可以采用预处理模块113和/或预处理模块114进行处理)后输入该目标模型/规则101，即可得到机器译文的译文质量。本申请实施例中的目标模型/规则101具体可以为神经网络。需要说明的是，在实际的应用中，所述数据库130中维护的训练数据不一定都来自于数据采集设备160的采集，也有可能是从其他设备接收得到的。另外需要说明的是，训练设备120也不一定完全基于数据库130维护的训练数据进行目标模型/规则101的训练，也有可能从云端或其他地方获取训练数据进行模型训练，上述描述不应该作为对本申请实施例的限定。
根据训练设备120训练得到的目标模型/规则101可以应用于不同的系统或设备中，如应用于图4所示的执行设备110，所述执行设备110可以是终端，如手机终端，平板电脑，笔记本电脑，增强现实(augmented reality，AR)/虚拟现实(virtual reality，VR)，车载终端等，还可以是服务器或者云端等。在图4中，执行设备110配置输入/输出(input/output，I/O)接口112，用于与外部设备进行数据交互，用户可以通过客户设备140向I/O接口112输入数据，所述输入数据在本申请实施例中可以包括：客户设备输入的原文和机器译文。
预处理模块113和预处理模块114用于根据I/O接口112接收到的输入数据(如原文和机器译文)进行预处理(具体可以是对原文和机器译文进行处理，得到词向量)，在本申请实施例中，也可以没有预处理模块113和预处理模块114(也可以只有其中的一个预处理模块)，而直接采用计算模块111对输入数据进行处理。
在执行设备110对输入数据进行预处理，或者在执行设备110的计算模块111执行计算等相关的处理过程中，执行设备110可以调用数据存储系统150中的数据、代码等以用于相应的处理，也可以将相应处理得到的数据、指令等存入数据存储系统150中。
最后，I/O接口112将处理结果，例如，机器译文的译文质量反馈给客户设备140。
值得说明的是，训练设备120可以针对不同的下游系统，生成该下游系统对应的目标模型/规则101，该相应的目标模型/规则101即可以用于实现上述目标或完成上述任务，从而为用户提供所需的结果。
在图4中所示情况下，用户可以手动给定输入数据(例如，输入一段文字)，该手动给定可以通过I/O接口112提供的界面进行操作。另一种情况下，客户设备140可以自动地向I/O接口112发送输入数据(例如，输入一段文字)，如果要求客户设备140自动发送输入数据需要获得用户的授权，则用户可以在客户设备140中设置相应权限。用户可以在客户设备140查看执行设备110输出的结果，具体的呈现形式可以是显示、声音、动作等具体方式(例如，输出结果可以是机器译文是否可以接受)。客户设备140也可以作为数据采集端，采集如图所示输入I/O接口112的输入数据及输出I/O接口112的输出结果 作为新的样本数据，并存入数据库130。当然，也可以不经过客户设备140进行采集，而是由I/O接口112直接将如图所示输入I/O接口112的输入数据及输出I/O接口112的输出结果，作为新的样本数据存入数据库130。
值得注意的是，图4仅是本申请实施例提供的一种系统架构的示意图，图中所示设备、器件、模块等之间的位置关系不构成任何限制。例如，在图4中，数据存储系统150相对执行设备110是外部存储器，在其它情况下，也可以将数据存储系统150置于执行设备110中。
如图4所示，根据训练设备120训练得到目标模型/规则101，该目标模型/规则101可以是本申请实施例中的译文质量检测模型，具体的，本申请实施例提供的神经网络可以是CNN，深度卷积神经网络(deep convolutional neural network,DCNN)，循环神经网络(recurrent neural network，RNN)等等。
由于CNN是一种非常常见的神经网络，下面结合图5重点对CNN的结构进行详细的介绍。如上文的基础概念介绍所述，卷积神经网络是一种带有卷积结构的深度神经网络，是一种深度学习(deep learning)架构，深度学习架构是指通过机器学习的算法，在不同的抽象层级上进行多个层次的学习。作为一种深度学习架构，CNN是一种前馈(feed-forward)人工神经网络，该前馈人工神经网络中的各个神经元可以对输入其中的图像作出响应。
如图5所示，卷积神经网络(CNN)200可以包括输入层110，卷积层/池化层120(其中池化层为可选的)，以及神经网络层130。下面对这些层的相关内容做详细介绍。
卷积层/池化层120：
卷积层：
如图5所示卷积层/池化层120可以包括如示例121-126层，在一种实现中，121层为卷积层，122层为池化层，123层为卷积层，124层为池化层，125为卷积层，126为池化层；在另一种实现方式中，121、122为卷积层，123为池化层，124、125为卷积层，126为池化层。即卷积层的输出可以作为随后的池化层的输入，也可以作为另一个卷积层的输入以继续进行卷积操作。
以卷积层121为例，卷积层121可以包括很多个卷积算子，卷积算子也称为核，其在自然语言处理中的作用相当于一个从输入的语音或语义信息中提取特定信息的过滤器，卷积算子本质上可以是一个权重矩阵，这个权重矩阵通常被预先定义。
这些权重矩阵中的权重值在实际应用中需要经过大量的训练得到，通过训练得到的权重值形成的各个权重矩阵可以从输入图像中提取信息，从而帮助卷积神经网络100进行正确的预测。
当卷积神经网络100有多个卷积层的时候，初始的卷积层(例如121)往往提取较多的一般特征，该一般特征也可以称之为低级别的特征；随着卷积神经网络100深度的加深，越往后的卷积层(例如126)提取到的特征越来越复杂，比如高级别的语义之类的特征，语义越高的特征越适用于待解决的问题。
池化层：
由于常常需要减少训练参数的数量，因此卷积层之后常常需要周期性的引入池化层，即如图5中120所示例的121-126各层，可以是一层卷积层后面跟一层池化层，也可以是 多层卷积层后面接一层或多层池化层。在自然语言数据处理过程中，池化层的唯一目的就是减少数据的空间大小。
神经网络层130：
在经过卷积层/池化层120的处理后，卷积神经网络100还不足以输出所需要的输出信息。因为如前所述，卷积层/池化层120只会提取特征，并减少输入数据带来的参数。然而为了生成最终的输出信息(所需要的类信息或别的相关信息)，卷积神经网络100需要利用神经网络层130来生成一个或者一组所需要的类的数量的输出。因此，在神经网络层130中可以包括多层隐含层(如图5所示的131、132至13n)以及输出层140，该多层隐含层中所包含的参数可以根据具体的任务类型的相关训练数据进行预先训练得到，例如该任务类型可以包括语音或语义识别、分类或生成等等。
在神经网络层130中的多层隐含层之后，也就是整个卷积神经网络100的最后层为输出层140，该输出层140具有类似分类交叉熵的损失函数，具体用于计算预测误差，一旦整个卷积神经网络100的前向传播(如图5由110至140的传播为前向传播)完成，反向传播(如图5由140至110的传播为反向传播)就会开始更新前面提到的各层的权重值以及偏差，以减少卷积神经网络100的损失及卷积神经网络100通过输出层输出的结果和理想结果之间的误差。
需要说明的是，如图5所示的卷积神经网络100仅作为一种卷积神经网络的示例，在具体的应用中，卷积神经网络还可以以其他网络模型的形式存在。
如图6所示，卷积神经网络(CNN)200可以包括输入层110，卷积层/池化层120(其中池化层为可选的)，以及神经网络层130，在图6中，卷积层/池化层120中的多个卷积层/池化层并行，将分别提取的特征均输入给全神经网络层130进行处理。
图7为本申请实施例提供的一种芯片的硬件结构的示意图。该芯片包括神经网络处理器(neural processing unit，NPU)50。该芯片可以被设置在如图4所示的执行设备110中，用以完成计算模块111的计算工作。该芯片也可以被设置在如图4所示的训练设备120中，用以完成训练设备120的训练工作并输出目标模型/规则101。如图5和图6所示的卷积神经网络中各层的算法均可在如图7所示的芯片中得以实现。
本申请实施例的译文质量检测方法的具体可以在NPU 50中的运算电路503和/或向量计算单元507中执行，从而得到机器译文的译文质量。
下面对NPU 50中的各个模块和单元进行简单的介绍。
NPU 50作为协处理器可以挂载到主CPU(host CPU)上，由主CPU分配任务。NPU 50的核心部分为运算电路503，在NUP 50工作时，NPU 50中的控制器504可以控制运算电路503提取存储器(权重存储器或输入存储器)中的数据并进行运算。
在一些实现中，运算电路503内部包括多个处理单元(process engine,PE)。在一些实现中，运算电路503是二维脉动阵列。运算电路503还可以是一维脉动阵列或者能够执行例如乘法和加法这样的数学运算的其它电子线路。在一些实现中，运算电路503是通用的矩阵处理器。
举例来说，假设有输入矩阵A，权重矩阵B，输出矩阵C。运算电路从权重存储器502中取矩阵B相应的数据，并缓存在运算电路中每一个PE上。运算电路从输入存储器501中取矩阵A数据与矩阵B进行矩阵运算，得到的矩阵的部分结果或最终结果，保存在累 加器(accumulator)508中。
向量计算单元507可以对运算电路的输出做进一步处理，如向量乘，向量加，指数运算，对数运算，大小比较等等。例如，向量计算单元507可以用于神经网络中非卷积/非全连接层(fully connected layers，FC)层的网络计算，如池化(pooling)，批归一化(batch normalization)，局部响应归一化(local response normalization)等。
在一些实现种，向量计算单元507能将经处理的输出的向量存储到统一缓存器506。例如，向量计算单元507可以将非线性函数应用到运算电路503的输出，例如累加值的向量，用以生成激活值。在一些实现中，向量计算单元507生成归一化的值、合并值，或二者均有。在一些实现中，处理过的输出的向量能够用作到运算电路503的激活输入，例如用于在神经网络中的后续层中的使用。
统一存储器506用于存放输入数据以及输出数据。
权重数据直接通过存储单元访问控制器505(direct memory access controller，DMAC)将外部存储器中的输入数据搬运到输入存储器501和/或统一存储器506、将外部存储器中的权重数据存入权重存储器502，以及将统一存储器506中的数据存入外部存储器。
总线接口单元(bus interface unit，BIU)510，用于通过总线实现主CPU、DMAC和取指存储器509之间进行交互。
与控制器504连接的取指存储器(instruction fetch buffer)509，用于存储控制器504使用的指令；
控制器504，用于调用指存储器509中缓存的指令，实现控制该运算加速器的工作过程。
一般地，统一存储器506，输入存储器501，权重存储器502以及取指存储器509均可以为片上(on-chip)存储器。NPU的外部存储器可以为该NPU外部的存储器，该外部存储器可以为双倍数据率同步动态随机存储器(double data rate synchronous dynamic random access memory，DDR SDRAM)、高带宽存储器(high bandwidth memory，HBM)或其他可读可写的存储器。
下面结合附图对本申请实施例的译文质量检测方法进行详细介绍。本申请实施例的译文质量检测方法可以由图1中的数据处理设备、图2中的用户设备、图3中的执行设备210以及图4中的执行设备110等设备执行。
图8是本申请实施例提供的一种应用场景的示意图。如图8所示，通过机器翻译系统对原文进行翻译能够得到机器译文，机器译文可以送入到下游系统中进行处理。本申请实施例的译文质量检测方法能够(在机器译文送入到下游系统进行处理之前)对机器译文的译文质量进行检测或者评估，从而获取机器译文的译文质量，便于根据机器译文的译文质量来确定是否将机器译文送入到下游系统中进行处理。
图9是本申请实施例提供的一种译文质量检测过程的示意图。如图9所示，机器翻译系统对原文进行机器翻译后得到机器译文，接下来，机器译文和原文被送入到译文质量检测系统中进行质量检测，以确定机器译文的译文质量，当机器译文的译文质量满足要求时(例如，机器译文的译文质量较好)，可以将机器译文送入到下游系统进行处理，而当机器译文的译文质量不能满足要求时(例如，机器译文的译文质量较差)，可以将机器译文送入到其他处理模块中进行其他处理(例如，可以将机器译文舍弃掉不做处理)。
机器译文的译文质量一般与机器译文的具体应用场景存在很大的关联关系，对于同一条机器译文，在面对不同的下游任务(下游系统对机器译文进行处理的任务可以称为下游任务)时，该机器译文的可接受性可能不太相同。也就是说，机器译文的译文质量标准跟机器译文的具体应用场景相关，因此，在本申请实施例中，通过结合机器译文的应用场景，能够更有针对性地确定机器译文的译文质量。
图10是本申请实施例提供的译文质量检测方法的示意性流程图。图10所示的方法可以由译文质量检测系统(具体可以是如图9中所示的译文质量检测系统)执行，该译文质量检测系统具体可以是图1中的数据处理设备，也可以是图2中的用户设备，也可以是图3中的执行设备210，也可以是图4中的执行设备110。图10所示的方法包括步骤1001和步骤1002，下面分别对这两个步骤进行详细的介绍。
1001、获取原文和机器译文。
其中，上述机器译文是原文经过机器翻译系统翻译得到的译文，也就是说，上述机器译文是机器翻译系统对原文进行机器翻译得到的译文。
当10所示的方法由图9所示的译文质量检测系统执行时，上述步骤1001中的机器译文可以是经过图9中的机器翻译系统进行机器翻译得到的译文。
上述原文和机器译文均可以属于自然语言，其中，自然语言通常是指一种自然地随文化演化的语言。
上述原文可以是采用某种自然语言书写得到的一段文字，例如，上述原文可以是一段英文或者中文等等。
可选地，上述原文属于第一自然语言，上述机器译文属于第二自然语言，第一语言和第二语言为不同的自然语言。
应理解，上述原文属于第一自然语言可以是指上述原文是采用第一自然语言表达的一段文字，上述机器译文属于第二自然语言，可以是指上述机器译文是采用第二自然语言表达的一段文字。上述原文和机器译文可以属于任意两种不同的自然语言。
例如，上述原文可以是一段中文，该原文的机器译文可以是一段英文(通过机器翻译系统对这段中文进行翻译，得到了一段英文)。
再如，上述原文可以是一段中文，该中文的机器译文可以是一段日文(通过机器翻译系统对这段中文进行翻译，得到了一段日文)。
1002、根据原文、机器译文以及机器译文的应用场景，确定机器译文的译文质量。
当10所示的方法由图9所示的译文质量检测系统执行时，下游系统能够体现出机器译文的应用场景，步骤1002中的机器译文的应用场景可以是后续对机器译文进行处理的下游系统的应用场景，该下游系统的应用场景具体可以是指下游系统的处理任务。
例如，下游系统的处理任务可以是情感分类、垃圾邮件检测、意图识别以及命名实体识别等用途中一种或者多种。
其中，命名实体识别(named entity recognition，NER)，又可以称为“专名识别”，具体是指识别文本中具有特定意义的实体，该实体主要包括人名、地名、机构名、专有名词等。
本申请中，在确定机器译文的译文质量时，通过结合机器译文的应用场景，能够更有针对性地对机器译文的译文质量进行检测，可以提高机器译文质量的检测效率。
可选地，上述步骤1002中根据原文、机器译文以及机器译文的应用场景，确定机器译文的译文质量，具体包括：采用下游系统对应的译文质量检测模型对原文和机器译文进行处理，得到机器译文的译文质量。
其中，上述下游系统用于对机器译文进行后续处理，该下游系统能够反映机器译文的应用场景，因此，在根据下游系统对应的译文质量检测模型对原文和机器译文进行处理时相当于考虑到了机器译文的应用场景。
下游系统对应的译文质量检测模型可以是根据训练样本和训练目标训练得到的，其中，该训练样本包括训练原文和训练机器译文，该训练目标包括训练机器译文的译文质量。
上述训练机器译文可以是训练原文经过机器翻译系统翻译得到的译文，也就是说，上述训练机器译文是机器翻译系统对训练原文进行机器翻译后得到的译文，上述训练机器译文的译文质量可以是根据下游系统对训练机器译文的处理结果以及下游系统对训练原文的参考译文的处理结果确定的。
具体地，上述训练机器译文的译文质量可以是根据下游系统对训练机器译文的处理结果与下游系统对训练原文的参考译文的处理结果之间的差异确定的。也就是说，可以根据下游系统对训练机器译文的处理结果与下游系统对训练原文的参考译文的处理结果之间的差异来确定训练机器译文的译文质量。
进一步的，上述训练机器译文的译文质量可以是根据第一处理结果和第二处理结果之间的差异来确定的。也就是说，可以根据第一处理结果与第二处理结果之间的差异来确定训练机器译文的译文质量，其中，第一处理结果为下游系统对训练机器译文进行处理的处理结果，第二处理结果为下游系统对训练原文的参考译文进行处理的处理结果。
例如，当上述第一处理结果与第二处理结果之间的差异较小时，训练机器译文的译文质量比较好，当上述第一处理结果与第二处理结果之间的差异较大时，训练机器译文的译文质量比较差。
上述下游系统对应的译文质量检测模型可以是预先训练好的模型。在对译文质量进行检测时，需要采用(后续对机器译文进行处理的)下游系统对应的译文质量检测模型对译文质量进行检测。
上述训练原文、训练机器译文和训练机器译文的译文质量可以统称为训练数据，通过该训练数据对译文质量检测模型进行训练，能够得到译文质量检测模型的模型参数，进而得到训练后的译文质量检测模型，该训练后得到的译文质量检测模型可以对原文和机器译文进行处理，从而得到机器译文的译文质量。
上述训练原文和训练机器译文可以称为训练输入数据，在对机器译文的译文质量检测模型进行训练时，通过输入训练输入数据，使得译文质量检测模型的输出与训练目标尽可能地接近，并在译文质量检测模型的输出与训练目标的差满足预设要求(例如，译文质量检测模型的输出与训练目标的差小于预设阈值)时的模型参数确定为译文质量检测模型的最终模型参数。
可选地，上述训练原文和训练原文的参考译文来自于已知的双语平行语料。
上述已知的双语平行语料可以来自于本地存储的双语平行语料库，也可以来自于云端存储的双语平行语料库。
通过从已知的双语平行语料库中获取原文和原文的参考译文，能够减少训练数据的获 取难度，进而能够简化根据训练数据得到译文质量检测模型的过程。
可选地，上述训练原文的参考译文为人工编辑后的译文。
一般来说，由于人工编辑者的水平较高，能够确保参考译文的准确性，在参考译文数目相当的情况下，更准确的参考译文才能使译文质量检测模型更精准。因此，在获取到训练原文之后，通过人工编辑获得训练原文的参考译文，能够获取到准确的参考译文，便于后续得到更为准确的译文质量检测模型。
在本申请中，译文质量检测模型可以是根据训练数据训练得到的，这里的训练数据包括训练原文，训练机器译文和训练机器译文的译文质量，下面对结合附图对训练数据的获取过程进行简单的介绍。
图11是本申请实施例提供的获取训练数据的示意图。
如图11所示，机器翻译系统对训练原文进行机器翻译，得到训练机器译文，下游系统对训练机器译文进行处理，得到第一处理结果，另外，下游系统还需要对训练原文的参考译文进行处理，得到第二处理结果；接下来，可以再根据第一处理结果与第二处理结果的差异来确定训练机器译文的译文质量。
其中，根据第一处理结果与第二处理结果的差异来确定训练机器译文的译文质量具体可以由下游系统本身来执行，也可以由下游系统之外的其他设备或者装置来执行。
图11中的训练原文和训练原文的参考译文可以来自于已知的双语平行语料，并且，该训练原文可以包含多段文字(此时，训练原文的参考译文是该多段文字分别对应的参考译文)。
由于训练原文一般会包含大量的数据，因此，通过图11所示的过程能够得到大量的训练数据，该训练数据可以表示为(<训练原文训练机器译文>，训练机器译文的译文质量)，这些训练数据可以用于后续对译文质量检测模型进行训练。
通过图11所示的过程得到了训练数据之后，可以对译文质量检测模型进行训练，并在后续利用译文质量检测模型对译文质量进行检测，以确定机器译文的译文质量。
下面结合例一和例二来说明如何确定训练译文的译文质量，进而得到训练数据。
例一：下游系统执行的是主观性分类任务。
在例一中，原文和原文的参考译文如下所示：
原文1：我以为昨天夜里下雨了。
原文1的参考译文：I thought it rained last night.
下游系统对原文1的参考译文的处理结果如下：
原文1的参考译文：I thought it rained last night.(分类结果：主观)。
机器翻译系统对原文1进行机器翻译后得到的译文具体为机器译文1a或者机器译文1b，这两个机器译文具体如下所示：
机器译文1a：I thought it rained yesterday.
机器译文1b：It rained last night.
采用下游系统分别对机器译文1a和机器译文1b进行处理，得到的具体处理结果和可接受性标注信息如下所示：
机器译文1a：I thought it rained yesterday.(分类结果：主观；可接受性标注：可接受)；
机器译文1b：It rained last night.(分类结果：客观；可接受性标注：不可接受)。
其中，采用下游系统对对机器译文1a进行处理，得到的处理结果为主观，与原文1的处理结果相同。可以认为机器译文1a的译文质量满足要求，机器译文1a的译文质量为可接受。
采用下游系统对对机器译文1b进行处理，得到的处理结果为客观，与原文1的处理结果不同。可以认为机器译文1b的译文质量不能满足要求，机器译文1b的译文质量为不可接受。
通过以上处理可以得到以下训练数据：
(<原文1机器译文1a>，可接受)；
(<原文1机器译文1b>，不可接受)。
在上面的例子中，机器译文1a中的“夜里”没有被翻译出来，但是不影响后续的分类结果，下游系统对机器译文1a的分类结果仍然为主观，机器译文1a的分类结果与原文1的参考译文的分类结果相同，因此，对于下游系统的分类任务来说，机器译文1a是可以接受的。
而在机器译文1b中，由于“我认为”没有被翻译出来，下游系统对机器译文1b的分类结果是客观，与原文1的参考译文的分类结果不同，因此，机器译文1b是不可接受的。
例一中是以下游系统执行的是主观性分类任务为例对获取数据的过程进行了说明，下面结合例二以下游系统执行的是命名实体的识别为例对获取数据的过程进行说明。
例二：下游系统执行的是命名实体的识别。
在例二中，原文和原文的参考译文如下所示：
原文2：布什与沙龙在纽约进行会谈。
原文2的参考译文：Bush held a talk with Sharon in New York.
下游系统对原文2的参考译文的识别结果为：(命名实体：Bush,Sharon,New York)。
机器翻译系统对原文2进行机器翻译后得到的译文是机器译文2a、机器译文2b和机器译文2c中的任意一种，这些机器译文具体如下所示：
机器译文2a：Bush and Sharon in New York.
机器译文2b：Bush gave a talk at a salon in New York.
机器译文2c：Bush held a talk with Shalong in New York.
下游系统对以上三个机器译文的处理结果以及三个机器译文的可接受性标注信息如下所示：
机器译文2a：Bush and Sharon in New York.(命名实体：Bush,Sharon,New York；可接受性标注：可接受)；
机器译文2b：Bush gave a talk at a salon in New York.(命名实体：Bush,New York；可接受性标注：不可接受)；
机器译文2c：Bush held a talk with Shalong in New York.(命名实体：Bush,Shalong,New York；可接受性标注：不可接受)。
根据下游系统对原文2的参考译文的识别结果可知，参考译文中共有3个命名实体。在机器译文2a中，3个命名实体均翻译正确，可以将机器译文2a标注为可接受。在机器译文2b中，“沙龙”这个命名实体没有被翻译出来，可以将机器译文2b标注为不可接受。在机器译文2c中，“沙龙”翻译有误，可以将机器译文2b也标注为不可接受。这样标注后 就可以得到三个训练数据，具体如下所示：
(<原文2机器译文2a>，可接受)；
(<原文2机器译文2b>，不可接受)；
(<原文2机器译文2c>，不可接受)。
在以上这个例子中，判断机器译文2a、机器译文2b和机器译文2c是否可以接受均以参考译文与机器译文经过下游系统得到的结果相同为可接受的准则。
事实上，也可以根据实际情况来确定机器译文的可接受准则。例如，如果对命名实体在翻译过程中的遗漏十分敏感，而对命名实体的错误可以容忍的话，则可根据命名实体结果集中的元素数目判断是否可接受。在这种情况下，上述机器译文2a和机器译文2c中进行命名实体识别得到的都是3个命名实体。因此，可以将这两个机器译文都标注为可接受。在这种情况下，标注后得到的三个训练数据，具体如下：
(<原文2机器译文2a>，可接受)；
(<原文2机器译文2b>，不可接受)；
(<原文2机器译文2c>，可接受)。
应理解，在上述例一和例二中，均是以机器译文的译文质量为机器译文的可接受性为例进行的说明，事实上，机器译文的译文质量还可以有多种表现形式，这里不对机器译文的具体表现形式做任何限定。
图12是本申请实施例提供的一种译文质量检测过程的示意图。
图12所示的过程可以分为两个阶段，一个是训练阶段，一个是检测阶段。在训练阶段中，当获取到训练数据(<训练原文训练机器译文>，训练机器译文的译文质量)之后，可以根据该训练数据对译文质量检测系统中的译文质量检测模型进行训练，在完成译文质量检测模型的训练之后，接下来就可以利用该译文质量检测系统进行译文质量检测了。
应理解，在图12所示的训练阶段采用的训练数据与下面检测阶段中的下游系统是对应的，也就是说，训练数据中的训练机器译文的译文质量是根据下游系统对训练机器译文的处理结果以及下游系统对训练原文的参考译文的处理结果的差异确定的。
在检测阶段，机器翻译系统对原文进行机器翻译得到机器译文，译文质量检测系统根据原文和机器译文来确定机器译文的译文质量，当机器译文的译文质量满足要求时(例如，机器译文的译文质量为可接受)，可以将机器译文送入到下游系统中进行处理，当机器译文的译文质量不能满足要求时(例如，机器译文的译文质量为不可接受)，那么，可以将机器译文送入到其他处理模块中进行处理。
在本申请中，机器译文的译文质量可以有多种表现形式，下面对机器译文的译文质量的几种可能的表现形式进行介绍。
可选地，机器译文的译文质量可以是下列信息中的任意一种。
(1)机器译文的可接受性信息；
(2)机器译文与标准参考译文之间的差异值。
其中，机器译文的可接受性信息既可以用于指示所述机器译文是否可以接受，也可以指示机器译文被接受的概率或者不被接受的概率。
当机器译文的可接受性信息用于指示所述机器译文是否可以接受时，机器译文的可接受性信息可以包含一个标识位，该标识位的取值用于表示机器译文是否可以接受，例如， 该标识位的取值为1时表示该机器译文可以被接受(能够用于下游系统的处理)，该标识位的取值为0时表示该机器译文不可以被接受(或者称为不可接受，这种情况下，机器译文不能够用于下游系统的处理)。
当机器译文的可接受性信息用于指示机器译文被接受的概率或者不被接受的概率时，机器译文的可接受性信息具体可以是机器译文被接受的概率的概率值或者机器译文不被接受的概率的概率值。
当上述机器译文的可接受性信息用于指示机器译文被接受的概率时，如果机器译文的被接受的概率大于某个预设数值，那么，该机器译文可以送入到下游系统进行后续处理，否则，该机器译文不能够送入到下游系统进行后续处理。
例如，机器译文被接受的概率为85％，85％大于80％(该预设数值也可以是其它数值，这里仅仅是为了举例说明)，该机器译文可以送入到下游系统进行后续处理。
本申请中的译文质量检测模型有多种可能的实现方式，一般而言，凡是能从两种语言的输入句子中提取特征，并执行二分类的模型均可使用。
可选地，上述译文质量检测模型既可以是神经网络模型，也可以是基于支持向量机的模型。
当上述译文质量检测模型为神经网络模型时，由于神经网络模型处理数据的能力较强，因而能够在对原文和机器译文进行处理时的处理效果更好。
下面结合具体的实例对译文质量检测模型对原文和机器译文进行处理，得到机器译文的译文质量的过程进行描述。
图13示出了译文质量检测模型对原文和机器译文的处理过程。
在图13中，在获取到原文和机器译文之后，可以分别将原文和机器译文中的每个词都映射为词向量(word embedding)，然后分别通过双向循环神经网络(recurrent neural network,RNN)对原文的词向量和机器译文的词向量进行处理，分别得到原文的中间向量和机器译文的中间向量，接下来，再采用注意力机制(attention mechanism)对原文的中间向量和机器译文的中间向量进行处理，得到一个目标向量，最后再采用前馈神经网络(feed-forward neural network)对该目标向量进行处理，从而得到机器译文相对于原文是否可接受的估计。其中，RNN中的网络单元可以为门限循环单元(gated recurrent unit,GRU)。
上述计算过程可以简化为将原文和机器译文代入到函数f(x,z；φ)中，从而得到机器译文的译文质量，其中，x表示原文，z表示机器译文，φ为译文质量检测模型中的参数，包括词向量参数、循环神经网络参数、注意力机制参数、前馈神经网络参数。在获取译文质量检测模型时，可以将可接受性标注语料
Figure PCTCN2020079964-appb-000011
作为训练数据集合，采用反向传播(backpropagation)算法进行训练，从而得到译文质量检测模型。其中，m表示可接受性标注语料的条目数，x i为第i个原文，z i为第i个机器译文(z i是对x i进行机器翻译得到的机器译文)，y i为z i的可接受性标注(此时，训练数据集合中的机器译文的译文质量采用可接受性标注来表示)。
下面结合图14来更详细地描述译文质量检测模型对原文和机器译文的处理过程。
如图14所示，采用译文质量检测模型对原文和机器译文进行处理具体包括以下步骤：
2001、分别将原文s和机器译文t中的每个词映射为词向量(word embedding)。
在步骤2001中可以采用公式(1)来获得原文s中的每个词的词向量，采用公式(2)来获得机器译文t中的每个词的词向量。
Figure PCTCN2020079964-appb-000012
Figure PCTCN2020079964-appb-000013
其中，
Figure PCTCN2020079964-appb-000014
表示原文s中的第i个词向量，
Figure PCTCN2020079964-appb-000015
表示源语言词向量参数矩阵，s i表示原文s中第i个词以one-hot形式表示的向量(这种向量只有一个特征不为0的，其他都是0)，
Figure PCTCN2020079964-appb-000016
表示机器译文t中的第i个词向量，
Figure PCTCN2020079964-appb-000017
表示目标语言词向量参数矩阵，t i表示机器译文t中第i个词以one-hot形式表示的向量。
2002、采用RNN分别对原文s中的每个词的词向量和机器译文t中的每个词的词向量进行处理，得到原文s的第一中间向量和机器译文t的第二中间向量。
在步骤2002中可以采用公式(3)对原文s中的每个词的词向量进行处理，采用公式公式(4)对机器译文t中的每个词的词向量进行处理。
Figure PCTCN2020079964-appb-000018
Figure PCTCN2020079964-appb-000019
其中，在上述公式(3)和(4)中，
Figure PCTCN2020079964-appb-000020
表示原文s的第一中间向量，BiGRU S表示源语言双向GRU网络对原文词向量执行的运算，
Figure PCTCN2020079964-appb-000021
表示机器译文t的第二中间向量，BiGRU T表示目标语言双向GRU网络对机器译文词向量执行的运算。
2003、利用非线性层对步骤2002处理得到的原文s的第一中间向量进行非线性处理，得到第三中间向量。
在步骤2003中，可以采用公式(5)对第一中间向量进行非线性处理，得到第三中间向量。
Figure PCTCN2020079964-appb-000022
其中，h i S表示第三中间向量，ReLU是一种非线性激活函数，W g S表示一个参数矩阵，b g S表示一个参数向量。
2004、利用非线性层对步骤2002处理得到的机器译文t的第二中间向量进行非线性处理，得到第四中间向量。
Figure PCTCN2020079964-appb-000023
其中，h i T表示第四中间向量，ReLU是一种非线性激活函数，W g T表示一个参数矩阵，b g T表示一个参数向量。
2005、采用注意力机制对上述步骤2003和步骤2004处理得到的两组句子向量进行处理，得到一个目标向量。
具体地，可以将步骤2003和步骤2004得到的两组句子向量{h i S} i＝1 L和{h i T} i＝1 L组合起来表示为{h i} i＝1 2L，然后利用公式(7)和公式(8)得到目标向量。
Figure PCTCN2020079964-appb-000024
Figure PCTCN2020079964-appb-000025
其中，w表示一个参数向量。
2006、通过非线性层和线性层对目标向量进行处理，得到机器译文相对于原文是否可接受的估计。
在步骤2006中，可以根据公式(9)和(10)来计算机器译文相对于原文是否可接受的估计。
v＝ReLU(W Uu+b u)                 (9)
p＝sigmoid(W vv+b v)               (10)
其中，W u,W v表示两个参数矩阵，b u,b v表示两个参数向量。
上文结合附图对本申请实施例的译文质量检测方法进行了详细的描述，下面结合图15对本申请实施例的译文质量检测装置进行描述，应理解，图15中描述的译文质量检测装置2000能够执行本申请实施例的译文检测方法的各个步骤，下面在介绍本申请实施例的译文检测装置时适当省略重复的描述。
图15所示的译文检测装置3000包括：
存储器3001，用于存储程序；
处理器3002，用于执行所述存储器存储的程序，当所述存储器存储的程序被所述处理器3002执行时，所述处理器3002用于：
获取原文和机器译文，所述机器译文是所述原文经过机器翻译系统翻译得到的译文；
根据所述原文、所述机器译文以及所述机器译文的应用场景，确定所述机器译文的译文质量。
应理解，上述译文检测装置3000除了包含存储器3001和处理器3002之外，还可以包括输入输出接口3003，通过该输入输出接口3003能够从其他设备(例如，机器翻译系统)获取原文和机器译文，在获取到原文和机器译文之后，可以采用处理器3002对原文和机器译文进行处理，得到机器译文的译文质量。进一步的，在得到机器译文的译文质量之后，译文装置3000还可以通过输入输出接口3003将机器译文的译文质量传输给其他设备(例如，下游系统)。
上述译文检测装置3000相当于图9和图12中所示的译文质量检测系统，能够用于检测机器翻译系统翻译得到的机器译文的译文质量。
上述译文检测装置3000可以相当于是图1所示的数据处理设备或者图2所示的用户设备。译文检测装置3000可以相当于图3所示的执行设备210、图4所示的执行设备110。
图16是本申请实施例提供的机器翻译系统的示意性框图。
图16所示的机器翻译系统4000包括：
存储器4001，用于存储程序；
处理器4002，用于执行所述存储器存储的程序，当所述存储器存储的程序被所述处理器4002执行时，所述处理器4002用于：
获取原文；
对所述原文进行翻译，得到所述原文的机器译文；
根据所述原文、所述机器译文以及所述机器译文的应用场景，确定所述机器译文的译文质量。
上述机器翻译系统4000还可以包含输入输出接口4003，机器翻译系统4000通过输入输出接口4003能够获取原文，具体地，通过输入输出接口4003可以从其他设备(例如， 终端设备)获取原文，在获取到原文之后，通过处理器4002的处理能够最终得到机器译文的译文质量。机器翻译系统4000通过输入输出接口4003能够将机器译文的质量传输给其他设备(例如，下游系统)。
在这里，机器翻译系统4000不仅可以实现对原文的翻译，还能够对自身翻译得到的机器译文进行检测，从而得到机器译文的译文质量。这里的机器翻译系统4000相当于图9、图12中所示的机器翻译系统和译文质量检测系统的组合。
机器翻译系统4000可以相当于是图1所示的数据处理设备或者图2所示的用户设备。机器翻译系统4000可以相当于图3所示的执行设备210、图4所示的执行设备110。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read-only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (16)

  1. 一种译文质量检测方法，其特征在于，包括：
    获取原文和机器译文，所述机器译文是所述原文经过机器翻译系统翻译得到的译文；
    根据所述原文、所述机器译文以及所述机器译文的应用场景，确定所述机器译文的译文质量。
  2. 如权利要求1所述的方法，其特征在于，所述根据所述原文、所述机器译文以及所述机器译文的应用场景，确定所述机器译文的译文质量，包括：
    采用下游系统对应的译文质量检测模型对所述原文和所述机器译文进行处理，得到所述机器译文的译文质量；
    其中，所述下游系统用于对所述机器译文进行后续处理，所述下游系统对应的译文质量检测模型是根据训练样本和训练目标训练得到的，所述训练样本包括训练原文和训练机器译文，所述训练目标包括所述训练机器译文的译文质量；
    所述训练机器译文是所述训练原文经过所述机器翻译系统翻译得到的译文，所述训练机器译文的译文质量是根据第一处理结果与第二处理结果之间的差异确定的，其中，所述第一处理结果为所述下游系统对所述训练机器译文进行处理的处理结果，所述第二处理结果为所述下游系统对所述训练原文的参考译文进行处理的处理结果。
  3. 如权利要求2所述的方法，其特征在于，所述训练原文和所述训练原文的参考译文来自于已知的双语平行语料。
  4. 根据权利要求1-3中任一项所述的方法，所述机器译文的译文质量为所述机器译文的可接受性信息。
  5. 如权利要求4所述的方法，其特征在于，所述机器译文的可接受性信息用于指示所述机器译文是否可以接受。
  6. 如权利要求4所述的方法，其特征在于，所述机器译文的可接受性信息用于指示所述机器译文被接受的概率或者不被接受的概率。
  7. 如权利要求1-6中任一项所述的方法，其特征在于，所述译文质量检测模型为神经网络模型。
  8. 一种译文质量检测装置，其特征在于，包括：
    存储器，用于存储程序；
    处理器，用于执行所述存储器存储的程序，当所述存储器存储的程序被所述处理器执行时，所述处理器用于：
    获取原文和机器译文，所述机器译文是所述原文经过机器翻译系统翻译得到的译文；
    根据所述原文、所述机器译文以及所述机器译文的应用场景，确定所述机器译文的译文质量。
  9. 如权利要求8所述的装置，其特征在于，所述处理器用于：
    采用下游系统对应的译文质量检测模型对所述原文和所述机器译文进行处理，得到所述机器译文的译文质量；
    其中，所述下游系统用于对所述机器译文进行后续处理，所述下游系统对应的译文质 量检测模型是根据训练样本和训练目标训练得到的，所述训练样本包括训练原文和训练机器译文，所述训练目标包括所述训练机器译文的译文质量；
    所述训练机器译文是所述训练原文经过所述机器翻译系统翻译得到的译文，所述训练机器译文的译文质量是根据第一处理结果与第二处理结果之间的差异确定的，其中，所述第一处理结果为所述下游系统对所述训练机器译文进行处理的处理结果，所述第二处理结果为所述下游系统对所述训练原文的参考译文进行处理的处理结果。
  10. 如权利要求9所述的装置，其特征在于，所述训练原文和所述训练原文的参考译文来自于已知的双语平行语料。
  11. 根据权利要求8-10中任一项所述的装置，所述机器译文的译文质量为所述机器译文的可接受性信息。
  12. 如权利要求11所述的装置，其特征在于，所述机器译文的可接受性信息用于指示所述机器译文是否可以接受。
  13. 如权利要求11所述的装置，其特征在于，所述机器译文的可接受性信息用于指示所述机器译文被接受的概率或者不被接受的概率。
  14. 如权利要求8-13中任一项所述的装置，其特征在于，所述译文质量检测模型为神经网络模型。
  15. 一种机器翻译系统，其特征在于，包括：
    存储器，用于存储程序；
    处理器，用于执行所述存储器存储的程序，当所述存储器存储的程序被所述处理器执行时，所述处理器用于：
    获取原文；
    对所述原文进行翻译，得到所述原文的机器译文；
    根据所述原文、所述机器译文以及所述机器译文的应用场景，确定所述机器译文的译文质量。
  16. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有程序代码，所述程序代码包括用于执行如权利要求1-7中任一项所述的方法的部分或全部步骤的指令。
PCT/CN2020/079964 2019-03-27 2020-03-18 译文质量检测方法、装置、机器翻译系统和存储介质 WO2020192523A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/479,420 US20220004721A1 (en) 2019-03-27 2021-09-20 Translation quality detection method and apparatus, machine translation system, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910237160.0A CN110083842B (zh) 2019-03-27 2019-03-27 译文质量检测方法、装置、机器翻译系统和存储介质
CN201910237160.0 2019-03-27

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/479,420 Continuation US20220004721A1 (en) 2019-03-27 2021-09-20 Translation quality detection method and apparatus, machine translation system, and storage medium

Publications (1)

Publication Number Publication Date
WO2020192523A1 true WO2020192523A1 (zh) 2020-10-01

Family

ID=67413536

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/079964 WO2020192523A1 (zh) 2019-03-27 2020-03-18 译文质量检测方法、装置、机器翻译系统和存储介质

Country Status (3)

Country Link
US (1) US20220004721A1 (zh)
CN (1) CN110083842B (zh)
WO (1) WO2020192523A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110083842B (zh) * 2019-03-27 2023-10-03 华为技术有限公司 译文质量检测方法、装置、机器翻译系统和存储介质
CN112163434B (zh) * 2020-10-20 2024-03-08 腾讯科技（深圳）有限公司 基于人工智能的文本翻译方法、装置、介质及电子设备
CN117744837A (zh) * 2023-12-29 2024-03-22 支付宝(杭州)信息技术有限公司 一种模型训练、文本检测方法、装置、存储介质及设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120209587A1 (en) * 2011-02-16 2012-08-16 Kabushiki Kaisha Toshiba Machine translation apparatus, machine translation method and computer program product for machine tranalation
CN103714054A (zh) * 2013-12-30 2014-04-09 北京百度网讯科技有限公司 翻译方法和翻译装置
CN104731777A (zh) * 2015-03-31 2015-06-24 网易有道信息技术（北京）有限公司 一种译文评价方法及装置
CN106649282A (zh) * 2015-10-30 2017-05-10 阿里巴巴集团控股有限公司 基于统计的机器翻译方法、装置及电子设备
CN109062912A (zh) * 2018-08-08 2018-12-21 科大讯飞股份有限公司 一种翻译质量评价方法及装置
CN110083842A (zh) * 2019-03-27 2019-08-02 华为技术有限公司 译文质量检测方法、装置、机器翻译系统和存储介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9606988B2 (en) * 2014-11-04 2017-03-28 Xerox Corporation Predicting the quality of automatic translation of an entire document
JP6910012B2 (ja) * 2017-01-11 2021-07-28 パナソニックＩｐマネジメント株式会社 文章評価装置、及び文章評価方法
US10552547B2 (en) * 2017-10-10 2020-02-04 International Business Machines Corporation Real-time translation evaluation services for integrated development environments
KR102516363B1 (ko) * 2018-01-26 2023-03-31 삼성전자주식회사 기계 번역 방법 및 장치
CN108829684A (zh) * 2018-05-07 2018-11-16 内蒙古工业大学 一种基于迁移学习策略的蒙汉神经机器翻译方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120209587A1 (en) * 2011-02-16 2012-08-16 Kabushiki Kaisha Toshiba Machine translation apparatus, machine translation method and computer program product for machine tranalation
CN103714054A (zh) * 2013-12-30 2014-04-09 北京百度网讯科技有限公司 翻译方法和翻译装置
CN104731777A (zh) * 2015-03-31 2015-06-24 网易有道信息技术（北京）有限公司 一种译文评价方法及装置
CN106649282A (zh) * 2015-10-30 2017-05-10 阿里巴巴集团控股有限公司 基于统计的机器翻译方法、装置及电子设备
CN109062912A (zh) * 2018-08-08 2018-12-21 科大讯飞股份有限公司 一种翻译质量评价方法及装置
CN110083842A (zh) * 2019-03-27 2019-08-02 华为技术有限公司 译文质量检测方法、装置、机器翻译系统和存储介质

Also Published As

Publication number Publication date
CN110083842A (zh) 2019-08-02
CN110083842B (zh) 2023-10-03
US20220004721A1 (en) 2022-01-06

Similar Documents

Publication Publication Date Title
WO2021047286A1 (zh) 文本处理模型的训练方法、文本处理方法及装置
WO2022007823A1 (zh) 一种文本数据处理方法及装置
WO2019228317A1 (zh) 人脸识别方法、装置及计算机可读介质
CN111368993B (zh) 一种数据处理方法及相关设备
WO2022068627A1 (zh) 一种数据处理方法及相关设备
WO2022057776A1 (zh) 一种模型压缩方法及装置
WO2021184902A1 (zh) 图像分类方法、装置、及其训练方法、装置、设备、介质
WO2020192523A1 (zh) 译文质量检测方法、装置、机器翻译系统和存储介质
EP4163831A1 (en) Neural network distillation method and device
WO2023160472A1 (zh) 一种模型训练方法及相关设备
WO2022001724A1 (zh) 一种数据处理方法及装置
WO2022253074A1 (zh) 一种数据处理方法及相关设备
WO2021057884A1 (zh) 语句复述方法、训练语句复述模型的方法及其装置
WO2021051987A1 (zh) 神经网络模型训练的方法和装置
EP4198807A1 (en) Audio processing method and device
EP4361843A1 (en) Neural network searching method and related device
WO2023165361A1 (zh) 一种数据处理方法及相关设备
CN116432019A (zh) 一种数据处理方法及相关设备
WO2021129411A1 (zh) 文本处理方法及装置
CN116821307B (zh) 内容交互方法、装置、电子设备和存储介质
WO2023279921A1 (zh) 神经网络模型的训练方法、数据处理的方法及装置
WO2021083312A1 (zh) 训练语句复述模型的方法、语句复述方法及其装置
WO2021129410A1 (zh) 文本处理方法及装置
WO2024082891A1 (zh) 一种数据处理方法及相关设备
WO2023143262A1 (zh) 一种数据处理方法及相关设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20779198

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20779198

Country of ref document: EP

Kind code of ref document: A1