CN113392659A - Machine translation method, device, electronic equipment and storage medium - Google Patents
Machine translation method, device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113392659A CN113392659A CN202110714213.0A CN202110714213A CN113392659A CN 113392659 A CN113392659 A CN 113392659A CN 202110714213 A CN202110714213 A CN 202110714213A CN 113392659 A CN113392659 A CN 113392659A
- Authority
- CN
- China
- Prior art keywords
- translation
- word sequence
- text
- target entity
- entity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013519 translation Methods 0.000 title claims abstract description 153
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000013136 deep learning model Methods 0.000 claims abstract description 37
- 238000007781 pre-processing Methods 0.000 claims abstract description 9
- 230000015654 memory Effects 0.000 claims description 19
- 238000012549 training Methods 0.000 claims description 14
- 230000001537 neural effect Effects 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000002372 labelling Methods 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 3
- 230000006403 short-term memory Effects 0.000 claims description 3
- 230000002457 bidirectional effect Effects 0.000 claims description 2
- 230000007787 long-term memory Effects 0.000 claims 1
- 230000000694 effects Effects 0.000 abstract description 12
- 238000012545 processing Methods 0.000 abstract description 9
- 238000010586 diagram Methods 0.000 description 9
- 238000011156 evaluation Methods 0.000 description 6
- 238000012360 testing method Methods 0.000 description 5
- 230000014509 gene expression Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000003643 water by type Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/40—Processing or translation of natural language
- G06F40/58—Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
- G06F40/295—Named entity recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Machine Translation (AREA)
Abstract
The invention relates to the technical field of language processing, and provides a machine translation method, a machine translation device, electronic equipment and a storage medium. The machine translation method comprises the following steps: preprocessing a text to be translated to obtain a word sequence of the text to be translated; identifying whether a target entity is contained in the word sequence through a deep learning model; if so, replacing a target entity in the word sequence with a placeholder, translating the replaced word sequence and the target entity respectively, and replacing the placeholder in the translation result of the replaced word sequence with the translation result of the target entity to obtain the translation result of the text to be translated; if not, performing text translation on the word sequence to obtain a translation result of the text to be translated. According to the method, the target entity in the text to be translated is accurately identified through the deep learning model, the text and the entity are translated respectively, the entity translation accuracy is improved, and the text translation effect is further improved.
Description
Technical Field
The invention relates to the technical field of language processing, in particular to a machine translation method, a device, electronic equipment and a storage medium.
Background
With the deepening of internationalization, communication among countries is deepened more and more, and the requirement for language translation is also larger and larger. With the rise of deep learning, the development of automatic machine translation has made a breakthrough. Among the different machine translation models, neural machine translation models with attention mechanism are becoming popular because they are able to focus on the most relevant parts of the source text when translating.
However, the neural machine translation model is limited by training data, and the translation effects under different scenes are greatly different. For example, when a neural machine translation model in the general field is used for translating a text in the travel field, a significant error occurs in the translation of the entity part, and the guarantee of the accuracy of the entity translation is a very important part in the adaptive process of the vertical field.
In the current text translation in the tourism field, an entity part is captured in a dictionary and regular expression mode, the text and the entity part are translated respectively, and finally translation results are spliced. Due to the diversity of entity expressions, dictionaries can not be completely covered usually, and regular dictionary easily causes problems such as false recalls and the like.
Therefore, it is necessary to develop a new machine translation method to replace the dictionary and the regular manner to realize the accurate recognition of the entity in the text.
It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the invention and therefore may include information that does not constitute prior art that is already known to a person of ordinary skill in the art.
Disclosure of Invention
In view of this, the invention provides a machine translation method, a machine translation device, an electronic device and a storage medium, which can accurately identify a target entity in a text to be translated through a deep learning model, respectively translate the text and the entity, improve the accuracy of entity translation, and further improve the text translation effect.
One aspect of the present invention provides a machine translation method, including: preprocessing a text to be translated to obtain a word sequence of the text to be translated; identifying whether a target entity is contained in the word sequence through a deep learning model; if so, replacing a target entity in the word sequence with a placeholder, translating the replaced word sequence and the target entity respectively, and replacing the placeholder in the translation result of the replaced word sequence with the translation result of the target entity to obtain the translation result of the text to be translated; if not, performing text translation on the word sequence to obtain a translation result of the text to be translated.
In some embodiments, the deep learning model includes an embedding layer, an encoder, and a decoder arranged in sequence.
In some embodiments, the identifying whether the word sequence includes a target entity by a deep learning model includes: performing vector conversion on the word sequence through the embedding layer to obtain vector representation of each word; performing context feature extraction on the word sequence represented by the vector through the encoder to obtain feature representation of each word; marking and predicting the character sequence represented by the characteristics through the decoder to obtain an entity marking sequence of the character sequence; and judging whether the word sequence contains a target entity labeled by the corresponding target entity according to the entity labeling sequence.
In some embodiments, the encoder is built based on a bidirectional long-short term memory network; the decoder is constructed based on conditional random fields.
In some embodiments, the deep learning model is generated based on training data training of a travel domain; the target entity is a travel entity, and the travel entity comprises: store name, company name, person name, place name, and time.
In some embodiments, the translating the replaced word sequence and the target entity separately includes: performing text translation on the replaced word sequence to obtain a translation result of the replaced word sequence; and performing entity translation on the target entity to obtain a translation result of the target entity.
In some embodiments, the replaced word sequence is text translated by a neural machine translation model; and performing entity translation on the target entity according to a preset translation rule.
Another aspect of the present invention provides a machine translation apparatus, comprising: the system comprises a preprocessing module, a translation module and a translation module, wherein the preprocessing module is configured to preprocess a text to be translated to obtain a word sequence of the text to be translated; the entity identification module is configured to identify whether the word sequence contains a target entity through a deep learning model; the combined translation module is configured to replace a target entity in the word sequence with a placeholder when the word sequence comprises the target entity, respectively translate the replaced word sequence and the target entity, and replace the placeholder in the translation result of the replaced word sequence with the translation result of the target entity to obtain the translation result of the text to be translated; and the text translation module is configured to perform text translation on the word sequence to obtain a translation result of the text to be translated when the word sequence does not contain the target entity.
Yet another aspect of the present invention provides an electronic device including: a processor; a memory having executable instructions stored therein; wherein the executable instructions, when executed by the processor, implement the machine translation method of any of the above embodiments.
Yet another aspect of the present invention provides a computer-readable storage medium storing a program which, when executed by a processor, implements the machine translation method of any of the embodiments described above.
Compared with the prior art, the invention has the beneficial effects that:
according to the method, entity recognition is carried out through the deep learning model, a dictionary and a regular mode are replaced, and a target entity in the text to be translated can be accurately recognized; the placeholder is used for replacing the target entity, the text and the target entity are respectively translated, and the translated target entity is replaced into the translated text, so that the entity translation accuracy can be improved, the whole translation fluency can be improved, and the text translation effect can be further improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention. It is obvious that the drawings described below are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
FIG. 1 is a diagram illustrating the steps of a machine translation method in one embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating steps for identifying a target entity through a deep learning model according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a network structure of a deep learning model according to an embodiment of the present invention;
FIG. 4 is a block diagram of a machine translation device according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of an electronic device according to an embodiment of the invention;
fig. 6 is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.
The drawings are merely schematic illustrations of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
In addition, the flow shown in the drawings is only an exemplary illustration, and not necessarily includes all the steps. For example, some steps may be divided, some steps may be combined or partially combined, and the actual execution sequence may be changed according to the actual situation. It should be noted that features of the embodiments of the invention and of the different embodiments may be combined with each other without conflict.
Fig. 1 shows the main steps of a machine translation method in an embodiment, and referring to fig. 1, the machine translation method includes: step S110, preprocessing a text to be translated to obtain a word sequence of the text to be translated; step S120, identifying whether the word sequence contains a target entity through a deep learning model; if yes, execute step S130, including: replacing a target entity in the word sequence with a placeholder, translating the replaced word sequence and the target entity respectively, replacing the placeholder in the translation result of the replaced word sequence with the translation result of the target entity, and obtaining the translation result of the text to be translated; if not, executing step S140, performing text translation on the word sequence, and obtaining a translation result of the text to be translated.
Entities in texts are very important and often contain a lot of key information, and in the process of using machine translation services, the translation accuracy of the entities receives more and more attention, which is also a difficulty of self-adaptation in the field of machine translation. According to the machine translation method, entity recognition is carried out through the deep learning model, a dictionary and a regular mode are replaced, and a target entity in a text to be translated can be accurately recognized; the placeholder is used for replacing the target entity, the text and the target entity are respectively translated, and the translated target entity is replaced into the translated text, so that the entity translation accuracy can be improved, the whole translation fluency can be improved, and the text translation effect can be further improved.
In one embodiment, the deep learning model includes an embedded layer, an encoder, and a decoder arranged in sequence. Fig. 2 shows the main steps of identifying a target entity through a deep learning model, and referring to fig. 2, identifying whether a word sequence contains the target entity through the deep learning model includes: step S210, carrying out vector conversion on the word sequence through the embedding layer to obtain the vector representation of each word; step S220, carrying out context feature extraction on the word sequence represented by the vector through an encoder to obtain feature representation of each word; step S230, marking and predicting the character sequence represented by the characteristics through a decoder to obtain an entity marking sequence of the character sequence; step S240, determining whether the word sequence includes a target entity labeled with a corresponding target entity according to the entity labeling sequence.
Fig. 3 shows a network structure of a deep learning model, and in combination with fig. 3, in this embodiment, an Embedding layer 310 of the deep learning model adopts a trained Embedding model, and can perform vector conversion on a text and output a vector representation of each word; the encoder 320 adopts a Bi-directional Long Short-Term Memory network (Bi-directional Long Short-Term Memory, Bi-LSTM for Short) to learn the context characteristics, enhance the representation of the context, and output the vector representation of the context environment of each word; the decoder 330 adopts a Conditional Random Field (CRF for short), has strong sequence control capability, and can solve the labeled sequence by constructing a CRF loss and combining the obtained CRF features with a Viterbi algorithm. Through the deep learning model of Bi-LSTM + CRF, the excellent entity recognition effect can be achieved.
As shown in FIG. 3, in one particular example, the text to be translated is pre-processed to obtain a word sequence of "Mark" Watney "viewed" Mars' ". The sequence of words is input into the embedding layer 310, resulting in a vector representation of each word. The vector-represented word sequences are input to the forward LSTM layer (l) of the encoder 320, respectively1→l2→l3→l4) And a backward LSTM layer (r)4→r3→r2→r1) Obtaining a vector representation of contextual features for each word and passing through an output layer (c)1～c4) And output to the decoder 330. The decoder 330 performs label prediction on the character sequence represented by the features and outputs a physical label sequence "B-PER" "E-PER" "O" "S-LOC'". Wherein, the 'B-PER' represents the prefix of the name entity of the person, the 'E-PER' represents the suffix of the name entity of the person, the 'O' represents a non-entity, and the 'S-LOC' represents a single component word of the name entity of the place. Therefore, according to the entity labeling sequence, the target entity labels corresponding to the 'Mark', 'Watney' and 'Mars' in the word sequence 'Mark', 'Watney' and 'Mars' can be accurately identified as the target entities.
Further, after the target entity is identified, translating the replaced word sequence and the target entity respectively, including: performing text translation on the replaced word sequence, specifically performing text translation through a neural machine translation model, and obtaining a translation result of the replaced word sequence; and performing entity translation on the target entity, specifically performing entity translation according to a preset translation rule, and obtaining a translation result of the target entity. A Neural Machine Translation model (NMT) is an existing algorithm model, a preset Translation rule is usually to forcibly translate a target entity, and different target entities can set different Translation rules, for example, the name "Mark Watney" is translated into "Mark waters", and the name "Mars" is translated into "Mars".
In one embodiment, the deep learning model is generated based on training data training in the travel domain; the target entity is a tourism entity, and the tourism entity comprises: store name, company name, person name, place name, and time. Specifically, during training, a batch of high-quality corpus data in the tourism field is screened out through data cleaning, then target entities in each corpus data (one corpus data corresponds to one sentence or one text) are marked through marking, and then the marked corpus data is used for training a deep learning model.
Table 1 shows the data amounts of a training data set and a test data set during deep learning model training according to the present invention, where the training data set includes 30721 pieces of chinese corpus data and 27442 pieces of english corpus data, and the test data set includes 8000 pieces of chinese corpus data and 6000 pieces of english corpus data.
Table 1:
| training data set | Test data set | |
| Chinese character | 30721 | 8000 |
| English | 27442 | 6000 |
Table 2 shows the evaluation results of the traditional CRF model for the chinese target entity identification in the travel field, specifically shows that the accuracy, recall rate, and F1 value of the evaluation results for the address target entity, the hotel target entity, and the company target entity are all to be improved.
Table 2:
| entity type | Rate of accuracy | Recall rate | F1 value |
| Address | 78.38 | 66.02 | 71.67 |
| Hotel | 74.13 | 67.19 | 70.49 |
| Company(s) | 85.01 | 80.74 | 82.82 |
Table 3 shows the evaluation results of the deep learning model based on Bi-LSTM + CRF of the invention on the recognition of Chinese target entities in the travel field, and particularly shows that the evaluation results of address target entities, hotel target entities, company target entities and time target entities all have high accuracy, recall rate and F1 value.
Table 3:
| entity type | Rate of accuracy | Recall rate | F1 value |
| Address | 77.08 | 79.51 | 78.28 |
| Hotel | 73.67 | 77.79 | 75.67 |
| Company(s) | 85.11 | 89.31 | 87.16 |
| Time | 86.92 | 90.95 | 88.89 |
Table 4 shows the evaluation results of the Bi-LSTM + CRF-based deep learning model of the present invention for identifying english target entities in the travel field, specifically showing that the evaluation results for address target entities, hotel target entities, company target entities and time target entities all have high accuracy, recall rate and F1 value.
Table 4:
| entity type | Rate of accuracy | Recall rate | F1 value |
| Address | 83.05 | 84.00 | 83.52 |
| Hotel | 74.33 | 74.33 | 74.33 |
| Company(s) | 83.91 | 85.63 | 84.76 |
| Time | 86.15 | 90.88 | 88.45 |
Table 5 shows the service throughput during deep learning model training of the present invention, where the test environment is a CPU dual-core Docker container environment, and the memory usage is about 335M.
Table 5:
the tests and the online effect show that the method adopts the deep learning model to identify the target entity, and can effectively improve the translation effect of the tourism entity in machine translation. Compared with a mode of adopting a dictionary and a regular expression, the coverage rate and the recall rate of entity recognition can be effectively improved. The F1 value can be effectively improved relative to the traditional CRF model. The recognition speed block of the deep learning model can meet the real-time requirement of the online translation service.
In conclusion, the method carries out entity recognition through the deep learning model, replaces a dictionary and a regular mode, and can accurately recognize the target entity in the text to be translated; the placeholder is used for replacing the target entity, the text and the target entity are respectively translated, and the translated target entity is replaced into the translated text, so that the entity translation accuracy can be improved, the whole translation fluency can be improved, and the text translation effect can be further improved.
The embodiment of the invention also provides a machine translation device which can be used for realizing the machine translation method described in any embodiment. The features and principles of the machine translation method described in any of the above embodiments may be applied to the machine translation apparatus embodiments below. In the following embodiments of the machine translation apparatus, the features and principles that have been elucidated with respect to machine translation and entity recognition will not be repeated.
Fig. 4 shows the main blocks of the machine translation apparatus in an embodiment, and referring to fig. 4, the machine translation apparatus 400 includes: the preprocessing module 410 is configured to preprocess the text to be translated to obtain a word sequence of the text to be translated; an entity identification module 420 configured to identify whether the word sequence contains a target entity through a deep learning model; the combined translation module 430 is configured to, when the word sequence includes a target entity, replace the target entity in the word sequence with a placeholder, translate the replaced word sequence and the target entity respectively, and replace the placeholder in the translation result of the replaced word sequence with the translation result of the target entity to obtain a translation result of the text to be translated; and the text translation module 440 is configured to perform text translation on the word sequence to obtain a translation result of the text to be translated when the word sequence does not include the target entity.
Further, the machine translation apparatus 400 may further include modules for implementing other process steps of the above embodiments of the machine translation method, and specific principles of the modules may refer to the description of the above embodiments of the machine translation method, and will not be repeated here.
As described above, the machine translation device of the present invention can perform entity recognition through the deep learning model, replace the dictionary and the regular mode, and accurately recognize the target entity in the text to be translated; the placeholder is used for replacing the target entity, the text and the target entity are respectively translated, and the translated target entity is replaced into the translated text, so that the entity translation accuracy can be improved, the whole translation fluency can be improved, and the text translation effect can be further improved.
The embodiment of the present invention further provides an electronic device, which includes a processor and a memory, where the memory stores executable instructions, and when the executable instructions are executed by the processor, the machine translation method described in any of the above embodiments is implemented.
As described above, the electronic device of the present invention can perform entity recognition through the deep learning model, replace the dictionary and the regular mode, and accurately recognize the target entity in the text to be translated; the placeholder is used for replacing the target entity, the text and the target entity are respectively translated, and the translated target entity is replaced into the translated text, so that the entity translation accuracy can be improved, the whole translation fluency can be improved, and the text translation effect can be further improved.
Fig. 5 is a schematic structural diagram of an electronic device in an embodiment of the present invention, and it should be understood that fig. 5 only schematically illustrates various modules, and these modules may be virtual software modules or actual hardware modules, and the combination, the splitting, and the addition of the remaining modules of these modules are within the scope of the present invention.
As shown in fig. 5, the electronic device 600 is embodied in the form of a general purpose computing device. The components of the electronic device 600 include, but are not limited to: at least one processing unit 610, at least one memory unit 620, a bus 630 connecting the different platform components (including the memory unit 620 and the processing unit 610), a display unit 640, etc.
Wherein the storage unit stores program code that can be executed by the processing unit 610, such that the processing unit 610 performs the steps of the machine translation method described in any of the embodiments above. For example, processing unit 610 may perform the steps shown in fig. 1 and 2.
The storage unit 620 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM)6201 and/or a cache memory unit 6202, and may further include a read-only memory unit (ROM) 6203.
The memory unit 620 may also include programs/utilities 6204 including one or more program modules 6205, such program modules 6205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 600 may also communicate with one or more external devices 700, and the external devices 700 may be one or more of a keyboard, a pointing device, a bluetooth device, and the like. The external devices 700 enable a user to interactively communicate with the electronic device 600. The electronic device 600 may also be capable of communicating with one or more other computing devices, including routers, modems. Such communication may occur via an input/output (I/O) interface 650. Also, the electronic device 600 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via the network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via the bus 630. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, to name a few.
The embodiment of the present invention further provides a computer-readable storage medium for storing a program, and the program implements the machine translation method described in any of the above embodiments when executed. In some possible embodiments, the various aspects of the invention may also be implemented in the form of a program product comprising program code for causing a terminal device to perform the machine translation method described in any of the embodiments above, when the program product is run on the terminal device.
As described above, the computer-readable storage medium of the present invention can perform entity recognition through the deep learning model, replace a dictionary and a regular manner, and accurately recognize a target entity in a text to be translated; the placeholder is used for replacing the target entity, the text and the target entity are respectively translated, and the translated target entity is replaced into the translated text, so that the entity translation accuracy can be improved, the whole translation fluency can be improved, and the text translation effect can be further improved.
Fig. 6 is a schematic structural diagram of a computer-readable storage medium of the present invention. Referring to fig. 6, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited in this regard and, in the present document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of readable storage media include, but are not limited to: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device, such as through the internet using an internet service provider.
The foregoing is a more detailed description of the invention in connection with specific preferred embodiments and it is not intended that the invention be limited to these specific details. For those skilled in the art to which the invention pertains, several simple deductions or substitutions can be made without departing from the spirit of the invention, and all shall be considered as belonging to the protection scope of the invention.
Claims (10)
1. A method of machine translation, comprising:
preprocessing a text to be translated to obtain a word sequence of the text to be translated;
identifying whether a target entity is contained in the word sequence through a deep learning model;
if so, replacing a target entity in the word sequence with a placeholder, translating the replaced word sequence and the target entity respectively, and replacing the placeholder in the translation result of the replaced word sequence with the translation result of the target entity to obtain the translation result of the text to be translated;
if not, performing text translation on the word sequence to obtain a translation result of the text to be translated.
2. The machine translation method of claim 1, wherein said deep learning model comprises an embedded layer, an encoder and a decoder arranged in sequence.
3. The method of machine translation according to claim 2, wherein said identifying whether the word sequence includes a target entity by a deep learning model comprises:
performing vector conversion on the word sequence through the embedding layer to obtain vector representation of each word;
performing context feature extraction on the word sequence represented by the vector through the encoder to obtain feature representation of each word;
marking and predicting the character sequence represented by the characteristics through the decoder to obtain an entity marking sequence of the character sequence;
and judging whether the word sequence contains a target entity labeled by the corresponding target entity according to the entity labeling sequence.
4. The machine translation method of claim 2, wherein said encoder is constructed based on a bidirectional long and short term memory network;
the decoder is constructed based on conditional random fields.
5. The machine translation method of claim 1, wherein said deep learning model is generated based on training data training of travel domain;
the target entity is a travel entity, and the travel entity comprises: store name, company name, person name, place name, and time.
6. The machine translation method of claim 1, wherein said separately translating the replaced word sequence and the target entity comprises:
performing text translation on the replaced word sequence to obtain a translation result of the replaced word sequence;
and performing entity translation on the target entity to obtain a translation result of the target entity.
7. The method of machine translation according to claim 6 wherein said replaced word sequence is text translated by a neural machine translation model;
and performing entity translation on the target entity according to a preset translation rule.
8. A machine translation device, comprising:
the system comprises a preprocessing module, a translation module and a translation module, wherein the preprocessing module is configured to preprocess a text to be translated to obtain a word sequence of the text to be translated;
the entity identification module is configured to identify whether the word sequence contains a target entity through a deep learning model;
the combined translation module is configured to replace a target entity in the word sequence with a placeholder when the word sequence comprises the target entity, respectively translate the replaced word sequence and the target entity, and replace the placeholder in the translation result of the replaced word sequence with the translation result of the target entity to obtain the translation result of the text to be translated;
and the text translation module is configured to perform text translation on the word sequence to obtain a translation result of the text to be translated when the word sequence does not contain the target entity.
9. An electronic device, comprising:
a processor;
a memory having executable instructions stored therein;
wherein the executable instructions, when executed by the processor, implement the machine translation method of any of claims 1-7.
10. A computer-readable storage medium storing a program, wherein the program, when executed by a processor, implements the machine translation method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110714213.0A CN113392659A (en) | 2021-06-25 | 2021-06-25 | Machine translation method, device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110714213.0A CN113392659A (en) | 2021-06-25 | 2021-06-25 | Machine translation method, device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113392659A true CN113392659A (en) | 2021-09-14 |
Family
ID=77624006
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110714213.0A Pending CN113392659A (en) | 2021-06-25 | 2021-06-25 | Machine translation method, device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113392659A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110543644A (en) * | 2019-09-04 | 2019-12-06 | 语联网（武汉）信息技术有限公司 | Machine translation method and device containing term translation and electronic equipment |
| CN111259672A (en) * | 2020-02-12 | 2020-06-09 | 新疆大学 | Chinese tourism field named entity identification method based on graph convolution neural network |
| CN111310485A (en) * | 2020-03-12 | 2020-06-19 | 南京大学 | Machine translation method, device and storage medium |
| CN111310471A (en) * | 2020-01-19 | 2020-06-19 | 陕西师范大学 | Travel named entity identification method based on BBLC model |
| CN111539229A (en) * | 2019-01-21 | 2020-08-14 | 波音公司 | Neural machine translation model training method, neural machine translation method and device |
| CN111611802A (en) * | 2020-05-21 | 2020-09-01 | 苏州大学 | Multi-field entity identification method |
| KR20200142851A (en) * | 2019-06-13 | 2020-12-23 | 주식회사 누아 | Device, method and computer program for machine translation of geograohic name |
| WO2021072852A1 (en) * | 2019-10-16 | 2021-04-22 | 平安科技（深圳）有限公司 | Sequence labeling method and system, and computer device |
| CN112766001A (en) * | 2021-01-14 | 2021-05-07 | 语联网（武汉）信息技术有限公司 | Enterprise name translation method and device |
| KR102261710B1 (en) * | 2020-11-04 | 2021-06-07 | (주)휴먼아이티솔루션 | Method, apparatus and computer readable medium for managing multilingual tourism contents based on artificial intelligence |
-
2021
- 2021-06-25 CN CN202110714213.0A patent/CN113392659A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111539229A (en) * | 2019-01-21 | 2020-08-14 | 波音公司 | Neural machine translation model training method, neural machine translation method and device |
| KR20200142851A (en) * | 2019-06-13 | 2020-12-23 | 주식회사 누아 | Device, method and computer program for machine translation of geograohic name |
| CN110543644A (en) * | 2019-09-04 | 2019-12-06 | 语联网（武汉）信息技术有限公司 | Machine translation method and device containing term translation and electronic equipment |
| WO2021072852A1 (en) * | 2019-10-16 | 2021-04-22 | 平安科技（深圳）有限公司 | Sequence labeling method and system, and computer device |
| CN111310471A (en) * | 2020-01-19 | 2020-06-19 | 陕西师范大学 | Travel named entity identification method based on BBLC model |
| CN111259672A (en) * | 2020-02-12 | 2020-06-09 | 新疆大学 | Chinese tourism field named entity identification method based on graph convolution neural network |
| CN111310485A (en) * | 2020-03-12 | 2020-06-19 | 南京大学 | Machine translation method, device and storage medium |
| CN111611802A (en) * | 2020-05-21 | 2020-09-01 | 苏州大学 | Multi-field entity identification method |
| KR102261710B1 (en) * | 2020-11-04 | 2021-06-07 | (주)휴먼아이티솔루션 | Method, apparatus and computer readable medium for managing multilingual tourism contents based on artificial intelligence |
| CN112766001A (en) * | 2021-01-14 | 2021-05-07 | 语联网（武汉）信息技术有限公司 | Enterprise name translation method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11157698B2 (en) | Method of training a descriptive text generating model, and method and apparatus for generating descriptive text | |
| CN110196894B (en) | Language model training method and language model prediction method | |
| CN108363790B (en) | Method, device, equipment and storage medium for evaluating comments | |
| CN111309915B (en) | Method, system, device and storage medium for training natural language of joint learning | |
| CN109933662B (en) | Model training method, information generation method, device, electronic equipment and computer readable medium | |
| CN107729313B (en) | Deep neural network-based polyphone pronunciation distinguishing method and device | |
| US8380488B1 (en) | Identifying a property of a document | |
| CN110163181B (en) | Sign language identification method and device | |
| US9400787B2 (en) | Language segmentation of multilingual texts | |
| CN111414745A (en) | Text punctuation determination method and device, storage medium and electronic equipment | |
| CN108763230B (en) | Neural machine translation method using external information | |
| CN111814493B (en) | Machine translation method, device, electronic equipment and storage medium | |
| CN113743101A (en) | Text error correction method and device, electronic equipment and computer storage medium | |
| CN111753532B (en) | Error correction method and device for Western text, electronic equipment and storage medium | |
| CN113761845A (en) | Text generation method and device, storage medium and electronic equipment | |
| JP7194759B2 (en) | Translation data generation system | |
| CN111666405B (en) | Method and device for identifying text implication relationship | |
| CN111783435B (en) | Shared vocabulary selection method, device and storage medium | |
| CN116579327A (en) | Text error correction model training method, text error correction method, device and storage medium | |
| CN111460224A (en) | Comment data quality labeling method, device, equipment and storage medium | |
| CN116306690A (en) | Machine translation quality evaluation method, device, equipment and storage medium | |
| CN111339760A (en) | Method and device for training lexical analysis model, electronic equipment and storage medium | |
| CN113392659A (en) | Machine translation method, device, electronic equipment and storage medium | |
| CN115906854A (en) | Multi-level confrontation-based cross-language named entity recognition model training method | |
| CN112100335B (en) | Problem generation method, model training method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |