JP2022152805A - Simultaneous translation system and method - Google Patents

Abstract

To provide a simultaneous translation system and method which can realize simultaneous translation with high reliability.SOLUTION: A simultaneous translation system 50 for translating voice of a first language into a second language, includes: devices 64, 66, 68, and 70 for carrying out voice recognition of a voice signal of the first language, dividing the recognized voice into chunks, and outputting a chunk sequence formed of chunks of the first language; an automatic translation device 72 for automatically translating each of the chunks of the first language output by the devices 64, 66, 68, and 70 into the second language to output chunks of the second language; and a translation result editing device 74 having a display device for performing display so that the chunks of the second language output from the automatic translation device 72 and the chunks of the first language corresponding to the chunks of the second language can be contrasted with each other, and editing by a first editor can be performed with respect to the chunks of the second language, and for outputting the edited text of the second language in response to reception of completion of the edition.SELECTED DRAWING: Figure 1

Description

This technology relates to a simultaneous translation system, and more particularly to a simultaneous translation system and method capable of efficiently editing translated sentences.

Recently, opportunities to watch movies, TV dramas, news, etc. from various countries have increased explosively. This is due to the development of communication technology including the Internet. In addition, as typified by web conferencing, systems for people from different countries to have conversations or conferences by communication, and to listen to lectures are spreading at a very rapid pace.

The problem with these systems is language. There is no problem in conversations where only people of the same language participate, but in meetings where people from different countries participate, especially meetings where three or more languages are used, it is useful unless there is simultaneous translation. Results are hard to come by.

Conventionally, in such meetings, simultaneous interpreters who are fluent in both one language and another are provided for the number of language combinations, and each participant receives one of the interpretations by the simultaneous interpreters. was using.

However, due to the increase in opportunities for international conferences and viewing of real-time programs such as news from various countries, it is necessary to prepare the necessary number of simultaneous interpreters. It has become extremely difficult, if not impossible.

Therefore, for such simultaneous translation, it is conceivable to use speech recognition and automatic translation, which have recently made remarkable progress. If the final output is not text but speech, speech synthesis may be performed on the result of automatic translation. Recent speech recognition and automatic translation are faster and more accurate than ever before. Therefore, it is very effective to use these techniques for simultaneous translation.

As a simultaneous translation device using automatic translation, there is one disclosed in Patent Document 1 listed below. The simultaneous translation apparatus disclosed in Patent Document 1 receives input of speech in a first language, automatically translates it into a second language, and outputs the result in speech of the second language. In Patent Literature 1, the same automatic translation device performs two-way simultaneous translation from a first language to a second language and from the second language to a first language.

JP 2018-195276 A

However, when using an automatic translation device, including the automatic translation device disclosed in Patent Document 1, especially when it is difficult to listen to the other party's remarks on the spot, such as in a meeting, the final translation is done manually. , i.e. post-editing is required. This is because, if there is a mistranslation or an expression that is difficult to interpret in the middle, there is a high possibility that the user will not be able to understand the rest of the story correctly, or that the proceedings will go off track. It's for.

The translation device disclosed in Patent Document 1 does not disclose the need for post-editing at all. If post-editing is performed by the translation device disclosed in Patent Document 1, an editor who compares the pre-translation speech or text with the post-translation text to determine the accuracy of the translation and makes necessary corrections necessary.

However, the number of such interpreters who are fluent in both languages is small. Therefore, the translation device disclosed in Patent Document 1 has the problem that it is difficult to achieve highly reliable simultaneous translation.

SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide a simultaneous translation system and method capable of realizing highly reliable simultaneous translation.

A simultaneous translation system according to a first aspect of the present invention is a simultaneous translation system for translating speech in a first language into a second language, wherein a speech signal in the first language is speech-recognized, divided into chunks, and A chunk string output means for outputting a chunk string composed of chunks in one language, and each chunk in the first language output by the chunk string output means is automatically translated into a second language, and a chunk in the second language is output. so that the automatic translation means, the second language chunk output by the automatic translation means, and the first language chunk corresponding to the second language chunk can be compared, and the first language chunk for the second language chunk and post-editing means for outputting the edited text in the second language in response to receiving an instruction to end editing, the display device having a display for editing by an editor.

Preferably, the simultaneous translation system further includes back-translation means for translating chunks of the second language into the first language, and the display device further displays the back-translation result by the back-translation means.

More preferably, the post-editing means further includes matching degree calculation means for calculating a degree of matching between the result of translation by the reverse translation means and the chunk in the first language, which is the original text of the chunk in the second language, and the display device further includes: Display the degree of agreement as confidence in the translation of the chunk in the second language.

Further preferably, the simultaneous translation system includes first and second post-editing means, the simultaneous translation system further distributes the output of the automatic translation means to the first or second post-editing means and inputs the input and first merging means for merging in proper order the outputs of the first and second post-editing means for .

Preferably, the chunk sequence output means includes speech recognition text output means for recognizing speech signals in the first language and outputting text in the first language, and automatic text output in the first language output by the speech recognition text output device. an automatic text dividing means for dividing into chunks and outputting a chunk string of the first language, and a chunk string of the first language outputted by the automatic text dividing means are displayed so that a second editor can edit the chunks. and chunk string editing means for outputting the displayed chunk string in the first language in response to an instruction to finish editing by the second editor.

More preferably, the chunk sequence output means includes first and second chunk sequence editing means, and the simultaneous translation system further distributes the output of the automatic text segmentation means to the first or second chunk sequence editing means. A second integration means for receiving the input and integrating the outputs of the first and second chunk sequence editing means for the input in the correct order.

More preferably, the speech recognition text output means includes a speech recognition device for recognizing a speech signal in a first language and outputting a text in the first language; Text editing means for inputting the displayed text in the first language to the automatic text division means in response to an instruction to end editing.

Preferably, the speech recognition text output means includes first and second text editing means, and the simultaneous translation system further distributes and inputs the output of the speech recognition device to the first or second text editing means, It includes third merging means for merging, in correct order, the outputs of the first and second text editing means for said input.

More preferably, the chunk string output means includes speech recognition means for recognizing speech signals in the first language and outputting text in the first language, and text in the first language output by the speech recognition device for performing third editing. text editing means for outputting the displayed text in the first language in response to an instruction to end editing; and the first language output by the text editing means. text segmentation means for segmenting the text of the language into chunks and outputting a sequence of chunks in the first language.

More preferably, the chunk string output means includes first and second text editing means, and the simultaneous translation system further distributes the output of the speech recognition means to the first or second text editing means and inputs it, It includes second merging means for merging the outputs of the first and second text editing means for the input in the correct order.

A simultaneous translation method according to a second aspect of the present invention is a simultaneous translation method for translating speech in a first language into a second language, wherein a computer recognizes a speech signal in the first language and divides it into chunks. a step of automatically translating each of the first language chunks into a second language by a computer and outputting the second language chunks; A computer displays second language chunks and first language chunks corresponding to the second language chunks for comparison and for editing by a first editor on the second language chunks. a computer accepting edits to the chunk in the second language by the first editor; and a computer accepting edits in response to receiving an instruction to end editing. and outputting the second language text.

The above and other objects, features, aspects and advantages of the present invention will become apparent from the following detailed description of the invention taken in conjunction with the accompanying drawings.

FIG. 1 is a block diagram showing a schematic configuration of a simultaneous translation system according to the first embodiment of the invention. FIG. 2 is a schematic diagram showing processing steps in the system shown in FIG. FIG. 3 is a diagram showing an editing screen before editing the speech recognition result. FIG. 4 is a diagram showing an editing screen after editing the speech recognition result. FIG. 5 is a diagram showing an editing screen before editing a chunk, which is a unit of automatic translation. FIG. 6 is a diagram showing an editing screen after editing a chunk, which is a unit of automatic translation. FIG. 7 is a diagram showing an editing screen before editing the translation result. FIG. 8 is a diagram showing an editing screen after editing the translation result. FIG. 9 is a flow chart showing the control structure of a program that implements the simultaneous translation system shown in FIG. FIG. 10 is a flow chart showing the control structure of a program that implements the speech recognition result editing apparatus shown in FIG. FIG. 11 is a flow chart showing the control structure of a program that implements the translation unit editing device shown in FIG. FIG. 12 is a flow chart showing the control structure of a program that implements the translation result editing apparatus shown in FIG. FIG. 13 is a flow chart showing the control structure of a program that implements the translation result detail processing shown in FIG. FIG. 14 is a flow chart showing the control structure of a program that implements the central server according to the second embodiment of this invention. FIG. 15 is a diagram showing a translation result editing screen according to the first modification of the second embodiment. FIG. 16 is a flow chart showing the control structure of a program that implements the translation result editing device according to the first modified example of the second embodiment. FIG. 17 is a diagram showing a translation result editing screen in the translation result editing device according to the second modification of the second embodiment. FIG. 18 is a flow chart showing the control structure of a program that implements the translation result editing device according to the second modified example of the second embodiment. FIG. 19 is a flow chart showing the control structure of a program that implements the translation result editing device according to the third modified example of the second embodiment.

In the following description and drawings, identical parts are provided with identical reference numerals. Therefore, detailed description thereof will not be repeated.

1st Embodiment 1 Configuration (1) Overall Configuration FIG. 1 is a block diagram showing the overall configuration of a simultaneous translation system 50 according to the first embodiment of the present invention. Referring to FIG. 1, a simultaneous translation system 50 is connected to a network 62, and a central server 60 that centrally controls each functional unit of the simultaneous translation system 50 by communication through the network 62. Both are connected to the network 62. The translation result by the speech recognition device 64, the speech recognition result editing device 66, the speech segmentation device 68, the translation unit editing device 70, the automatic translation device 72, the translation result editing device 74, the speech synthesis device 76, and the translation result editing device 74 is and an automatic translation device 78 for back-translating.

The speech recognition device 64 and the automatic translation devices 72 and 78 may be of any type as long as they can be used according to the application (field and type of language). The automatic translation device 78 should have the relationship between the original language and the translation language reversed from that of the automatic translation device 72 . In this embodiment, an automatic translation device 78 is provided separately from the automatic translation device 72 . However, it is not limited to such an embodiment, and the automatic translation device 72 may be used to translate from the target language to the original language.

The speech recognition result editing device 66 receives the text of the speech recognition result output by the speech recognition device 64 in real time, edits it by the first editor, and immediately outputs it to the speech division device 68 when the editing is completed. It is. Since the text output by the speech recognition device 64 is the result of speech recognition, it often contains speech recognition errors. Therefore, in order to automatically translate the original text with high accuracy, it is necessary to correct errors contained in this text. Although this correction may be performed by automatic processing, in this embodiment, accuracy is emphasized and it is performed manually.

Since the processing performed by the speech recognition result editing device 66 is simple text editing, the speech recognition result editing device 66 is sufficient with a normal computer and does not require a particularly complicated configuration.

The utterance dividing device 68 has a function of dividing the text divided into utterance units output by the speech recognition result editing device 66 into units called "chunks" suitable for translation so that the translation can be performed with high accuracy. Where the text is divided into chunks depends on the language and domain. In the case of speech, one sentence is often long. It is known that the longer the sentence, the lower the accuracy of automatic translation. Therefore, by dividing the text into chunks in this way and translating each chunk as a chunk sequence, the accuracy of automatic translation can be improved. A so-called "section" or the like corresponds to a chunk.

The speech division device 68 determines how to divide the text into chunks by so-called machine learning. For this learning, a large number of sentences are prepared as training data in which each token is labeled to indicate whether or not the sentence is to be divided into chunks. The speech segmentation device 68 can be realized by training an SVM (Support Vector Machine) using this training data. A neural network (DNN) trained by deep learning can also be used.

The translation unit editing device 70 is for the second editor to manually edit and output the chunk sequence output by the speech segmentation device 68 in real time. Speech segmenter 68 can be trained, but its decisions cannot always match perfectly with human judgment. If the original text for translation is incorrectly divided, the translation will naturally be incorrect. Therefore, in this embodiment, the second editor visually confirms the result of division by the speech division device 68, and edits it if necessary.

Since the processing performed by the speech segmentation device 68 is only simple text editing (segmentation and integration of chunks, which is referred to as "chunk editing" for convenience), the speech segmentation device 68 can be implemented by a normal computer. and a particularly complicated configuration is not required.

The translation result editing device 74 is for the third editor to visually check the translation result by the automatic translation device 72 and correct it if necessary. In this embodiment, as will be described later, the original text to be translated and the translated text by the automatic translation device 72 are displayed in comparison. Therefore, the editor can easily judge whether the translation is appropriate or not by comparing the two, and make corrections if necessary. Further, in this embodiment, as will be described later, the details of the translation result can be displayed at the request of the editor. In addition to the original text and the translated text, the detailed screen displays the result of back-translating the translated text into the original language and its score (reliability). Therefore, the editor can judge whether the translation is appropriate or not and correct the translation after taking all of these into consideration.

(2) Overall Processing Configuration FIG. 2 shows the overall processing configuration of this simultaneous translation system 50 . Referring to FIG. 2, source language speech signal 110 is input to speech recognition engine 112 of speech recognizer 64 and source language text 114 is output. The source language text 114 is subjected to speech recognition result editing 116 by a first editor using a speech recognition result editing device 66 to become a source language text 118 after speech recognition correction. The corrected source language text 118 is input to the translation result segmentation engine 120 of the speech segmentation device 68 , becomes the source language text 122 segmented into chunk strings, and is input to the translation unit editing device 70 . The source language text 122 undergoes translation unit segmentation editing 124 by a second editor using a translation unit editor 70, resulting in source language text 126 modified into the correct sequence of chunks suitable for translation and input to an automatic translation engine 128. be.

The automatic translation engine 128 automatically translates the source language text 126 corrected into a correct chunk sequence suitable for translation in this way, and outputs a target language text 130 . How many errors exist in the target language text 130 depends on the performance of the automatic translation engine 128 . However, since the input is corrected to the correct sequence of chunks suitable for translation, the accuracy is at least higher than without such corrections. The target language text 130 is post-edited 132 by a third editor using the translation result editing device 74 shown in FIG. As a result, even if the translation result contains an error, it is corrected by the third editor and output as the translation language text 134 .

In this embodiment, the translated language text 134 post-edited in this manner is converted into a translated language speech signal 138 by a speech synthesis engine 136 of the speech synthesizer 76 and output.

As described above, in the simultaneous translation processing 100 according to this embodiment, speech recognition of the source language speech signal 110, correction of the speech recognition result, division of the corrected text into chunks, correction of the chunk sequence, automatic translation, automatic translation and speech synthesis are pipelined in units of speech recognition by the speech recognition engine 112 . As a result, simultaneous translation can be performed with high accuracy.

(3) Edit Screens a. Editor screen for speech recognition results FIG. 3 shows an editor screen 200 used for correcting speech recognition results. In this embodiment, the editor screen 200 includes an edit field 210, an audio signal display portion 212, a reverse button 214 that slightly reverses the played audio, a pause button 216, and a forward button 218 that slightly advances the played audio, and a “Next” button 220 .

Edit field 210 displays the text of the speech recognition result. As shown in FIG. 3, although the text of the speech recognition result is a mixture of kanji and kana, it does not contain punctuation marks and is a single character string that is connected as a whole. In addition, among these character strings, there are character strings indicating mere sounds inserted during speech, such as characters/character strings 230, 232, and 234, and characters/character strings 236 and 238, which are inserted interjectively. It contains meaningless characters/strings.

Edit field 210 allows text editing by the first editor. For example, by deleting mere sounds as shown in edited portions 250, 252 and 254 shown in FIG. be a string.

An image showing the audio signal corresponding to the edit field 210 is displayed in the audio signal display section 212 . If you move the pointer to a point in this image and click on it, the sound will start playing from the position corresponding to that point. By clicking the return button 214, the playback position can be returned for a short period of time (for example, 1 second). By clicking the advance button 218, the playback position can be advanced by a short period of time (for example, 3 seconds). If the pause button 216 is clicked, reproduction is paused, and a reproduction start button (not shown) is displayed instead of the pause button 216 button. Therefore, if necessary, the first editor can easily confirm the target speech for speech recognition and correct the speech recognition result.

Editing by the editor screen 200 is only simple deletion and insertion of character strings. Further, the processing performed on the audio signal is only the display of the audio signal, the detection of the clicked position, and the reproduction, stop, and change of the reproduction position of the audio signal according to the clicked position. Therefore, this editing process can be realized using a well-known program.

When the editor clicks on the "Next" button 220, the edited text displayed in the edit field 210 is output for subsequent processing.

B. Translation Unit Editor Screen FIG. 5 shows the translation unit (chunk) editor screen 300 . Referring to FIG. 5, an editor screen 300 includes an edit field 302 in which the text of the corrected speech recognition result, which is divided by the utterance dividing device 68 shown in FIG. and Next buttons 304 and 306 .

The output of the speech splitter 68 includes line feed codes inserted immediately after each location determined to be the end of the chunk (split position). The segmentation by the speech segmentation device 68 can achieve some degree of accuracy through training, but does not completely eliminate errors. For example, in FIG. 5, the first sentence 310 and the second sentence 312 are originally one sentence. However, the speech splitter 68 has split this sentence into a first sentence 310 and a second sentence 312 .

The second editor, who has determined that the first sentence 310 and the second sentence 312 should be put together, deletes the line feed code attached to the end of the first sentence 310 . As a result, the first sentence 310 and the second sentence 312 are correctly grouped into one sentence, like the edited part 320 shown in FIG.

When "Next" button 304 or "Next" button 306 is clicked, the text displayed in edit field 302 is saved and output for subsequent processing.

C Editor Screen of Translation Result Shown is the editor screen 350 of the translation result editing device 74 shown in FIG. Referring to FIG. 7, editor screen 350 includes an original text field 352 displaying the original text of automatic translation, a translated text editing field 354 displaying the translation result, and a “next” button 356 .

The third editor compares the displayed contents of the original text field 352 and the displayed contents of the translated text editing field 354 . If there is an error in the translation, the third editor corrects the error in the translation displayed in the translation edit field 354 .

In the case of FIG. 7, for example, when it is determined that the translation 360 displayed in the translation edit field 354 corresponding to the original sentence 358 displayed in the original text field 352 is incorrect or difficult to understand, the third The editor clicks any part of the translation 360 . Then, an editor screen 400 for translation result details shown in FIG. 8 is displayed, and the translation 360 can be corrected as described later. After correction, this editor screen 350 is displayed.

When the "Next" button 356 is clicked, the translated text displayed in the translated text edit field 354 is output as the correct translated text for subsequent processing.

D. Editor screen for translation result details FIG. 8 shows an editor screen 400 for translation result details that is displayed when the third editor clicks on the translation 360 in FIG. Referring to FIG. 8, editor screen 400 displays original text 410 displaying the original text, translated text 414 displaying the translated text in an editable manner, and reverse translation 412 resulting from back-translating the translated text 414 into the original language. It includes a back-translation 412 displayed with degrees 420 , an “apply modifications and close” button 416 and a cancel button 418 .

The reverse translation 412 is the result of reverse translation of the translated sentence 414 in the target language into the original language by the automatic translation device 78 shown in FIG. 1 in this embodiment. If the original sentence 410 and the reverse translation 412 express exactly the same or substantially the same meaning, then the translation 414 is highly reliable. Conversely, if the two are different, there is a possibility that the translation 414 is incorrect. By contrasting and displaying the original text 410 and the reverse translation 412 in this manner, the third editor can easily judge the validity of the translated text 414 .

As the reliability 420, in this embodiment, the degree of matching between words appearing in both the original sentence 410 and the reverse translation 412 is calculated. It is assumed that the third editor understands both the original language and the translated language, although the degree of coincidence of words alone is not sufficient. You should be able to determine how.

When the "reflect corrections and close" button 416 is clicked, the translation displayed in the translation 414 is correct, and the editor screen 350 shown in FIG. 7 is redisplayed with the translation 360 shown in FIG. 7 corrected. When the cancel button 418 is clicked, even if the translation 414 has been corrected, the correction is discarded, and the editor screen 350 of FIG. 7 is displayed without changing the translation.

(4) Program Configuration The structure and functions of each functional unit described above are realized by computer hardware and a computer program executed by the computer hardware. The control structure of a program that implements each functional unit, particularly the control structure of a program that implements editing work, will be described below.

A Central server 60
Referring to FIG. 9, the program that implements central server 60 shown in FIG. Step 452 initiates communication with each element of simultaneous translation system 50 over network 62, and an output buffer in storage for use as a first-in-first-out buffer for use in communicating with each element of simultaneous translation system 50. and _initializes to 0 a write pointer _PW indicating the address to which data is to be written next and a read pointer PR indicating the address to which data is to be read next for each of the buffers for the above elements. It includes step 454 and step 456 for waiting for an operation start instruction after completion of step 454 and proceeding to the next process. An output buffer is provided for each of the speech recognition device 64, the speech recognition result editing device 66, the speech _segmentation device 68, the translation unit editing device 70, the automatic translation device 72, and the translation result editing device 74. Note that pointers _PR are also provided for the number of buffers.

The program further waits in step 458 for receiving the text of the speech recognition result from the speech recognition device 64 via the network 62 in response to receiving the operation start instruction in step 456, and performs speech recognition in step 458. A step 460, executed in response to receiving the resulting text, for writing the received speech recognition results into a buffer for speech recognition results and incrementing the value of the buffer write pointer _PW for speech recognition results by one. step 462, and the start of simultaneous translation processing is sent to the speech recognition result editing device 66, the speech segmentation device 68, the translation unit editing device 70, the automatic translation device 72, the translation result editing device 74, the speech synthesis device 76, and the automatic translation device 78. and step 464 of notifying.

The program further includes a step 466 that waits for a request to send or write data from each device on network 62 and branches the control flow depending on the type of request received. Note that only three branches at step 466 are shown in FIG. However, it should be noted that of these three branches, write and read branches exist for each type of device on network 62 .

Further, in response to the fact that the received request is a data write request, the program checks whether the value of the write pointer _PW of the output buffer and the value of the read pointer match according to the type of data. Step 468 for judging and branching the flow of control; Step 470 for displaying on the screen or writing to a predetermined log file; Step 472 for writing the data received in step 466 to the address indicated by the write _{pointer PW ;} and step 474 which proceeds to step 466 .

The reason why the determination of step 468 is necessary is as follows. In this embodiment, a ring buffer is assumed as the buffer. For ring buffers, the pointer wraps around when it reaches the end of the buffer. Normally, when data is written to a buffer, data is read from that location. Therefore, the write position always precedes the read position. If the written data is read and processed smoothly, there is no particular problem. It may catch up. Of course, this possibility is small, but not impossible. In such a case, the data written in the buffer may be overwritten before it is read out. When such trouble occurs, a warning is displayed or a log is recorded in this way so as not to stop the process, and the process proceeds.

In response to the request received at step 466 being a request to read data, the program further determines whether the read pointer _{PR of the buffer is equal to the value of the write pointer PW ,} and according to the result of the determination. step 476 to branch control flow, and step 478, in response to an affirmative determination of step 476, notifying the source of the read request that there is no data to read and returning control to step 466; , in response to a negative determination in step 476, step 480 of reading data from the address indicated by the read _{pointer PR} and transmitting it to the source of the request; to step 466;

An affirmative determination at step 476 is typically at the beginning of the process. No data has been written at the start of processing. Therefore, the value of the write pointer _PW is zero. At this time, the value of the read pointer _PR is also zero. Therefore, there is no data to send when a read request is received. This is also the case when writing and reading proceed synchronously (at the same speed).

In the case of the central server 60, the operator may give an instruction to stop the entire system. In this embodiment, when such a stop request is received at step 466, at the determination of step 466, necessary processing such as instructing each element of the simultaneous translation system 50 to end the operation is performed. End execution.

B Voice recognition device 64
Any speech recognition device 64 may be used as described above. At present, one using a so-called neural network is preferred. There are already many programs that implement speech recognition devices, and there are many textbooks and patent documents relating to them. Therefore, for the sake of brevity, the details of the program that implements speech recognizer 64 will not be repeated here.

C Speech recognition result editing device 66
Referring to FIG. 10, the program that implements speech recognition result editing device 66 executes initialization processing when activated, step 552 that initiates communication with central server 60, and execution of a predetermined initial screen. step 554 to display , step 556 to wait for an instruction to start operation of the simultaneous translation system 50 from the central server 60, and in response to receiving the instruction to start the operation, supervise a transmission request for speech recognition results. It includes step 558 of sending to server 60 and step 560 of waiting for data from central server 60 . In step 560 , if no reply is received from the central server 60 and if a reply is received stating that the speech recognition result does not yet exist, control returns to step 558 . The processing of this part is common to each part of the simultaneous translation system 50 other than the network 62 .

Further, in response to receiving the text data of the speech recognition result from the central server 60 in step 560, this program further generates and displays an editor screen 200 as shown in FIG. step 564 to wait for an instruction by step 564; step 566 to branch the flow of control according to the contents of the instruction when the instruction is received in step 564; in response to sending the text displayed in edit field 210 to central server 60 as an edit result and returning control flow to step 558 .

The program further responds to a negative determination at step 566 by branching the flow of control to step 570 and step 570 depending on whether the indication received in step 564 indicates an end of operation. Step 574 performs post-processing and terminates execution of the program in response to an affirmative determination of step 574, and executes a process according to the instruction and controls in response to a negative determination in step 570. and step 572 returning to step 564 . When the determination in step 570 is negative, the text displayed in the edit field 210 is edited, the voice signal display section 212 is clicked, the return button 214, the pause button 216, the forward button 218 and the playback start button (not shown) , and so on.

D Speech division device 68
Speech segmentation device 68 is implemented by a trained SVM or DNN, as described above. There are already many programs that implement SVMs and DNNs, and there are also many textbooks, papers and patent documents on the details of SVMs and DNNs. Therefore, the details of the speech splitter 68 and the program that implements it will not be repeated here for the sake of brevity.

E translation unit editing device 70
Referring to FIG. 11, the program that implements translation unit editing device 70 includes step 600 for performing initialization processing, step 602 for starting communication with central server 60, and step 602 for displaying an initial screen at the start of operation. 604, a step 606 of waiting for an operation start instruction from the central server 60, and in response to receiving an operation start instruction from the central server 60, a translation unit segmentation result (speech segmentation device 68 processing result) and a step 610 of waiting until data is received from the central server 60 .

Further, in response to receiving data to be processed from the central server 60 in step 610, this program further generates an editor screen 300 shown in FIG. a step 614 that awaits an instruction, and a step 616 that, in response to receiving any instruction from the second editor at step 614, branches control flow depending on whether the instruction is the "next" button 306; , in response to an affirmative determination in step 616, sending the text displayed in edit field 302 to central server 60 as a chunk string of translation unit (chunk) results, and returning control to step 608; 618.

The program further responds to a negative determination at step 616 by branching control flow at step 620 according to whether the indication indicates the end of processing; In response to step 624, necessary post-processing is performed to end program execution, and in response to a negative determination in step 620, processing corresponding to the received instruction is performed and control is stepped. and step 622 returning to 614 . The processing in step 622 is processing for editing the text displayed in the edit field 302 in FIG.

F Automatic translation device 72
As for the automatic translation device 72, an existing one can be used as described above. There are already many programs that implement automatic translation devices, and there are many textbooks, papers, and patent documents related to the details of automatic translation devices. Therefore, for the sake of brevity, the details of the program that implements the automatic translation device 72 will not be repeated here. However, at present, so-called neural machine translation is most desirable as the automatic translation device 72 in terms of accuracy and processing speed.

G translation result editing device 74
Referring to FIG. 12, the program that implements translation result editing device 74 performs initialization processing at step 650, starts communication with central server 60 at step 652, and displays an initial screen indicating the start of processing. Step 654; Step 656 for waiting for an operation start instruction from the central server 60; It includes a step 658 of sending to the results editor 74 and a step 660 of waiting until data is received from the central server 60 .

Further, in response to receiving data from the central server 60 in step 660, the program displays the editor screen 350 shown in FIG. step 666 for branching the control flow depending on whether the instruction is the "Next" button 356 in response to receiving any instruction at , and in response to an affirmative determination at step 666, and a step 668 of transmitting the translated text displayed in the edit field 354 to the central server 60 as an edited result and returning control to step 658 .

The program further responds to a negative determination at step 666 by branching control flow at step 670 depending on whether any of the translations in translation edit field 354 have been clicked; In response to the affirmative response, start a program that displays the translation details editor screen 400 shown in FIG. a step 672 for updating the original translation with the edited data in response to receiving the editing result from the editor screen 400 in step 672; and a display of the editor screen 350 according to the updated translation in step 674. and step 676 which updates and returns control to step 664 . Details of step 672 will be described later with reference to FIG.

The program further responds to a negative determination at step 670 by branching control flow at step 678 depending on whether the received indication is a termination indication, and responsive to a positive determination at step 678. Then, step 682 executes necessary post-processing and terminates execution of the program, and step 680 executes processing according to the operation and returns control to step 664 in response to a negative determination in step 678. including. Although there is nothing particularly important as the processing performed in step 680, there are, for example, operation cancellation, operation re-execution, help display, pointer movement, and the like.

Referring to FIG. 13, the program that implements step 672 of FIG. 12 includes step 700 of executing necessary initialization processing, step 702 of temporarily storing the translation received from the program shown in FIG. Step 704 for providing the received translation to the automatic translation device 78 shown in FIG. Translated sentence 414 in FIG. 8), its back-translation obtained in step 704 (back-translation 412 in FIG. 8), and reliability indicating the degree of matching between the original sentence 410 and the back-translation 412 are calculated. and a step 708 of waiting for an instruction by a third editor.

Further, in response to the instruction received in step 708 being an instruction to edit the translated text, the program updates the translated text stored in the temporary storage device with the edited translated text in step 710, and automatically updates the edited translated text in step 710. Step 712 for back-translating by the translation device 78; Step 714 for updating the display of the editor screen 400 with the edited translation and the new back-translation and returning the control to Step 708; and a step 716 of, in response to the button 416, setting the translation text saved in the temporary storage as the return value and ending the execution of the program. When the instruction received at step 708 is the cancel button 418, the temporarily saved translation is discarded and the execution of the program ends. At step 672 in FIG. 12, the translated sentence normally delivered by the program shown in FIG. 13 is used as the new translated sentence, and when canceled, the output of the program shown in FIG. 13 is ignored.

H Speech synthesizer 76
As for the speech synthesizer 76, an existing one can be used as described above. There are already many programs that implement speech synthesizers, and there are many textbooks, papers, and patent documents on the details of speech synthesizers. Therefore, the details of the program that implements the speech synthesizer 76 will not be repeated here for the sake of brevity.

2. Operation (1) Start of Overall Operation Referring to FIG. Device 70 , automatic translation device 72 , translation result editing device 74 , speech synthesizer 76 , and automatic translation device 78 are initialized and start communicating with central server 60 . Among these devices, the devices other than the voice recognition device 64 wait for an instruction to start processing from the central server 60 .

The speech recognition device 64 detects a speech portion from an input speech signal, and recognizes the speech from the start to the end of the speech portion as one unit. The speech recognition device 64 transmits the speech recognition result as a text to the central server 60 as a data write request, and then waits for detection of the next speech part.

When the central server 60 receives this speech recognition result (FIG. 9, YES at step 458), it writes it to the address indicated by the write pointer _PW in the storage area as the output buffer of the speech recognition device 64 (step 460). Add 1 to _PW (step 462). The central server 60 transmits an instruction to start processing to each unit (step 464). In response to this, each device other than the speech recognition device 64 starts processing and transmits a data transmission request to the central server 60 (step 558 in FIG. 10, step 608 in FIG. 11, step 608 in FIG. 12). step 658), and waits for data from the central server 60 (step 560,
610, and 660).

At the start of operation, data is stored only in the output buffer of the speech recognition device 64 (YES at step 476 in FIG. 9). The central server 60 reads the speech recognition result text from the address indicated by the read pointer in the output buffer of the speech recognition device 64 (step 478 in FIG. 9), and transmits it to the speech recognition result editing device 66 (step 480). Further, the central server 60 adds 1 to the read pointer _PR of the output buffer of the speech recognition device 64 and returns the control to step 658 .

Below, each part operates as follows.

(2) Operation of Speech Recognition Device 64 The speech recognition device 64 detects the speech portion of the input speech signal, performs speech recognition of the speech portion, and transmits the text as a block of text to the central server 60 repeatedly. . The central server writes this data to the output buffer of the speech recognition device 64 , and upon receiving a read request from the speech recognition result editing device 66 , reads the data in a first-in, first-out manner and transmits it to the speech recognition result editing device 66 . This basic operation is common to any output buffer.

(3) Operation of voice recognition result editing device 66 The voice recognition result editing device 66 transmits a request for transmission of the voice recognition result to the central server 60 and waits until the voice recognition result is transmitted (step 560 in FIG. 10). ).

Upon receiving the text from the central server 60, the speech recognition result editing device 66 generates and displays the editor screen 200 shown in FIG. 3 (step 562). When the first editor makes some edits to the text, the corresponding editing process is performed through steps 564, 566, 570, 572 in FIG. 10, and the text is updated if necessary (step 570). 572). Control returns to step 564 to await the next instruction by the first editor.

When the first editor clicks the "next" button 220 shown in FIG. 3, the edited text (or the pre-edited text if no edits have been made) is sent to the central server 60. FIG. The central server 60 stores the data in the output buffer for the voice recognition result editing device 66 .

(4) Operation of Speech Splitting Device 68 The speech splitting device 68 sends a transmission request for the text edited by the speech recognition result editing device 66 to the central server 60 and waits until the text is transmitted from the central server 60 . The central server 60 reads the text from the output buffer for the speech recognition result editor 66 and sends it to the speech segmentation device 68 . The speech segmentation device 68 detects the end of the chunk by SVM or DNN for this text, inserts a line feed code immediately after the character at the end of the chunk, and transmits it to the central server 60 . The central server 60 stores this text in an output buffer for the speech splitter 68 .

(5) Operation of translation unit editing device 70 The translation unit editing device 70 sends a transmission request for the text resulting from the segmentation by the speech segmentation device 68 to the central server 60 (step 608 in FIG. 11). is sent (step 610). The central server 60 reads the text from the output buffer for the speech splitter 68 and sends it to the translation unit editor 70 . When the division result by the speech division device 68 is received from the central server 60 (YES at step 610 in FIG. 11), the translation unit editing device 70 generates the editor screen 300 shown in FIG. 5 and displays it on the display device (step 612). Translation unit editor 70 waits for instructions from the second editor (step 614), and if so, determines what the instructions are (steps 616 and 620).

If the instruction is the "next" button 306 shown in FIG. 5, the translation unit editing device 70 transmits the edited text to the central server 60 (step 618), and waits for the next data (steps 608 and 610). The central server 60 stores this data in an output buffer for the translation unit editing device 70 . If the instruction is not an instruction to end the operation, it is an instruction for some kind of editing. Therefore, according to the instruction, the text of the translation unit segmentation is edited, and processing such as updating the text is performed (step 622), and the next data is awaited (steps 608 and 610). If the instruction is to end the operation, necessary post-processing is executed (step 624), and execution of the program ends. If the instruction is not an instruction to end the operation, the designated editing process is executed to update the text of the translation unit division (step 622), and wait for the next instruction (step 614).

(6) Operation of Automatic Translation Device 72 The automatic translation device 72 transmits a transmission request for the text to be translated to the central server 60 . When the text to be translated is received from the central server 60, the automatic translation device 72 automatically translates the text and outputs the text in the translated language. The automatic translation device 72 transmits this text to the central server 60 together with the original text. Central server 60 stores these texts in an output buffer for automatic translation device 72 .

(7) Operation of Translation Result Editing Device 74 The translation result editing device 74 sends a transmission request for the text output by the automatic translation device 72 (original text and translation result) to the central server 60 (step 658 in FIG. 12). , waits for data transmission from the central server 60 (step 660). Upon receiving the text from the central server 60, the translation result editing device 74 generates an editor screen 350 shown in FIG. 7 by comparing the original text in the source language and the translated text in the target language (step 662). ,indicate. Thereafter, the translation result editing device 74 waits for instructions from the third editor (step 664).
a. Editing on the Overview Screen When the editor gives some instruction, the translation result editing device 74 determines whether or not the instruction is the "next" button 356 at step 666 in FIG. If the determination is affirmative (YES at step 666), the translation result editing device 74 transmits the translation displayed in the translation editing field 354 of FIG. data is awaited (steps 658 and 660). The central server 60 stores this data in the output buffer for the translation result editing device 74 .

If the determination in step 666 is negative, the translation result editing device 74 determines in step 670 whether any part of the translated text displayed in the translated text editing field 354 shown in FIG. A program whose control structure is shown in FIG. 13 is started with the translation and the corresponding original as arguments (step 672). As a result, the editor screen 400 shown in FIG. 8 is displayed and the result returns to step 672 . The operation of the translation result editing device 74 in FIG. 8 will be described later.

The translation result editing device 74 updates the translation of the text received from the central server 60 with the edited data received in step 672 of FIG. 12 (step 674), and displays the editor screen 350 with the updated data. update and return control to step 664 (step 676).

If the determination in step 670 is negative, the translation result editing device 74 determines in step 678 whether or not the instruction is an instruction to end processing. If the instruction is an end instruction (YES at step 678), necessary post-processing is performed and execution of the program is terminated (step 682). If the instruction is not an end instruction (NO at step 678), the process according to the instruction is executed at step 680, and control is returned to step 664 to wait for the next instruction from the third editor.

B. Editing on the detailed screen When the translation result editing device 74 is called at step 672 in FIG. 12, it operates as follows. Referring to FIG. 13, step 700 is performed first, and the translated text and the original text passed as arguments in step 672 are temporarily stored in the storage device. At step 704, the translated sentence is input to the automatic translation device 78 and back-translated into the sentence in the original language. Then, an editor screen 400 shown in FIG. 8 is generated and displayed using the original text, the reverse translation, and the translated text (step 706).

The process waits for an instruction from the third editor (step 708), and if there is an instruction, it is determined whether the instruction is an instruction to edit the translation, the "reflect corrections and close" button 416, or the cancel button 418, and the flow of control proceeds. is branched according to the judgment result.

If it is an instruction to edit the translated text, the translated text temporarily stored in the storage device is updated with the edited translated text (step 710), and the edited translated text is input to the automatic translation device 78 for reverse translation (step 712). The editor screen 400 shown in FIG. 8 is updated and re-displayed (step 714) using the edited translation and its reverse translation obtained in this way, and the original text (step 714), and the process returns to waiting for the third editor's instruction ( step 708).

If the "reflect corrections and close" button 416 is clicked, the temporarily saved translation is set as the return value in step 716, and the control returns to step 672 in FIG. 12 (step 716).

If the cancel button 418 has been clicked, a value indicating that fact is set as the return value, and the control returns to step 672 in FIG.

(8) Operation of speech synthesizer 76 The speech synthesizer 76 requests the central server 60 to transmit the result of editing by the translation result editing device 74 . When text in the target language, which is the result of editing by the translation result editing device 74, is transmitted from the central server 60, the speech synthesizer 76 generates a speech signal from the text using any known speech synthesis method. and output to an appropriate output device.

Under the overall control of the central server 60, the simultaneous translation system 50 repeats the above-described processing in a pipeline manner for each character string obtained as a result of speech recognition by the speech recognition device 64. FIG. As a result, the speech signal in the original language input to the speech recognition device 64 is translated into the target language one after another, converted into text, and converted into a speech signal.

3 Effect According to the first embodiment described above, speech recognition, correction of speech recognition results, division of speech recognition results into chunk strings, correction of division of chunk strings, automatic translation, correction of automatic translation results, and Speech synthesis is performed in parallel in a pipelined manner. Since the supervising server 60 sorts and controls the storage destinations of each data so that the output buffers between the devices operate in a first-in, first-out manner, the results of speech recognition are translated into translation languages in the correct order and converted into speech signals. Also, the editor corrects the speech recognition result, corrects the division of the recognition result into chunk strings, and corrects the translation result. As a result, simultaneous translation can be achieved with high accuracy.

In addition, the first editor and the second editor who correct the speech recognition result and the division of the recognition result into chunk sequences need only have knowledge of the original language, and knowledge of the target language. is not required. Also, the third editor needs to have knowledge of both the source language and the target language, but since the source language has already been converted into text and chunked suitable for automatic translation, processing related to the source language burden will be much less. Moreover, since the original text and the translated text are displayed in comparison with each other as in the above embodiment, it is possible to easily judge and correct errors in the translated text. Furthermore, by displaying a back-translated version of the translated text as shown in FIG. 8, the validity of the translated text can be easily determined. Furthermore, since the degree of matching between the reverse translation and the translated text is displayed as the reliability of the translated text, the load on the third editor is further reduced.

Although there is no description of post-editing in Patent Document 1, if post-editing is performed, the post-editing editor must have substantially the same comprehension and expressive power as a native speaker in both the first and second languages. Without it, it is considered that sufficient editing cannot be performed. If the simultaneous translation system 50 according to the above embodiment is used, editors (first editor and second editor) who need to have sufficient knowledge about the source language and editors who have knowledge about both languages Only an editor (third editor) is required, and the third editor does not require native-level skills in the source language. Therefore, the simultaneous translation system 50 according to the above embodiment can make up for the lack of personnel necessary for simultaneous translation.

Note that in the first embodiment, the central server 60 functions as an output buffer between each device. However, the invention is not limited to such embodiments. Separate output buffers may be provided between each device. In this case, these output buffers may be provided in the device on the output side or in the device on the input side. Also, a storage device independent of both the output-side device and the input-side device may be provided between the two.

2nd Embodiment 1 Outline In the above embodiment, it is assumed that there is one editor for each of the first editor, the second editor, and the third editor. However, if multiple editors are available, simultaneous translation can be performed faster and more accurately. The second embodiment is such an embodiment.

2. Hardware Configuration As hardware, the simultaneous translation system according to the second embodiment can use the same hardware as the simultaneous translation system 50 of the first embodiment shown in FIG. Therefore, the description will not be repeated here.

3 Program configuration (1) Differences from the first embodiment The difference from the first embodiment is that any of the speech recognition result editing device 66, the translation unit editing device 70, and the translation result editing device 74 shown in FIG. and that the program executed by the central server corresponding to the central server 60 in the second embodiment has a control structure different from the control structure shown in FIG. Therefore, only the control structure of the program executed by the central server 60 will be described below. It should be noted that among the speech recognition result editing device 66, the translation unit editing device 70, and the translation result editing device 74, the plural types are not limited to any one type. Any combination of these devices may be plural.

(2) Program of Central Server FIG. 14 shows the control structure of the program executed by the central server according to the second embodiment in the form of a flowchart. 14 differs from that shown in FIG. 9 in that the simultaneous translation system includes a plurality of speech recognition result editors 66, translation unit editors 70 or translation result editors 74 instead of step 464 in FIG. , and if the request is determined to be a write request in step 466, before step 468, each device (especially the voice recognition result editing Regarding the data received from the device 66, the translation unit editing device 70, and the translation result editing device 74), the received data is temporarily stored and arranged in order so that the order matches the order in which the data was sent to each device. As a result of step 756 and the processing of step 756, if the received data is all ordered and can be written to the output buffer, the control proceeds to step 468, and if not yet ordered, the data is temporarily stored and the control proceeds to step 468. and step 758 to return to 466. 14, instead of step 480 in FIG. 9, data is read from the position of the read pointer _PR , the value of the read pointer _PR is attached to the data, and the data is transmitted to the device that sent the transmission request. It also differs from FIG. 9 in that step 760 is included. In this embodiment, each device attaches the token indicating the reading order attached to the data received by itself to the data of the processing result and transmits it to the central server 60 .

(3) Differences in Operation When there are a plurality of devices that perform the same processing, for example, when there are a plurality of translation result editing devices 74 in the system, the translation results are translated into the translation results of the plurality of translation results at step 760 in FIG. They are sorted and transmitted to the editing device 74 and edited by each editor. That is, the first time step 760 is executed, the data is sent to one editor's device, and the next time the data is sent to another editor's device. In this case, if different editors have different processing speeds, or if communication delays occur for some reason, the editing results are returned from the plurality of translation result editing devices 74 in an order different from the order in which they were sent. Sometimes it comes. In this second embodiment, multiple devices performing the same process share the same output buffer, so storing their outputs in the output buffer in the order they were received ensures that the data is in order when passed to the next process. Back and forth.

Therefore, in step 756 of this embodiment, the token attached to the data received so far is compared with the token of the newly received data, and if the two are consecutive, the newly received data is output. Store in a buffer, otherwise temporarily store in a separate storage area from the output buffer until data with tokens for that discontinuous portion is received. Then, when data with tokens to fill the discontinuous portion is received, the data is stored in the output buffer and the temporarily stored data is subsequently stored. Of course, there may be cases where there are multiple pieces of data to be temporarily saved.

By doing this, when the edited data is sent back in a certain order to multiple editors who perform the same processing, and the edited data is returned in a different order than when it was sent, the edited data will be sent in the correct order. You can rearrange the order of the data and store it in the buffer. Therefore, there is no adverse effect on subsequent processing. In addition, since the editing process is performed by a plurality of editors, the amount of data processed by each editor is reduced, the processing speed is improved as a whole, and the burden on each editor is reduced.

In this case, it is possible that the tokens for replenishing the discontinuous portion may not be sent within the predetermined time. In such a case, in order to keep the processing speed, it is possible to store the temporarily stored data in the output buffer even if all the data is not complete. In this case, when the missing data is received later, the data can be ignored.

Note that in the second embodiment, the central server 60 implements the output buffer of each device in the form of a ring buffer using the memory area of the central server 60 . However, the invention is not limited to such embodiments. The central server 60 may implement the output buffer of each device in the form of a database.

Third Modification 1 First Modification FIG. 15 shows an editor screen of the translation result editing device that changes from the translation result editor screens 350 and 400 shown in FIGS. 7 and 8 in the translation result editing device 74 according to the first embodiment. 800 is shown. In this example, all the translation results to be processed are processed on one screen without dividing the processing like the editor screen 350 and the editor screen 400 for details of the translation results.

A. Edit Screen Referring to FIG. 15, editor screen 800 includes a display field 814 for displaying an object to be edited, an end button 810 and a “next” button 812 .

The display field 814 displays sentence display portions 820, 822, 824, 826, etc. each having three sentence units each including an original sentence, a translated sentence, and a reverse translation of the translated sentence. When these sentence display portions cannot be displayed in the display field 814, a scroll button 828 is displayed at the right end of the display field 814, and by operating the scroll button 828, all translated sentences to be processed can be displayed and processed. .

In the sentence display section 820 and the like, the translated sentence portion (translated sentence edit field) can be edited. In addition, it is possible to move between the respective translation edit fields by, for example, tab keys, arrow keys, mouse operation, or the like. Therefore, if necessary, all translated sentences can be updated at once by correcting all translated sentences at once and then clicking the “Next” button 812 .

B. Program Referring to FIG. 16, the control structure of the program for realizing this modified example is shown in the form of a flow chart. The flowchart shown in FIG. 16 is a modification of that shown in FIG. FIG. 16 differs from FIG. 12 in that when data (a pair of an original text and its translated text) is received in step 650, each translated text contained in the received data is back-translated, for example, by the automatic translation device 78 shown in FIG. a step 850 for generating and displaying an editor screen 800 shown in FIG. including step 854 which waits for an instruction by and advances the control to step 666 when the instruction is received; and step 856 for executing the processing including editing of the entire sentence display portion 824 and returning control to step 854 .

In this modified example, each sentence is corrected and reflected on the editor screen 800 at once, so there is an advantage that necessary corrections can be made at once.

2 Second Modification The second modification is a modification of FIGS. 17 and 18, similar to the first modification, and is an example in which all the original sentences, translated sentences, and their reverse translations are displayed on one screen in the translation result editing device. be.

A. Edit Screen Referring to FIG. 17, an editor screen 900 of the translation result editing apparatus according to the second modified example includes, as in FIG. ' button 812 is displayed.

In the display field 910, sentence display portions 920, 922, 924, 926, etc. are displayed, each having three sentences each including an original sentence, a translated sentence, and a reverse translation of the translated sentence, as in the first modification. In each of these, the original text, the translated text, and the reverse translation of the translated text are displayed in this order from the top. 15 is also the same as FIG.

However, in this example, a button called an OK button is displayed at the left end of the sentence display area. For example, an OK button 930 is displayed at the left end of the sentence display portion 920 . This OK button 930 is for indicating that the translation text displayed in the text display section 920 has been edited.

In this embodiment, only the translation edit field in the sentence display section 920 is editable, and the other translation edit fields are not editable. When the editor edits the translation and clicks the OK button 930, the OK button 930 is highlighted and the translation edit field is disabled. Then, the translation editing field of the next sentence display section 922 becomes editable.

In the example shown in FIG. 17, editing of the sentence display portions 920 and 922 has been completed, and the sentence display portion 924 is currently being edited.

This process is performed for all the translated texts, and after all the translated texts are highlighted by clicking the OK button, the editing is completed by clicking the "Next" button 812 . Therefore, in this embodiment, the "Next" button 812 may not be clickable until all OK buttons are highlighted.

B. Program Referring to FIG. 18, the program for realizing the translation result editing apparatus according to this modification differs from that shown in FIG. 16 in that the display screen shown in FIG. a step 950 of creating an OK flag array for each translated text and initializing all values to 0 (before approval); a step 954 for branching the flow of control depending on whether or not the instruction is the OK button when the determination in step 666 is NO; and when the determination in step 954 is affirmative, the Step 956 of substituting "1" (after approval) into the element of the OK flag array corresponding to the translation, and as a result of the processing in step 956, the value of the element of the OK flag array corresponding to all the sentence display parts is "1". '', and when the determination in step 958 is negative, the editing of the sentence display portion being processed is disabled and the editing of the next sentence display portion is enabled. and step 960 which edits and redisplays editor screen 900 and returns control to step 854 . If the determination in step 954 is negative, control proceeds to step 678, and the rest is the same as in FIG. If the determination in step 958 is affirmative, it means that editing for all translations has been completed, so control proceeds to step 668 to transmit the editing results to central server 60 and control returns to step 658 .

C. Operation As is clear from the above description, in this second modification, as in the first modification, all original texts, translated texts, and their reverse translations are displayed on one screen. However, unlike the first modified example, editing is completed for each translated text, and editing of the next translated text cannot be completed until all preceding translated texts have been edited. Then, when the editing of all translated sentences is completed, the edited result is transmitted to the central server 60 .

According to the second modified example, as in the first modified example, the editor can edit the translated text while comparing the original text and the reverse translation for the entire translated text. Furthermore, in this second modification, translations are edited one by one, unlike the first modification. Therefore, there is also the effect that the editor can concentrate his attention on one translation to be processed. In addition, in this second modification, once editing of one translated sentence is completed, it is not possible to go back to editing that translated sentence. Therefore, there is also the effect of reducing the possibility that the flow of translation will stagnate.

3 Third Modification a Edit Screen In this third modification, the edit screen shown in FIG. 17 is used as in the second modification.

B. Program FIG. 19 is a flow chart showing the control structure of a program that implements the translation result editing apparatus according to the third modification. 19, this program differs from the second modification shown in FIG. 18 in that after step 850, step 1000 for generating the display screen shown in FIG. is included in step 1002 for executing step 1004 in order from the first line to the last line for the translated text. Unlike FIG. 18, in this embodiment, at step 1000, the process of displaying the first row so that it can be entered is not performed. Instead, within step 1004, the screen is displayed after enabling input for each line to be processed.

Step 1004 activates (makes input possible) the OK button and the translation edit field to be processed, and deactivates (makes input impossible) the other lines to update the display screen. A step 1022 that waits until an instruction is received and branches the control according to the instruction, and in response to the received instruction being an OK button, the translation of the edited result of the corresponding line (the line being processed) is integrated. Step 1026 of deleting the processed line from the editing screen, and Step 1028 of redisplaying the editing screen to reflect the changes and ending the processing of step 1004.

Step 1004 further includes step 1030 of executing processing according to the operation when the instruction received in step 1022 is editing, and step 1030 of updating the screen display after step 1030 so that the editing result is reflected on the screen and control is executed. Step 1032 for returning to 1022, and Step 1034 for performing necessary post-processing and terminating the execution of this program when the instruction received in step 1022 is to terminate processing.

C. Operation In the second modified example, editing is performed line by line, but all the editing results are collectively sent to the central server 60 at the end. In this third modification, the initial screen display is the same as in the second modification. However, the translated text is made editable in order from the top, and when the editing of a certain translated text is completed and the OK button is pressed, only the data of the translated text is immediately transmitted to the central server 60 as the data after editing in response. This is different from the second modified example in that respect. After that, the display related to the translation is erased from the screen. Therefore, the editor's attention can be focused on the translation to be processed more than in the second modification. Since the editor cannot reconsider the translation that has already been edited, there is an effect that the risk of the flow of simultaneous translation becoming stagnant can be made even smaller than in the second modification.

Note that the screen examples used in the above embodiments are merely examples. An editing screen different from the screen examples described above may be adopted.

Also, in the above embodiment, at least three editors are used from the first to the third. However, the invention is not limited to such embodiments. If the reliability of the speech recognition result is very high, there is a possibility that the first editor can be omitted. Also, when the accuracy of dividing the speech recognition result into chunks is very high, there is a possibility that the second editor can be omitted. Of course, even when the accuracy of each process is high, it is preferable from the standpoint of reliability to assign an editor. Further, in the above embodiment, the first editor and the second editor have knowledge of at least the first language, and the third editor has knowledge of both the first language and the second language. However, according to the present invention, the third editor does not need to have a high ability as a simultaneous interpreter. Therefore, a person who has knowledge of both the first language and the second language can play any role from the first editor to the third editor. Therefore, the work sharing may be switched among these editors.

The embodiments disclosed this time are merely examples, and the present invention is not limited to the embodiments described above. The scope of the present invention is indicated by each claim in the scope of claims after taking into consideration the description of the detailed description of the invention, and all changes within the meaning and range of equivalents to the wording described therein include.

50 Simultaneous translation system 60 Central server 62 Network 64 Speech recognition device 66 Speech recognition result editing device 68 Speech segmentation device 70 Translation unit editing device 72, 78 Automatic translation device 74 Translation result editing device 76 Speech synthesis device 100 Simultaneous translation processing 210, 302 Edit field 212 Audio signal display section 352 Original text field 354 Translation edit fields 358, 410 Original text 360, 414 Translation text 412 Reverse translation 814, 910 Display fields 820, 822, 824, 826, 920, 922, 924, 926 Text display section

Claims (6)

A simultaneous translation system for translating speech in a first language into a second language, comprising:
chunk string output means for recognizing the speech signal in the first language, dividing it into chunks, and outputting a chunk string consisting of chunks in the first language;
automatic translation means for automatically translating each of the chunks in the first language output by the chunk sequence output means into the second language and outputting the chunks in the second language;
a first editing of the chunk in the second language so that the chunk in the second language output by the automatic translation means can be compared with the chunk in the first language corresponding to the chunk in the second language; a post-editing means for outputting the edited text in the second language in response to receiving an instruction to end editing; . The simultaneous translation system further includes reverse translation means for translating the chunks of the second language into the first language;
2. The simultaneous translation system according to claim 1, wherein said display device further displays a back-translation result by said back-translation means. The post-editing means further includes matching degree calculation means for calculating a degree of matching between the translation result by the reverse translation means and the chunk in the first language, which is the original text of the chunk in the second language,
3. The simultaneous translation system of claim 2, wherein the display device further displays the degree of matching as a degree of confidence in the translation of the second language chunk. The simultaneous translation system includes first and second post-editing means,
The simultaneous translation system further sorts and inputs the output of the automatic translation means to the first or second post-editing means, and outputs the outputs of the first and second post-editing means for the input in the correct order. 4. The simultaneous translation system according to any one of claims 1 to 3, comprising a first integration means for integrating with. The chunk string output means is
speech recognition text output means for recognizing speech signals in the first language and outputting text in the first language;
automatic text division means for automatically dividing the text in the first language output from the speech recognition text output device into the chunks and outputting the chunk string in the first language;
a display device for displaying the chunk string in the first language output by the automatic text division means so that a second editor can edit the chunks, and responds to an instruction to finish editing by the second editor; 5. The simultaneous translation system according to any one of claims 1 to 4, further comprising chunk string editing means for outputting the displayed chunk string in the first language. A simultaneous translation method for translating speech in a first language into a second language, comprising:
a computer performing voice recognition on the first language audio signal, dividing it into chunks and outputting a chunk sequence of chunks in the first language;
a computer performing an automatic translation into the second language for each of the chunks in the first language and outputting chunks in the second language;
so that the computer can compare the chunks in the second language with the chunks in the first language corresponding to the chunks in the second language, and the editing by the first editor to the chunks in the second language displaying on a display device such that
a computer accepting edits to the chunk in the second language by the first editor;
and a computer, in response to receiving an instruction to end editing, outputting said second language text edited in said step of accepting edits.

Images