TW201941084A - Translation system, translation method, translation device, and speech input/output device - Google Patents

Abstract

The present invention provides a translation method comprising: a step wherein a translation device 3 identifies a first language used by a first user and a second language used by a second user; a step wherein a first user terminal U1 used by the first user receives speech input in the first language; a step wherein the first user terminal converts the speech in the first language into text data and transmits the obtained text data to the translation device 3; a step wherein the translation device 3 converts the text data into translation data in the second language; a step wherein the translation device 3 transmits the translation data to a second user terminal used by the second user; and a step wherein the second user terminal outputs a speech in the second language into which the translation data has been converted.

Description

Translation system, translation method, and translation device

The invention relates to a translation system, a translation method, a translation device, and a voice input / output device for translating an input voice.

Conventionally, a technique for translating an input voice is known.

Japanese Patent Application Laid-Open No. 2013-201505 discloses a technique of translating an input voice and generating a character code string, and transmitting the character code string to a device on the communication partner side, and in a device on the communication partner side Display the character code string in the form of subtitles.

[Problems to be Solved by the Invention]

In the prior art, the translated character code string is displayed in the form of subtitles. Therefore, people who communicate must visually confirm the subtitles displayed on the display. In this case, there is a problem that conversation cannot be performed while observing the expression of the other party.

Therefore, the present invention has been made in view of these aspects, and an object thereof is to provide a translation system, a translation method, a translation device, and a voice input / output device that can improve the quality of communication with a counterpart in a different language. [Means for solving problems]

A translation system according to a first aspect of the present invention includes a first user terminal used by a first user and a translation device capable of communicating with the first user terminal. The first user terminal includes: a voice input unit that receives input of a voice in a first language; and a terminal transmission unit that converts the voice in the first language received by the voice input unit into a first Text data is sent to the translation device; a terminal receiving unit receives, from the translation device, the translation of the language information of the second language input in the second user terminal used by the second user into the first language A first language translation material obtained from one language; and a sound output unit that outputs a sound of the first language obtained by converting the first language translation material received by the terminal receiving unit.

The translation device includes a determination unit that determines a first language used by the first user and a second language used by the second user; and a device receiving unit that receives the first language from the first user terminal. Language text data; a language conversion unit that converts the first language text data received by the device receiving unit into a second language translation data; and a device transmission unit that sends the second user terminal to the second user terminal Language translation materials.

Alternatively, the language conversion unit converts the second language translation data into the first language to generate retranslated data, and the device transmitting unit sends the retranslated data to the first user terminal, The voice output unit outputs the retranslated data while the voice input unit receives an input of a voice in the first language.

Alternatively, the device transmitting unit may suspend the language conversion unit when the language conversion unit has converted the language information of the second language input in the second user terminal into the first language translation data. The retranslated material is transmitted, and the first language translated material is transmitted.

Alternatively, after the retranslated material is transmitted to the first user terminal, the device receiving unit may receive the first language text material including a predetermined word, and the device transmitting unit may stop sending the data to the first user terminal. The second user terminal sends the second language translation material.

It may also be that the device receiving unit receives second language text data obtained by converting the sound of the second language from the second user terminal, and the language converting unit converts all the data received by the device receiving unit. The second language text material in the second language is converted into the first language translation material. When the first language translation material is sent, the language conversion unit ends the generation of the new first language translation material. At this time, the device transmitting unit suspends the transmission of the first language translation material in the transmission process and starts the transmission of a new first language translation material.

The sound output unit may include a bone conduction speaker, and the sound input unit may output the sound of the first language from the bone conduction speaker while the sound input unit receives the input of the sound of the first language.

It may also be that the sound input unit has a plurality of main microphones for acquiring sounds and a sub microphone for acquiring surrounding sounds, and the terminal transmitting unit removes the sounds obtained from the plurality of main microphones. The text data corresponding to the sound after the surrounding sound acquired by the sub microphone is transmitted as the first language text data corresponding to the sound in the first language.

Alternatively, in a state in which the first user terminal is worn by a user, a distance between the sub microphone and the mouth of the first user is greater than the plurality of main microphones and the mouth of the first user the distance between.

The first user terminal may further include an extraction unit that compares the sound input to each of the plurality of main microphones with the sound input to the sub microphones, and extracts data from the plurality of main microphones. The sound input in at least one of the microphones extracts the sound of the first language; and a text conversion unit that converts the sound of the first language extracted by the extraction unit into the first language text material .

Alternatively, the extraction unit may determine a sound estimated to be the sound of the first language based on a result of comparing the sounds input to each of the plurality of main microphones, and based on the determined sound and As a result of comparing the sounds inputted in the sub microphones, the sounds of the first language are extracted.

Alternatively, the terminal transmitting unit may send user identification information for determining the second user to the translation device in association with the first language text data, and the determining unit may refer to A memory unit that memorizes the languages used by the plurality of users in a manner associated with the plurality of users, and determines to be associated with the second user indicated by the user identification information sent by the terminal transmitting unit. Ground in the second language of the memory unit.

Alternatively, the terminal sending unit may send the language information used to determine the language of the second user to the translation device before the sending of the first language text material is started, and the determining unit is based on the The second language is determined by the language information sent by the terminal transmitting unit.

The determination unit may determine the second language based on a word included in the first language text.

A translation method according to a second aspect of the present invention includes a step of a translation device determining a first language used by a first user and a second language used by a second user; and a first user terminal used by the first user accepts the first language A step of inputting a voice of the user; a step of the first user terminal converting the voice of the first language into a text material, and sending the converted text material to the translation device; the translation device The step of converting the text material into the translation material in the second language; the step of the translation device transmitting the translation material to a second user terminal used by the second user; and the second user terminal outputting a pair A step of converting the translation data into a voice in the second language.

A translation apparatus according to a third aspect of the present invention includes: a determining unit that specifies a first language used by a first user and a second language used by a second user; and a device receiving unit, from the first language used by the first user. The user terminal receives the first language text data obtained by converting the sound in the first language; the language conversion unit converts the first language text data received by the device receiving unit into a second language translation data And a device transmitting unit that transmits the second language translation data to a second user terminal used by the second user.

A voice input / output device according to a fourth aspect of the present invention includes a voice input unit that receives voice input in a first language, and a terminal transmission unit that converts the voice in the first language received by the voice input unit. The obtained text data is sent to a translation device that translates the first language into a second language; and a terminal receiving unit receives, from the translation device, the first text inputted in the second user terminal used by the second user. A first language translation material obtained by translating language information in two languages into the first language; and a sound output unit, during inputting sound to the sound input unit, outputting the received information to the terminal receiving unit The sound in the first language obtained by converting the first language translation data. [Effect of the invention]

According to the present invention, there is an effect that the quality of communication with a partner using a different language can be improved.

[Outline of Translation System S1] FIG. 1 is a diagram showing a configuration of a translation system S1 of the present embodiment. The translation system S1 includes a head-mounted device 1 (1a, 1b, 1c) and an information terminal 2 (2a, 2b, 2c), and a translation device 3 that functions as a translation device. The headset 1 and the information terminal 2 function as a user terminal which is a sound input / output device by operating in cooperation.

In FIG. 1, a user U1 who speaks a first language (for example, Japanese), a user U2 who speaks a second language (for example, English), and a user U3 who speaks a third language (for example, Chinese) are shown. In order to enable multiple users who speak different languages to understand the words spoken by other users, the translation system S1 translates the words spoken by each user into words spoken by other users, and translates the translated words into Output in the form of sound. In the following description, it is assumed that the user U1 is a guide that guides tourists at a tourist attraction, and the users U2 and U3 are tourists who receive a description of the user U1.

In the example shown in FIG. 1, the user U1, the user U2, and the user U3 are respectively wearing a head-mounted device 1a, a head-mounted device 1b, and a head-mounted device 1c. In addition, the user U1, the user U2, and the user U3 respectively hold an information terminal 2a, an information terminal 2b, and an information terminal 2c. The information terminal 2a, the information terminal 2b, and the information terminal 2c are, for example, smart phones. The head-mounted device 1a, the head-mounted device 1b, and the head-mounted device 1c can transmit and receive data to and from the information terminal 2a, the information terminal 2b, and the information terminal 2c via the wireless channel B1, the wireless channel B2, and the wireless channel B3, respectively. The wireless channel is, for example, Bluetooth (registered trademark).

The head-mounted device 1a, the head-mounted device 1b, and the head-mounted device 1c have the same configuration. In the following description, when it is not necessary to distinguish each one, the head-mounted device 1 may be referred to as the head-mounted device 1 in some cases. Similarly, the information terminal 2a, the information terminal 2b, and the information terminal 2c have the same configuration. In the following description, the information terminal 2 may be referred to as the information terminal 2 if there is no need to distinguish between various cases.

The head-mounted device 1 is configured to be worn on the head by a user, to accept input of voices spoken by the user, and to convert the input voices into digital voice signals. The head-mounted device 1 sends a digital audio signal to the pre-associated information terminal 2 via the wireless channel B1.

The information terminal 2 recognizes the sound contained in the digital sound signal received from the head-mounted device 1 and converts it into text data. When a voice in a first language is input into the head-mounted device 1, the information terminal 2 creates a text material in the first language. For example, when a Japanese voice is input into the head-mounted device 1, the information terminal 2 creates a Japanese text material.

The information terminal 2 sends the created text material in the first language to the translation device 3. In the example shown in FIG. 1, the information terminal 2 sends the text material in the first language to the translation device 3 via the wireless communication line W, the access point 4 and the network N. The wireless communication line W is, for example, a Wi-Fi (registered trademark) line or a line using another wireless communication method.

After receiving the text material in the first language, the translation device 3 converts the received text material in the first language into the text material in the second language determined in advance. The translation device 3 generates, for example, a text material in English that can be understood by the user U2 selected as the counterpart of the user U1 after receiving the Japanese text material by the information terminal 2a used by the Japanese-speaking user U1. In this specification, the text data generated by the translation device 3 is referred to as translation data. The translation device 3 sends the created translation material in the second language to the information terminal 2b.

The information terminal 2b converts the received text data in the second language into a digital audio signal, and sends the converted digital audio signal to the head-mounted device 1b via the wireless channel B2. The head-mounted device 1b converts the received digital sound signal into an analog sound signal and outputs it in a manner that can be recognized by the user U2.

The translation device 3 generates the text data in the second language, converts the text data in the second language into the text data in the first language, and generates retranslated data in the first language. The translation device 3 sends the created retranslated material to the information terminal 2a. The information terminal 2a converts the received retranslated data in the first language into a digital audio signal, and sends the digital audio signal to the head-mounted device 1a via the wireless channel B1. The head-mounted device 1a converts the received digital sound signal into an analog sound signal and outputs it in a manner that can be recognized by the user U1.

For example, after the head-mounted device 1a sends the text data based on the sound input by the user U1 to the translation device 3 to the translation device 3, after the time required for the translation device 3 to generate the retranslated data, the output is Retranslated data sounds. The predetermined period is, for example, a period that is set when the user U1 operates the information terminal 2 a and a utterance to be translated is input. The details of the operation for the user U1 to set a predetermined period will be described later.

By making the retranslated data by the translation device 3 and outputting the sound based on the retranslated data by the head-mounted device 1, the user U1 can compare his own utterance with the utterance indicated by the sound based on the retranslated data to confirm himself Whether the spoken words were translated correctly. Therefore, when the user U1 confirms that the translation is not correctly performed, the user U1 and the user U3 and the user U3 who are the counterparties of the conversation can make a gesture or make a change.

As described above, the translation system S1 will output the text data obtained by translating the sound input in the head-mounted device 1 from the head-mounted device 1 in the form of sound, so the user wearing the head-mounted device 1 can use it while watching Speak with your opponent's face in a different language to understand what the other person is saying. Therefore, the translation system S1 can improve the quality of communication with counterparts who use different languages.

Details will be described later, and the head-mounted device 1 includes a bone conduction speaker. Therefore, the user can listen to the original voice of the conversation partner through the ear, and listen to the content obtained by translating the voice produced by the other party through the bone conduction speaker and the content obtained by re-translating the voice emitted by the user. With the above-mentioned configuration of the head-mounted device 1, the user can listen to the original voice of the other party with his ears, and listen to the translated voice through the bone guide, so that he can grasp the emotion of the other party and understand what the other party is saying. As a result, the quality of communication with a partner using a different language can be further improved. Hereinafter, details of the head-mounted device 1, the information terminal 2, and the translation device 3 will be described.

[Appearance of Head Mounted Device 1] FIG. 2 is a diagram showing the appearance of the head mounted device 1. As shown in FIG. 2, the headset 1 includes a first main microphone 11, a second main microphone 12, a sub microphone 13, a bone conduction speaker 14, a control unit 15, a cable 16, a microphone receiving portion 17, a connection member 18, and a main body. Department 19.

The first main microphone 11, the second main microphone 12, and the sub microphone 13 function as a sound input unit. The first main microphone 11 and the second main microphone 12 are main microphones for acquiring sounds emitted by the user U, and the sub microphone 13 is a sub microphone for acquiring surrounding sounds. In a state where the user U is wearing the head-mounted device 1, the distance between the sub microphone 13 and the mouth of the user U is greater than the distance between the first main microphone 11 and the second main microphone 12 and the mouth of the user U, which are a plurality of main microphones. Distance.

The first main microphone 11 and the second main microphone 12 are juxtaposed in a microphone accommodating portion 17 connected to the main body portion 19 via a flexible connecting member 18. On the other hand, the sub microphone 13 is provided in the vicinity of the bone conduction speaker 14 which is worn in contact with the vicinity of the lower portion of the user's ear. The user uses the microphone receiving portion 17 near the mouth. Therefore, the first main microphone 11 and the second main microphone 12 acquire the sound emitted by the user U at a position closer to the mouth of the user U than the sub microphone 13. the sound of.

The sub microphone 13 is provided on the outside of the bone conduction speaker 14L on the opposite side from the bone conduction speaker 14R on the side to which the connection member 18 is connected (that is, on the side not in contact with the user U). In this way, the sub microphone 13 is provided in the bone conduction speaker 14L which is electrically away from the first main microphone 11 and the second main microphone 12 compared to the bone conduction speaker 14R. Therefore, the sub microphone 13 is input to the first main microphone 11 and the second main microphone The sound signal of 12 and the sound signal input to the sub microphone 13 do not easily interfere with each other, so the noise removal performance described later can be improved.

The bone conduction speaker 14 is a speaker that can transmit sound to the user U by vibrating the bone using sound pressure. The bone conduction speaker 14R is, for example, worn so as to contact the ridge-like protrusions on the lower part of the user's right ear, the bone conduction speaker 14L is worn so as to contact the ridge-like protrusions on the lower part of the user's left ear, the bone-conduction speaker 14R and the bone conduction The wearing position of the speaker 14L is arbitrary. The bone conduction speaker 14 outputs the sound of the first language while the first main microphone 11 and the second main microphone 12 receive the sound of the language (for example, the first language) used by the user. The sound in the first language output from the bone conduction speaker 14 is a sound based on a material obtained by translating the sound of another user who speaks the second language, or a sound based on a retranslated material.

The head-mounted device 1 has a bone conduction speaker 14 so that the user can listen to the other party's original voice with his ear and listen to the translated voice through the bone conduction, so he can grasp the emotion of the other party to understand what the other party is saying.

The control unit 15 houses various electrical circuits electrically connected to the first main microphone 11, the second main microphone 12, the sub microphone 13, the bone conduction speaker 14R, and the bone conduction speaker 14L via a cable 16. The electrical circuit includes, for example, a circuit for removing noise inputted from the first main microphone 11, the second main microphone 12, and the sub microphone 13, and extracting a voice input section functioning from a user to function, and converting an analog voice signal. It is a codec circuit for digital audio signals, and a communication circuit for transmitting and receiving digital audio signals to and from the information terminal 2.

[Internal Configuration of Headset 1 and Information Terminal 2] FIG. 3 is a diagram showing the internal configuration of the headphone 1 and the information terminal 2. The control unit 15 includes a sound processing unit 151, a communication unit 152, and a control unit 153. In the following four descriptions related to the configuration of the head-mounted device 1, the head-mounted device 1 is used as the head-mounted device 1a used by the user U1 who speaks the first language.

The sound processing unit 151 extracts the sounds emitted by the user U1 by removing surrounding sounds other than the sounds emitted by the user U1 based on the sound signals input from the first main microphone 11, the second main microphone 12, and the sub microphone 13. Extraction section to function. The audio processing unit 151 generates a digital audio signal by performing pulse code modulation (PCM) encoding on the extracted audio, for example. The audio processing unit 151 inputs the generated digital audio signals to the communication unit 152.

The audio processing unit 151 decodes the digital audio signal input from the communication unit 152 to generate an analog audio signal. The sound processing unit 151 outputs the generated analog sound signal through the bone conduction speaker 14.

Furthermore, the sound processing unit 151 has a function of removing noises such as ambient sound from the sound signals input from the first main microphone 11 and the second main microphone 12. In order to remove noise, the sound processing unit 151 first determines the sound of the sound estimated to be the first language based on a result of comparing the sounds input to each of the first main microphone 11 and the second main microphone 12.

The audio processing unit 151 extracts, for example, a signal in a predetermined range from a phase difference between the analog audio signal input to the first main microphone 11 and the analog audio signal input to the second main microphone 12, and removes the difference in amplitude to a predetermined value. A signal outside the range is used to generate an estimated sound signal obtained by extracting the sound of the sound estimated to be the first language. The predetermined range is, for example, the distance from the mouth of the user U to the first main microphone 11 and the distance from the mouth of the user U to the second main microphone 12 when the user U is wearing the head-mounted device 1. The range below the difference in the conceivable amplitude at the maximum conceivable maximum.

Then, the sound processing unit 151 determines the sound based on the comparison between the analog sound signal input to the first main microphone 11 and the analog sound signal input to the second main microphone 12, and the sound input to the sub microphone 13. As a result of the comparison, the sound of the first language is extracted from the first sound signal after the components of the surrounding sound are removed. Specifically, the sound processing unit 151 removes from the generated estimated sound signal a component of at least a part of the surrounding sound signal input to the sub microphone 13.

Before removing the surrounding sound signal from the estimated sound signal, the sound processing unit 151 inputs the signal to the sub microphone so that the amount of attenuation of the signal which is clearer than the level of the user U's sound in the estimated sound signal is within a predetermined range. The surrounding sound signal attenuates. By doing so, the sound processing unit 151 can remove the components of the surrounding sound to the maximum while keeping the components of the sound emitted by the user as much as possible.

The communication unit 152 includes a wireless communication module for transmitting the digital sound signals generated by the sound processing unit 151 to the information terminal 2 and receiving the digital sound signals from the information terminal 2. The communication unit 152 functions as a terminal transmission unit that transmits the first language text data obtained by converting the sound in the first language received by the first main microphone 11 and the second main microphone 12 to the translation device 3. In addition, the communication unit 152 functions as a terminal receiving unit that receives the first language translation data obtained by translating the language information of the second language input in the headphone 1b used by the user U2 from the translation device 3 into the first language. Features. Furthermore, the communication unit 152 receives the retranslated data obtained by translating the second language translation data obtained by translating the first language text data transmitted from the communication unit 152 to the translation device 3 into the first language.

The control unit 153 is, for example, a one-chip microcomputer (Central Processing Unit, CPU), Read Only Memory (ROM), and Random Access Memory (RAM). ). The control unit 153 executes the program stored in the ROM by the CPU, and writes the data values described in the program to the registers of the sound processing unit 151 and the communication unit 152, so that the sound processing unit 151 and the communication The unit 152 performs control.

Next, the configuration of the information terminal 2 will be described. The information terminal 2 includes a first communication section 21, a second communication section 22, a display section 23, an operation section 24, a memory section 25, and a control section 26. The control unit 26 includes a text conversion unit 261 and a user interface (UI) processing unit 262.

The first communication unit 21 includes a wireless communication module for receiving digital audio signals from the head-mounted device 1 via the wireless channel B, and transmitting the digital audio signals output from the audio processing unit 151 to the head-mounted device 1. The second communication unit 22 includes a wireless communication module for transmitting and receiving text data to and from the access point 4 via the wireless communication line W. The second communication unit 22 uses, for example, text data corresponding to the sound input from the first main microphone 11 and the second main microphone 12 after removing the surrounding sound input to the sub microphone 13 as the sound corresponding to the first language. Send the first language text.

The display unit 23 is a display that displays various information. The display unit 23 displays the first language text data generated by the text conversion unit 261 based on the digital sound signal transmitted from the head-mounted device 1 under the control of the UI processing unit 262, and the second communication unit 22 receives the text data from the translation device 3. 1st language translation. Examples of the materials displayed on the display unit 23 will be described later.

The operation unit 24 is a device for receiving a user's operation, and is, for example, a touch panel provided on the display unit 23. The operation unit 24 inputs an electric signal generated by a user's operation to the UI processing unit 262.

The storage unit 25 is a storage medium such as a ROM or a RAM. The memory unit 25 memorizes a program executed by the control unit 26. Then, the memory unit 25 memorizes a language name that can be spoken by a user using the information terminal 2 inputted through the operation unit 24. Further, a text recognition dictionary 261 for converting digital audio signals into text data and a speech synthesis dictionary for converting text data into digital voice signals are stored. The storage unit 25 stores a plurality of speech recognition dictionaries and a plurality of speech synthesis dictionaries in association with a plurality of language names.

The control unit 26 is, for example, a CPU, and functions as a text conversion unit 261 and a UI processing unit 262 by executing a program stored in the storage unit 25.

The text conversion unit 261 converts the voice in the first language extracted by the voice processing unit 151 that functions as an extraction unit into the first language text material. Specifically, first, the text conversion unit 261 analyzes the digital audio signal input from the first communication unit 21 to determine a phoneme. Then, the text conversion unit 261 refers to the memory unit 25 and uses a voice recognition dictionary corresponding to a language name that can be spoken by a user using the information terminal 2 to determine words included in the digital voice signal, thereby converting the digital voice signal. Converted to 1st language text.

The text conversion unit 261 associates the account name of the user identification information for identifying the user and the account names of the users U2 and U3 as the counterparty with the second communication unit 22 in association with the generated first language text material. Send to translation device 3. The text conversion unit 261 transmits the account names of the user U2 and the user U3 at the same time as transmitting the first language text material in response to receiving the instruction to start the translation of the input voice from the UI processing unit 262, for example. In addition, when the text conversion unit 261 obtains the second language name that can be used by the user U2 via the UI processing unit 262, it can also determine the language of the language that can be used by the user U2 before starting to send the first language text material. The language information is sent to the translation device 3.

Then, the text conversion unit 261 converts the first language text data received from the translation device 3 via the second communication unit 22 into a digital audio signal by referring to the speech synthesis dictionary stored in the storage unit 25. The text conversion unit 261 sends the generated digital audio signal to the head-mounted device 1 via the first communication unit 21.

The UI processing unit 262 displays the text data acquired from the text conversion unit 261 on the display unit 23. The UI processing unit 262 determines the operation content indicated by the electrical signal input from the operation unit 24 and notifies the text conversion unit 261 of the determined operation content. For example, the UI processing unit 262 notifies the text conversion unit 261 of the account name of the user U1 and the account names of the users U2 and U3 set by the user, and causes the memory unit 25 to memorize it.

[Configuration of Translation Apparatus 3] FIG. 4 is a diagram showing a configuration of the translation apparatus 3. The translation device 3 includes a communication section 31, a memory section 32, and a control section 33.

The communication unit 31 includes a communication interface for transmitting and receiving text data between the network N and the information terminal 2. The communication unit 31 includes, for example, a local area network (LAN) controller.

The storage unit 32 includes a storage medium such as a ROM, a RAM, and a hard disk. The storage unit 32 stores programs executed by the control unit 33. Further, the memory unit 32 stores a dictionary for converting the text data in the first language into text data in other languages. Further, the storage unit 32 stores a use language table that associates the account names of a plurality of users who can use the translation system S1 with the language names that can be used by each user.

The control unit 33 is, for example, a CPU, and functions as a translation control unit 331 and a language conversion unit 332 by executing a program stored in the storage unit 32.

The translation control unit 331 controls the language conversion unit 332 so as to convert the first language text data received from the communication unit 31 into the second language translation data. The translation control unit 331 controls the language conversion unit 332 so as to convert the second language text data received from the information terminal 2b via the communication unit 31 into the first language translation data. The translation control unit 331 sends the second language translation data generated by the language conversion unit 332 to the information terminal 2b via the communication unit 31, and transmits the first language translation data generated by the language conversion unit 332 through the communication unit 31 And it is sent to the information terminal 2a.

In addition, the translation control unit 331 controls the language conversion unit 332 to translate the acquired second language translation data into the first language after obtaining the second language translation data generated based on the first language text data from the language conversion unit 332. Text material and re-translated material. The translation control unit 331 sends the retranslated data generated by the language conversion unit 332 to the information terminal 2a via the communication unit 31.

Furthermore, the translation control unit 331 functions as a determination unit that specifies a first language used by the user U1 and a second language used by the user U2. The translation control unit 31 refers to the used language table stored in the memory unit 32 to determine the correspondence with the account name of the user U1 sent from the second communication unit 22 via the first communication unit 21 in a manner associated with the first language text. The first language name and the second language name corresponding to the account name of the user U2 received in a manner associated with the first language text. The translation control section 331 notifies the determined result to the language conversion section 332.

The translation control unit 331 may determine the second language that can be used by the user U2 based on the language information transmitted from the information terminal 2a. The translation control unit 331 determines, for example, a first language that can be used by the user U1 and a second language that can be used by the user U2 based on information input when the user U1 using the information terminal 2a performs a log in process. In this case, the translation control unit 331 causes the language conversion unit 332 to perform translation based on the determined first language and second language until the user U1 logs off.

The translation control unit 331 may also analyze the received first language text data to determine that the language used by the user U1 is the first language. Furthermore, the translation control unit 331 may determine the second language based on the words contained in the first language text. The translation control unit 331 determines, for example, that the language used by the user U2 is the second language based on the account name included in the received first language text material. Specifically, the translation control unit 331 detects that “Tom” is the account name included in the language table when the content of the first language text is “Tom, first meeting”, and determines that “Tom” is capable of The language used is English, and it is determined that the second language is English.

The language conversion unit 332 converts the first language text data received by the first communication unit 21 from the information terminal 2a into the second language translation data based on the language type notified from the translation control unit 331. The language conversion unit 332 receives the input of the first language text data from the translation control unit 331, and then generates the second language translation data by translating the input first text language data, and translates the generated second language The data is notified to the translation control section 331. Then, the language conversion unit 332 converts the second language text data received by the first communication unit 21 from the information terminal 2b into the first language translation data. After the language conversion unit 332 receives the input of the second language text data from the translation control unit 331, it translates the inputted second language text data to generate the first language translation data, and translates the generated first language The data is notified to the translation control section 331.

Further, the language conversion unit 332 translates the second language translation data generated based on the first language text data into the first language based on the instruction from the translation control unit 331 to generate retranslated data. The language conversion unit 332 notifies the translation control unit 331 of the retranslated data.

[Operation Screen of Information Terminal 2] Next, an operation when the user U1 starts a conversation with the user U2 and the user U3 will be described. 5 (a) and 5 (b) are diagrams showing a user selection screen displayed on the display unit 23 when the user U1 starts using the information terminal 2a. In the present embodiment, it is assumed that the information terminal 2a is not a terminal of the user U1 but a rented terminal. Therefore, the user U1 first needs to set a language that the user U1 can use.

As shown in FIG. 5 (a), on the user selection screen, the account name and language name of a user who can use the translation service of the translation device 3 are displayed in association with each other. As shown in FIG. 5 (b), the user U1 can use the translation service using the information terminal 2a by touching a check box to the left of his account name "Taro". When the radio button on the left side of "Taro" is touched by the UI processing unit 262, the memory unit 25 remembers that the account name of the user U1 is "Taro" and the language used is Japanese.

FIGS. 6 (a) and 6 (b) are diagrams showing a partner selection screen displayed on the display unit 23 when a partner engaged in a conversation is selected. In FIG. 6A, similar to FIG. 5A, the account name and language name of a user who can use the translation service of the translation device 3 are displayed in association with each other. As shown in FIG. 6 (b), after the user U1 touches the check box to the left of "Tom" and the check box to the left of "Week", the UI processing unit 262 causes the memory unit 25 to memorize the user U2 as "Tom" 2. User U3 is "week".

7 (a) and 7 (b) are diagrams showing a conversation screen displayed on the display unit 23 during a conversation. As shown in FIG. 7 (a), the conversation screen includes: a first area R1, which displays the first language text data obtained by converting the sounds made by the user; a second area R2, which displays the retranslated data; and The third area R3 displays the first language translation data obtained by translating the voice made by the other party.

The conversation screen includes a "speech" icon operated by the user while inputting a voice. The text conversion unit 261 converts the sound input from the headset 1 into the first language text material while the finger touches the "speech" icon, and ends to the first language when the finger leaves the "speech" icon. Conversion processing of text data. Then, the text conversion unit 261 sends the first language text material corresponding to the sound inputted from the time when the finger touches the "talk" icon until the finger leaves, to the translation device 3. By doing so, only the period specified by the user becomes the target of translation, so that it is possible to prevent erroneous translation of the surrounding sound input to the head-mounted device 1 while the user has not input the sound.

As shown in FIG. 7 (b), when the user U1 makes a sound while touching the "speech" icon, a first language text material indicating the content of the sound made by the user U1 is displayed in the first area R1. The user U1 can confirm that the voice is correctly recognized by visually confirming the text material in the first language indicated.

When the information terminal 2a has an error in the text material of the first language visually confirmed by the user U1, the information terminal 2a can also operate by canceling the input voice by the user U1 issuing a predetermined word. For example, when the word "retransmission" is detected in the voice input from the head-mounted device 1a, the text conversion unit 261 starts from touching the "speech" icon until the word "retransmission" is detected. The first language texts generated during the period are deleted. The text conversion unit 261 instructs the UI processing unit 262, and also deletes the first language text data displayed on the display unit 23. By doing so, the text conversion unit 261 can prevent the incorrect first language text material from being transmitted to the translation device 3 if the voice of the user U1 cannot be converted into the correct first language text material.

8 (a) and 8 (b) are diagrams showing a screen of the information terminal 2a and a screen of the information terminal 2b after the user U1 releases his or her finger from the "speech" icon. In the second area R2 of the screen of the information terminal 2a shown in FIG. 8 (a), retranslated materials are displayed. FIG. 8 (b) shows a screen of the information terminal 2b. In the third area R3 of the information terminal 2b, English is obtained by translating the Japanese language "first meet" issued by the user U1.

The user U1 can visually confirm the retranslated material displayed in the information terminal 2a to confirm whether it is correctly translated. When the word "retransmission" issued by the user is detected while the retranslated material is displayed, the text conversion unit 261 may notify the translation device 3 that there is an error in the second language translation material. After the translation device 3 recognizes that there is an error in the translation, the translation device 3 may notify the information terminal 2b of the error in the translation, and the information terminal 2b may erase the English language indicated in the third area R3 and display the display Words that have been canceled (for example, canceled). By doing so, the translation device 3 can prevent the incorrect second language translation data from being continuously displayed on the information terminal 2b if the voice from the user U1 cannot be converted into the correct first language text material.

The translation control unit 331 may also send the second language translation data to the information terminal 2b after waiting for the time required for the user U1 to confirm the content of the retranslated data after the retranslated data is transmitted to the information terminal 2a. In this case, after the translation control unit 331 sends the retranslated data to the information terminal 2a, the communication unit 31 may terminate the text in the first language including the predetermined word (for example, "retransmission"), and then suspend it. Send the second language translation material to user U2. By doing so, the translation device 3 can prevent the incorrect second language translation material from being transmitted to the information terminal 2b if the voice from the user U1 cannot be converted into the correct first language text material.

FIGS. 9 (a) and 9 (b) are screens of the information terminal 2a and the screen of the information terminal 2b after the user U2 speaks a voice in the second language, following the state of FIG. 8 (a) and FIG. Illustration. In the first area R1 of the information terminal 2b shown in FIG. 9 (b), text information in the second language corresponding to the voice made by the user U2 is displayed. The information in the first area R1 of the information terminal 2a shown in FIG. 9 (a) is displayed. The first region R1 displays first language translation data generated by translating the second language text material. In the second area R2 of the information terminal 2b shown in FIG. 9 (b), retranslated data obtained by translating the first language translation data is displayed.

The first language sound corresponding to the retranslated material shown in the second region R2 of FIG. 9 (a) and the first language corresponding to the first language translation material shown in the third region R3 are output from the head-mounted device 1a. Voice of language. At this time, if the user U2 emits the sound of the second language and sends it to the translation device 3 while the sound of the first language is output from the head-mounted device 1a, the translation control section 331 sends the head-mounted device 1b to the language conversion section 332 When the language information input in the second language is converted into the first language translation material, the transmission of the retranslated material is suspended and the first language translation material is transmitted to the information terminal 2a. By doing so, the user U1 can grasp the content of the sounds emitted by the user U2 without giving priority to the content of the data.

Similarly, when the first language translation data based on the voice of the second language issued by the user U2 is transmitted to the information terminal 2a, the language conversion unit 332 newly translates the voice of the second language issued by the user U2 and ends When the new first language translation material is generated, the translation control unit 331 may suspend the transmission of the first language translation material in the transmission process and start transmitting the new first language translation material. By doing so, the user U2 promptly issues a sound of the correct content when a sound containing an erroneous content is issued, thereby transmitting the correct content to the user U1.

While transmitting the first language translation data based on the voice in the second language issued by the user U2 to the information terminal 2a, the language conversion unit 332 newly translated the voice in the third language issued by the user U3, ending the new When the first language translation material is generated, the translation control unit 331 may suspend the transmission of the first language translation material in the transmission process and start transmitting the first language translation material based on the voice of the third language issued by the user U3. In this case, the translation control unit 331 may notify the information terminal 2b that the transmission of the first language translation material based on the voice made by the user U2 has been interrupted, and the information terminal 2b may indicate that the transmission of the first language translation material is interrupted. By doing so, the user U2 can recognize that the content he or she has not communicated to the user U1, and can take appropriate measures such as speaking again.

[Processing Procedure in Translation System S1] FIG. 10 is a diagram showing a processing procedure in the translation system S1. FIG. 10 is from the time point when the user U1 starts inputting the sound of the first language in the head-mounted device 1a (step S11). After the head-mounted device 1a is input with the voice in the first language, it transmits digital audio data corresponding to the voice in the first language to the information terminal 2.

The information terminal 2a converts the received digital audio data into the first language text data (step S12). During this period, the UI processing unit 262 of the information terminal 2a monitors whether the voice input has been completed (step S13), and the text conversion unit 261 continues to generate the first language text material until the voice input is completed. When the text conversion unit 261 determines that the voice input has been completed (YES in step S13), the text conversion unit 261 sends the generated first language text data to the translation device 3 via the second communication unit 22. .

Then, the language conversion unit 332 converts the first language text data received through the communication unit 31 into the second language text data to generate a second language translation data (step S14). The translation control unit 331 sends the second language translation data generated by the language conversion unit 332 to the information terminal 2b via the communication unit 31.

After receiving the second language translation data, the text conversion unit 261 of the information terminal 2b converts the received second language translation data into a digital audio signal in the second language (step S15). The text conversion unit 261 of the information terminal 2b transmits the digital audio signal in the second language to the head-mounted device 1b via the first communication unit 21. The sound processing unit 151 of the head-mounted device 1b converts the digital sound signal received from the information terminal 2b into an analog sound signal, and outputs the sound of the second language from the bone conduction speaker 14 (step S16).

After generating the second language translation data in step S14, the translation control unit 331 causes the language conversion unit 332 to translate the second language translation data into the first language to create the retranslated data (step S17), and sends the retranslated data to the information Terminal 2a. After receiving the retranslated data, the text conversion unit 261 of the information terminal 2a converts the received retranslated data into a digital audio signal in the first language (step S18). The text conversion unit 261 of the information terminal 2a transmits the digital audio signal in the first language to the head-mounted device 1a via the first communication unit 21. The sound processing unit 151 of the head-mounted device 1a converts the digital sound signal received from the information terminal 2a into an analog sound signal, and outputs the sound of the first language from the bone conduction speaker 14 (step S19).

In addition, as described above, the text conversion unit 261 may also send the retranslated data to the information terminal 2a before sending the second language translation data to the information terminal 2b after step S14, and then, after the user U1 confirms the retranslation, After the time required for the document, the second language translation document is sent to the information terminal 2b.

FIG. 11 is a diagram showing a configuration of a translation system S2 according to a modification of the present embodiment. The translation system S2 is different from the translation system S1 in that the user U1 uses a head-mounted device 10 having a part of the functions of the information terminal 2a instead of the head-mounted device 1a. Moreover, the difference from the translation system S1 is that the user U2 and the user U3 do not use the headgear 1b and the headgear 1c, and use the information terminal 20b and the information terminal 20c instead of the information terminal 2b and the information terminal 2c.

The head-mounted device 10 has a function of the text conversion unit 261 in addition to the functions of the head-mounted device 1 shown in FIG. 3. The second communication unit 22 has a function instead of the communication unit 152 in the head-mounted device 1. As a result, the user U1 can use the headphone 10 without using the information terminal 2a to talk with the user U2 using the second language and the user U3 using the third language.

In addition, the head-mounted device 10 may be configured to acquire the operation content in the information terminal 2 connected via a wireless channel, so that the user U1 can use the information terminal 2 to set a time point for starting the translation. Moreover, the head-mounted device 10 may cause the retranslated data and the first language translation data received from the translation device 3 via the access point 4a to be displayed on the information terminal 2 connected via the wireless channel, so that the user U1 can visually confirm.

In addition to the information terminal 2, the information terminal 20 also includes a microphone for inputting sound and a speaker for outputting sound. The user U2 and the user U3 can use the information terminal 20b and the information terminal 20c to talk to the user U1 without wearing the head-mounted device 1b and the head-mounted device 1c.

[Effects of Translation System S1 and Translation System S2] As described above, in the translation system S1 and translation system S2 of this embodiment, the translation device 3 translates the voice of the first language input in the head-mounted device 1a into The sound in the second language is then transmitted to the head-mounted device 1b, and the head-mounted device 1b outputs the sound translated into the second language. Then, the translation device 3 translates the sound of the second language input into the head-mounted device 1b into the sound of the first language and sends it to the head-mounted device 1a, and the head-mounted device 1a outputs the sound translated into the first language. By doing so, a user wearing the head-mounted device 1a and the head-mounted device 1b can speak while looking at the face of the other party who speaks a language different from that of the other party, so as to understand what the other party is saying. The quality of communication between language counterparts.

Moreover, the head-mounted device 1 has a bone conduction speaker 14. Therefore, the user can listen to the other party's original voice with his ear, and listen to the translated voice through the bone guide, so he can grasp the other party's feelings and understand what the other party is saying. As a result, the quality of communication with a partner using a different language can be further improved.

In the above description, the case where the translation device 3 has the language conversion unit 332 is exemplified. However, the translation device 3 may use an external server such as a cloud server having a translation function as the language conversion.部 332. FIG. 12 is a diagram showing a configuration of the translation device 3 when the function of the language conversion unit 332 is realized by a language conversion server 5 different from the translation device 3. In this way, the translation control unit 331 may execute the translation process by interoperating with the external language conversion server 5 that operates in the same manner as the language conversion unit 332 via the communication unit 31.

As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range described in the said embodiment, Various deformation | transformation and change are possible within the range of the summary. For example, the specific embodiment of the device dispersion and integration is not limited to the above embodiment, and all or a part of the device can be configured by functionally or physically dispersing and integrating in any unit. In addition, a new embodiment produced by an arbitrary combination of a plurality of embodiments is also included in the embodiments of the present invention. The effects of the new embodiment produced by the combination have the effects of the original embodiment.

In the above description, it is assumed that the information terminal 2 is a terminal leased to the user U, but the information terminal 2 may also be a terminal signed by the user U and a mobile phone operator. In this case, the information terminal 2 may also store the language information used by the user U in advance, without displaying the user selection screen shown in FIG. 5.

1, 1a, 1b, 1c ‧‧‧ headset

2, 2a, 2b, 2c‧‧‧ information terminal

3‧‧‧ translation device

4, 4a, 4b ‧‧‧ access points

5‧‧‧ Language Conversion Server

10‧‧‧ Headset

11‧‧‧1st main microphone

12‧‧‧ 2nd main microphone

13‧‧‧ secondary microphone

14, 14L, 14R‧‧‧ bone conduction speakers

15‧‧‧Control unit

16‧‧‧cable

17‧‧‧Microphone storage

18‧‧‧ connecting member

19‧‧‧ main body

20, 20b, 20c‧‧‧ Information Terminal

21‧‧‧The first communication department

22‧‧‧ 2nd Ministry of Communications

23‧‧‧Display

24‧‧‧Operation Department

25‧‧‧Memory Department

26‧‧‧Control Department

31‧‧‧Communication Department

32‧‧‧Memory Department

33‧‧‧Control Department

151‧‧‧Sound Processing Department

152‧‧‧Ministry of Communications

153‧‧‧Control Department

261‧‧‧Text Conversion Department

262‧‧‧Processing Department

331‧‧‧ Translation Control Department

332‧‧‧Language Conversion Department

B, B1, B2, B3‧‧‧ wireless channels

N‧‧‧Internet

U1, U2, U3‧‧‧ users

R1‧‧‧Area 1

R2‧‧‧ Zone 2

R3‧‧‧Region 3

S1, S2‧‧‧‧ Translation System

S11 ～ S19‧‧‧step

W, W1, W2, W3‧‧‧ wireless communication lines

FIG. 1 is a diagram showing a configuration of a translation system according to the present embodiment. FIG. 2 is a diagram showing an appearance of a headset. FIG. 3 is a diagram showing the internal configuration of a headset and an information terminal. FIG. 4 is a diagram showing a configuration of a translation device. 5 (a) and 5 (b) are diagrams showing a user selection screen displayed on a display section when a user starts using an information terminal. FIGS. 6 (a) and 6 (b) are diagrams showing a counterpart selection screen displayed on the display unit when a counterpart is selected for a conversation. 7 (a) and 7 (b) are diagrams showing a conversation screen displayed on the display unit during a conversation. 8 (a) and 8 (b) are diagrams showing a screen of an information terminal after the user releases his or her finger from the "speech" icon. FIGS. 9 (a) and 9 (b) are diagrams showing the screen of the information terminal after the user speaks a second language in the state of FIG. 8 (a) and FIG. 8 (b). FIG. 10 is a diagram showing a processing procedure in a translation system. FIG. 11 is a diagram showing a configuration of a translation system according to a modification of the embodiment. FIG. 12 is a diagram showing a configuration of a translation device when a function of a language conversion unit is realized by a language conversion server.

Claims (16)

A translation system includes a first user terminal used by a first user and a translation device capable of communicating with the first user terminal. The first user terminal includes: a voice input unit that accepts voice input in a first language. A terminal sending unit that sends first language text data obtained by converting the sound in the first language received by the voice input unit to the translation device; a terminal receiving unit that receives from the translation device A first language translation material obtained by translating language information of a second language input in a second user terminal used by a second user into the first language; and a sound output unit that outputs information to the terminal receiving unit. The translation in the first language, which is obtained by converting the received first language translation data, includes: a determining unit that determines a first language used by the first user and the second user A second language used; a device receiving unit that receives the first language text data from the first user terminal; a language conversion unit that receives the data received by the device receiving unit Converting the first-language text data into a second-language translation data, and converting the second-language translation data into the first language to generate retranslated data; and the device transmitting unit, for the second user, When the terminal sends the translation material in the second language, the retranslated material is sent to the first user terminal, and the second language input in the second user terminal has been transmitted by the language conversion unit. When the language information of the first language translation material is converted into the first language translation material, the transmission of the retranslation material is suspended and the first language translation material is transmitted, and the voice output unit receives the first language translation unit in the voice input unit During the input of the voice of the language, the retranslated data sent by the device transmitting unit is output. A translation system includes a first user terminal used by a first user and a translation device capable of communicating with the first user terminal. The first user terminal includes: a voice input unit that accepts voice input in a first language. A terminal sending unit that sends first language text data obtained by converting the sound in the first language received by the voice input unit to the translation device; a terminal receiving unit that receives from the translation device A first language translation material obtained by translating language information of a second language input in a second user terminal used by a second user into the first language; and a sound output unit that outputs information to the terminal receiving unit. The translation in the first language, which is obtained by converting the received first language translation data, includes: a determining unit that determines a first language used by the first user and the second user A second language used; a device receiving unit that receives the first language text data from the first user terminal; a language conversion unit that receives the data received by the device receiving unit The first language body text data is converted into a second language translation data, and the second language translation data is converted into the first language to generate retranslated data; and the device transmitting section is When the terminal sends the translation material in the second language, the retranslated material is sent to the first user terminal. When the retranslated material is sent to the first user terminal, the device receiving unit When receiving the first language text data including a predetermined word, the sending of the second language translation data to the second user terminal is suspended, and the voice output unit receives the first language in the voice input unit. During the input of the voice, the retranslated data sent by the device transmitting unit is output. A translation system includes a first user terminal used by a first user and a translation device capable of communicating with the first user terminal. The first user terminal includes: a voice input unit that accepts voice input in a first language. A terminal sending unit that sends first language text data obtained by converting the sound in the first language received by the voice input unit to the translation device; a terminal receiving unit that receives from the translation device A first language translation material obtained by translating language information of a second language input in a second user terminal used by a second user into the first language; and a sound output unit that outputs information to the terminal receiving unit. The translation in the first language, which is obtained by converting the received first language translation data, includes: a determining unit that determines a first language used by the first user and the second user The second language used; the device receiving unit receives the text material of the first language from the first user terminal, and receives the sound of the second language from the second user terminal The second language text data obtained by the conversion; the language conversion unit converts the first language text data received by the device receiving unit into a second language translation data, and receives the second language translation data by the device receiving unit The second language text material in the second language is converted into a first language translation material; and a device transmitting unit sends the second language translation material to the second user terminal, and when sending the first language translation material, During the 1 language translation, when the language conversion unit finishes generating the new first language translation material, the device transmission unit suspends the transmission of the first language translation material in the transmission process and starts a new Sending of the first language translation material. A translation system includes a first user terminal used by a first user and a translation device capable of communicating with the first user terminal. The first user terminal includes: a voice input unit that accepts voice input in a first language. A terminal transmitting unit that converts the first language text data obtained by converting the voice in the first language received by the voice input unit, and determines a second user different from the first user The user identification information is transmitted to the translation device in association with the first language text material; and the terminal receiving unit receives, from the translation device, the first input from the second user terminal used by the second user. A first language translation material obtained by translating language information in 2 languages into the first language; and a sound output unit that outputs the translation information obtained by converting the first language translation material received by the terminal receiving unit The sound in the first language is described, and the translation device includes a determination unit that determines a first language used by the first user, and determines a second language used by the second user, and The second language refers to a memory unit that memorizes the languages used by the plurality of users in a manner associated with the plurality of users, and is indicated by the user identification information transmitted by the terminal transmitting unit. The second user is memorized in the memory unit in association; the device receiving unit receives the first language text data from the first user terminal; the language conversion unit stores the data received by the device receiving unit Converting the first language text data into a second language translation data; and a device transmitting unit transmitting the second language translation data to the second user terminal. A translation system includes a first user terminal used by a first user and a translation device capable of communicating with the first user terminal. The first user terminal includes: a voice input unit that accepts voice input in a first language. A terminal sending unit that sends first language text data obtained by converting the sound in the first language received by the voice input unit to the translation device; a terminal receiving unit that receives from the translation device A first language translation material obtained by translating language information of a second language input in a second user terminal used by a second user into the first language; and a sound output unit that outputs information to the terminal receiving unit. The translation in the first language, which is obtained by converting the received first language translation data, includes: a determining unit that determines a first language used by the first user, and based on the first language The words contained in the 1 language text material to determine the second language used by the second user; the device receiving unit receives the first language text material from the first user terminal The language conversion unit converts the receiving unit received by means of the first language text language translation data for the second data; and a transmitting section means, the second user terminal transmits the translated second language information. The translation system according to any one of claims 1 to 5, wherein the sound output unit has a bone conduction speaker, and the sound input unit receives an input of a sound of the first language during the period And outputting the sound of the first language from the bone conduction speaker. A translation method includes: a translation device determining a first language used by a first user and a second language used by a second user; a first user terminal used by the first user accepting voice input in the first language Step; step of converting, by the first user terminal, the sound in the first language into text material in the first language, and sending the converted text material in the first language to the translation device; A step of converting the first language text data into a second language translation data; the translation device converting the second language translation data into the first language to generate retranslated data; the translation device A step in which a second user terminal used by the second user sends the second language translation material and sends the retranslated material to the first user terminal; the second user terminal outputs a response to the first A step of converting the second language translation data into the sound of the second language; a step of outputting the retranslated data by the first user terminal while receiving the input of the first language voice; And in the step of transmitting the retranslated material to the first user terminal, the translation device converts the language information of the second language input in the second user terminal into the first language translation material. In this case, the step of transmitting the first-language translation material is suspended by transmitting the re-translation material. A translation method includes: a translation device determining a first language used by a first user and a second language used by a second user; a first user terminal used by the first user accepting voice input in the first language Step; step of converting, by the first user terminal, the sound in the first language into text material in the first language, and sending the converted text material in the first language to the translation device; A step of converting the first language text data into a second language translation data; the translation device converting the second language translation data into the first language to generate retranslated data; the translation device A step in which a second user terminal used by the second user sends the second language translation material and sends the retranslated material to the first user terminal; the second user terminal outputs a response to the first A step of converting the second language translation data into the sound of the second language; a step of outputting the retranslated data by the first user terminal while receiving the input of the first language voice; And in the step of transmitting the retranslated data to the first user terminal by the translation device, after transmitting the retranslated data to the first user terminal, the translation device receives the first In the case of the language text material, the step of sending the second language translation material to the second user terminal is suspended. A translation method includes: a translation device determining a first language used by a first user and a second language used by a second user; a first user terminal used by the first user accepting voice input in the first language Step; the step of converting, by the first user terminal, the sound in the first language into the text material in the first language, and sending the converted text material in the first language to the translation device; A step in which the first user terminal receives the text material in the first language; the translation device receives, from the second user terminal used by the second user, the second language text material obtained by converting the voice in the second language Step of the translation device converting the text material in the first language into translation material in the second language; the translation device converting the received text material in the second language into the first language Step of language translation data; step of the translation device sending the second language translation data to the second user terminal; the second user terminal output converting the second language translation data A step of obtaining the voice in the second language; when the translation device ends the generation of the new translation material in the first language during the transmission of the translation material in the first language, the translation device suspends all transmission in the transmission process. The process of transmitting the first language translation material is described, and a new step of transmitting the first language translation material is started. A translation method includes: a translation device determining a first language used by a first user and a second language used by a second user; a first user terminal used by the first user accepting voice input in the first language Step; step of converting, by the first user terminal, the sound in the first language into text material in the first language, and sending the converted text material in the first language to the translation device; the first user A step of sending, by the terminal, the user identification information for determining the second user in association with the text material in the first language; the translation device converting the text material in the first language into the translation material in the second language Step; the step of sending, by the translation device, the second language translation data to a second user terminal used by the second user; the second user terminal output obtained by converting the second language translation data In the step of sound in the second language, in the step of determining the second language, the translation device determines a second language used by the second user, and the second language is related to a plurality of users by referring to the second language. Link A memory unit that memorizes the languages used by the plurality of users, and memorizes the second user indicated by the user identification information sent by the first user terminal in the memory. Memory department. A translation method includes: a translation device determining a first language used by a first user and a second language used by a second user; a first user terminal used by the first user accepting voice input in the first language Step; step of converting, by the first user terminal, the sound in the first language into text material in the first language, and sending the converted text material in the first language to the translation device; A step of converting the first language text material into a second language translation material; a step of the translation device transmitting the second language translation material to a second user terminal used by the second user; the second user A step of outputting a voice in the second language obtained by converting the second language translation data to the terminal, and the step of determining the second language by the translation device is based on the information contained in the first language text data. Words to determine the second language used by the second user. A translation device includes: a determining unit that determines a first language used by a first user and a second language used by a second user; a device receiving unit that receives the first language from the first user terminal used by the first user The first language text data obtained by converting the sound in the first language; the language conversion unit converts the first language text data received by the device receiving unit into a second language translation data, and converts the first language text data Converting the 2-language translation data into the first language to generate re-translation data; and the device transmitting unit, while transmitting the second-language translation data to a second user terminal used by the second user, the The translation data is sent to the first user terminal, and when the language conversion unit has converted the language information of the second language input into the second user terminal into the first language translation material, The transmission of the retranslated material is suspended, and the first language translated material is transmitted. A translation device includes: a determining unit that determines a first language used by a first user and a second language used by a second user; a device receiving unit that receives the first language from the first user terminal used by the first user The first language text data obtained by converting the sound in the first language; the language conversion unit converts the first language text data received by the device receiving unit into a second language translation data, and converts the first language text data Converting the 2-language translation data into the first language to generate re-translation data; and the device transmitting unit, while transmitting the second-language translation data to a second user terminal used by the second user, the re-translation data The translation data is sent to the first user terminal, and after the retranslated data is sent to the first user terminal, when the device receiving unit receives the first language text material containing a predetermined word, Stop sending the second language translation material to the second user terminal. A translation device includes: a determining unit that determines a first language used by a first user and a second language used by a second user; a device receiving unit that receives the first language from the first user terminal used by the first user A first language text material obtained by converting a sound in a first language, and receiving a second language text material obtained by converting a sound in a second language from a second user terminal used by the second user; a language conversion unit; , Converting the first language text data received by the device receiving unit into a second language translation data, and converting the second language text data received by the device receiving unit into the second language Converting to the first language translation material; and a device transmitting unit that transmits the second language translation material to the second user terminal, and during the period when the first language translation material is transmitted, the language conversion unit ends the new language translation material When the first language translation material is generated, the device transmitting unit suspends the transmission of the first language translation material in the transmission process, and starts the new transmission of the first language translation material. . A translation device includes: a determining unit that determines a first language used by a first user and a second language used by a second user; a device receiving unit that receives an address from a first user terminal used by the first user The first language text data obtained by converting the sound in the first language; a language conversion unit that converts the first language text data received by the device receiving unit into a second language translation data; and a device transmission unit Sending the second language translation data to a second user terminal used by the second user, the determining unit determines a second language used by the second user, and the second language is referred to by referring to A memory unit that memorizes the languages used by the plurality of users in a manner associated with the plurality of users, and the memory indicated by the user identification information received from the first user terminal used by the first user; The second user is memorized in the memory unit in association. A translation device includes: a determining unit that determines a first language used by a first user and a second language used by a second user; a device receiving unit that receives an address from a first user terminal used by the first user The first language text data obtained by converting the sound in the first language; a language conversion unit that converts the first language text data received by the device receiving unit into a second language translation data; and a device transmission unit Sending the second language translation data to a second user terminal used by the second user, and the determining unit determines the words used by the second user based on the words contained in the first language text Second language.