JP7429107B2 - Speech translation device, speech translation method and its program - Google Patents

Description

The present disclosure relates to a speech translation device, a speech translation method, and a program using the speech translation method.

For example, Patent Document 1 discloses a voice input unit that converts the voice uttered by a first language speaker and a second language speaker who is a conversation partner of the first language speaker into voice data and outputs the voice data; An input switch that is entered while the first language speaker is making a sound, and an input switch that is entered even when the first language speaker is not making a sound, and an input switch that translates the input speech data into speech. An interpretation system is disclosed that includes an audio output unit that converts and outputs the audio.

Patent No. 3891023

However, in the technology disclosed in Patent Document 1, when the first speaker and the second speaker have a conversation, the input switch is operated each time the first speaker and the second speaker speak. operation becomes cumbersome. When the first speaker and the second speaker have a conversation, the input switch must be operated frequently, which increases the frequency and period of use of the interpretation system.

Furthermore, when the first speaker and the second speaker mutually operate the interpretation system, non-owners of the interpretation system usually do not understand how to operate the interpretation system. Therefore, it takes time to operate the interpretation system, which accelerates the length of time the interpretation system is used. As a result, conventional interpretation systems have the problem of consuming energy due to an increase in the period of use.

Therefore, an object of the present disclosure is to provide a speech translation device, a speech translation method, and a program therefor, which can suppress an increase in energy consumption of the speech translation device by simplifying the operation.

A speech translation device according to an aspect of the present disclosure includes a first speaker who speaks in a first language, and a second speaker who is a conversation partner of the first speaker and who speaks in a second language different from the first language. A voice translation device for having a conversation with a person, the voice detection unit detecting a voice section uttered by the first speaker and the second speaker from the sounds input to the voice input unit; By performing voice recognition on the voice in the voice section detected by the voice detection unit, displaying a translation result translated from the first language indicated by the voice into the second language, and displaying the result of translation from the second language to the first language. A display unit that displays the translation result translated into a language, and content that prompts the second speaker to speak after the first speaker speaks, after displaying the translation result, or at the same time, through the display unit, After displaying the translation result via the display unit, or at the same time, the first speaker outputs content in the second language and prompts the first speaker to speak after the second speaker speaks. and a speech instruction unit that outputs in language.

Note that some specific aspects of these may be realized using a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM. It may be implemented using any combination of integrated circuits, computer programs, and storage media.

According to the speech translation device and the like of the present disclosure, by simplifying the operation, it is possible to suppress an increase in energy consumption of the speech translation device.

FIG. 1A is a diagram illustrating the appearance of the speech translation device according to the first embodiment and an example of a usage scene of the speech translation device between the first speaker and the second speaker when the first speaker speaks. FIG. 1B is a diagram illustrating an example of the external appearance of the speech translation device according to the first embodiment and a usage scene of the speech translation device between the first speaker and the second speaker when the second speaker speaks. FIG. 1C is a diagram showing another example of a usage scene of the speech translation device when a first speaker and a second speaker have a conversation. FIG. 2 is a block diagram showing the speech translation device in the first embodiment. FIG. 3 is a flowchart showing the operation of the speech translation device in the first embodiment. FIG. 4 is a block diagram showing a speech translation device according to the second embodiment. FIG. 5 is a flowchart showing the operation of the speech translation device in the second embodiment. FIG. 6 is a flowchart showing the operation of the speech translation device in a modification of the second embodiment. FIG. 7 is a block diagram showing a speech translation device according to the third embodiment. FIG. 8 is a flowchart showing the operation of the speech translation device in the third embodiment. FIG. 9 is a block diagram showing a speech translation device according to a modification of the third embodiment. FIG. 10 is a block diagram showing a speech translation device according to the fourth embodiment. FIG. 11 is a flowchart showing the operation of the speech translation device according to the fourth embodiment.

A speech translation device according to an aspect of the present disclosure includes a first speaker who speaks in a first language, and a second speaker who is a conversation partner of the first speaker and who speaks in a second language different from the first language. A voice translation device for having a conversation with a person, the voice detection unit detecting a voice section uttered by the first speaker and the second speaker from the sounds input to the voice input unit; By performing voice recognition on the voice in the voice section detected by the voice detection unit, displaying a translation result translated from the first language indicated by the voice into the second language, and displaying the result of translation from the second language to the first language. a display unit that displays a translation result translated into a language; and a display unit that outputs, in the second language, content that prompts the second speaker to speak after the first speaker speaks, through the display unit, and and a speech instruction section that outputs, in the first language, via the display section, content that prompts the first speaker to speak after the second speaker has uttered the speech.

According to this, by detecting each speech interval from a conversation between a first speaker and a second speaker, a translation result of the detected speech from the first language to the second language can be obtained, It is also possible to obtain a translation result obtained by translating the detected voice from the second language to the first language. In other words, this speech translation device automatically converts the language of the detected speech into a different language for each utterance by the first speaker and the second speaker, without any input operations for translation. can be translated into

In addition, the speech translation device outputs content that prompts the second speaker to speak after the first speaker speaks, or outputs content that prompts the first speaker to speak after the second speaker speaks. can do. As a result, with this speech translation device, the first speaker and the second speaker can communicate without having to perform an input operation to start each utterance. be able to recognize when to do so.

As described above, the speech translation device does not require input operations to start speaking, input operations to switch languages, etc., and is excellent in operability. In other words, since the operation of this speech translation device does not take much time, it is possible to suppress an increase in the period of use.

Therefore, by simplifying the operation of the speech translation device, it is possible to suppress an increase in energy consumption of the speech translation device.

In particular, since this speech translation device can be operated easily, it is also possible to suppress erroneous operations.

A speech translation method according to another aspect of the present disclosure includes a first speaker who speaks in a first language, and a second speaker who is a conversation partner of the first speaker and who speaks in a second language different from the first language. A voice translation method for having a conversation with a speaker, the method comprising: detecting a voice section uttered by the first speaker and the second speaker from sounds input to a voice input section; By performing voice recognition on the voice in the voice section, displaying the translation result translated from the first language indicated by the voice into the second language, and displaying the translation result translated from the second language into the first language. outputting, in the second language via the display unit, content that the display unit displays and prompts the second speaker to speak after the first speaker speaks; and outputting content for prompting the first speaker to speak in the first language via the display unit after the speaker speaks.

This speech translation method also provides the same effects as the above-mentioned speech translation device.

Further, a program according to another aspect of the present disclosure is a program for causing a computer to execute a speech translation method.

This program also has the same effects as the above-mentioned speech translation device.

The speech translation device according to another aspect of the present disclosure further provides that, when the first speaker or the second speaker utters and the speech is recognized, the first speaker or the second speaker who uttered the utterance again A priority utterance input unit is provided that prioritizes and recognizes the utterances of the speaker.

According to this, for example, when the first and second speakers make a mistake in saying something, or when a voice that they hesitate to say is translated midway through, by operating the priority speech input section, the utterance can be corrected. Since priority is given to the speaker, the speaker who has uttered the utterance can have an opportunity to utter the utterance again (can rephrase the utterance). For this reason, after the priority speech input unit finishes recognizing the voice uttered by one of the first speaker and the second speaker, it shifts to the process for recognizing the voice of the other speaker. Even if one speaker speaks, the voice uttered by one speaker can be returned to the voice recognition process. Thereby, the speech translation device can reliably acquire the voices of the first speaker and the second speaker, and therefore can output a translation result translated based on the voices.

The speech translation device according to another aspect of the present disclosure further includes a speech input section into which the speech of conversation between the first speaker and the second speaker is input, and a speech section detected by the speech detection section. a speech recognition unit that converts speech into a text sentence by speech recognition; a speech recognition unit that converts the text sentence converted by the speech recognition unit from the first language to the second language; It includes a translation unit that translates into a first language, and a voice output unit that outputs the result translated by the translation unit as a voice.

According to this, it is possible to perform speech recognition on input speech and then translate the language of the speech into another language. In other words, the speech translation device can perform processing from obtaining the speech of a conversation between the first speaker and the second speaker to outputting the result of translating the speech. Therefore, the speech translation device can mutually translate the voices of the first speaker and the second speaker, without communicating with an external server. It can be applied even in environments where it is difficult for the speech translation device to communicate with an external server.

In the speech translation device according to another aspect of the present disclosure, a plurality of the speech input sections are provided, and further performs signal processing on speech input to at least some of the plurality of speech input sections. The first beam former unit controls the directivity of sound collection in the direction of the sound source of the voice of the first speaker, and the voice input to at least some of the voice input units of the plurality of voice input units. a second beamformer section that controls the directivity of sound collection in the direction of the sound source of the voice of the second speaker by signal processing; and the output signal of the first beamformer section, or an input switching unit that switches to an output signal of the second beamformer unit; and a sound source direction estimation unit that estimates a sound source direction by signal processing the audio input to the plurality of audio input units; The unit causes the input switching unit to switch between acquiring the output signal of the first beamformer unit and acquiring the output signal of the second beamformer unit.

According to this, the direction of the speaker relative to the speech translation device can be estimated by the sound source direction estimation section. Therefore, the input switching section can switch to either the output signal of the first beamformer section or the output signal of the second beamformer section suitable for the direction of the speaker. In other words, since the directionality of the sound collection of the beamformer unit can be directed toward the sound source, the speech translation device can collect the sounds of the first speaker and the second speaker while reducing ambient noise. can.

In the speech translation device according to another aspect of the present disclosure, a plurality of the speech input sections are provided, and the sound source direction is further configured to estimate a sound source direction by signal processing the speech inputted to the plurality of speech input sections. an estimation unit; displaying the first language in a display area of the display unit corresponding to the position of the first speaker with respect to the speech translation device; and displaying the first language in a display area of the display unit corresponding to the position of the second speaker with respect to the speech translation device; a control unit that displays the second language in a display area of a display unit, the control unit configured to display the second language in a display direction from the display unit of the speech translation device toward the first speaker or the second speaker; , when the display direction displayed in any display area of the display section and the sound source direction estimated by the sound source direction estimation section are compared, and the display direction and the estimated sound source direction substantially match; , the speech recognition section and the translation section are executed, and when the display direction and the estimated sound source direction are different, the speech recognition section and the translation section are stopped.

According to this, when the display direction of the language displayed in the display area of the display unit and the sound source direction of the sound uttered by the speaker substantially match, the first episode uttered by the speaker in the first language It is possible to identify whether the user is speaking in the second language or the second speaker speaking in the second language. In this case, the first speaker's voice can be recognized in the first language, and the second speaker's voice can be recognized in the second language. Further, when the display direction and the sound source direction are different, by stopping the translation of the input voice, it is possible to prevent the input voice from not being translated or being mistranslated.

Thereby, the voice translation device can reliably recognize the first language voice and the second language voice, and therefore can reliably translate the voice. As a result, this speech translation device can suppress an increase in the processing amount of the speech translation device by suppressing mistranslations and the like.

In the speech translation device according to another aspect of the present disclosure, when the control section stops the speech recognition section and the translation section, the speech instruction section outputs content encouraging speech in the instructed language again.

According to this, even if the display direction and the sound source direction are different, the speech instruction section outputs the content encouraging speech again, so that the target speaker speaks. Therefore, the speech translation device can reliably acquire the speech of the target speaker, and therefore can translate the speech more reliably.

In the speech translation device according to another aspect of the present disclosure, when the display direction and the estimated sound source direction are different, the speech instruction section again performs the speech instruction after a predetermined period has elapsed since the control section made the comparison. Outputs content that encourages speaking in the specified language.

According to this, by leaving a specified period after comparing the display direction and the sound source direction, it is possible to suppress the voices of the first speaker and the second speaker from being input together. . As a result, after the predetermined period of time has elapsed, the target speaker speaks by outputting the content encouraging him to speak again. Therefore, the speech translation device can more reliably acquire the speech of the target speaker, and therefore can translate the speech more reliably.

In the speech translation device according to another aspect of the present disclosure, a plurality of the speech input sections are provided, and further performs signal processing on speech input to at least some of the plurality of speech input sections. The first beam former unit controls the directivity of sound collection in the direction of the sound source of the voice of the first speaker, and the voice input to at least some of the voice input units of the plurality of voice input units. a second beamformer unit that controls the directivity of sound collection in the direction of the sound source of the voice by the second speaker by signal processing the output signal of the first beamformer unit; and a sound source direction estimation section that estimates the direction of the sound source by signal processing the output signal of the section.

According to this, the direction of the speaker relative to the speech translation device can be estimated by the sound source direction estimation section. Therefore, the sound source direction estimation section processes the output signal of the first beamformer section and the output signal of the second beamformer section suitable for the direction of the speaker, so that the calculation cost due to signal processing can be reduced. .

In the speech translation device according to another aspect of the present disclosure, the speech instruction section may display content prompting the first speaker to speak in the first language via the display section when the speech translation device is activated. output, and after the voice uttered by the first speaker is translated from the first language to the second language and the translation result is displayed on the display unit, content that prompts the second speaker to speak. , output in the second language via the display section.

According to this, if you register in advance that the second speaker will speak in the second language after the first speaker speaks in the first language, when the speech translation device starts up, the first speaker will If the content prompting speech is output in the first language, the first speaker can start speaking. Therefore, it is possible to suppress mistranslation caused by the second speaker speaking in the second language when the speech translation device is activated.

In the speech translation device according to another aspect of the present disclosure, the speech instruction section causes the speech output section to output a voice for encouraging speech a predetermined number of times after starting translation, After outputting the voice, the display section is caused to output a message to encourage speaking.

According to this, by limiting the voice for prompting speech to a specified number of times, it is possible to suppress an increase in energy consumption of the speech translation device.

In the speech translation device according to another aspect of the present disclosure, the speech recognition unit outputs a result of speech recognition of the speech and a reliability score of the result, and the speech instruction unit acquires the result from the speech recognition unit. If the reliability score obtained is less than or equal to a threshold value, content that prompts the user to speak is outputted via at least one of the display unit and the audio output unit without translating the voice whose reliability score is less than or equal to the threshold value. .

According to this, if the reliability score indicating the accuracy of speech recognition is equal to or less than the threshold value, the speech instruction section outputs the content encouraging speech again, so that the target speaker speaks again. Therefore, the speech translation device can reliably recognize the speech of the target speaker, and therefore can translate the speech more reliably.

Particularly, if the voice output unit outputs the content that prompts the speaker to speak, the speaker will be more likely to notice that the voice is not being recognized correctly.

Note that some specific aspects of these may be realized using a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM. It may be implemented using any combination of integrated circuits, computer programs, or recording media.

The embodiments described below are all specific examples of the present disclosure. The numerical values, shapes, materials, components, arrangement positions of the components, etc. shown in the following embodiments are merely examples, and do not limit the present disclosure. Further, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims will be described as arbitrary constituent elements. Moreover, in all embodiments, the contents of each can be combined.

Hereinafter, a speech translation device, a speech translation method, and a program thereof according to one aspect of the present disclosure will be specifically described with reference to the drawings.

(Embodiment 1)
<Configuration: Voice translation device 1>
FIG. 1A is a diagram showing the appearance of the speech translation device 1 according to the first embodiment and an example of a usage scene of the speech translation device 1 between the first speaker and the second speaker when the first speaker speaks. be. FIG. 1B is a diagram showing the appearance of the speech translation device 1 according to the first embodiment and an example of a usage scene of the speech translation device 1 between the first speaker and the second speaker when the second speaker speaks. be.

As shown in FIGS. 1A and 1B, the speech translation device 1 includes a first speaker who speaks in a first language, and a conversation partner of the first speaker who speaks in a second language different from the first language. This is a device that bidirectionally translates a conversation between a first speaker and a second speaker so that the two speakers can have a conversation. In other words, the speech translation device 1 recognizes the respective languages uttered (uttered) by the first speaker and the second speaker between the two different languages of the first speaker and the second speaker, and It is a device that translates content into each other's language. For example, the speech translation device 1 translates a first language uttered by a first speaker into a second language and outputs the result, and translates a second language uttered by a second speaker into the first language and outputs the result. Further, the first language and the second language are, for example, Japanese, English, French, German, Chinese, etc.

FIGS. 1A and 1B of this embodiment illustrate a situation in which one first speaker and one second speaker have a conversation while facing each other. Note that it may be used when a plurality of first speakers and a plurality of second speakers have a conversation.

Note that the first speaker and the second speaker may have a conversation while facing each other using the speech translation device 1, or may have a conversation side by side side by side as shown in FIG. 1C. FIG. 1C is a diagram showing another example of a usage scene of the speech translation device 1 when a first speaker and a second speaker have a conversation. In this case, the speech translation device 1 may change the display mode. Such a speech translation device 1 is used in a vertical or horizontal orientation, as shown in FIGS. 1A, 1B, and 1C.

The speech translation device 1 is a mobile terminal that can be carried by the first speaker, such as a smartphone or a tablet terminal.

FIG. 2 is a block diagram showing the speech translation device 1 in the first embodiment.

As shown in FIG. 2, the speech translation device 1 includes a speech input section 21, a speech detection section 22, a priority speech input section 24, a speech instruction section 25, a speech recognition section 23, a translation section 26, a display section 27, an audio output section 28, and a power supply section 29.

[Voice input section 21]
The voice input section 21 is a microphone into which the voice of a first speaker and a second speaker having a conversation is input, and is communicably connected to the voice detection section 22 . That is, the audio input section 21 acquires (collects) sound, converts the acquired sound into an electrical signal, and outputs the acoustic signal, which is the converted electrical signal, to the audio detection section 22 . Note that the acoustic signal acquired by the audio input section 21 may be stored in a storage section or the like.

Note that the audio input section 21 may be configured as an adapter. In this case, the voice input unit 21 functions when a microphone is attached to the voice translation device 1, and acquires the acoustic signal acquired by the microphone.

[Sound detection unit 22]
The voice detection unit 22 is a device that detects voice sections uttered by the first speaker and the second speaker from the sound input to the voice input unit 21, and is capable of communicating with the voice input unit 21 and the voice recognition unit 23. connected to. Specifically, the audio detection unit 22 considers the moment when the volume increases and the moment when the volume decreases from the volume indicated in the audio signal obtained from the audio input unit 21 as a break in the audio, and detects the sound. The start and end points of the voice section in the signal are detected (end of speech detection). Here, the audio section indicates the audio of each episode uttered by the speaker, but may also include the period from the start point to the end point of the audio of one episode.

The voice detection section 22 detects the voice section detected from the acoustic signal, that is, the voice of each conversation between the first speaker and the second speaker from the acoustic signal, and transmits voice information indicating the detected voice to the voice recognition section. Output to 23.

[Speech instruction section 25]
The utterance instruction unit 25 outputs content that prompts the second speaker to speak after the first speaker has uttered the content in the second language via the display unit 27, and This is a device that outputs content in the first language that prompts the user to speak. In other words, the speech instruction section 25 displays speech instruction text information that prompts the first speaker or the second speaker to speak at respective timings so that the first speaker and the second speaker can have a conversation. output to section 27. Furthermore, the speech instruction section 25 outputs speech instruction audio information, which is content that prompts the first speaker or the second speaker to speak, to the audio output section 28 . In this case, the speech instruction section 25 outputs speech instruction audio information having the same content as the content shown in the speech instruction text information output to the display section 27 to the audio output section 28 . Note that the speech instruction section 25 does not need to output the speech instruction audio information to the audio output section 28, and it is not essential to output content encouraging speech by voice.

Here, the utterance instruction text information is a text sentence indicating content that prompts the first speaker or the second speaker to speak. Furthermore, the speech instruction audio information is audio indicating content that prompts the first speaker or the second speaker to speak.

Furthermore, the speech instruction section 25 outputs an instruction command for the translation section 26 to translate the first language into the second language, or for the translation section 26 to translate the second language into the first language. For example, since the second speaker speaks after the first speaker speaks, the speech instruction section 25 outputs an instruction command to the speech recognition section 23 to recognize the speech uttered by the second speaker in the second language. Then, an instruction command for translating the recognized speech from the second language to the first language is output to the translation unit 26. The same applies when the first speaker speaks.

Furthermore, after one of the first speaker and the second speaker speaks, the speech instruction section 25 outputs speech instruction text information to the display section 27, which is the content of urging the other speaker to speak. . At the time or after the translation unit 26 outputs the translation result of the voice uttered by one of the speakers, the utterance instruction unit 25 outputs the utterance instruction text information to the display unit 27, and outputs the utterance instruction voice information. It is output to the audio output section 28.

Further, when the speech instruction section 25 acquires an instruction command from the priority speech input section 24 to be described later, it outputs speech instruction text information to the display section 27, which is content that urges the speaker who has most recently spoken to speak again. Then, the speech instruction audio information is output to the audio output section 28.

Furthermore, when the speech translation device 1 is activated, the speech instruction section 25 outputs content that prompts the first speaker to speak in the first language via the display section 27 . That is, when the first speaker is the owner of the voice translation, the speech instruction unit 25 prompts the first speaker to start speaking. The utterance instruction unit 25 also provides content that prompts the second speaker to speak after the voice uttered by the first speaker is translated from the first language to the second language and the translation result is displayed on the display unit 27. is output in the second language via the display unit 27. After the first speaker's utterances of the first language are translated into the second language, the second speaker speaks in the second language, and the second language utterances are translated into the first language. By repeating this, the conversation between the first speaker and the second speaker becomes more lively.

Furthermore, after the start of translation, the speech instruction section 25 causes the speech output section 28 to output a voice for encouraging speech a specified number of times. In other words, since there are cases where the second speaker does not speak immediately or cannot be heard, the speech instruction unit 25 outputs the voice to encourage speech a specified number of times. After outputting a sound for encouraging speech a predetermined number of times, speech instruction section 25 causes display section 27 to output a message for encouraging speech. That is, if there is no effect even after outputting the voice to encourage speech a specified number of times, a message to encourage speech is displayed on the display unit 27 in order to suppress power consumption.

The speech instruction section 25 is communicably connected to the speech recognition section 23, the priority speech input section 24, the translation section 26, the display section 27, and the speech output section 28.

[Priority speech input section 24]
When the first speaker or the second speaker makes an utterance and the speech is recognized, the priority utterance input unit 24 again gives priority to the utterance of the first speaker or the second speaker who made the utterance (or consecutively). ) This is a device that allows the voice recognition unit 23 to perform voice recognition. In other words, the priority speech input unit 24 provides an opportunity for the speaker who has most recently spoken and whose voice has been recognized to speak again to the first or second speaker. can be given. In other words, after the priority speech input unit 24 finishes recognizing the voice uttered by one of the first speaker and the second speaker, the priority speech input unit 24 shifts to the process of recognizing the voice of the other speaker. Even if one speaker speaks, the voice uttered by one speaker can be returned to the voice recognition process.

The priority speech input unit 24 is an operation input unit that receives input from the operator of the speech translation device 1. For example, if the speaker makes a mistake in speaking, if the voice that he/she hesitates to say is translated midway through, or if the interval in which the voice detection unit 22 does not detect voice exceeds a specified interval, the voice translation device 1 will terminate the utterance. There are times when the speaker who spoke most recently wants to continue speaking, such as when there is a risk that the speaker may recognize that the person has spoken. For this reason, the priority speech input unit 24 gives priority to the voice uttered by the most recent speaker, causes the voice recognition unit 23 to recognize the voice, and causes the translation unit 26 to translate the voice. As a result, the priority speech input section 24 sends an instruction command to the speech instruction section 25 to cause the speech instruction section 25 to output speech instruction text information and speech instruction voice information that are content to prompt the speech instruction section 25 again. Output to. The operator is at least one of the first speaker and the second speaker, but in this embodiment, the operator is mainly the first speaker.

In this embodiment, the priority speech input section 24 is a touch sensor that is provided integrally with the display section 27 of the speech translation device 1. In this case, the display unit 27 of the speech translation device 1 may display an operation button as the priority speech input unit 24 that accepts an operation by one of the speakers.

In this embodiment, when the speech recognition unit 23 switches the speech recognition from the first language to the second language, the first language is changed so that the first language before the switching is given priority for speech recognition and translation. A priority speech input section 24, which is a priority button, is displayed on the display section 27. In addition, when the speech recognition unit 23 switches the speech recognition from the second language to the first language, a second language priority button is provided in order to prioritize the speech recognition and translation of the second language before switching. The priority speech input section 24 is displayed on the display section 27. Such a priority button is displayed on the display unit 27 at least after translation.

[Voice recognition unit 23]
The speech recognition section 23 performs speech recognition on the speech in the speech section detected by the speech detection section 22 and converts it into a text sentence. Specifically, when the voice recognition unit 23 acquires the voice information detected by the voice detection unit 22, the voice recognition unit 23 performs voice recognition on the voice indicated by the voice information. For example, if the voice shown in the voice information is in the first language, the voice is recognized in the first language, and if the voice shown in the voice information is in the second language, the voice is recognized in the second language. do. When the speech recognition unit 23 performs speech recognition on speech in the first language, the speech recognition unit 23 generates a first text sentence indicating the content of the recognized speech, and outputs the generated first text sentence to the translation unit 26 . Furthermore, when the speech recognition unit 23 performs speech recognition on speech in the second language, the speech recognition unit 23 generates a second text sentence indicating the content of the recognized speech, and outputs the generated second text sentence to the translation unit 26 .

[Translation Department 26]
The translation unit 26 is a translation device that translates the text sentence converted by the speech recognition unit 23 from the first language to the second language, and from the second language to the first language. Specifically, upon acquiring a first text sentence from the speech recognition unit 23, the translation unit 26 translates it from the first language into the second language. That is, the translation unit 26 generates a second translated text sentence by translating the first text sentence into the second language. Furthermore, upon acquiring the second text sentence from the speech recognition unit 23, the translation unit 26 translates it from the second language into the first language. That is, the translation unit 26 generates a first translated text sentence by translating the second text sentence into the first language.

Here, the content of the first text sentence shown in the first language matches the content of the second translated text sentence shown in the second language. Also, the content of the second text sentence shown in the second language matches the content of the first translated text sentence shown in the first language.

After generating the second translated text sentence, the translation unit 26 recognizes the contents of the second translated text sentence and generates translated speech in the second language indicating the recognized contents of the second translated text sentence. Moreover, when the translation unit 26 generates the first translated text sentence, it recognizes the contents of the first translated text sentence and generates translated speech in the first language indicating the recognized contents of the first translated text sentence. Note that the voice output unit 28 may generate the translated voice based on the first translated text sentence and the second translated text sentence.

After generating the second translated text sentence or the first translated text sentence, the translation unit 26 outputs the generated second translated text sentence or the first translated text sentence to the display unit 27 . Further, when the translation unit 26 generates the translated voice in the second language or the translated voice in the first language, the translation unit 26 generates the translated voice in the second language or outputs the generated translated voice in the first language to the voice output unit 28. do.

The translation section 26 is communicably connected to the speech instruction section 25, the speech recognition section 23, the display section 27, and the speech output section 28.

[Display section 27]
The display section 27 is, for example, a monitor such as a liquid crystal panel or an organic EL panel, and is communicably connected to the speech instruction section 25 and the translation section 26. Specifically, the display unit 27 displays the translation result translated from the first language indicated by the voice into the second language by voice recognition of the voice in the voice section detected by the voice detection unit 22, and , a monitor that displays translation results from a second language to a first language. The display unit 27 displays the first text sentence, the second text sentence, the first translated text sentence, and the second translated text sentence acquired from the translation unit 26. In addition, after displaying these text sentences or at the same time, the display unit 27 displays speech instruction text information that prompts the first speaker or the second speaker to speak.

Note that the display unit 27 changes the screen layout for displaying the text sentence according to the positional relationship between the first speaker and the second speaker with respect to the speech translation device 1. For example, as shown in FIGS. 1A and 1B, when the first speaker speaks, the display unit 27 displays the first text sentence whose voice has been recognized in the display area of the display unit 27 located on the first speaker's side. Then, the translated second translated text sentence is displayed in the display area of the display unit 27 located on the second speaker side. Furthermore, when the second speaker speaks, the display unit 27 displays the second text sentence that has been voice-recognized in the display area of the display unit 27 located on the second speaker's side, and The translated first translated text sentence is displayed in the display area of the display unit 27. In these cases, the display unit 27 displays the first text sentence and the second translated text sentence in an upside-down manner, and the first translated text sentence and the second translated text sentence in an upside-down manner. Note that, as shown in FIG. 1C, when the first speaker and the second speaker converse side by side, the display unit 27 shows that the orientation of the characters in the first text sentence and the second text sentence are the same. Display as shown.

[Audio output section 28]
The audio output unit 28 is a speaker that acquires translated audio as a result of translation by the translation unit 26 from the translation unit 26 and outputs the acquired translated audio, and is communicably connected to the translation unit 26 and the speech instruction unit 25. Ru. That is, when the first speaker speaks, the audio output unit 28 reproduces and outputs translated audio having the same content as the second translated text displayed on the display unit 27. Further, when the second speaker speaks, the audio output unit 28 reproduces and outputs translated audio having the same content as the first translated text sentence displayed on the display unit 27.

Furthermore, upon acquiring the speech instruction audio information, the audio output unit 28 reproduces and outputs the audio that urges the first speaker or the second speaker to speak as indicated in the speech instruction audio information. After outputting the translated voice of the first translated text sentence or the second translated text sentence, the voice output unit 28 reproduces and outputs the voice indicated by the speech instruction voice information.

[Power supply section 29]
The power supply section 29 is, for example, a primary battery or a secondary battery, and is connected via wiring to the voice input section 21, voice detection section 22, priority speech input section 24, speech instruction section 25, speech recognition section 23, translation section 26, It is electrically connected to the display section 27, audio output section 28, and the like. The power supply unit 29 supplies power to the voice detection unit 22, priority speech input unit 24, speech instruction unit 25, voice recognition unit 23, translation unit 26, display unit 27, voice output unit 28, and the like.

<Operation>
The operation performed by the speech translation device 1 configured as described above will be explained using FIG. 3.

FIG. 3 is a flowchart showing the operation of the speech translation device 1 in the first embodiment.

The speech translation device 1 is set in advance so that the first speaker speaks in the first language, and it is set in advance that the second speaker speaks in the second language. Here, it is assumed that one of the first speaker and the second speaker starts speaking. The first speaker activates the speech translation device 1, and the speech translation device 1 starts translating the conversation between the first speaker and the second speaker.

First, as shown in FIG. 3, when a first speaker and a second speaker have a conversation, the speech translation device 1 is activated before uttering a voice. The speech translation device 1 acquires a sound (S11) and generates an acoustic signal indicating the acquired sound. In this embodiment, when one speaker starts speaking, the speech translation device 1 acquires the voice uttered by the other speaker. As shown in FIG. 1A, when one speaker is the first speaker and utters "What are you looking for?", the voice input unit 21 acquires the uttered voice. The audio input unit 21 acquires sound, converts the acquired sound into an electrical signal, and outputs an acoustic signal, which is the converted electrical signal, to the audio detection unit 22 .

Next, when the audio detection unit 22 acquires the audio signal from the audio input unit 21, the audio detection unit 22 detects the audio section of one speaker from the sound indicated in the audio signal (S12), thereby converting the detected audio into one of the speakers. Extract the voice as the person's voice. For example, as shown in FIG. 1A, the first speaker's voice section “What are you looking for?” is detected from the sound input to the voice input unit 21, and the detected voice is extracted. . The voice detection unit 22 outputs voice information indicating the extracted voice of one speaker to the voice recognition unit 23.

The speech instruction unit 25 outputs instruction commands for recognizing speech in the language spoken by one speaker to the speech recognition unit 23, and outputs instructions for translating the recognized speech from one language to another language. The command is output to the translation section 26. That is, the speech instruction section 25 outputs an instruction command for switching the language recognized by the speech recognition section 23 so that the speech recognition section 23 can recognize the language spoken by one of the speakers. Furthermore, the speech instruction section 25 outputs an instruction command for switching the translation language so that the translation section 26 can translate in a desired language based on the language voice recognized by the speech recognition section 23 .

For example, upon acquiring the instruction command, the speech recognition unit 23 switches the recognition language from the second language to the first language, or from the first language to the second language. Moreover, upon acquiring the instruction command, the translation unit 26 switches the translation language from the second language to the first language or from the first language to the second language.

Next, upon acquiring the instruction command and the voice information, the voice recognition unit 23 recognizes the voice indicated by the voice information (S13). For example, if the language of one speaker is the first language, the speech recognition unit 23 selects the first language as the recognition language and recognizes the speech indicated in the speech information in the selected first language. That is, the speech recognition section 23 converts the speech indicated by the speech information into a text sentence in the first language, and outputs the converted first text sentence to the translation section 26 . Further, if the language of one speaker is the second language, the speech recognition unit 23 selects the second language as the recognition language, and performs speech recognition of the speech indicated in the speech information in the selected second language. That is, the speech recognition section 23 converts the speech indicated by the speech information into a text sentence in the second language, and outputs the converted second text sentence to the translation section 26 .

For example, as shown in FIG. 1A, the voice recognition unit 23 converts the voice "What are you looking for?" shown in the voice information into the first text sentence "What are you looking for?" do.

Next, upon acquiring the text sentence from the speech recognition unit 23, the translation unit 26 translates it from one of the first language and the second language into the other language (S14). That is, if the text sentence is a first text sentence in the first language, the translation unit 26 translates it into the second language, and generates a second translated text sentence as a result of the translation. Furthermore, if the text sentence is a second text sentence in the second language, the translation unit 26 translates it into the first language, and generates a first translated text sentence as a result of the translation. For example, as shown in FIG. 1A, the translation unit 26 translates the first text sentence "What are you looking for?" in the first language into the second language, and translates the first text sentence "What are you looking for?" in the first language into the second translated text sentence "What are you looking for?" "What are you looking for?"

Next, the translation unit 26 outputs the generated second translated text sentence in the second language or the first translated text sentence in the first language to the display unit 27. The display unit 27 displays the second translated text sentence or the first translated text sentence (S15). For example, as shown in FIG. 1A, the display unit 27 displays the second translated text sentence "What are you looking for?".

Moreover, when the translation unit 26 generates the second translated text sentence, it generates translated speech in the second language by converting the second translated text sentence into speech. Moreover, when the translation unit 26 generates the first translated text sentence, it generates translated audio in the first language by converting the first translated text sentence into audio. The translation section 26 outputs the generated second language translated speech or first language translated speech to the speech output section 28 . The audio output unit 28 outputs the translated audio in the second language or the translated audio in the first language (S16). For example, as shown in FIG. 1A, the voice output unit 28 outputs the second translated text sentence "What are you looking for?" in voice. Note that the processing in steps S15 and S16 may be performed at the same timing, or may be performed at opposite times.

Next, the speech instruction section 25 determines whether an instruction command has been obtained from the priority speech input section 24 (S17). For example, if one of the speakers wants to speak again, the operator of the speech translation device 1 operates the priority speech input section 24. Thereby, the priority speech input section 24 outputs an instruction command to the speech instruction section 25 upon receiving the operation.

When the speech instruction section 25 acquires the instruction command from the priority speech input section 24 (YES at S17), the speech recognition section 23 and the translation section 26 terminate and interrupt the speech recognition and translation processing of one speaker, or , even if the process has shifted to the process of recognizing the voice of the other speaker, the process returns to the process of recognizing and translating the voice uttered by one speaker. The utterance instruction unit 25 sends a message to one of the speakers again, in order to give priority to the voice uttered by the one speaker whose voice has been voice-recognized most recently. Speech instruction text information, which is content to encourage speech, is output to the display unit 27. The display section 27 displays the speech instruction text information acquired from the speech instruction section 25 (S18). For example, the display unit 27 displays speech instruction text information "Please speak again."

Further, when the speech instruction section 25 acquires an instruction command from the priority speech input section 24, it outputs speech instruction voice information that prompts one speaker to speak to the voice output section 28. The audio output unit 28 outputs the speech instruction audio information acquired from the speech instruction unit 25 as a sound (S19). For example, the audio output unit 28 outputs the utterance instruction audio information "Please speak again" as a voice.

In this case, the speech translation device 1 may display or output a message such as "Thank you for your patience" to the other speaker, or may output nothing. Note that the processes in steps S18 and S19 may be performed simultaneously, or the processes may be reversed.

Further, the speech instruction section 25 may cause the speech output section 28 to output the speech instruction voice information a prescribed number of times. After the speech instruction section 25 outputs the speech instruction voice information a predetermined number of times, the speech instruction section 25 may cause the display section 27 to output the message of the speech instruction voice information.

Then, the speech translation device 1 ends the process. As a result, one of the speakers speaks again, and the speech translation device 1 starts processing from step S11.

On the other hand, if the utterance instruction unit 25 cannot obtain the instruction command from the priority utterance input unit 24 (NO in S17), the utterance instruction unit 25 outputs utterance instruction text information to the display unit 27, which is the content that prompts the other speaker to speak. For example, in this case, one speaker does not need to speak again and the speech is correctly recognized. The display section 27 displays the speech instruction text information acquired from the speech instruction section 25 (S21). For example, as shown in FIG. 1A, the display unit 27 displays speech instruction text information "Your Turn!".

Furthermore, when the instruction command cannot be obtained from the priority speech input section 24, the speech instruction section 25 outputs speech instruction voice information that urges the other speaker to speak to the voice output section 28. The audio output unit 28 outputs the speech instruction audio information acquired from the speech instruction unit 25 as a sound (S22). For example, the audio output unit 28 outputs the utterance instruction audio information "Your Turn!" as a voice. Note that the processes in steps S21 and S22 may be performed simultaneously, or the processes may be reversed.

Furthermore, the speech instruction section 25 may cause the speech output section 28 to output a voice for encouraging speech a specified number of times. The utterance instruction unit 25 may output a voice to encourage the user to speak a specified number of times, and then cause the display unit 27 to output a message to encourage the user to speak.

Then, the speech translation device 1 ends the process. As a result, one of the speakers speaks again, and the speech translation device 1 starts processing from step S11.

In this way, the speech translation device 1 can translate the conversation between the first speaker and the second speaker just by the first speaker operating the speech translation device 1 for the first time.

Note that the same process is performed when one speaker speaks in response to the other speaker's utterance, so the explanation thereof will be omitted.

<Effect>
Next, the effects of the speech translation device 1 in this embodiment will be explained.

As described above, the speech translation device 1 in this embodiment is a first speaker who speaks in a first language, and a conversation partner of the first speaker who speaks in a second language different from the first language. The speech translation device 1 is for a conversation between two speakers, and includes a speech detection section 22 that detects the speech sections uttered by the first speaker and the second speaker from the sounds input to the speech input section 21. Then, the voice in the voice section detected by the voice detection unit 22 is voice recognized, and the result of translation from the first language indicated by the voice to the second language is displayed, and the translation result from the second language to the first language is displayed. a display section 27 that displays the translation result translated into a second language; The speech instructing section 25 outputs, via the display section 27, the content for prompting the first speaker to speak in the first language after the speech.

According to this, by detecting each speech interval from a conversation between a first speaker and a second speaker, a translation result of the detected speech from the first language to the second language can be obtained, It is possible to obtain a translation result obtained by translating the detected voice from the second language to the first language. In other words, this speech translation device 1 automatically converts the language of the detected speech into a different language for each utterance by the first speaker and the second speaker, without any input operation for translation. Can be translated into languages.

The speech translation device 1 also outputs content that prompts the second speaker to speak after the first speaker speaks, or outputs content that prompts the first speaker to speak after the second speaker speaks. You can As a result, in this speech translation device 1, the first speaker and the second speaker do not have to perform an input operation to start the utterance for each utterance by the first speaker and the second speaker. be able to recognize when to do something.

As described above, the speech translation device 1 does not require input operations to start speaking, input operations to switch languages, etc., and is excellent in operability. In other words, since the operation of this speech translation device 1 does not take much time, it is possible to suppress an increase in the period of use.

Therefore, the speech translation device 1 can suppress an increase in energy consumption of the speech translation device 1 by simplifying the operation. In particular, with this speech translation device 1, since the operation can be simplified, it is also possible to suppress erroneous operations.

In addition, the speech translation method in the present embodiment includes a first speaker who speaks in a first language, and a second speaker who is a conversation partner of the first speaker and who speaks in a second language different from the first language. is a voice translation method for carrying out a conversation, which includes detecting a voice section uttered by a first speaker and a second speaker from sounds input to a voice input section 21, and translating the voice of the detected voice section. By recognizing the voice, the display unit 27 displays the translation result translated from the first language indicated by the voice to the second language, and displays the translation result translated from the second language to the first language. In addition, content that prompts the second speaker to speak after the first speaker speaks is outputted in the second language via the display unit 27, and the content prompting the first speaker to speak after the second speaker speaks This includes outputting the prompting content in the first language via the display unit 27.

This voice translation method also provides the same effects as the voice translation device 1 described above.

Further, the program in this embodiment is a program for causing a computer to execute a speech translation method.

This program also has the same effects as the above-mentioned speech translation device 1.

Furthermore, when the first speaker or the second speaker makes an utterance and the speech is recognized, the speech translation device 1 according to the present embodiment gives priority to the utterance of the first speaker or the second speaker who made the utterance. A priority speech input unit 24 is provided for speech recognition.

According to this, for example, when the speakers who are the first speaker and the second speaker make a mistake in saying something, or when a voice that they hesitate to say is translated halfway, etc., by operating the priority speech input section 24, the utterance can be made. Since priority is given to the speaker who has spoken, the speaker who has spoken can have an opportunity to speak again (can rephrase). For this reason, after the priority speech input unit 24 has finished recognizing the voice uttered by one of the first speaker and the second speaker, the priority speech input unit 24 performs processing for voice recognition of the voice of the other speaker. Even after the transition, the voice uttered by one speaker can be returned to the voice recognition process. Thereby, the speech translation device 1 can reliably acquire the voices of the first speaker and the second speaker, and therefore can output a translation result translated based on the voices.

The speech translation device 1 according to the present embodiment further includes a speech input section 21 into which the speech of the conversation between the first speaker and the second speaker is input, and a speech detection section 22 that converts the speech of the speech section detected by the speech detection section 22 into speech. A speech recognition unit 23 that converts the text into a text sentence by recognition, and a translation that translates the text text converted by the speech recognition unit 23 from a first language to a second language, and from the second language to the first language. unit 26, and an audio output unit 28 that outputs the result translated by the translation unit 26 as a voice.

According to this, it is possible to perform speech recognition on input speech and then translate the language of the speech into another language. In other words, the speech translation device 1 can perform processing from obtaining the speech of a conversation between the first speaker and the second speaker to outputting the result of translating the speech. Therefore, the speech translation device 1 can mutually translate the voices of the first speaker and the second speaker in conversation without communicating with an external server. The present invention can be applied even in environments where it is difficult for the speech translation device 1 to communicate with an external server.

In the speech translation device 1 according to the present embodiment, the speech instruction section 25 outputs content that prompts the first speaker to speak in the first language via the display section 27 when the speech translation device 1 is started, After the voice uttered by the first speaker is translated from the first language to the second language and the translation result is displayed on the display unit 27, content that prompts the second speaker to speak is displayed on the display unit 27. Output in the second language.

According to this, if it is registered in advance that the second speaker speaks in the second language after the first speaker speaks in the first language, when the speech translation device 1 is started, the first speaker The first speaker can start speaking by outputting content in the first language that prompts the first speaker to speak. Therefore, when the speech translation device 1 is activated, mistranslation caused by the second speaker speaking in the second language can be suppressed.

In the speech translation device 1 according to the present embodiment, after the start of translation, the speech instruction section 25 causes the speech output section 28 to output a voice for encouraging speech a specified number of times, and outputs a voice for encouraging speech a specified number of times. After that, a message to prompt the user to speak is outputted on the display unit 27.

According to this, an increase in energy consumption of the speech translation device 1 can be suppressed by limiting the number of times the voice for prompting the user to speak is limited to the specified number of times.

(Embodiment 2)
<Configuration>
The configuration of the speech translation device 1a of this embodiment will be explained using FIG. 4.

FIG. 4 is a block diagram showing a speech translation device 1a in the second embodiment.

This embodiment differs from Embodiment 1 in that the direction of the sound source is estimated.

Other configurations in this embodiment are the same as those in Embodiment 1 unless otherwise specified, and the same configurations are denoted by the same reference numerals and detailed explanations regarding the configurations will be omitted.

As shown in FIG. 4, the speech translation device 1a includes a speech detection section 22, a priority speech input section 24, a speech instruction section 25, a speech recognition section 23, a translation section 26, a display section 27, a speech output section 28, and a power supply section 29. In addition, it includes a plurality of audio input units 21 and a sound source direction estimation unit 31.

[Multiple audio input units 21]
The plurality of audio input units 21 constitute a microphone array. Specifically, the microphone array is made up of two or more microphone units arranged apart from each other, and acquires audio, and acquires an acoustic signal converted from the acquired audio into an electrical signal.

The plurality of audio input units 21 output the acquired acoustic signals to the sound source direction estimation unit 31. Furthermore, at least one of the plurality of audio input units 21 outputs an acoustic signal to the audio detection unit 22. In this embodiment, one audio input section 21 is communicably connected to the audio detection section 22 and outputs an acoustic signal to the audio detection section 22 .

In this embodiment, two voice input units 21 are provided in the voice translation device 1a, and one voice input unit 21 is spaced apart from the other voice input unit 21 by a distance that is equal to or less than 1/2 wavelength of the voice. It will be placed in

[Sound source direction estimation unit 31]
The sound source direction estimating unit 31 estimates the sound source direction by signal processing the sounds input to the plurality of audio input units 21 . Specifically, when the sound source direction estimation unit 31 acquires the audio information from the audio detection unit 22 and the acoustic signals from the multiple audio input units 21, the sound source direction estimation unit 31 detects each of the multiple audio input units 21 constituting the microphone array. The time difference (phase difference) between the voices reaching the destination is calculated, and the direction of the sound source is estimated using, for example, a delay time estimation method. In other words, if the voice detection section 22 is able to detect a voice section, it means that the voice of the first speaker or the second speaker has been input to the voice input section 21. Therefore, the sound source direction estimation section 31 acquires voice information. is used as a trigger to start estimating the direction of the sound source.

The sound source direction estimation unit 31 outputs sound source direction information indicating the sound source direction, which is the result of the estimation, to the speech instruction unit 25.

[Speech instruction section 25]
The speech instruction section 25 includes a control section 31a that controls the display mode on the display section 27. Specifically, the control unit 31a displays the first language in the display area of the display unit 27 corresponding to the position of the first speaker with respect to the voice translation device 1a, and displays the first language in the display area of the display unit 27 corresponding to the position of the first speaker with respect to the voice translation device 1a. The second language is displayed in the display area of the corresponding display section 27. For example, as shown in FIG. 1A, the display area of the display unit 27 corresponding to the position of the first speaker is the display area of the display unit 27 on the first speaker's side where Japanese is displayed. Further, the display area of the display unit 27 corresponding to the position of the second speaker is the display area of the display unit 27 on the second speaker side where English is displayed.

The control unit 31a is a display direction from the display unit 27 of the speech translation device 1a toward the first speaker or the second speaker, and a display direction that is displayed in any display area of the display unit 27; The sound source direction estimated by the sound source direction estimation unit 31 is compared. When the display direction and the sound source direction substantially match, the control section 31a causes the speech recognition section 23 and the translation section 26 to execute. For example, as shown in FIG. 1A, when the first speaker speaks, a first text sentence indicating the content of the first speaker's voice input into the speech translation device 1a is transmitted to the first speaker (or the first speaker). displayed in the display area (on the side facing the person). In this case, the display direction is a direction from the display section 27 toward the first speaker, and the sound source direction estimated by the sound source direction estimation section 31 is also a direction from the display section 27 toward the first speaker.

On the other hand, if the display direction and the sound source direction are different, the control section 31a stops the speech recognition section 23 and the translation section 26. When the first speaker speaks, even if the first text sentence indicating the content of the first speaker's voice is displayed in the first speaker's display area, the sound source direction estimated by the sound source direction estimation unit 31 is displayed on the display area. 27 toward the second speaker, the display direction and the estimated sound source direction do not match. For example, if the first speaker continues speaking without operating the priority speech input section 24 after the first speaker has spoken, the voice input section 21 may have picked up surrounding sounds unrelated to the conversation. Cases etc.

Further, when the control section 31a stops the speech recognition section 23 and the translation section 26, the speech instruction section 25 outputs the content encouraging speech in the instructed language again. For example, since the display direction and the estimated sound source direction do not match, it is not possible to determine which speaker made the utterance. I don't know if it's okay to use voice recognition. Further, even if the first speaker speaks, if the voice cannot be recognized, translation cannot be performed. Therefore, the control section 31a stops the speech recognition section 23 and the translation section 26.

<Operation>
The operation performed by the speech translation device 1a configured as described above will be explained using FIG. 5.

FIG. 5 is a flowchart showing the operation of the speech translation device 1a in the second embodiment.

Processes similar to those in FIG. 5 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

The speech translation device 1a acquires a sound (S11) and generates an acoustic signal indicating the acquired sound.

Next, the sound source direction estimation unit 31 determines whether or not audio information has been acquired from the audio detection unit 22 (S12a).

If the sound source direction estimation unit 31 does not acquire audio information from the audio detection unit 22 (NO in S12a), this means that the audio detection unit 22 cannot detect audio from the acoustic signal, so the sound source direction estimation unit 31 acquires the audio information. cannot be obtained. That is, this is a case where the first speaker and the second speaker are not having a conversation. In this case, the process of step S12a is repeated.

When the sound source direction estimation unit 31 acquires audio information from the audio detection unit 22 (YES in S12a), this is a case where at least one of the first speaker and the second speaker speaks. In this case, the sound source direction estimating unit 31 calculates the time difference (phase difference) of sounds included in the acoustic signals acquired from each of the plurality of audio input units 21, and estimates the sound source direction (S31). The sound source direction estimation unit 31 outputs sound source direction information indicating the sound source direction, which is the result of the estimation, to the speech instruction unit 25.

Next, the control unit 31a of the sound source direction estimation unit 31 determines whether the display direction and the estimated sound source direction substantially match (S32).

If the display direction and the sound source direction are different (NO in S32), the control unit 31a stops the speech recognition unit 23 and the translation unit 26. When the control unit 31a stops the speech recognition unit 23 and the translation unit 26, the speech instruction unit 25 again outputs content encouraging speech in the instructed language.

Specifically, the speech instruction section 25 outputs speech instruction text information that prompts one speaker to speak to the display section 27. The display section 27 displays the speech instruction text information acquired from the speech instruction section 25 (S33).

Furthermore, the speech instruction section 25 outputs speech instruction voice information that prompts one speaker to speak to the voice output section 28 . The audio output unit 28 outputs the speech instruction audio information acquired from the speech instruction unit 25 as a sound (S34).

Then, the speech translation device 1a ends the process. As a result, one of the speakers speaks again, and the speech translation device 1a starts processing from step S11.

When the display direction and the sound source direction substantially match (YES in S32), the control unit 31a causes the speech recognition unit 23 and the translation unit 26 to execute. The speech translation device 1a then proceeds to step S13 and performs the same process as in FIG. 3.

<Effect>
Next, the effects of the speech translation device 1a in this embodiment will be explained.

As described above, in the speech translation device 1a according to the present embodiment, a plurality of speech input units 21 are provided. The speech translation device 1a further includes a sound source direction estimation unit 31 that estimates a sound source direction by signal processing the speech input to the plurality of speech input units 21, and a first speaker for the speech translation device 1a. a control unit 31a that displays a first language in a display area of the display unit 27 corresponding to the position of the speech translation device 1a, and displays a second language in a display area of the display unit 27 that corresponds to the position of the second speaker with respect to the speech translation device 1a; Equipped with. The control unit 31a controls the display direction from the display unit 27 of the speech translation device 1a toward the first speaker or the second speaker, and the display direction in which the display is displayed in any display area of the display unit 27. and the sound source direction estimated by the sound source direction estimating unit 31, and if the display direction and the sound source direction substantially match, the speech recognition unit 23 and the translation unit 26 are executed, and the display direction and the sound source direction are If different, the speech recognition unit 23 and translation unit 26 are stopped.

According to this, when the display direction of the language displayed in the display area of the display unit 27 and the sound source direction of the sound uttered by the speaker substantially match, the first language uttered by the speaker in the first language is It is possible to identify whether the speaker is the speaker or the second speaker speaking in the second language. In this case, the first speaker's voice can be recognized in the first language, and the second speaker's voice can be recognized in the second language. Further, when the display direction and the sound source direction are different, by stopping the translation of the input voice, it is possible to prevent the input voice from not being translated or being mistranslated.

Thereby, the speech translation device 1a can reliably recognize the first language speech and the second language speech, and therefore can reliably translate the speech. As a result, this speech translation device 1a can suppress an increase in the processing amount of the speech translation device 1a by suppressing mistranslations and the like.

In the speech translation device 1a according to the present embodiment, when the control section 31a stops the speech recognition section 23 and the translation section 26, the speech instruction section 25 again outputs content encouraging speech in the instructed language.

According to this, even if the display direction and the sound source direction are different, the speech instruction unit 25 outputs the content encouraging speech again, so that the target speaker speaks. Therefore, the speech translation device 1a can reliably acquire the speech of the target speaker, and therefore can translate the speech more reliably.

The speech translation device 1a in this embodiment also has the same effects as in the first embodiment.

(Modification of Embodiment 2)
Unless otherwise specified, other configurations in this modification are the same as those in Embodiment 1, and the same configurations are denoted by the same reference numerals and detailed explanations regarding the configurations will be omitted.

The operation performed by the speech translation device 1a configured in this way will be explained using FIG. 6.

FIG. 6 is a flowchart showing the operation of the speech translation device 1a in a modification of the second embodiment.

Processes similar to those in FIG. 5 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

In the processing of the speech translation device 1a, after going through the processing of steps S11 to S31, if NO in step S32, the control unit 31a compares the display direction and the sound source direction and determines whether a specified period has elapsed. Determination is made (S32a).

If the specified period has not elapsed since the comparison between the display direction and the sound source direction (NO in S32a), the control unit 31a returns the process to step S32a.

If the specified period has elapsed since the comparison between the display direction and the sound source direction (YES in S32a), the control unit 31a advances the process to step S33 and performs the same process as in FIG. 5.

In this way, in the speech translation device 1a according to the present modification, when the display direction and the sound source direction are different, the speech instruction section 25 issues an instruction again after a specified period has elapsed since the control section 31a made the comparison. Output content that encourages verbal utterances.

According to this, by leaving a specified period after comparing the display direction and the sound source direction, it is possible to suppress the voices of the first speaker and the second speaker from being input together. . As a result, after the predetermined period of time has elapsed, the target speaker speaks by outputting the content encouraging him to speak again. Therefore, the speech translation device 1a can more reliably acquire the speech of the target speaker, and therefore can translate the speech more reliably.

The speech translation device 1a in this modification also provides the same effects as in the second embodiment.

(Embodiment 3)
<Configuration>
The configuration of the speech translation device 1b of this embodiment will be explained using FIG. 7.

FIG. 7 is a block diagram showing a speech translation device 1b in the third embodiment.

This embodiment differs from Embodiment 1 etc. in that the direction of the sound source is estimated.

Unless otherwise specified, other configurations in this embodiment are the same as those in Embodiment 1, etc., and the same configurations are denoted by the same reference numerals and detailed explanations regarding the configurations will be omitted.

The speech translation device 1b includes a speech detection section 22, a priority speech input section 24, a speech instruction section 25, a speech recognition section 23, a translation section 26, a display section 27, a speech output section 28, a power supply section 29, and a sound source direction estimation section 31. In addition, it includes a plurality of audio input sections 21, a first beam former section 41, a second beam former section 42, and an input switching section 32.

[Multiple audio input units 21]
The plurality of audio input units 21 constitute a microphone array. Each of the plurality of audio input units 21 outputs the acquired acoustic signal to the first beam former unit 41 and the second beam former unit 42 . In this embodiment, an example is shown in which two audio input units 21 are used.

[First beamformer section 41 and second beamformer section 42]
The first beamformer unit 41 performs signal processing on the acoustic signal of the voice input to at least some of the voice input units 21 of the plurality of voice input units 21, thereby directing the voice toward the source of the voice of the first speaker. Controls the directionality of sound collection. Furthermore, the second beam former unit 42 processes the acoustic signals of the voices input to at least some of the voice input units 21 of the plurality of voice input units 21 to generate a sound source of the voice by the second speaker. Controls the directivity of sound collection. In this embodiment, the first beamformer section 41 and the second beamformer section 42 perform signal processing on the acoustic signals obtained from each of the plurality of audio input sections 21.

Thereby, the first beamformer section 41 and the second beamformer section 42 suppress input of sound in directions other than the predetermined direction by controlling the directivity of sound collection in the predetermined direction. The predetermined directions are, for example, respective sound source directions of voices uttered by the first speaker and the second speaker.

In this embodiment, the first beamformer section 41 is arranged on the first speaker side and is communicably connected to each of the plurality of audio input sections 21, and the second beamformer section 42 is arranged on the side of the second speaker. It is disposed on the side and is communicably connected to each of the plurality of audio input units 21 . Each of the first beamformer section 41 and the second beamformer section 42 outputs, to the input switching section 32, an acoustic processing signal that is a result of signal processing of the acoustic signal acquired from each of the plurality of audio input sections 21.

[Speech instruction section 25]
The speech instruction section 25 causes the input switching section 32 to switch between acquiring the output signal of the first beamformer section 41 and the output signal of the second beamformer section 42 . Specifically, when the speech instruction unit 25 acquires the sound source direction information indicating the sound source direction that is the result of estimation from the sound source direction estimation unit 31, the speech instruction unit 25 calculates the direction of the sound source indicated by the sound source direction information and the direction of the sound collected by the beamformer unit. The direction is compared with a predetermined direction. The speech instruction unit 25 selects a beamformer unit in which the sound source direction and the predetermined direction substantially match or are close to each other.

The speech instruction section 25 outputs a switching command to the input switching section 32 so as to output the output signal of the beamformer section selected from the first beamformer section 41 and the second beamformer section 42 .

[Input switching section 32]
The input switching unit 32 is a device that acquires the output signal of the first beamformer unit 41 and the output signal of the second beamformer unit 42 and switches the output signal to be output to the audio detection unit 22. The input switching unit 32 switches the signal to be acquired to the output signal of the first beamformer unit 41 or the output signal of the second beamformer unit 42. Specifically, by acquiring a switching command from the speech instruction section 25, the input switching section 32 changes the output signal of the first beamformer section 41 from the output signal of the second beamformer section 42 or the second beam The output signal of the former section 42 is switched to the output signal of the first beam former section 41. The input switching unit 32 outputs the output signal of the first beamformer unit 41 to the audio detection unit 22 or outputs the output signal of the second beamformer unit 42 to the audio detection unit 22 according to a switching command.

The input switching section 32 is communicably connected to the first beamformer section 41, the second beamformer section 42, the voice detection section 22, and the speech instruction section 25.

<Operation>
The operation performed by the speech translation device 1b configured as described above will be explained.

FIG. 8 is a flowchart showing the operation of the speech translation device 1b in the third embodiment.

Processes similar to those in FIG. 5 and the like are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

As shown in FIG. 8, in the processing of the speech translation device 1b, after the processing of steps S11, S12a, S31, and S32, if the control unit 31a determines that the display direction and the sound source direction substantially match (S32 (YES), the speech instruction section 25 outputs a switching command to the input switching section 32 (S51).

Specifically, when the first speaker and the second speaker speak, the first beamformer unit 41 in the two audio input units 21 is configured to listen to the first speaker's utterances more than the second speaker's utterances. The second beamformer unit 42 has high sensitivity to speech, and the second beam former unit 42 has higher sensitivity to speech by the second speaker than to speech by the first speaker.

Therefore, if the display direction is the display area of the display section 27 on the side of the first speaker, the first beamformer section 41 has higher sensitivity to the first speaker's utterance, so the utterance instruction section 25 outputs a switching command to the input switching unit 32 so as to output the output signal of the first beamformer unit 41. In this case, the input switching section 32 outputs the output signal of the first beamformer section 41 upon acquiring the switching command.

Furthermore, if the display direction is the display area of the display section 27 on the second speaker's side, the second beamformer section 42 has higher sensitivity to the second speaker's speech, so the speech instruction section 25 , outputs a switching command to the input switching unit 32 so as to output the output signal of the second beamformer unit 42. In this case, the input switching section 32 outputs the output signal of the second beamformer section 42 upon acquiring the switching command.

The speech translation device 1b then proceeds to step S12 and performs the same process as in FIG. 5.

<Effect>
Next, the effects of the speech translation device 1b in this embodiment will be explained.

As described above, in the speech translation device 1b according to the present embodiment, a plurality of speech input units 21 are provided. Further, the speech translation device 1b further performs signal processing on the speech input to at least some of the speech input sections 21 of the plurality of speech input sections 21, thereby converging the speech from the first speaker in the direction of the sound source. The first beamformer section 41 that controls the directivity of sound and the sound input to at least some of the plurality of sound input sections 21 are subjected to signal processing to generate the sound of the second speaker. a second beamformer section 42 that controls the directivity of sound collection in the direction of the sound source; and an input switch that switches the acquired signal to the output signal of the first beamformer section 41 or the output signal of the second beamformer section 42. section 32, and a sound source direction estimation section 31 that estimates the direction of the sound source by signal processing the sounds input to the plurality of sound input sections 21. Then, the speech instruction section 25 causes the input switching section 32 to switch between acquiring the output signal of the first beamformer section 41 and the output signal of the second beamformer section 42 .

According to this, the direction of the speaker relative to the speech translation device 1b can be estimated by the sound source direction estimation unit 31. Therefore, the input switching section 32 can switch to either the output signal of the first beamformer section 41 or the output signal of the second beamformer section 42 suitable for the direction of the speaker. In other words, since the directionality of the sound collection of the beamformer unit can be directed toward the direction of the sound source, the speech translation device 1b can collect the sounds of the first speaker and the second speaker while reducing ambient noise. Can be done.

The speech translation device 1b in this embodiment also has the same effects as in the first embodiment.

(Modification of Embodiment 3)
The speech translation device 1c of this modification will be explained using FIG. 9.

FIG. 9 is a block diagram showing a speech translation device 1c in a modification of the third embodiment.

Unless otherwise specified, other configurations in this modification are the same as those in Embodiment 1, etc., and the same configurations are denoted by the same reference numerals and detailed explanations regarding the configurations will be omitted.

As shown in FIG. 9, the first beamformer section 41 and the second beamformer section 42 are communicably connected to each of the plurality of audio input sections 21, and are connected to the sound source direction estimation section 31 and the input switching section 32. Connected for communication.

Acoustic signals from each of the plurality of audio input sections 21 are input to the first beam former section 41 and the second beam former section 42 . The first beamformer section 41 and the second beamformer section 42 perform signal processing on each of the input acoustic signals, and transfer the respective acoustic processed signals that are the result of the signal processing to the sound source direction estimation section 31 and the input switching section. The output signal is output to each of the sections 32.

That is, in this modification, each of the plurality of audio input units 21 is communicably connected to the first beamformer unit 41 and the second beamformer unit 42, and is communicably connected to the sound source direction estimation unit 31. do not have.

In this way, the sound source direction estimating unit 31 receives an acoustic signal with enhanced directivity of sound collection in the direction of the sound source of the speaker's voice by the first beam former unit 41 and the second beam former unit 42 .

In such a speech translation device 1c in this modification, a plurality of speech input sections 21 are provided. Further, the speech translation device 1c further performs signal processing on the speech input to at least some of the plurality of speech input sections 21, so that the speech translated by the first speaker is focused in the direction of the sound source. The first beamformer section 41 that controls the directivity of sound and the sound input to at least some of the plurality of sound input sections 21 are subjected to signal processing to generate the sound of the second speaker. By signal processing the output signals of the second beamformer section 42, which controls the directivity of sound collection in the direction of the sound source, the output signals of the first beamformer section 41, and the output signals of the second beamformer section 42, and a sound source direction estimation section 31 that estimates the direction of the sound source.

According to this, the sound source direction estimation unit 31 can estimate the direction of the speaker relative to the speech translation device 1c. For this reason, the sound source direction estimation unit 31 performs signal processing on the output signal of the first beamformer unit 41 and the output signal of the second beamformer unit 42 that are suitable for the direction of the speaker, thereby reducing the calculation cost due to signal processing. be able to.

The speech translation device 1c in this modification also provides the same effects as in the first embodiment described above.

(Embodiment 4)
<Configuration>
The configuration of the speech translation device 1d of this embodiment will be explained using FIG. 10.

FIG. 10 is a block diagram showing a speech translation device 1d in the fourth embodiment.

This embodiment differs from Embodiment 1 in that the speech translation device 1d includes a score calculation unit 43.

Unless otherwise specified, the configuration in this embodiment is the same as that in Embodiment 1, etc., and the same components are denoted by the same reference numerals, and detailed explanations regarding the configuration will be omitted.

As shown in FIG. 10, the speech recognition section 23 of the speech translation device 1d includes a score calculation section 43.

[Score calculation unit 43]
The score calculation unit 43 calculates the result of voice recognition of the voice and the reliability score of the result, and outputs the calculated reliability score to the speech instruction unit 25. The reliability score indicates the accuracy (similarity) of speech recognition when the speech indicated by the speech information acquired from the speech detection unit 22 is speech recognized. For example, the score calculation unit 43 compares a text sentence obtained by converting the voice shown in the voice information with the voice shown in the voice information, and calculates a reliability score representing the degree of similarity between the text sentence and the voice.

Note that the score calculation section 43 does not need to be included in the speech recognition section 23, and may be a separate device independent from the speech recognition section 23.

[Speech instruction section 25]
The speech instruction unit 25 determines the accuracy of speech recognition by evaluating the reliability score obtained from the score calculation unit 43 of the speech recognition unit 23. Specifically, the speech instruction unit 25 determines whether the reliability score obtained from the score calculation unit 43 of the speech recognition unit 23 is less than or equal to a threshold value. When the reliability score is less than or equal to the threshold, the utterance instruction unit 25 displays content that prompts utterance through at least one of the display unit 27 and the audio output unit 28 without translating the voice whose reliability score is less than or equal to the threshold. and output it. The speech instruction unit 25 translates the speech when the reliability score is higher than the threshold.

<Operation>
The operation performed by the speech translation device 1d configured as described above will be explained.

FIG. 11 is a flowchart showing the operation of the speech translation device 1d in the fourth embodiment.

Processes similar to those in the figures are given the same reference numerals, and descriptions thereof will be omitted as appropriate.

In the processing of the speech translation device 1d, after the processing of steps S11 to S13, the score calculation section 43 of the speech recognition section 23 calculates the reliability score of the speech recognition result, and sends the calculated reliability score to the utterance instruction section 25. (S61).

Next, upon acquiring the reliability score from the score calculation unit 43 of the speech recognition unit 23, the speech instruction unit 25 determines whether the acquired reliability score is less than or equal to a threshold value (S62).

If the reliability score is less than or equal to the threshold (YES in S62), the speech instruction unit 25 displays speech instruction text information that encourages speech again without translating the voice whose reliability score is less than or equal to the threshold. It is outputted via the section 27 (S18). The speech translation device 1d then proceeds to step S19 and performs the same processing as in FIG. 3 and the like.

If the reliability score is higher than the threshold (NO in S62), the speech instruction unit 25 proceeds to step S14 and performs the same process as in FIG. 3 and the like.

<Effect>
Next, the effects of the speech translation device 1d in this embodiment will be explained.

As described above, in the speech translation device 1d according to the present embodiment, the speech recognition unit 23 outputs the result of speech recognition of speech and the reliability score of the result, and the speech instruction unit 25 outputs the result of speech recognition of the speech and the reliability score of the result. If the reliability score obtained from 23 is less than the threshold, content to encourage speaking is outputted via at least one of the display unit 27 and the audio output unit 28 without translating the voice whose reliability score is less than the threshold. do.

According to this, if the reliability score indicating the accuracy of speech recognition is equal to or less than the threshold value, the speech instruction unit 25 outputs the content encouraging speech again, so that the target speaker speaks again. Therefore, the speech translation device 1d can reliably recognize the speech of the target speaker, and therefore can translate the speech more reliably.

Particularly, if the voice output unit 28 outputs the content that prompts the speaker to speak, the speaker will be more likely to notice that the voice is not being recognized correctly.

The speech translation device 1d in this embodiment also has the same effects as in the first embodiment described above.

(Other variations, etc.)
Although the present disclosure has been described above based on Embodiments 1 to 4 and Embodiments 2 and 4, the present disclosure is limited to Embodiments 1 to 4 and Embodiments 2 and 4, etc. isn't it.

For example, in the speech translation device, speech translation method, and program thereof according to each of the first to fourth embodiments and the second and fourth embodiments, the respective voices of the first speaker and one or more second speakers are transmitted over the network. may be stored in the cloud server by transmitting it to the cloud server via the network, and transmitting only the first text sentence and the second text sentence in which the respective voices are recognized to the cloud server via the network. You can also save it on a cloud server.

Furthermore, in the speech translation device, speech translation method, and program thereof according to each of the first to fourth embodiments and the modifications of the second and fourth embodiments, the speech recognition unit and the translation unit are not installed in the speech translation device. You can. In this case, the speech recognition section and the translation section may be engines installed in the cloud server. The speech translation device may send the acquired speech information to the cloud server, and the speech translation device may send the obtained speech information to the cloud server, and the text sentence, the translated text sentence, and the translated speech, which are the results of speech recognition and translation performed by the cloud server based on the speech information. , may be obtained from a cloud server.

Further, the speech translation methods according to the first to fourth embodiments and the modified examples of the second and fourth embodiments are realized by a program using a computer, and such a program may be stored in a storage device. .

Further, each processing unit included in the speech translation device, speech translation method, and program thereof according to each of the first to fourth embodiments and the modifications of the second and fourth embodiments is typically realized as an LSI, which is an integrated circuit. be done. These may be integrated into one chip individually, or may be integrated into one chip including some or all of them.

Further, circuit integration is not limited to LSI, and may be realized using a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells inside the LSI may be used.

In each of the first to fourth embodiments and the modified examples of the second and fourth embodiments, each component is configured with dedicated hardware or by executing a software program suitable for each component. May be realized. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

Furthermore, all the numbers used above are exemplified to specifically explain the present disclosure, and Embodiments 1 to 4 of the present disclosure and modified examples of Embodiments 2 and 4 are based on the exemplified numbers. not limited to.

Furthermore, the division of functional blocks in the block diagram is just an example; multiple functional blocks may be realized as one functional block, one functional block may be divided into multiple functional blocks, or some functions may be moved to other functional blocks. You can. Further, functions of a plurality of functional blocks having similar functions may be processed in parallel or in a time-sharing manner by a single piece of hardware or software.

Further, the order in which the steps in the flowchart are executed is for illustrative purposes to specifically explain the present disclosure, and may be in an order other than the above. Further, some of the above steps may be executed simultaneously (in parallel) with other steps.

In addition, embodiments 1 to 4 and modifications of embodiments 2 and 4 may be modified in various ways that those skilled in the art can think of, and embodiments 1 to 4 and modifications may be made without departing from the spirit of the present disclosure. The present disclosure also includes forms realized by arbitrarily combining the components and functions of the modified examples of Embodiments 2 and 4.

The present disclosure can be applied to a speech translation device, a speech translation method, and a program thereof, which are used for a plurality of speakers of different languages to communicate through conversation.

1, 1a, 1b, 1c, 1d Speech translation device 21 Speech input section 22 Speech detection section 23 Speech recognition section 24 Priority speech input section 25 Speech instruction section 26 Translation section 27 Display section 28 Speech output section 31 Sound source direction estimation section 31a Control Section 32 Input switching section 41 First beamformer section 42 Second beamformer section

Claims (13)

A voice translation device for carrying out a conversation between a first speaker who speaks in a first language and a second speaker who is a conversation partner of the first speaker and who speaks in a second language different from the first language. There it is,
a voice detection unit that detects a voice section uttered by the first speaker and the second speaker from the sound input to the voice input unit;
By performing voice recognition on the voice in the voice section detected by the voice detection unit, displaying a translation result translated from the first language indicated by the voice into the second language, and displaying the translation result from the second language to the second language. a display section that displays translation results translated into one language;
outputting content that prompts the second speaker to speak after the first speaker speaks in the second language via the display unit , after displaying the translation result, or at the same time , and and a speech instruction section that outputs content prompting the first speaker to speak in the first language via the display section , after displaying the translation result, or at the same time, after the speaker has uttered the speech translation device. . Further, when the voice uttered by the first speaker or the second speaker is voice recognized, the voice uttered by the first speaker or the second speaker whose voice has been recognized is prioritized again. The speech translation device according to claim 1, further comprising a priority speech input unit that performs speech recognition. moreover,
an audio input unit into which audio of the conversation between the first speaker and the second speaker is input;
a voice recognition unit that converts the voice in the voice section detected by the voice detection unit into a text sentence by voice recognition;
a translation unit that translates the text sentence converted by the speech recognition unit from the first language to the second language, and from the second language to the first language;
The speech translation device according to claim 1 or 2, further comprising a speech output section that outputs the result translated by the translation section as a voice. A plurality of the audio input units are provided,
moreover,
a first beam that controls the directivity of sound collection in the direction of the source of the voice of the first speaker by signal processing the voice input to at least some of the voice input units of the plurality of voice input units; Forma part and
a second beam that controls the directivity of sound collection in the direction of the sound source of the voice by the second speaker by signal processing the voice input to at least some of the voice input units of the plurality of voice input units; Forma part and
an input switching unit that switches a signal to be acquired to an output signal of the first beamformer unit or an output signal of the second beamformer unit;
a sound source direction estimation unit that estimates a sound source direction by signal processing the audio input to the plurality of audio input units;
The speech translation according to claim 3, wherein the speech instruction section causes the input switching section to switch between acquiring the output signal of the first beamformer section and acquiring the output signal of the second beamformer section. Device. A plurality of the audio input units are provided,
moreover,
a sound source direction estimation unit that estimates a sound source direction by signal processing the audio input to the plurality of audio input units;
Displaying the first language in a display area of the display unit corresponding to the position of the first speaker with respect to the voice translation device, and displaying the first language on the display unit corresponding to the position of the second speaker with respect to the voice translation device. a control unit that displays the second language in the area;
The control unit includes:
a display direction from the display section of the speech translation device toward the first speaker or the second speaker, the display direction being displayed in one of the display areas of the display section; and the sound source direction estimating section. Compare the sound source direction estimated by
If the display direction and the estimated sound source direction substantially match, causing the speech recognition unit and the translation unit to execute;
The speech translation device according to claim 3, wherein the speech recognition section and the translation section are stopped when the display direction and the estimated sound source direction are different. The speech translation device according to claim 5, wherein when the control unit stops the speech recognition unit and the translation unit, the speech instruction unit outputs content that prompts speech in the instructed language again. If the display direction and the estimated sound source direction are different, the speech instruction section outputs content encouraging speech in the instructed language again after a prescribed period has elapsed since the comparison was made by the control section. 6. The speech translation device according to 5 or 6. A plurality of the audio input units are provided,
moreover,
a first beam that controls the directivity of sound collection in the direction of the source of the voice of the first speaker by signal processing the voice input to at least some of the voice input units of the plurality of voice input units; Forma part and
a second beam that controls the directivity of sound collection in the direction of the sound source of the voice by the second speaker by signal processing the voice input to at least some of the voice input units of the plurality of voice input units; Forma part and
The speech translation device according to claim 3, further comprising: a sound source direction estimating unit that estimates a sound source direction by signal processing an output signal of the first beamformer unit and an output signal of the second beamformer unit. The speech instruction section is
When the voice translation device is activated, content that prompts the first speaker to speak is output in the first language via the display unit,
After the voice uttered by the first speaker is translated from the first language to the second language and the translation result is displayed on the display unit, the display displays content that prompts the second speaker to speak. The speech translation device according to any one of claims 1 to 8, wherein the speech translation device outputs the speech in the second language via a section. The speech instruction section is
After the translation starts, the audio output unit outputs a voice to encourage speaking a predetermined number of times,
The speech translation device according to any one of claims 3 to 8, wherein the display unit outputs a message to prompt the user to speak after outputting the voice to prompt the user to speak the predetermined number of times. The voice recognition unit outputs a result of voice recognition of the voice and a reliability score of the result,
When the reliability score acquired from the speech recognition unit is below a threshold value, the speech instruction unit displays content encouraging speech to the display unit and the above, without translating the speech whose reliability score is below the threshold value. The speech translation device according to claim 3, wherein the speech translation device outputs the speech through at least one of the speech output units. A voice translation method for carrying out a conversation between a first speaker who speaks in a first language and a second speaker who is a conversation partner of the first speaker and who speaks in a second language different from the first language. There it is,
detecting audio sections uttered by the first speaker and the second speaker from the sounds input to the audio input unit;
Displaying a translation result translated from the first language indicated by the voice into the second language by performing voice recognition on the voice in the detected voice section, and translating the voice from the second language into the first language. The display unit that displays the results displays;
outputting content that prompts the second speaker to speak after the first speaker speaks in the second language via the display unit , after displaying the translation result, or at the same time , and A speech translation method comprising: outputting content that prompts the first speaker to speak after the speaker has uttered the content in the first language via the display unit , after displaying the translation result, or at the same time . A program for causing a computer to execute the speech translation method according to claim 12.

Images