JP7284400B2 - Voice communication system, repeater and voice relay method - Google Patents

Description

The present invention relates to a voice communication system, a relay device, and a voice relay method for relaying voice communications in different languages.

A combination of AI translation, speech recognition and speech synthesis has been proposed and put into practical use. Portable translation devices (for example, Pocketalk (registered trademark)) have been put into practical use (Non-Patent Document 1), and automatic translation systems that translate languages during voice communication have been proposed (for example, Patent Document 1).

JP 2006-146732 A

“Poketalk”, [online], Sourcenext Co., Ltd., [searched on August 14, 2019], Internet <URL: https://pocketalk.jp/>

A portable translation device has a function of translating speech input through a microphone, and is assumed to be used in face-to-face conversations. Further, the automatic translation system of Patent Document 1 determines the language used by subscriber terminal A and the language used by subscriber terminal B, and translates the voice during transmission of the call voice. What language is translated into what language is determined by recognizing the utterances of both subscriber terminals A and B. FIG. That is, in Patent Document 1, translation cannot be started until communication is once started and the languages of both subscriber terminals A and B are recognized.

On the other hand, in the case of so-called wireless communication, radio waves normally reach directly from one user terminal to the user terminal of the communication partner, so translation cannot be interposed in the middle.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a voice communication system, a relay device, and a voice relay method that relay wireless communication and enable translation without checking the language of the spoken voice.

A voice communication system of the present invention comprises a plurality of repeaters communicating with corresponding communication terminals on different channels, and a repeater to which the plurality of repeaters are connected. The repeater has a language table that assigns different languages to a plurality of repeaters, and when voice received from a communication terminal corresponding to a certain repeater is input, the received voice is assigned to at least one other repeater. language, and transmit this translated voice through the repeater to which that language is assigned.

The relay device of the present invention includes a plurality of interfaces to which repeaters communicating with corresponding communication terminals on different channels are connected respectively, a language table storing the language used in the communication of the repeaters connected to each interface, A translation engine for translating communication voice and a control unit are provided. When a received voice, which is voice received by a repeater connected to this interface, is input from one of the interfaces, the control unit sets the language assigned to this interface as the original language and at least one other interface. with the language assigned to the target language as the target language, the received speech is input to the translation engine and translated into the target language speech, and the translated speech translated by the translation engine is connected to the interface to which the target language is assigned. Output to repeater.

In the voice relay method of the present invention, a relay device that uses a plurality of repeaters to relay voice communications of a plurality of communication terminals executes the following procedures. Different languages are assigned to a plurality of repeaters communicating with corresponding communication terminals on different channels. When voice received from a corresponding communication terminal is input from a certain repeater, the received voice is translated into voice of a language assigned to at least one other repeater. Transmit this translated speech through the repeater to which the language is assigned.

In the above invention, translation may be performed by an external translation server. A session table may be further provided for registering a communication session in which the called channel (repeater, interface) and the responding channel (repeater, interface) are participating channels (repeater, interface).

It translates the received speech in the original language into the translated speech in the target language and outputs it from the channel (repeater, interface) corresponding to this language. As a result, speech can be translated and relayed without confirming the languages used by the calling user and the responding user. Also, by providing a large number of channels, multilingual communication using a transceiver becomes possible.

FIG. 1 is a block diagram of a voice communication system. FIG. 2 is a diagram showing a language table provided in the relay device. FIG. 3 is a diagram showing a session table provided in the relay device. FIG. 4 is a flow chart showing the operation of the relay device. FIG. 5 is a flow chart showing the operation of the relay device.

An embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a configuration diagram of a voice communication relay system. This system enables multilingual communication by a communication terminal (UA) 5 via the relay device 1 . A plurality of repeaters 4 (4-1 to 5) are connected to the relay device 1. FIG. Each repeater 4-1 to 5 communicates with a communication terminal 5 (5-1 to 5). Each repeater 4-1 to 5 and each communication terminal 5-1 to 5 communicate without mutual interference. For example, they communicate using different channels. A channel can be a frequency, time slot, PN code, IP address, etc., or anything that can identify and separate one channel from another. Further, the communication method between each repeater 4 and communication terminal 5 may be different. For example, analog wireless communication, digital wireless communication, cellular system, VoIP communication, and the like. In the following description, it is assumed that communication is performed using different channels (frequencies). That is, the repeater 4-1 communicates with the communication terminal 5-1 using channel 1, the repeater 4-2 communicates with the communication terminal 5-2 using channel 2, and the repeater 4-3 communicates with the communication terminal 5-3 using channel 3, repeater 4-4 communicates with communication terminal 5-4 using channel 4, and repeater 4-5 communicates with communication terminal 5-5 using channel 5. do.

The repeaters 4 are connected to the relay device 1 respectively. Relay device 1 is connected to translation server 3 via Internet 2 . The translation server 3 is a server that provides a so-called cloud translation service. A translation service is a service in which, when a user specifies a translation language and uploads voice, the voice is translated into the specified language, synthesized into voice, and downloaded to the user. In FIG. 1 , relay device 1 uploads speech to translation server 3 as a user and downloads translated speech from translation server 3 .

The relay device 1 has an interface 12 (12-1 to 5) for connecting the repeaters 4-1 to 4-5. The interface 12 is connected to the control section 10 . The control unit 10 is composed of a computer and controls the operation of the relay device 1 . In addition to the interface 12 , the storage unit 11 and the network communication unit 13 are connected to the control unit 10 . The storage unit 11 is configured by a RAM, a flash memory, or the like, and is set with a buffer area for audio signals, a language table 30, a session table 31, and the like shown in FIGS. Network communication unit 13 communicates with translation server 3 via Internet 2 .

A language is assigned to each of the interfaces 12-1 to 5, that is, to the communication channels 1 to 5 of the repeaters 4-1 to 4-5. The assignments are stored in the language table 30 of FIG. In FIG. 2, channel 1 is assigned Japanese, channel 2 is assigned English, channel 3 is assigned Chinese (Mandarin), channel 4 is assigned Chinese (Cantonese), Channel 5 is assigned Korean. That is, the communication terminal 5-1 that communicates on channel 1 is owned by a user who communicates in Japanese, and the communication terminal 5-2 that communicates on channel 2 is owned by a user who communicates in English and communicates on channel 3. The communication terminal 5-3 is owned by a user who communicates in Mandarin, the communication terminal 5-4 which communicates on channel 4 is owned by a user who communicates in Cantonese, and the communication terminal 5-5 which communicates on channel 5 is Possessed by a user who communicates in Korean. Note that the language assigned to each channel is not limited to the example in FIG. The number of communication terminals 5 for each channel is not limited to one.

When the repeater 4 receives voice from the communication terminal 5 communicating on the same channel, the repeater 4 inputs the voice to the relay device 1 via the interface 12 . For example, when the repeater 4-1 receives a voice spoken in Japanese and inputs the voice from the interface 12-1 to the control unit 10, the control unit 10 sets the translation source language (original language) to "Japanese. ', and specify 'English', 'Mandarin', 'Cantonese', and 'Korean' as the translation destination language (target language), and upload the received speech to the translation server 3 . The translation server 3 recognizes the language of the received speech, translates it into the target language, and synthesizes the speech to generate the speech of the target language (translated speech). Then, this translated voice is sent back to the relay device 1 .

The control unit 10 of the relay device 1 outputs the translated speech to the repeaters 4-2 to 4-5 from the interfaces 12-2 to 5 to which the respective languages are assigned, and causes the repeaters 4-2 to 4-5 to transmit the translated speech. When at least one of the communication terminals 5-2 to 5-5 responds to the transmission of the translated voice of the repeaters 4-2 to 5, the communication is started (a communication session is established), and the participant participates in this communication. The channel of the communication terminal 5 that is present is registered in the session table 31 . While the channel in communication is registered in the session table 31, even if there is a call from another channel, the channel in communication cannot be called.

FIG. 3 is a diagram showing the session table 31. As shown in FIG. In this table, the channel of the called communication terminal 5 (corresponding to the interface 12 in the relay device 1) and the communication session whose members are the channels of the communication terminal 5 that responded to this call are registered. A session number, participating channel, and elapsed time are registered in the session table for each communication session. Elapsed time indicates the time that has passed without communication after the session is established. When the elapsed time reaches 10 seconds (10000 ms), it is determined that the communication has ended and the communication session is canceled.

In the example of FIG. 3, a communication session (session 1) of participating channels 1 and 2 and a communication session (session 2) of participating channels 3 and 4 are registered, and the elapsed time of no communication is 1 second and 3.5 seconds, respectively. is. When session 1 is registered first, when a call is received from channel 3, relay device 1 inputs translated speech to repeaters 4-4 and 4-5 of channels 4 and 5, which are vacant channels. Of these, only the communication terminal 5-4 that received the translated voice transmitted by the repeater 4-4 responded and session 2 was established. In this embodiment, vacant channels are determined from participating channels in the session table, but vacant channels may be stored separately in a table or the like.

FIG. 4 is a flow chart showing the operation of the control unit 10 of the relay device 1. As shown in FIG. This flow chart shows the operation of call processing from the communication terminal 5 . The control unit 10 waits until receiving a call from the communication terminal 5 in S1. This call can be from any of channels 1-5. If a call voice is received from any channel (YES in S1), the control unit 10 temporarily registers that channel in the session table (S2), and searches for an empty channel that is a communicable channel (S3). . If no other communication session has been established, all channels other than the channel that received the calling voice (calling channel) are idle. If another session has already been established, a channel other than the calling channel that is not participating in the other session is an empty channel.

The control unit 10 designates the language (original language) assigned to the call channel and the language (target language) assigned to each empty channel, and transmits the received voice to the translation server 3 (S4). Then, the translated speech is received from the translation server 3 (S5).

The control unit 10 inputs the translated voice from the corresponding interface 12 to the repeater 4 so as to transmit it on the channel to which the language is assigned. The control unit 10 may input voice to the repeater 4 and input a signal for switching the repeater 4 to the transmission mode. Further, the repeater 4 may automatically switch to the (VOX function) transmission mode by voice input. The repeater 4 transmits the input translated speech through the channel assigned to that language (S6).

After this transmission, it waits for a certain period of time (S7). The certain period of time is, for example, 10 seconds. If a reply is received from the channel that transmitted the translated voice during this period (YES in S8), that channel is registered in the session table (S9). As a result, the calling channel and the channel that sent the reply are registered in the session table, and a communication session is established and officially registered. Receiving the reply (S8) continues until a certain period of time elapses. If there are multiple channels that have sent replies, all of them should be registered in the session table.

After a certain period of time has passed (YES in S7), it is determined whether there is a reply from another channel, that is, whether the communication session has been officially registered (S11). If the communication session has been formally registered (YES in S11), the process proceeds to the session management operation in FIG. 5 (S12). If there is no reply and the session remains provisionally registered (NO in S11), the provisional registration is deleted (S13), and the process returns to the call waiting operation (S1).

FIG. 5 is a flowchart showing session management operations. It waits whether a voice is received from any of the participating terminals (S21) or whether a certain period of time (for example, 10 seconds) elapses without receiving voice (S22). When the voice signal from the participating channel is received (YES at S21), the language of the received channel is set as the original language and the language of the other participating channel is set as the target language, and the received voice is transmitted to the translation server 3 (S23). Then, the translated speech is received from the translation server 3 (S24). The received translated voice is transmitted from the participation channel to which each language is assigned (S25). Note that the reception of this voice resets the fixed-time timer.

On the other hand, if a certain period of time (10 seconds) has passed without receiving any sound (YES in S22), the corresponding session data is deleted from the session table (S26), thereby making the participating channel of this session an empty channel. (channel release). Then, this processing ends.

The above flowchart shows the processing operation focused on one call, but the above processing may be executed in parallel for a plurality of calls.

In the above embodiment, the call channel and the communication session for communicating only with the channel that responds to this call are registered in the session table. good too. That is, the voice received on one channel may be translated into the languages of all other channels, and the translated voice may be transmitted from all other channels. In this case, if multiple voices overlap, they may be mixed. In this way, if emergency broadcasts such as evacuation instructions are made in one language, they are also transmitted in other languages, so emergency broadcasts can be made in multiple languages at the same time.

In this embodiment, one language is assigned to one channel, but one language may be assigned to a plurality of channels. Thereby, multiple communication sessions using the same language can be established. Also, in this embodiment, a predetermined language is fixedly assigned to each channel, but this assignment may be switched by a user's operation or the like.
In the above embodiment, almost all processing related to translation is entrusted to the translation server 3, but some processing, such as final speech synthesis, may be performed locally.

1 relay device 3 translation server 4 (4-1 to 5) repeater 5 (5-1 to 5) communication terminal

Claims (7)

a plurality of repeaters communicating with corresponding communication terminals on different channels;
a relay device to which the plurality of repeaters are connected;
with
The relay device
a language table that assigns different languages to the plurality of repeaters;
When the voice received from the corresponding communication terminal is input from a certain repeater, the received voice is translated into the voice of the language assigned to at least one other repeater, and the translated voice is translated into the voice of that language. A voice communication system that transmits through a repeater that has an assigned language. Equipped with an external translation server,
2. The voice communication system according to claim 1, wherein the relay device transmits the received voice to the translation server , and the translation server returns the translated voice to the relay device. 3. The relay device further comprises a session table in which a communication session is registered in which the certain repeater and the repeater that has received the voice of the communication terminal responding to the transmitted translated voice are participating repeaters. 3. A voice communication system according to claim 2. a plurality of interfaces to which repeaters communicating with corresponding communication terminals on different channels are respectively connected;
a language table for storing languages used in communication of repeaters connected to each interface;
a translation engine that translates the voice of communication;
When a received voice, which is voice received by a repeater connected to the interface, is input from one of the interfaces, the language assigned to this interface is the original language, and the language assigned to at least one other interface is the original language. inputting the received speech into the translation engine and translating it into speech of the target language, with the target language being the target language;
a control unit that outputs the translated speech translated by the translation engine to a repeater connected to an interface to which the target language is assigned;
relay device with The translation engine is a translation server provided on a network,
5. The relay device according to claim 4, wherein said control section further comprises a network communication section for communicating with said translation server. Claim 4 or 5, further comprising a session table for registering a communication session having an interface to which the received voice is input and an interface to which the voice responding to the translated voice is input from the repeater as participation interfaces. The relay device described in . A relay device that relays voice communication of a plurality of communication terminals using a plurality of repeaters,
assigning different languages to a plurality of repeaters communicating with corresponding communication terminals on different channels;
When voice received from the corresponding communication terminal is input from a certain repeater, translating the received voice into voice of a language assigned to at least one other repeater,
A voice relay method for transmitting the translated voice through a repeater to which the language is assigned.

Images