KR20220152672A - Providing multilingual translation in real time - Google Patents

Abstract

The present invention relates to a method for providing multilingual interpretation/translation in real time. The method for providing multilingual interpretation/translation in real time, according to the present invention, comprises the steps of: (a) activating an interpretation/translation application installed in a first terminal; (b) tethering at least one guest terminal with the first terminal; (c) transmitting interpretation/translation application installation information to N terminals; (d) installing and activating the interpretation/translation application; (e) displaying a host ID of the first terminal, a guest ID, and corresponding used language on the N terminals; (f) transmitting inputted spoken language to the interpretation/translation application of the first terminal and then transmitting the inputted spoken language to an interpretation/translation server; (g) transmitting multilingual information to the first terminal; (h) displaying the multilingual information on the interpretation/translation application of the first terminal and simultaneously transmitting the multilingual information to the N terminals; and (i) displaying text translation information corresponding to each used language or outputting the multilingual information as voice interpretation information. Therefore, a plurality of people who speak different foreign languages can conduct a natural conversation.

Description

A method for providing real-time multilingual translation in real time}

The present invention relates to a method of providing real-time multilingual interpretation and translation, and more specifically, to a host terminal on which an interpreter and translation app is installed and a terminal such as a smart phone possessed by guests who do not have the interpreter and translation app installed and speak different mother tongues. After mutual tethering, if one of the host and guest inputs a predetermined voice into their terminal in their native language, the other party's terminal displays translation information translated into their native language or interprets information in real time. It relates to a "method of providing real-time multilingual interpretation and translation" that enables natural resolution of language barriers during multiple meetings using different languages by outputting the results.

In recent years, international exchanges have been actively carried out as transportation and communication technologies have developed. As more and more people visit countries where different languages are spoken, the demand for workers who can interpret and translate from the language of the country to their native language is rapidly increasing.

For example, due to the increase in international exchange between companies, institutions, and individuals with branches abroad, there is an urgent need for translation and interpretation services in various fields.

Accordingly, technologies for various translation devices and methods have been continuously developed, and a text translation service that interprets and translates a country's language into a corresponding country's language through the Internet already provides a considerable level of translation. In addition, a certain level of interpretation and translation is possible by using a dedicated interpretation/translation application or a dedicated translator by a smart device.

These smart devices or dedicated translators interpret and translate the native language into the voice of the country in real time by either the user setting the language of the country in advance or the dedicated translator recognizing the voice of the other party when the user meets a person in the country and meets face-to-face. .

However, when users of various languages from various countries gather at the same time, such as in a conference, the inconvenience of having to operate the translator directly occurs whenever the user has a conversation with a person who speaks a different language.

A technology related to an automatic translation system is disclosed in Korean Patent Publication No. 10-2016-0122371, "Automatic Interpretation and Translation System and Method Using Electronic Device" (hereinafter referred to as prior art).

The prior art is a system capable of automatically interpreting and translating a plurality of unspecified languages by using a native language in an electronic device used by the user even when the user is not familiar with the language of another country or the language is not communicated.

1 is a flowchart illustrating a method for using the interpretation/translation service proposed by the prior device.

As shown, Zigbee, Radar, WPAN (Wireless Personal Area Networks), Wi-Fi, 3G/4G/LTE, Bluetooth, It transmits information about service products to be provided through Ethernet, BcN, satellite communication, microwave, serial communication, PLC, and Beacon devices.

In addition, it is connected to the database and based on the information related to the interpretation and translation service registered in the server, information related to the provision of interpretation and translation services and related interpretation and translation service contents to be provided according to the interpretation and translation service system are created, and related to interpretation and translation services Transmits information related to the interpretation/translation service to be used to the user's electronic device.

When using the information related to the provision of interpretation and translation services provided here, it is automatically executed according to the method set for service use in advance, and the extracted information and results are stored in the database.

On the other hand, an input unit for automatic input of a request for interpretation and translation services and selection of additional functions so that information related to the provision of interpretation and translation services provided by the provider can be used, a display unit for displaying usage time, control method, and operation status of the device; , holding a usage table and storing information related to the use of the used service.

However, in the case of an interpretation and translation service disclosed in prior art or other known real-time automatic interpretation and translation services, a predetermined translation app must be installed on the terminal, and in particular, even when a plurality of participants using different native languages want to have a conversation, all participants in the conversation There was a hassle of having to install and use the same predetermined interpretation and translation app on one's own terminal.

1. Korean Patent Publication No. 10-2016-0122371, "Automatic Interpretation and Translation System and Method Using Electronic Device"

The present invention has been proposed to solve the above-described problems, and proposes a method for naturally enabling real-time interpretation and translation conversation when at least one of the conversation participants has an interpretation and translation app installed.

To this end, the present invention proposes a method of providing interpretation and translation services after tethering a plurality of guest terminals using different native languages with a host terminal in which an interpreter and translation app is installed.

That is, in the present invention, after tethering a host terminal on which an interpretation/translation app is installed and a terminal such as a smart phone possessed by guests who do not have the translation/interpretation app installed and speak different native languages, one of the host and the guest is If you input a predetermined voice into your terminal in your native language, the other party's terminal displays translation information translated into your native language or outputs interpretation information in real time. We want to provide "a method for providing real-time multilingual interpretation and translation" that can naturally resolve barriers.

A method for providing real-time multilingual interpretation and translation for at least one or more terminals on which an interpretation and translation app is not installed, which is a technical idea proposed by the present invention,

(a) activating an interpretation/translation app installed in a first terminal serving as a host;

(b) at least one or more guest terminals (N terminals; where N is a natural number of 1 or more; the languages used by the N terminals may be the same as or different from those of the first terminal) tethering with the first terminal;

(c) transmitting the translation app installation information from the first terminal to the N terminals;

(d) each of the N terminals executing the translation app installation information to install and activate the translation app from the translation server;

(e) The translation app of the first terminal collects and stores device information (including information on the language used by the terminal) of each of the N terminals from the N tethered terminals, and selects a predetermined guest for each of the N terminals. IDs are given and the languages used by the N terminals are displayed on the translation app of the first terminal, and the host ID and guest ID of the first terminal and the corresponding language are displayed on the tethered N terminals. step;

(f) After the voice language input through the first terminal and any one of the N terminals (random terminal a) is transmitted to the translation app of the first terminal, which is tethered, to the interpretation and translation server. transmission step;

(g) transmitting, by the interpretation/translation server, multilingual information obtained by translating the transmitted voice language into at least one multilingual language corresponding to the language of the first terminal and each of the N terminals to the first terminal;

(h) displaying the multilingual information in the translation app of the first terminal and simultaneously transmitting the multilingual information to the N tethered terminals;

(i) Characterized in that text translation information corresponding to each language used by the first terminal and the N terminals is displayed or output as voice interpretation information.

In the above step (e) of the present invention, the host ID and the guest ID of the first terminal and the corresponding language may be transmitted from the translation app of the first terminal to the translation server.

In addition, in the step (b) of the present invention, the first terminal selects an ID of a remote terminal located in a remote place and has the translation app installed thereon to participate in the translation conversation with the N tethered terminals. can make

In the case of implementing the method for providing real-time multilingual interpretation and translation, which is a technical idea proposed by the present invention, the following effects can be expected.

1. Multiple people who speak different foreign languages can have a natural conversation with real-time interpretation and translation.

2. If the interpretation and translation app according to the present invention is installed on the terminal of one person serving as the host of the meeting, the host proposed by the present invention even when the interpretation and translation app is not installed on the terminals possessed by a number of foreigners participating as guests Real-time interpretation and translation conversations are possible in a state where the terminal and the portable terminal of each guest participating as a guest are mutually tethered.

3. If the interpretation and translation app according to the present invention is installed on the terminal of a multilingual user participating in a meeting, real-time interpreter conversation is possible even in a remote place.

4. According to the present invention, a host and a guest can participate in a chat channel regardless of whether they are in the same space or remotely (however, tethering participants must be in the same space)

1 is an example of a conventional interpretation/translation method.
2 is an example of a method for providing real-time multilingual interpretation and translation proposed in the present invention.
3 shows an example of an interpretation/translation app proposed in the present invention. For reference, FIG. 3 shows a screen of a guest terminal and a tethered host terminal.
4 is an example of an interpretation/translation screen displayed on a host terminal (Orangei).
5 is a conceptual diagram for allowing a remote terminal user to participate in an interpretation/translation conversation according to the present invention.

Hereinafter, a method for providing real-time multilingual interpretation and translation proposed in the present invention will be described with reference to drawings and the like.

FIG. 2 is an embodiment of a method for providing real-time multilingual interpretation and translation proposed in the present invention, and is a diagram showing a situation in which a plurality of participants speaking different mother tongues talk to each other in a meeting place such as a conference hall. For reference, an embodiment of the present invention describes a method of providing multilingual translation in real time with at least one guest terminal user who does not have an interpreter/translation app installed.

First, in FIG. 2, the host terminal refers to a terminal such as a smartphone having the interpretation and translation app proposed in the present invention installed, and the remaining terminals (A, B..., E, F) speak different foreign languages participating in the meeting. It shows the terminals of each of the many foreign guests in use. Here, the interpretation and translation app of the present invention is not installed on the guest terminal possessed by the foreigner when participating in the first meeting.

First, a method of connecting a host terminal and guest terminal terminals (A, B..., E, F) using a tethering function will be described.

3 shows an example of an interpretation/translation app proposed in the present invention. For reference, FIG. 3 shows a screen of a guest terminal and a tethered host terminal.

First, as a multilingual real-time interpretation method with foreigners, activating the tethering activation button located at the bottom of the host terminal in which the interpretation and translation app of the present invention is installed enables the host through a tethering connection with each terminal owned by foreigners in the same space. connects to the terminal.

After tethering, the host terminal transmits information about installation of the translation app to N terminals, which are guest terminals. For example, it is a method of providing a URL through which the translation and interpretation app of the present invention can be installed. However, in another embodiment of the present invention, it may be possible to provide temporary program information through which an interpretation/translation service of the host terminal can be provided during tethering.

Next, the guests execute the translation app installation information for each of the N terminals they own, and install and activate the translation app from the translation server.

Next, the host terminal may use at least one or more guest terminals (A, B, ...E, F; or less, N terminals; where N is a natural number of 1 or more; languages used by the N terminals located in a tetherable radius) 1 terminal) downloads the interpretation/translation app, and then collects, stores, and stores device information (including information on the language used by the terminals) of the N tethered terminals. Here, it is assumed that the terminal possessed by the guest participating in the conference is a terminal registered in the home country, and during tethering, the host terminal collects and stores language information set in each guest terminal.

On the other hand, in the case of guest terminals, even if the interpretation and translation app is activated, their respective IDs may not have been determined yet.

For this reason, in the present invention, a predetermined guest ID is automatically assigned to each of the N terminals, which are the host terminal and the guest terminal, in the translation app installed in the host terminal.

For example, as shown in FIG. 3 , red, orange, yellow, green, blue, indigo, and purple can be automatically set. At this time, the language of each tethered terminal is displayed together with the participant ID.

That is, in case of red, English, in case of orange, Korean (eg, host terminal), in case of yellow, Japanese, etc.

That is, as shown in FIG. 3, the ID of each of the host terminal and the guest terminal participating in the meeting and the language set in the terminal are displayed on the terminal of the host.

The host ID displayed on the host terminal, the guest ID, and the corresponding language are also shared and displayed in the interpretation and translation apps of the N terminals, which are guest terminals tethered to the host terminal.

Meanwhile, the host terminal transmits its own ID, ID information of the N terminals, and device information (including information on the language used by the terminals) of the N terminals to the interpretation/translation server, and the interpretation/translation server transmits the device information of the N terminals. Based on (including information on the language used by the terminal), a corresponding language is set.

Next, one of the participants communicates with his/her native language voice on his/her own terminal. For reference, while one person is talking, the other participants must remain silent. This is because when two or more people try to communicate through their respective terminals, errors occur during interpretation and translation.

In the case of the present invention, after a voice language input through any one of the N terminals, which are a host terminal and a guest terminal (an arbitrary terminal a), is transmitted to the interpretation/translation app of the tethered first terminal, the transmitted to the translation server.

The interpretation/translation server translates the transmitted voice language into at least one or more multilingual languages corresponding to the languages used by the first terminal and the N terminals, and transmits multilingual information to the host terminal.

4 is an example of an interpretation/translation screen displayed on a host terminal (Orangei). For reference, the screen shown in Figure 4 is also displayed on the guest terminal accessed through tethering, and the guest can select his/her ID displayed at the top and input voice in his or her native language)

In the present invention, multilingual information transmitted to the host terminal is individually transmitted to the guest terminal corresponding to the language used by the participant terminals (ie, the host terminal and the GATES terminal).

That is, if you say "Hello, this is a patent test screen" using a host terminal using Korean (ID is orange, the language used is Korean), the host terminal displays all languages used by the participant terminal as shown in FIG. do. This information is received from the interpretation/translation server.

Meanwhile, the multilingual information is displayed on the interpretation/translation app of the host terminal, and the multilingual information is transmitted to the N terminals that are tethered guest terminals.

Next, in each guest terminal, translation information corresponding to the language used by each guest is displayed or translated into voice and output.

The translation app used in the present invention is equipped with a TTS function to output text information as voice.

Therefore, since translated text information or interpreted voice information corresponding to each language used is output through the participant terminal, there is an advantage in that even if foreigners who speak different native languages have a common meeting together, they can communicate in real time without language barriers.

The above described a method for meeting participants who speak multiple native languages to have a real-time conversation in a conference hall.

However, in the case of the present invention, real-time interpretation and translation conversations are possible even with foreigners in remote places. Hereinafter, a description will be made with reference to FIG. 5 showing a conceptual diagram of allowing a remote terminal user to participate in an interpretation/translation conversation according to the present invention.

To use this method, a search button displayed at the bottom of the translation and interpretation app shown in FIG. 3 is used to interconnect a host terminal in which the translation and interpretation app according to the present invention is installed and a guest terminal located in a remote location.

That is, when the (user) search button provided to the translation and interpretation app according to the present invention is selected using the host terminal, a list of nearby terminals is displayed after searching for a nearby terminal where the translation and interpretation app of the present invention is installed through GPS.

For reference, the ID of the terminal user and the language used are displayed in the list.

The host terminal can participate in the interpretation channel of the host terminal by selecting a predetermined ID from the neighboring terminals displayed in the list.

Meanwhile, even when not within a predetermined radius, the host terminal may participate in a conversation by inputting an ID of a user terminal of a specific interpretation/translation app.

As described above, when a remote terminal participates in an interpretation/translation conversation, the remote terminal directly receives interpretation/translation information of the conversation contents of the participants described in FIGS. It is provided to the tethered guest terminal through the host terminal described in 2 to 4.

Hereinafter, a real-time interpretation/translation conversation method between a host terminal and guest terminals in a state of being connected by tethering will be described.

First, a state in which a host participating in a conversation and a guest are connected to a conversation channel are as shown in FIG. 3 .

When voice information is input through a host terminal (eg, Orange) or a hands-free device connected to the host terminal, interpretation and translation information corresponding to the language used by the guest terminal is output through each guest terminal to another guest terminal through the above-described process. . The output here may include both translated text information and interpretation voice information.

For example, the host's channel opening time is 3:00:00 on time, and Orangei inputs the voice information "Hello, I am a person developing an interpreter application" into the host terminal at 3:00:01, and Yellowi enters 3 “Hello, nice to meet you” in Japanese at :00:03, and “Nice to meet you all,” in Chinese at 3:00:07. When a voice input says, "It's been a while", Orange's voice input is translated and interpreted, and the other party's terminal uses the TTS function to play the sound. Then, the voice inputs of Yellow and Blue are translated and interpreted, are delivered to each

Hosts and guests participating in the channel are not separately granted voice input authority, and are translated or interpreted in the order in which the voice input of the person who attempted voice input first started, and is delivered to the other user.

As an example, the voice input method of the present invention may be described as follows.

1. In the case of interruption or termination of voice input, if there is no input for more than 2 seconds from the user's voice input during voice input, the voice input is stopped, and the voice input until then is translated and interpreted and automatically played on the terminals of the channel participants ( interpret)

2. In addition, if the user during voice input inputs a predetermined voice indicating the end of the voice input, the voice is translated and interpreted from then on, and automatically played (interpreted) in the terminals of the chat channel participants.

The method for providing real-time multilingual interpretation and translation according to the present invention described above is summarized as follows.

1. Open a host channel using a host device

2. Connect guest terminals near the host terminal by tethering (For reference, the host terminal can invite by entering the terminal ID of the remote terminal where the interpretation and translation app of the present invention is installed, and the remote terminal within a predetermined radius is GPS You can also search and participate)

3. The host terminal may assign or set an arbitrary guest ID to a guest terminal that is tethered to the host terminal but does not have the interpretation/translation app of the present invention.

4. After assigning or setting a guest ID, the host terminal automatically sets the language used in the host and guest terminals (however, in the translation app of the present invention, the automatically set language may be changed in consideration of the terminal user's actual language). have)

5. If each voice is input sequentially through the host or guest terminal, it is transmitted to the translation server via the host terminal, and then translated into the native language used by the participant terminal, and then transmitted to the host terminal.

5. Translated text information is displayed on the host terminal, and the translated text information of each guest terminal is displayed through the host terminal to each guest terminal, and at the same time, an interpretation voice is automatically output in the language used by each guest terminal. In the terminal, you can check the text in your native language converted during voice input and check the error in the sentence or word input by voice.)

When the method according to the present invention is implemented through such a process, a plurality of participants who speak different mother tongues can communicate naturally without being constrained by language barriers.

When the method for providing real-time multilingual interpretation and translation according to the present invention described above is used, a plurality of people speaking different foreign languages can have a natural conversation through real-time interpretation and translation.

In addition, if the interpretation and translation app according to the present invention is installed on the terminal of one person serving as the host of the meeting, the host proposed by the present invention even when the interpretation and translation app is not installed on the terminals possessed by a number of foreigners participating as guests Real-time interpretation and translation conversations are possible in a state where the terminal and the portable terminal of each guest participating as a guest are mutually tethered.

In addition, if the interpretation and translation app according to the present invention is installed in the terminal of a multilingual user participating in a meeting, a real-time interpreter conversation is possible even in a remote place.

In addition, according to the present invention, the host and the guest can participate in the chat channel regardless of whether they are in the same space or at a remote location (however, tethering participants must be in the same space).

As described above, those skilled in the art will fully understand that the present invention has the advantage of being able to receive a real-time interpretation and translation function even if the interpretation and translation app of the present invention is not installed in the guest terminal.

(The host must have the 'Brit' APP installed, and guests connect to the host's system using the tethering connection of slide 2, STEP 1, and user access through tethering requires voice input and interpretation through the host's terminal. can proceed,

On the other hand, the translation server used to provide real-time interpretation and translation of the present invention can be implemented in various ways, and is application number 10-2020-0088521 filed by the present applicant, and the title of the invention is "real-time multilingual multilingual using automatic translation function" The server disclosed in "Translation Chatting Method" may be utilized as the interpretation/translation server of the present invention.

Although the embodiments of the present invention described above have been described with limited examples and drawings, those skilled in the art can make various modifications and variations from the above description.

For example, even if the described techniques are performed in a different order from the described method, combined or combined in a different form from the described method, or substituted or substituted by other components or equivalents, the implementation method proposed in the present invention It may be possible to achieve, and in this case, other implementations, other embodiments, and equivalents to the claims should be construed as belonging to the scope of the claims described later.

Claims (3)

A method of providing real-time multilingual interpretation and translation for at least one or more terminals on which the translation app is not installed, the method comprising:
(a) activating an interpretation/translation app installed in a first terminal serving as a host;
(b) at least one or more guest terminals (N terminals; where N is a natural number of 1 or more; the languages used by the N terminals may be the same as or different from those of the first terminal) tethering with the first terminal;
(c) transmitting the translation app installation information from the first terminal to the N terminals;
(d) each of the N terminals executing the translation app installation information to install and activate the translation app from the translation server;
(e) The translation app of the first terminal collects and stores device information (including information on the language used by the terminal) of each of the N terminals from the N tethered terminals, and selects a predetermined guest for each of the N terminals. IDs are given and the languages used by the N terminals are displayed on the translation app of the first terminal, and the host ID and guest ID of the first terminal and the corresponding language are displayed on the tethered N terminals. step;
(f) After the voice language input through the first terminal and any one of the N terminals (random terminal a) is transmitted to the translation app of the first terminal that is tethered, to the interpretation and translation server. transmission step;
(g) transmitting, by the interpretation/translation server, multilingual information obtained by translating the transmitted voice language into at least one multilingual language corresponding to the language of the first terminal and each of the N terminals to the first terminal;
(h) displaying the multilingual information in the translation app of the first terminal and simultaneously transmitting the multilingual information to the N tethered terminals;
(i) A method for providing real-time multilingual interpretation and translation, characterized in that text translation information corresponding to each language used by the first terminal and the N terminals is displayed or output as voice interpretation information. According to claim 1,
In step (e) above
The method of providing real-time multilingual interpretation and translation, characterized in that the host ID and the guest ID of the first terminal and the corresponding language are transmitted from the translation app of the first terminal to the translation server. According to claim 1,
In the step (b), the first terminal can participate in the translation conversation with the N tethered terminals by selecting an ID of a remote terminal located in a remote place and having the interpretation and translation app installed thereon. A method for providing real-time multilingual interpretation and translation.

Images