KR102349620B1 - Method and apparatus for providing interpretation and translation services using user terminal - Google Patents

Abstract

A method of providing an interpretation and translation service through a caller terminal includes the steps of receiving an input of a receiver terminal phone number from a caller, visually exposing at least one target language through an interpretation and translation application, receiving a selection input for a target language from the sender transmitting a selection input for the recipient terminal phone number and the target language to an interpretation/translation server along with a call setup request with the recipient terminal, and generating based on the original language voice received from the caller and transmitting audio information in one original language to the interpretation/translation server, wherein the interpretation/translation server transmits, to the receiver terminal, an interpretation result obtained by interpreting the audio information in the original language into the target language based on the selection input.

Description

Method and device for providing interpretation and translation service through user terminal

Disclosed are a method and an apparatus for providing both an interpretation and a translation service to both a sender and a receiver who use different languages through a user terminal.

When a caller and a receiver speak different languages during a call, and they do not speak the other's language, interpretation or translation is required for smooth communication between the caller and the receiver.

The existing interpretation and translation service through the user terminal provides only one service of interpretation or translation.

In addition, in order to use the interpretation and translation service through the existing user terminal, interpretation and translation applications must be installed in both the sender's terminal and the receiver's terminal.

In addition, in the existing interpretation and translation service through the user terminal, it is difficult to clearly distinguish the end point of input of voice information to use the interpretation and translation service.

In addition, in the existing interpretation and translation service through the user terminal, it is difficult to switch between activation and deactivation of the interpretation and translation service during a call.

Methods related to the provision of translation services are disclosed in Korean Patent Publication (KR 10-2016-0049947 A) and Korean Patent Publication (KR 10-2013-0043293 A).

Disclosed are a method and an apparatus for simultaneously providing an interpretation service and a translation service through a user terminal.

Disclosed are a method and an apparatus capable of providing an interpretation and translation service to a sender and a receiver even when an interpretation/translation application is installed only in the sender's terminal.

Disclosed are a method and apparatus for providing an interpretation/translation service through an interpretation/translation application capable of clearly distinguishing an end time point of inputting voice information to use an interpretation/translation service.

Disclosed are a method and apparatus for providing an interpretation and translation service capable of switching between activation and deactivation of an interpretation and translation service during a call.

A method of providing an interpretation and translation service through a caller terminal includes the steps of receiving an input of a receiver terminal phone number from a caller, visually exposing at least one target language through an interpretation and translation application, receiving a selection input for a target language from the sender transmitting a selection input for the recipient terminal phone number and the target language to an interpretation/translation server along with a call setup request with the recipient terminal, and generating based on the original language voice received from the caller and transmitting audio information in one original language to the interpretation/translation server, wherein the interpretation/translation server transmits, to the receiver terminal, an interpretation result obtained by interpreting the audio information in the original language into the target language based on the selection input.

The transmitting of the call setup request to the interpretation/translation server may include outputting an alarm indicating that the interpretation/interpretation service is started after the call call is established with the recipient terminal.

The receiver terminal may output an alarm informing that the interpretation and translation service is started after a call is established with the caller terminal.

The step of transmitting the original language voice information generated based on the original language voice received from the caller to the interpretation/translation server includes further receiving a predetermined code, and based on the original language voice received before the predetermined code is received. Thus, the original language voice information can be generated.

When the interpretation and translation application is installed in the receiver terminal, the STT processing result or the translation result for the original language voice information may be further transmitted to the receiver terminal.

The interpretation/translation application may be activated or deactivated according to an activation command or deactivation command of the interpretation/translation application input from the caller in a state in which a call is established with the recipient terminal.

In a state in which a call is established with the receiver terminal, the speakerphone mode of the caller terminal may be activated or deactivated according to a speakerphone mode activation command or a deactivation command input from the caller.

The method of providing an interpretation/translation service through a caller terminal is, through the interpretation/translation application, the STT processing result for the original speech information received from the interpretation/translation server, or the translation result of translating the original speech information into the target language. It may include further steps.

In the method of providing an interpretation and translation service through the caller terminal, when a release input of the call call setup is received from the caller, the STT processing received after the call call is established with the receiver terminal until the call call setup is released The method may further include generating a cumulative translation result based on the result or the translation result.

After generating the interpretation result, the interpretation/translation server may generate the translation result by performing speech-to-text (STT) processing on the interpretation result.

The interpretation/translation server may generate a speech to text (STT) processing result for the original language voice information, and then generate the translation result based on the STT processing result.

Transmitting the original language voice information generated based on the original language voice received from the caller to the interpretation and translation server until a release input for setting up the call is received from the caller, and through the interpretation and translation application, the The step of outputting an STT processing result for the original speech information received from the interpretation/translation server or a translation result obtained by translating the original speech information into the target language may be repeated.

The receiver terminal may transmit target language voice information generated based on the target language voice received from the receiver to the interpretation/translation server, and the interpretation/translation server may include, as a result of STT processing for the target language voice information, to the selection input. Based on the translation result of translating the target language voice information into the original language, or the translation result of interpreting the target language voice information into the original language, it is possible to transmit to the calling terminal, interpretation and translation through the calling terminal through the calling terminal The method of providing a service may further include outputting the STT processing result and the translation result through the interpretation/translation application, and outputting the interpretation result through the speaker.

The sender terminal, the receiver terminal, may transmit target language voice information generated based on the target language voice received from the receiver to the interpretation and translation server, the interpretation and translation server, STT processing result for the target language voice information , at least one of the translation results obtained by translating the target language voice information into the original language based on the original language information received from the sender terminal Transmitting the interpretation result obtained by interpreting the target language voice information into the original language to the sender terminal The method may further include, by the caller terminal, outputting the STT processing result, the translation result, or the interpretation result through the interpretation and translation application.

The interpretation and translation application may be activated according to an activation command input from the sender together with the recipient terminal phone number.

An interpretation/translation application that receives a selection input for a recipient terminal phone number and a target language from the sender, along with a call call setup request with the recipient terminal, transmits a selection input for the recipient's terminal phone number and the target language to the interpretation/translation server and a communication interface for transmitting original language voice information generated based on the original language voice received from the caller to the interpretation and translation server, wherein the interpretation and translation server selects the original language voice information based on the selection input to the target Transmitting the interpretation result translated into language to the receiver terminal.

The interpretation/translation application may output an alarm informing that the interpretation/interpretation service is started after a call is established with the recipient terminal.

The receiver terminal may output an alarm informing that the interpretation and translation service is started after a call is established with the caller terminal.

The interpretation/translation application may generate original language voice information based on the original language voice received from the caller until a predetermined code is received.

When the interpretation and translation application is installed in the receiver terminal, the original STT processing result or the translation result for the original language voice information may be further transmitted to the receiver terminal.

The interpretation/translation application may be activated or deactivated according to an activation command or deactivation command of the interpretation/translation application input from the caller in a state in which a call is established with the recipient terminal.

In a state in which a call is established with the receiver terminal, the speakerphone mode of the caller terminal may be activated or deactivated according to a speakerphone mode activation command or a deactivation command input from the caller.

The interpretation/translation application may output an STT processing result for the original speech information received from the interpretation/translation server or a translation result obtained by translating the original speech information into the target language.

The interpretation and translation application, when a release input of the call call setup is received from the caller, based on the STT processing result or the translation result received from after the call call is established with the receiver terminal until the call call setup is released You can create cumulative translation results.

After generating the interpretation result, the interpretation/translation server may generate the translation result by performing speech-to-text (STT) processing on the interpretation result.

The interpretation/translation server may generate a speech to text (STT) processing result for the original language voice information, and then generate the translation result based on the STT processing result.

The interpretation and translation application transmits, to the interpretation and translation server, the original language voice information generated based on the original language voice received from the caller until a release input for setting the call is received from the caller, and the interpretation and translation server The step of outputting the STT processing result for the original speech information received from the , or the translation result obtained by translating the original speech information into the target language may be repeated.

The caller terminal may further include a speaker for outputting an interpretation result, and the receiver terminal may transmit target language voice information generated based on the target language voice received from the receiver to the interpretation and translation server, and the interpretation and translation The server, based on the STT processing result for the target language voice information, the translation result of translating the target language voice information into the original language based on the original language information received from the caller terminal, and the target language voice information into the original language At least one of the interpreted interpretation results may be transmitted to the caller terminal, and the interpretation/translation application may output the STT processing result or the translation result, and output the interpretation result through the speaker.

Interpretation and translation services can be provided at the same time during a call.

Even if an interpretation and translation application is installed only in the sender's terminal, it is possible to provide an interpretation and translation service to both the sender and the receiver.

It is possible to clearly distinguish the end point of input of pitch information to use the interpretation and translation service.

It is possible to switch between activation and deactivation of the interpretation and translation service during a call.

1 is a conceptual diagram of a sender terminal, an interpretation/translation server, and a receiver terminal providing an interpretation/translation service.
2 is a block diagram of a caller terminal providing an interpretation and translation service.
3 is a flowchart of a method of providing an interpretation/translation service through an interpretation/translation application.
4 is a flowchart of a method of providing an interpretation and translation service through an interpretation and translation application.
5A and 5B show a user interface of an interpretation/translation application.
6A and 6B show a user interface of an interpretation/translation application.
7A and 7B show interpretation and translation results output through the sender terminal and the receiver terminal.
8 shows an STT processing result and a translation result output through the sender terminal.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Like reference numerals in each figure indicate like elements.

Various modifications may be made to the embodiments described below. It should be understood that the embodiments described below are not intended to limit the embodiments, and include all modifications, equivalents, and substitutes thereto.

Terms used in the examples are used only to describe specific examples, and are not intended to limit the examples. The singular expression includes the plural expression unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to designate that a feature, number, step, operation, component, part, or a combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiment belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

In addition, in the description with reference to the accompanying drawings, the same components are given the same reference numerals regardless of the reference numerals, and the overlapping description thereof will be omitted. In describing the embodiment, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the embodiment, the detailed description thereof will be omitted.

1 is a conceptual diagram of a sender terminal, an interpretation/translation server, and a receiver terminal providing an interpretation/translation service.

The sender terminal 100 , the interpretation/translation server 200 , and the receiver terminal 300 may transmit/receive information through a wired/wireless network. The sender terminal 100 and the receiver terminal 300 refer to all devices capable of installing an interpretation and translation application, and the device may be any one of a mobile terminal, a mobile terminal, a cell phone, a smart phone, a tablet PC, and the like. The caller terminal 100 transmits a call call setup request, a phone number of the receiver terminal 300 and a selection input for a target language to the interpretation/translation server 200 . The caller terminal 100 transmits, to the interpretation/translation server 200 , the original language voice information generated based on the original language voice received from the caller after a call is established with the receiver terminal 300 . The sender terminal 100 may output the STT processing result or the translation result received from the interpretation/translation server 200 .

The receiver terminal 300 may receive the interpretation result from the interpretation/translation server 200, and when the interpretation/translation application is installed, the STT processing result or the translation result for the original language voice information from the interpretation/translation server 200 is transmitted from the interpretation/translation server 200 can receive more.

The interpretation and translation server 200 may set up a call between the caller terminal 100 and the receiver terminal 300 using the phone number of the caller terminal 100 and the phone number of the receiver terminal 300 . The interpretation/translation server 200 may determine whether the interpretation/translation application is installed in the receiver terminal 300 by using the phone number of the receiver terminal 300 or by sending a request for confirmation of installation of the interpretation and translation application to the receiver terminal 300 . The interpretation and translation server 200 may determine whether to further transmit the STT processing result or translation result for the original language voice information to the receiver terminal 300 based on whether the interpretation and translation application is installed in the receiver terminal 300 .

The interpretation/translation server 200 may interpret, translate, or STT the original language voice information received from the caller terminal 100 (or the receiver terminal 300). The interpretation/translation server 200 holds a language database for a plurality of foreign languages (eg, English, Chinese, Japanese, Arabic, French, Indonesian, Spanish or Vietnamese, etc.) in order to interpret (or translate) voice information in the original language, Interpretation (or translation) can be performed by referring to the language database you have. When the target language is a language other than English, the interpretation/translation server 200 directly interprets or translates the original language voice information into the target language, or first interprets or translates the original language voice information into English, and then transmits the translation result An interpretation result or a translation result may be generated by re-interpreting or translating into the target language.

2 is a block diagram of a caller terminal providing an interpretation and translation service.

The caller terminal 100 includes a central processing unit 110 , a communication interface 120 , a microphone 130 , and a speaker 140 . The memory records the interpretation and translation application, and the central processing unit 110 operates according to the interpretation and translation application recorded in the memory. The interpretation/translation application includes the steps of receiving a recipient terminal phone number and an activation command of the interpretation/translation application from the sender, activating the interpretation/translation application according to the activation command of the interpretation/translation application, at least one target language visually through the interpretation/translation application The step of exposing, receiving a selection input for a target language from the sender, along with a call call setup request with the recipient terminal 300, to the interpretation and translation server 200 to the recipient's terminal phone number and the target language Transmitting a selection input for , transmitting the original voice information generated based on the original voice received from the sender to the interpretation and translation server 200 and through the interpretation and translation application, the interpretation and translation server 200 outputting the STT processing result for the original voice information received from the , or a translation result obtained by translating the original voice information into the target language.

The sender terminal 100 communicates with the interpretation/translation server 200 through the communication interface 120 . The communication interface 120 refers to an interface for information transmission between the transmitting side and the receiving side, and may include a mechanical cable or connector shape, an interface on electrical characteristics, and a procedure for a software interface. In addition, the communication interface 120 may be understood as a medium that processes a message received from another external simulation or transmits its own message, and may include an input channel and an output channel. The communication interface 120 may include all types of hardware interfaces such as plugs, connectors, cards, or other types that interconnect hardware devices, such as computers, to transmit information therebetween.

The caller terminal 100 may receive the caller's original language voice information through the microphone 130 , and may output the interpretation result received from the interpretation/translation server 200 through the speaker 140 . The interpretation result means that the target language voice information uttered by the receiver is interpreted into the original language of the sender.

3 is a flowchart of a method of providing an interpretation/translation service through an interpretation/translation application.

The sender terminal 100 receives a phone number of the receiver terminal and an activation command of the interpretation and translation application from the sender, and activates the interpretation and translation application according to the received activation command of the interpretation and translation application (S100).

The interpretation and translation application refers to an application for providing interpretation and translation services to the caller terminal 100 . The activation command of the interpretation/translation application may include an execution command for the interpretation/translation application in a pre-execution state and a wake-up command for the interpretation/translation application in an idle state.

Referring to FIG. 5A together, the command to activate the interpretation/translation application may be received through the sender's input to the display unit or input unit of the sender terminal. An “interpretation/translation” input area for inputting an interpretation/translation application activation command may be displayed together with a keypad (eg, a space between “#” and “*”) including 0 to 9, “#” and “*”. . The interpretation/translation application activation command may be input before or after inputting the phone number of the recipient terminal. The translation/interpretation application activation command may also be input through a specific key (eg, "*", "#" or a combination of other keys) of a folder phone having a physical button.

In one embodiment, when the caller has a phone number of 010-9999-1623 of the receiver terminal and wants an interpretation and translation service when talking with the receiver, after inputting the phone number of the receiver terminal 010-9999-1623, "interpretation and translation" You can input a command to activate the interpretation/translation application by touching the input area.

Referring back to FIG. 3 , the caller terminal visually exposes at least one target language through an interpretation and translation application ( S110 ).

The activated interpretation/translation application may visually expose at least one target language. The target language may mean a language used by a receiver or a target language in which a sender wants to convert their original voice. For example, when the receiver uses Chinese or the sender wants to interpret (or translate) his or her original voice into Chinese, the target language may mean Chinese. The interpretation and translation application may visually expose a plurality of target languages through a method such as a pop-up, a drop-down menu, or the like.

Referring to FIG. 5B together, the interpretation/translation application may expose target languages such as Vietnamese, Japanese, Spanish, Indonesian, English, French, Arabic, Chinese, and Russian. A new language may be added or changed in the exposed target language according to the interpretation/translation service provided by the interpretation/translation server 200 .

Referring back to FIG. 3 , the sender terminal receives a selection input for a target language from the sender ( S120 ).

The selection input may be input through an input area or an input unit using a part of the sender's body (eg, a finger, etc.) or an input tool (eg, a touch pen, a stylus pen, etc.). The region in which the selection input for the exposed target language is received may be displayed visually different from other regions (eg, visually transformed using hatching, shading, inverse image, color, etc.). Referring to FIG. 5B , a Chinese region in which a selection input is received among a plurality of target languages may be displayed with hatching.

Referring back to FIG. 3 , the caller terminal transmits a call call setup request, a phone number of the receiver terminal, and a selection input for a target language to the interpretation/translation server ( S130 ).

Call call setup refers to a communication line setup process between two terminals, including address ID required by the calling terminal of the data exchange network or path selection through the network and allowing connection to the receiving terminal. Call Call setup may be understood as a series of processes in which a certain data terminal equipment (DTE) transmits an ID signal and is connected to a receiving terminal (a terminal receiving call setup) via a network.

In one embodiment, the caller terminal 100 is a call call setup request to the receiver terminal 300 having a phone number of 010-9999-1623, Chinese, and a phone number of 010-9999-1623 of the receiver terminal can be transmitted to the interpretation and translation server 200 .

The interpretation/translation server may set up a call between the caller terminal and the receiver terminal based on the call call setup request received from the caller terminal and the phone number of the receiver terminal (S140).

In an embodiment, the interpretation/translation server 200 may set up a call between the caller terminal 100 and the receiver terminal 300 of the telephone number 010-9999-1623.

After a call is established with the receiver terminal 300 , the caller terminal 100 may visually or aurally output an alarm indicating that the interpretation/translation service is started. The alarm may be stored in the caller terminal 100 or received from the interpretation/translation server 200 .

In one embodiment, the caller terminal 100 outputs voice information such as "The simultaneous interpretation and translation service starts now. Please tell me exactly one sentence at a time." It may be visually output on the display of the terminal 100 .

After a call is established with the caller terminal 100 , the receiver terminal 300 may visually or audibly output an alarm indicating that the interpretation/translation service is started. The alarm may be stored in the receiver terminal 300 or received from the interpretation/translation server 200 .

In one embodiment, the receiver terminal 300 outputs voice information such as "Simultaneous interpretation and translation service is started at the request of the other party. Please tell me exactly one sentence at a time." It can be visually output to the display of (300).

In an embodiment, the caller terminal 100 may display a user interface (UI) when a call with the receiver terminal 300 is established. Referring to FIG. 6A , the user interface may indicate that the caller terminal 100 is currently providing an interpretation/translation service through an interpretation/translation application by displaying an “interpretation/interpretation service mode” on one end of the display unit.

Also, referring to FIG. 6A , the caller terminal 100 may display an input area for controlling the activation (or deactivation) of the speakerphone mode on one end of the display unit.

The sender terminal 100 may activate the speakerphone mode when a command to activate the speaker 140 phone mode is input from the sender while the interpretation and translation service is being provided. When the speakerphone mode is activated, the STT processing result or the translation result visually output by the interpretation/translation application can be checked together with the interpretation result. Also, when a speakerphone deactivation command is input from the sender while the speakerphone mode is activated, the caller terminal 100 may deactivate the speakerphone mode.

Also, referring to FIG. 6A , the caller terminal 100 may display an input area for controlling the activation (or deactivation) of the interpretation and translation service on one end of the display unit.

The interpretation/translation application may be switched to a deactivated state when a deactivation command of the interpretation/translation application is input from the sender in a state in which a call is established with the recipient terminal 300 . The deactivated interpretation and translation application may be activated again when an activation command of the interpretation and translation application is input from the sender. In addition, even if a call with the receiver terminal 300 is set in a state in which the interpretation and translation application is deactivated, when a request for activating the interpretation and translation application of the caller (or receiver) is received during a call, the caller terminal 100 (or the receiver terminal 300) ) installed in the interpretation and translation application can be activated.

The user interface may receive a predetermined code input from the caller in order to distinguish the original language voice for which the interpretation and translation service is provided. Referring to FIG. 6A , the user interface may further display an input area “Go” for inputting a predetermined code. The predetermined code will be described later in detail in step S150 below.

The sender terminal generates original language voice information based on the original language voice received from the sender (S150).

The original speech information may be generated by extracting the original speech information or encoding or recording the original speech information.

The sender terminal transmits the generated original language voice information to the interpretation/translation server (S160).

The original language may mean a language uttered by a caller or a language that is a target of an interpretation/translation service. For example, when the sender speaks Korean, the original language may mean Korean, and when the sender speaks English, the original language may mean English.

In one embodiment, the original language may be determined by referring to a user language (User Language) set in the caller terminal 100 by the interpretation and translation application. For example, when the language of the caller terminal is set to the Korean mode, the caller terminal may determine Korean as the original language.

In another embodiment, the original language may be selected through an interpretation and translation application. For example, the interpretation/translation application may further receive selection information for the original language from the sender. That is, when the language spoken by the caller is Korean, the caller terminal 100 may receive a source language selection input for Korean from the caller.

In another embodiment, the original language may be implemented to be determined by the interpretation/translation server 200 . For example, the interpretation/translation server 200 may determine which language the original language is based on the received original language voice information.

The caller terminal 100 may further receive a predetermined code after receiving the original language voice from the caller (not shown). The caller terminal 100 may generate original language voice information based on the received original language voice until a predetermined code is received. That is, the predetermined code may perform a function of a separator for separating the original language voice information to be interpreted and translated. The predetermined code may be composed of a combination of 0 to 9, "#", "*" or other keys through the keypad (or the area where the keypad is displayed) of the caller terminal. Referring to FIG. 6A together, the predetermined code may be received through an input area (eg, “Go”) displayed on one end of the caller terminal 100 .

Referring to FIG. 6B together, the sender terminal 100 determines that the original voice input is completed when receiving a predetermined code through the input area (“Go”) after receiving the original language voice “hello” from the sender can do. When it is determined that the original language voice input is completed, the caller terminal 100 may generate original language voice information from the original language voice and transmit it to the interpretation/translation server 200 .

The original language voice information may include original language information (eg, information that the original language is Korean). For example, the original language voice information including "Hello?" may further include information that the original language is Korean.

Referring back to FIG. 3 , the interpretation/translation server performs STT processing on the voice information received from the caller terminal, and translates and interprets the original voice information into the target language to generate an STT processing result, a translation result, or an interpretation result (S170). ).

In an embodiment, the interpretation/translation server 200 may first generate an interpretation result obtained by interpreting the source language voice information into the target language, and then may generate the translation result by processing the speech-to-text (STT) result of the interpretation. For example, the interpretation/translation server 200 interprets the original language voice information “How are you today” into a target language (eg, Chinese) to generate an interpretation result “最近得?” STT processing can produce a translation result of "最近得?".

The interpretation/translation server 200 may generate an STT processing result obtained by performing STT processing on the original language voice information, and then may generate a translation result based on the STT processing result. For example, the interpretation/translation server 200 performs STT processing on the original language voice information of “how are you these days” to generate an STT processing result of “how are you these days?” , Chinese) to produce a translation result of "最近得?".

The interpretation/translation server transmits the STT processing result or translation result to the sender terminal, and transmits the interpretation result to the receiver terminal (S180).

In one embodiment, the interpretation/translation server 200 transmits the STT processing result "How are you today" and the translation result "最近得?" to the sender terminal 100, and transmits the interpretation result, "最近得?" It may transmit to the receiver terminal 300 .

Referring to FIG. 4 together, the interpretation/translation server may determine whether an interpretation/translation application is installed in the receiver's terminal (S192).

In one embodiment, the interpretation and translation server 200 transmits a confirmation request as to whether the interpretation and translation application is installed to the receiver terminal 300, and when an Ack for the confirmation request is received from the receiver terminal 300, the receiver terminal ( 300), it can be determined that the interpretation and translation application is installed.

As a result of the determination, when the interpretation and translation application is installed in the receiver terminal, the interpretation and translation server may further transmit the STT processing result and/or the translation result to the receiver terminal (S194).

In one embodiment, the interpretation and translation server 200, when the interpretation and translation application is installed in the receiver terminal 300, the translation result of the original language voice information "How are you these days" "最近得?" How are you" may be further transmitted to the receiver terminal 300 .

Referring back to FIG. 3 , the sender terminal may output the STT processing result and/or the translation result received from the interpretation/translation server (S182), and the receiver terminal may output the interpretation result received from the interpretation/translation server (S198) .

Referring to FIG. 7A together, the caller terminal 100 transmits the STT processing result of the original language voice information "How are you today?" "How are you today?" and "最近得?", which is a translation result, can be visually output.

Referring to FIG. 7B together, the receiver terminal 300 in which the interpretation and translation application is installed may further receive the STT processing result and/or the translation result from the interpretation and translation server 200 . The receiver terminal 300 installed with the interpretation and translation application may visually output "How are you these days" and the translation result "最近得?", which are the STT processing results, and audibly output "最近得?", the interpretation result, have. The interpretation result may be output through the speaker of the receiver terminal 300 .

During call setup, the receiver terminal 300 may transmit target language voice information generated based on the target language voice received from the receiver to the interpretation/translation server 200 (not shown). In this case, the interpretation/translation server 200 translates the target language voice information into the original language based on the STT processing result for the target language voice information, the original language information received from the caller terminal 100, and/or the target language An interpretation result obtained by interpreting the language and voice information into the original language may be transmitted to the caller terminal 100 . The method of providing an interpretation/interpretation service through the caller terminal 100 may further include outputting the STT processing result and the translation result through an interpretation/translation application, and outputting the interpretation result through the speaker 140 .

When the caller terminal 100 (or the receiver terminal 300) receives an input to cancel the call setup from the caller (or the receiver) during call call setup, the received A cumulative translation result may be generated based on the STT processing result and/or the translation result.

The cumulative translation result may include both the STT processing result that is text for the original voice and/or the translation result that is the text of the target language. In addition, the cumulative translation result may further include an STT processing result and/or a translation result of voice information uttered by the counterpart.

Referring to FIG. 8 , when the sender utters "how are you today", the receiver utters "我好", and then the sender utters "what's the weather there" again, the cumulative translation result is the caller and Text information such as "How are you today / 最近得?", "我好 / I am well", "What's the weather there / 那天", which is the result of STT processing for all utterances of the recipient and/or its translation result may include all of them. According to an embodiment, the cumulative translation result may be implemented so that the original language is displayed before the target language, and only in the case of the original language voice information uttered by the caller, the original language may be displayed before the target language.

The caller terminal 100 (or the receiver terminal 300) transmits the original language voice information generated based on the original language voice received from the caller until a release input for call setup is received from the caller (or receiver) to the interpretation and translation server ( 200), and outputting the STT processing result for the original language voice information received from the interpretation/translation server 200 through the interpretation/translation application and/or the translation result obtained by translating the original voice information into the target language. can

The device described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

Software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or apparatus, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible from the above description by those skilled in the art. For example, the described techniques are performed in an order different from the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

100: caller terminal
110: central processing unit
120: communication interface
130: microphone
140: speaker
200: interpretation and translation server
300: receiver terminal

Claims (28)

receiving an input of a receiver terminal phone number from a sender;
visually exposing at least one target language through an interpretation/translation application;
receiving a selection input for a target language from the sender;
transmitting a selection input for the recipient terminal phone number and the target language to an interpretation/translation server along with a call setup request with the recipient terminal; and
Transmitting the original language voice information generated based on the original language voice received from the caller to the interpretation/translation server
including,
The interpretation and translation server is
Transmitting a request to confirm whether the interpretation and translation application is installed to the receiver terminal,
When it is determined that the interpretation and translation application is not installed in the receiver terminal, an interpretation result obtained by interpreting the original voice information into the target language is transmitted to the receiver terminal,
When it is determined that the interpretation and translation application is installed in the receiver terminal, the STT (Speech-To-Text) processing result for the original language voice information or the translated text translated from the original language voice information into the target language is further transferred to the receiver terminal to transmit,
A method of providing an interpretation and translation service through a caller's terminal.
According to claim 1,
The step of transmitting the call call setup request to the interpretation and translation server comprises:
Outputting an alarm informing that the interpretation and translation service is started after a call is established with the called party terminal
containing,
A method of providing an interpretation and translation service through a caller's terminal.
According to claim 1,
The receiver terminal outputs an alarm notifying that the interpretation and translation service is started after a call is established with the caller terminal,
A method of providing an interpretation and translation service through a caller's terminal.
According to claim 1,
Transmitting the original language voice information generated based on the original language voice received from the caller to the interpretation/translation server,
to receive more predetermined codes,
generating the original language voice information based on the received original language voice until the predetermined code is received,
A method of providing an interpretation and translation service through a caller's terminal.
delete According to claim 1,
The interpretation and translation application is activated or deactivated according to an activation command or deactivation command of the interpretation and translation application input from the caller in a state in which a call call is established with the recipient terminal,
A method of providing an interpretation and translation service through a caller's terminal.
According to claim 1,
In a state in which a call is established with the called terminal, the speakerphone mode of the caller terminal is activated or deactivated according to a speakerphone mode activation command or a deactivation command input from the caller,
A method of providing an interpretation and translation service through a caller's terminal.
According to claim 1,
The method further comprising the step of outputting, through the interpretation/translation application, the STT processing result for the original speech information received from the interpretation/translation server, or the translated text obtained by translating the original speech information into the target language,
A method of providing an interpretation and translation service through a caller's terminal.
9. The method of claim 8,
When an input for canceling the call setup is received from the caller, accumulated translation text is generated based on the STT processing result or the translation text received from after the call call is established with the receiver terminal until the call call setup is released step to do
further comprising,
A method of providing an interpretation and translation service through a caller's terminal.
According to claim 1,
After generating the interpretation result, the interpretation/translation server generates a translated text obtained by translating the voice information of the original language into the target language by processing the result of the interpretation by STT (Speech To Text);
A method of providing an interpretation and translation service through a caller's terminal.
According to claim 1,
The interpretation/translation server generates a translation text obtained by translating the STT processing result into the target language after generating the STT (Speech To Text) processing result for the voice information in the original language,
A method of providing an interpretation and translation service through a caller's terminal.
According to claim 1,
Until a release input for setting up the call is received from the caller,
transmitting the original language voice information generated based on the original language voice received from the caller to the interpretation/translation server; and
Repeating the step of outputting, through the interpretation and translation application, the STT processing result for the original speech information received from the interpretation and translation server or the translated text translated from the original speech information into the target language,
A method of providing an interpretation and translation service through a caller's terminal.
According to claim 1,
The receiver terminal transmits the target language voice information generated based on the target language voice received from the receiver to the interpretation/translation server,
The interpretation and translation server, as a result of the STT processing for the target language voice information, based on the original language information received from the caller terminal, translates the target language voice information into an original language and the target language voice information. Transmitting at least one of the interpretation results translated into the original language to the sender terminal,
outputting, by the sender terminal, the STT processing result, the translated text, or the interpretation result through the interpretation and translation application
further comprising,
A method of providing an interpretation and translation service through a caller's terminal.
According to claim 1,
The interpretation and translation application is activated (activation) according to the activation command input together with the receiver terminal phone number from the sender,
A method of providing an interpretation and translation service through a caller's terminal.
an interpretation/translation application for receiving a selection input for a recipient terminal phone number and a target language from the sender; and
Transmitting a selection input for the recipient's terminal phone number and the target language to the interpretation/translation server together with a call setup request with the recipient's terminal Communication interface for transmitting information to the interpretation and translation server
including,
The interpretation and translation server is
Transmitting a request to confirm whether the interpretation and translation application is installed to the receiver terminal,
When it is determined that the interpretation and translation application is not installed in the receiver terminal, an interpretation result obtained by interpreting the original voice information into the target language is transmitted to the receiver terminal,
When it is determined that the interpretation and translation application is installed in the receiver terminal, the STT (Speech-To-Text) processing result for the original language voice information or the translated text translated from the original language voice information into the target language is further transferred to the receiver terminal to transmit,
Caller terminal that provides interpretation and translation services.
16. The method of claim 15,
The interpretation and translation application is
After a call is established with the receiver terminal, outputting an alarm indicating that the interpretation and translation service is started,
Caller terminal that provides interpretation and translation services.
16. The method of claim 15,
The receiver terminal outputs an alarm notifying that the interpretation and translation service is started after a call is established with the caller terminal,
Caller terminal that provides interpretation and translation services.
16. The method of claim 15,
The interpretation and translation application generates original language voice information based on the original language voice received from the caller until a predetermined code is received,
Caller terminal that provides interpretation and translation services.
delete 16. The method of claim 15,
The interpretation and translation application is activated or deactivated according to an activation command or deactivation command of the interpretation and translation application input from the caller in a state in which a call call is established with the recipient terminal,
Caller terminal that provides interpretation and translation services.
16. The method of claim 15,
In a state in which a call is established with the called terminal, the speakerphone mode of the caller terminal is activated or deactivated according to a speakerphone mode activation command or a deactivation command input from the caller,
Caller terminal that provides interpretation and translation services.
16. The method of claim 15,
The interpretation/translation application outputs a translation text obtained by translating the STT processing result for the original speech information received from the interpretation/translation server or the original speech information into the target language,
Caller terminal that provides interpretation and translation services.
23. The method of claim 22,
The interpretation and translation application is
When an input for canceling the call setup is received from the caller, accumulated translation text is generated based on the STT processing result or the translation text received from after the call call is established with the receiver terminal until the call call setup is released doing,
Caller terminal that provides interpretation and translation services.
16. The method of claim 15,
After generating the interpretation result, the interpretation/translation server generates a translated text obtained by translating the voice information of the original language into the target language by processing the result of the interpretation by STT (Speech To Text);
Caller terminal that provides interpretation and translation services.
16. The method of claim 15,
The interpretation/translation server generates a translation text obtained by translating the STT processing result into the target language after generating the STT (Speech To Text) processing result for the voice information in the original language,
Caller terminal that provides interpretation and translation services.
16. The method of claim 15,
The interpretation and translation application is
Until a release input for setting up the call is received from the caller,
The original language voice information generated based on the original language voice received from the caller is transmitted to the interpretation and translation server, and the STT processing result for the original language voice information received from the interpretation and translation server or the original language voice information is converted into the target language repeating the step of outputting the translated text translated as
Caller terminal that provides interpretation and translation services.
16. The method of claim 15,
Further comprising a speaker for outputting the interpretation result,
The receiver terminal transmits the target language voice information generated based on the target language voice received from the receiver to the interpretation/translation server,
The interpretation and translation server, as a result of the STT processing for the target language voice information, based on the original language information received from the caller terminal, translates the target language voice information into an original language and the target language voice information. Transmitting at least one of the interpretation results translated into the original language to the sender terminal,
The interpretation and translation application outputs the STT processing result or the translated text, and outputs the interpretation result through the speaker,
Caller terminal that provides interpretation and translation services.
The interpretation and translation application performing the method of providing the interpretation and translation service of claim 1 is recorded,
computer readable recording medium.

Images