JP6985113B2 - How to provide an interpreter function for electronic devices - Google Patents

Description

The following description is stored in a computer-readable recording medium in order to cause the computer to execute the interpretation function providing method and device using the earset, and the interpreting function providing method according to the embodiment of the present invention in combination with the computer. Computer programs and their recording media.

An earset is a device equipped with earphones and a microphone that can be used to make a call without using hands by connecting to a telephone. For example, by connecting to a smartphone, it is possible to make a telephone call. , Music-like audio can be received and output, and various control signals for controlling the smartphone by physical buttons and sensors, or voice can be provided to the smartphone. For example, Patent Document 1 discloses a Bluetooth ear set having a built-in ear kernel microphone.

Korean Publication No. 10-2017-0030375 Gazette

A computer-readable record for causing a computer to execute an interpreter function providing method and device capable of providing an interpreter function using an ear set, and an interpreter function providing method according to an embodiment of the present invention in combination with a computer. A computer program stored in a medium and a recording medium thereof are provided.

The voice of the first language input through the first earset device and received by the electronic device is converted into the voice of the second language and provided to the second earset device, and the voice input through the second earset device and received by the electronic device is received. By converting the voice of the second language into the voice of the first language and providing it to the first earset device, the earset pair (pair) including the first earset device and the second earset device is used. To make a computer execute an interpreter function providing method and an apparatus capable of providing an interpreter function between two users who use different languages, and an interpreter function providing method according to an embodiment of the present invention in combination with a computer. A computer program stored in a computer-readable recording medium and the recording medium thereof are provided.

A method of providing an interpreter function for an electronic device, which is a step of receiving an application execution signal generated by the first earset device or the second earset device, a step of executing an application corresponding to the received application execution signal, and the above-mentioned. A step of providing translation into another language for voice input through at least one of the first earset device and the second earset device and received by the electronic device, subject to the control of the application being executed. Provides a method of providing an interpreter function, including.

A method of providing an interpreter function for a first earset device, in which the first language voice input to the first earset device is converted into a second language voice by the interpreter function of an application executed by the electronic device. At the stage of transmitting the voice of the first language to the electronic device, as provided to the second earset device forming an earset pair with the first earset device, the sound is input to the second earset device and said. In response to the electronic device converting the second language voice received by the electronic device into the first language voice by the interpreter function of the executed application, the converted first language from the electronic device. Provided is a method for providing an interpreter function, which includes a step of receiving a voice and a step of outputting the received voice of the first language.

Provided is a computer-readable recording medium in which a computer program for causing a computer to execute the interpretation function providing method is recorded.

Provided is a computer program stored in a computer-readable recording medium in combination with a computer to cause the computer to execute the interpretation function providing method.

The first earset device, which is a sound signal input unit for inputting sound, a sound signal output unit for outputting sound, and a second ear that forms an earset pair with the electronic device or the first earset device. A communication unit for communicating with a set device, a sound signal input unit, a sound signal output unit, and a control unit for controlling the communication unit are provided, and the control unit is input through the sound signal input unit. The sound of one language is converted into the sound of the second language according to the interpreting function of the application executed by the electronic device and provided to the second earset device forming an earset pair with the first earset device. The communication unit is controlled so as to transmit the sound of the first language to the electronic device, and the sound of the second language input to the second earset device and received by the electronic device is transmitted to the electronic device. Responds to converting to a first language voice according to the interpreting function of the executed application, and controls the communication unit to receive the converted first language sound from the electronic device, and the reception Provided is a first ear set device that controls the sound signal output unit so as to output the sound of the first language.

An earset can be used to provide an interpreter function, and an earset pair including a first earset device and a second earset device can be used to provide an interpreter function between two users in different languages. be able to.

By using the interpreting function, it is possible to provide the voice translated from the first language to the second language to the other party of the telephone call.

It is a figure which showed the example of the process of providing an interpreter function using an ear set in one Embodiment of this invention. It is a figure which showed the example of the ear set pair in one Embodiment of this invention. It is a figure which showed the communication example of the ear set pair and the electronic device in one Embodiment of this invention. It is a figure which showed the other communication example of an ear set pair and an electronic device in one Embodiment of this invention. It is a block diagram for demonstrating the internal structure of an electronic device, an ear set device, and a server in one Embodiment of this invention. It is a figure which showed the example of the cloud artificial intelligence platform in one Embodiment of this invention. It is a flowchart which showed the example of the interpretation function providing method in one Embodiment of this invention. It is a flowchart which showed the example which provides the interpretation function for self-interpretation and multi-person interpretation in one Embodiment of this invention. It is a flowchart which showed the example which switches to the music listening mode or the telephone call mode in the interpreting mode in one Embodiment of this invention. It is a flowchart which showed the example which provides the interpretation function during a telephone call in one Embodiment of this invention. It is a flowchart which showed the example of the interpretation function providing method of the ear set apparatus in one Embodiment of this invention.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

The method for providing an interpreter function according to an embodiment of the present invention may be executed by an electronic device or an ear set device described below. For example, a computer program according to an embodiment of the present invention may be installed and driven in an electronic device or an ear set device, and the electronic device or the ear set device is an embodiment of the present invention in accordance with the control of the driven computer program. The method of providing the interpreter function according to the form may be executed. The computer program described above may be stored in a computer-readable recording medium in order to combine it with an electronic device or an ear set device realized by a computer to cause the computer to execute an interpretation function providing method. The computer program described here may be in the form of one independent program package, or the form of one independent program package may be in the form of an operating system or other program package that is pre-installed and driven by the computer. It may be in a linked form.

FIG. 1 is a diagram showing an example of an interpreter function providing process using an ear set in one embodiment of the present invention. FIG. 1 shows an electronic device 110, a first earset device 120, and a second earset device 130, in which the first user 140 has the first earset device 120 as the right ear and the second user 150 has the second earset device 120. An example of wearing the ear set device 130 on the left ear is shown. Here, it is assumed that the first user 140 uses the first language and the second user 150 uses the second language.

At this time, the voice of the first language spoken by the first user 140 may be input to a sound signal input unit such as a microphone included in the first earset device 120, and the first earset device 120 is input. The voice of the first language may be transmitted to the electronic device 110. In this case, the electronic device 110 uses the interpreter function of the application installed in the electronic device 110 to convert the received voice of the first language into the voice of the second language and provide it to the second earset device 130. good. At this time, the second earset device 130 may output the provided second language sound through a sound signal output unit such as a speaker (earphone) included in the second earset device 130. Therefore, the second user 150 can hear the voice of the first language spoken by the first user 140 with the voice translated into the second language.

Further, the voice of the second language spoken by the second user 150 may be input to a sound signal input unit such as a microphone included in the second earset device 130, and the second earset device 130 may input the input second earset device 130. Bilingual audio may be transmitted to the electronic device 110. In this case, the electronic device 110 uses the interpreter function of the application installed in the electronic device 110 to convert the received voice of the second language into the voice of the first language and provide it to the first earset device 120. good. At this time, the first earset device 120 may output the provided voice of the first language through a sound signal output unit such as a speaker (earphone) included in the first earset device 120. Therefore, the first user 140 can hear the voice of the second language spoken by the second user 150 with the voice translated into the first language.

As described above, according to the embodiment of FIG. 1, the function of multi-person interpretation can be provided by using the first ear set device 120 and the second ear set device 130 forming one ear set pair.

In an embodiment different from the embodiment of FIG. 1, it may be considered that the first user 140 wears the first earset device 120 and the second earset device 130 to both ears, respectively. In this case, the voice of the first language spoken by the first user 140 may be input through at least one of the first earset device 120 and the second earset device 130 and transmitted to the electronic device 110. At this time, the electronic device 110 uses the interpreter function of the application installed in the electronic device 110 to convert the received voice of the first language into the voice of the second language, and the first ear set device 120 and the second ear. It may be transmitted to at least one of the set devices 130. At this time, the first earset device 120 and / or the second earset device 130 may output the voice converted into the second language, and the first user 140 may output the voice spoken by himself / herself in the first language. You will be able to hear the translated second language voice. Thus, unlike the embodiment of FIG. 1, other embodiments of the present invention can also provide a function for self-interpretation.

The function for multi-person interpretation and the function for self-interpretation may be provided by changing the mode in the application installed in the electronic device 110, respectively. For example, the application may control the electronic device 110 to provide a user interface capable of setting one of a multi-person interpretation mode and a self-interpretation mode, and utilize the provided user interface. Then, the user may directly set one of the multi-person interpretation mode and the self-interpretation mode. Alternatively, such a mode setting may be made by input to a physical button included in at least one of the first earset device 120 and the second earset device 130. As another example, the mode may be set by analyzing the voice input to the first ear set device 120 and the second ear set device 130. For example, if the pitch and volume of the voices input to the first earset device 120 and the second earset device 130 are similar, the self-interpretation mode may be set, and the first earset device 120 and the first earset device 120 may be set. 2 When the pitch and volume of the voice input to each of the earset devices 130 are completely different, the multi-person interpretation mode may be set.

FIG. 2 is a diagram showing an example of an ear set pair in one embodiment of the present invention. As shown in FIG. 2, the ear set pair according to the present embodiment includes the first ear set device 120 and the second ear set device 130. In the embodiments of FIGS. 1 and 2, the first earset device 120 is shown to be worn on the right ear, and the second earset device 130 is shown to be worn on the left ear, for the purpose of explaining the invention. It is only one embodiment and is not limited to this. For example, the first earset device 120 may be implemented to be worn on the left ear and the second earset device 130 may be worn on the right ear.

Further, in the embodiment of FIG. 2, an example of a cradle 210 that provides functions such as storage, deferment, and charging of the first earset device 120 and the second earset device 130 is shown. Such a cradle 210 is also merely an example for explaining an embodiment in which the first earset device 120 and the second earset device 130 form one earset pair, and its shape, function, and the like are shown in FIG. 2. It is not limited by the explanation of.

FIG. 3 is a diagram showing an example of communication between an ear set pair and an electronic device according to an embodiment of the present invention. In general, smartphones communicate with peripheral devices such as earsets and wireless speakers via short-range wireless communication protocols such as Bluetooth (Bluetooth), Zigbee, Wi-Fi and the like. Also in the embodiment of FIG. 3, an example in which the electronic device 110 communicates with the ear set pair by using such a short-range wireless communication protocol will be described. However, the electronic device 110 uses one short-range wireless communication protocol (as an example, Bluetooth 4.1) to communicate with only one device at a time (first earset device 120 in the embodiment of FIG. 3). There are cases where you have to. In other words, the case where the electronic device 110 cannot simultaneously communicate with two or more devices via one short-range wireless communication protocol is considered. In the present embodiment, the electronic device 110 may communicate with the first earset device 120 as one device using one short-range wireless communication protocol, and the second earset device 130 is the first earset. Communication may be performed by relaying the device 120. For this purpose, the first earset device 120 and the second earset device 130 may perform short-range wireless communication with each other. For example, the second earset device 130 may transmit the input second language voice to the electronic device 110 via the first earset device 120. On the contrary, the voice translated into the first language transmitted by the electronic device 110 may be transmitted to the second earset device 130 via the first earset device 120. As an example for such relay communication, a TWS (True Wireless Stereo) method may be used.

FIG. 4 is a diagram showing another communication example between the earset pair and the electronic device in one embodiment of the present invention. Unlike the embodiment of FIG. 3, the electronic device 110 may communicate with both the first earset device 120 and the second earset device 130 at the same time by using one short-range wireless communication protocol. For example, a short-range wireless communication protocol such as Bluetooth 5.0 provides a technology that allows a smartphone to simultaneously connect and communicate with two devices via Bluetooth. In the embodiment of FIG. 4, by utilizing the short-range wireless communication protocol capable of communicating with a plurality of devices at the same time, communication is performed simultaneously with both the first earset device 120 and the second earset device 130. Is also possible.

On the other hand, when the electronic device 110 provides the interpreting function by using the application, it is possible to provide the interpreting function voluntarily possessed by the application, but the electronic device 110 receives the interpreting function from the server linked with the application. Is also possible. For example, the electronic device 110 may connect to a server under the control of an application to transmit information about a first language voice and a second language for interpreting the first language voice. In this case, the server may convert the voice of the first language into the voice of the second language and provide it to the electronic device 110, and the electronic device 110 may translate the voice of the first language into the voice of the second language. You will be able to get voice.

FIG. 5 is a block diagram for explaining an internal configuration of an electronic device, an ear set device, and a server according to an embodiment of the present invention. The server 520 of FIG. 5 may be an external system that provides an interpreter function in cooperation with an application installed and driven in the electronic device 110.

The electronic device 110 and the server 520 may include a memory 511, 521, a processor 512, 522, a communication interface 513, 523, and an input / output interface 514, 524. The memory 511, 521 is a computer-readable recording medium and includes a RAM (random access memory), a ROM (read only memory), and a permanent mass memory device such as a disk drive. It's fine. Here, a permanent large-capacity storage device such as a ROM or a disk drive may be included in the electronic device 110 or the server 520 as another permanent storage device that is separated from the memory 511 and 521. Further, the memory 511 and 521 contain an operating system and at least one program code (for example, a code installed in an electric device 110 and driven by an electronic device 110 to provide a specific service). May be stored. Such software components may be loaded from a computer-readable recording medium separate from the memory 511, 521. Such computer-readable recording media may include computer-readable recording media such as floppy (registered trademark) drives, discs, tapes, DVD / CD-ROM drives, and memory cards. In other embodiments, the software components may be loaded into memory 511, 521 through communication interfaces 513, 523, which are not computer readable recording media. For example, at least one program is an electronic device 110 based on a computer program (eg, the application described above) installed by a file provided by a file distribution system that distributes a developer or application installation file over network 530. It may be loaded into the memory 511 of.

Processors 512 and 522 may be configured to process instructions in a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to processors 512 and 522 by memory 511, 521 or communication interfaces 513, 523. For example, the processor 512, 522 may be configured to execute an instruction received according to a program code stored in a recording device such as memory 511, 521.

The communication interfaces 513 and 523 may provide a function for the electronic device 110 and the server 520 to communicate with each other via the network 530, and the electronic device 110 and / or the server 520 may be another electronic device or another. It may provide a function for communicating with the server. As an example, a request generated by the processor 512 of the electronic device 110 according to a program code stored in a recording device such as a memory 511 may be transmitted to the server 520 via the network 530 under the control of the communication interface 513. On the contrary, control signals, instructions, contents, files, etc. provided under the control of the processor 522 of the server 520 are received by the electronic device 110 through the communication interface 513 of the electronic device 110 via the communication interface 523 and the network 530. May be done. For example, control signals, instructions, contents, files, etc. of the server 520 received through the communication interface 513 may be transmitted to the processor 512, the memory 511, and the contents, the files, etc. may be further included in the storage medium 110. It may be stored in (permanent storage device described above).

The input / output interface 514 may be a means for an interface with the input / output device 515. For example, the input device may include a device such as a microphone, keyboard, or mouse, and the output device may include a device such as a display or speaker. As another example, the input / output interface 514 may be a means for an interface with a device that integrates functions for input and output, such as a touch screen. The input / output device 515 may be composed of an electronic device 110 and one device. Also, the input / output interface 524 of the server 520 may be a means for interfacing with a device (not shown) for input or output that can be linked to or included in the server 520.

Also, in other embodiments, the electronic device 110 and the server 520 may include fewer or more components than the components of FIG. However, most prior art components need not be clearly shown in the figure. For example, the electronic device 110 may be realized to include at least a part of the above-mentioned input / output device 515, such as a transceiver, a GPS (Global Positioning System) module, a camera, various sensors, a database, and the like. Other components may be further included. As a more specific example, when the electronic device 110 is a smartphone, an acceleration sensor or gyro sensor, a camera module, various physical buttons, a button using a touch panel, an input / output port, which are generally included in the smartphone, are used. Various components such as accelerometers for vibration may be further incorporated into the electronic device 110.

The first earset device 120 forms an earset pair with a sound signal input unit 541 for receiving a sound input, a sound signal output unit 542 for outputting a sound, an electronic device 110, or a first earset device 120. 2. A communication unit 543 for communicating with the earset device 130, a sound signal input unit 541, a sound signal output unit 542 for outputting sound, and a control unit 544 for controlling the communication unit 543 may be provided. For example, the control unit 544 may control the sound signal input unit 541 so as to receive the input of the voice of the first language spoken by the user (for example, the first user 140 in FIG. 1). Further, the control unit 544 converts the input first language voice into the second language voice according to the interpreting function of the application executed by the electronic device 110 and provides the second earset device 130. , The communication unit 543 may be controlled so as to transmit the voice of the first language to the electronic device 110. Further, the control unit 544 converts the second language voice input to the second earset device 120 and received by the electronic device 110 into the first language voice according to the interpreter function of the application executed by the electronic device 110. In response to this, the communication unit 543 may be controlled so as to receive the voice of the first language converted from the electronic device 110. Further, the control unit 544 may control the sound signal output unit 542 so as to output the voice of the first language received from the electronic device 110.

FIG. 6 is a diagram showing an example of a cloud artificial intelligence platform according to an embodiment of the present invention. FIG. 6 shows an electronic device 610, a cloud artificial intelligence platform 620, and a content service 630.

As an example, the electronic device 610 may include, as an example, an in-home device connected via a smart home network, but each or at least one may correspond to the above-mentioned electronic device 110. Such an electronic device 610 or an application installed and driven by the electronic device 610 (hereinafter referred to as an application) may be linked with the cloud artificial intelligence platform 620 via the interface connect 640. Here, the Interface Connect 640 may provide the developer with an SDK (Software Development Kit) and / or a development document for developing an application installed and driven by the electronic device 610 or the electronic device 610. Further, the interface connect 640 may provide an API (Application Program Interface) that allows an application installed and driven by the electronic device 610 or the electronic device 610 to utilize the functions provided by the cloud artificial intelligence platform 620. As a specific example, the developer develops a device or application using the SDK (Software Development Kit) provided by Interface Connect 640 and / or the development document, and the device or application developed by using the API provided by Interface Connect 640 is used in the cloud. The function provided by the artificial intelligence platform 620 (as an example, the interpreter function) can be utilized.

Here, the cloud artificial intelligence platform 620 may provide not only an interpreter function but also a function for providing various voice-based services. For example, the cloud artificial intelligence platform 620 has an audio processing module 621 for recognizing received audio and synthesizing output audio, a vision processing module 622 for analyzing and processing received video and video, and received audio. Conversation processing module 623 for determining the appropriate conversation to output voice that fits according to (for example, voice translated into a second language), recommendation module for recommending a function that fits the received voice. 624, Includes various modules for providing speech-based services, such as Neural Machine Translation (NMT) 625, which supports artificial intelligence to translate languages on a sentence-by-sentence basis based on data learning. It's fine.

For example, the electronic device 110 may transmit information about the voice of the first language and the translated second language to the cloud artificial intelligence platform 620 by using the API provided by the interface connect 640. In this case, the cloud artificial intelligence platform 620 may recognize and analyze the received first language voice by utilizing the above-mentioned modules 621 to 625, and interprets the first language voice into the second language voice. , Appropriate response voice may be synthesized and provided.

Further, the expansion kit 650 may provide a development kit that allows a third-party content developer or company to realize a new voice infrastructure function based on the cloud artificial intelligence platform 620. For example, it may be developed using a development kit provided with another application and a server capable of utilizing the interpreting function of the cloud artificial intelligence platform 620.

Such a cloud artificial intelligence platform 620 is only a specific example of a server 520 for providing an interpreter function to the electronic device 110, and is a server 520 capable of providing an interpreter function to the electronic device 11. There is no limitation.

FIG. 7 is a flowchart showing an example of an interpreter function providing method in one embodiment of the present invention. The method for providing an interpreter function according to the present embodiment may be executed by a computer device such as the electronic device 110 described above. At this time, the processor 512 of the electronic device 110 may be realized to execute a control instruction (instruction) by the code of the operating system included in the memory 511 or the code of at least one program. Here, the processor 512 controls the electronic device 110 so that the electronic device 110 executes the steps 710 to 730 included in the interpretation function providing method of FIG. 7 according to the control instruction provided by the code stored in the electronic device 110. You can do it.

At step 710, the electronic device 110 may receive an application execution signal generated by the first earset device or the second earset device. Here, the first ear set device and the second ear set device may correspond to the first ear set device 120 and the second ear set device 130 described above. For example, the first earset device and the second earset device may consist of earset pairs for the left and right ears. The application execution signal includes input from a physical button included in at least one of the first earset device and the second earset device, or by at least one of the first earset device and the second earset device. It may be generated by the output value of the sensor. For example, the first earset device (or second earset device) senses and senses an input signal generated by the user pressing a button included in the first earset device (or second earset device). The application execution signal may be generated corresponding to the input signal. After that, the first earset device (or the second earset device) may transmit the generated application execution signal to the electronic device 110. As another example, the first earset device (or second earset device) may include an illuminance sensor. At this time, the first earset device (or the second earset device) detects the action of the user holding his / her finger over the measurement site of the illuminance sensor for a certain period of time or longer, and generates an application execution signal by detecting such an action. You may. As described above, those skilled in the art can easily understand that one of various methods for sensing a specific action of a user and generating an application execution signal can be utilized. Let's do it.

At step 720, the electronic device 110 may execute the application corresponding to the received application execution signal. At this time, the application to be executed may be a computer program for providing an interpreter function voluntarily or in cooperation with the above-mentioned server 520.

At step 730, the electronic device 110 is input through at least one of the first earset device and the second earset device and is received by the electronic device 110 in another language according to the control of the application to be executed. An interpreter function may be provided. At this time, as described above, the interpreting function may include a function for self-interpretation and a function for multi-person interpretation.

In this way, the user (or the users) can use the earset device to execute an application for the interpreting function of the electronic device 110, and other voices spoken using the earset device can be executed. You will be able to receive the provision of an interpreter function for the language.

FIG. 8 is a flowchart showing an example of providing an interpreting function for self-interpretation and multi-person interpretation in one embodiment of the present invention. Stages 81 to 890 of FIG. 8 show an example of a specific process for providing an interpreter function at stage 730 of FIG.

At step 810, the electronic device 110 may determine if the interpreter mode is running. For example, if the interpreting mode is not running due to the termination of the interpreting mode or the termination of the application, the process of FIG. 8 may be terminated. The electronic device 110 may execute step 820 when the interpreter mode is being executed in step 810.

At step 820, the electronic device 110 may determine whether the interpreting mode is the multi-party interpreting mode. If the interpreting mode is a multi-party interpreting mode, step 830 may be executed. If the interpreting mode is not the multi-person interpreting mode, the electronic device 110 may perform step 870.

At step 830, the electronic device 110 may determine whether audio has been received from the first earset device. Here, the first earset device may be any one of the two earset devices included in the earset pair. At this time, when the voice is received from the first ear set device, the electronic device 110 may execute the step 840. If no audio is received from the first earset device, the electronic device 110 may perform step 850.

At step 840, the electronic device 110 may convert the voice of the first language input through the first earset device and received by the electronic device 110 into the voice of the second language and provide it to the second earset device. Here, the second ear set device may be another ear set device excluding the ear set device that provided the sound in the stage 830. The second earset device outputs the converted voice of the second language so that the second user who wears the second earset device can receive the first user who wears the first earset device in the first language. The spoken voice may be translated into a second language voice and provided.

At step 850, the electronic device 110 may determine whether audio has been received from the second earset device. At this time, when the voice is received from the second ear set device, the electronic device 110 may execute the step 860. If no audio is received from the second earset device, the electronic device 110 may perform step 810 again.

At step 860, the electronic device 110 may receive the voice of the second language input through the second earset device, convert it into the voice of the first language, and provide it to the first earset device. The first earset device outputs the converted voice of the first language so that the first user who wears the first earset device can receive the second user who wears the second earset device in the second language. The spoken voice may be provided as an interpreter to the voice of the first language. After this, the electronic device 110 may execute step 810 again.

At step 870, the electronic device 110 may determine if the interpreting mode is the self-interpreting mode. If the interpreting mode is the self-interpreting mode, step 880 may be performed. If the interpreting mode is not the self-interpreting mode, the electronic device 110 may perform step 890.

At step 880, the electronic device 110 receives the voice of the first language input through at least one of the first earset device and the second earset device, converts it into the voice of the second language, and converts it into the voice of the second language. It may be provided to at least one of the ear set device and the second ear set device. In the self-interpretation mode, it is determined that one user is wearing both the first earset device and the second earset device, so that at least one of the first earset device and the second earset device is used. The audio of the first language may be received through the device and the translated audio of the second language may be provided to at least one of the first earset device and the second earset device. After step 880, the electronic device 110 may perform step 810 again.

At step 890, the electronic device 110 may set the mode. For example, the electronic device 110 may require the user to set one of the multi-person interpretation mode and the self-interpretation mode as the interpretation mode according to the control of the application. For example, the electronic device 110 may request the user to set one of the multi-person interpretation mode and the self-interpretation mode by using the user interface provided by the application, or the first ear set device. And the user may be required to input using a button or sensor included in at least one of the second earset devices. As another example, it is also possible to automatically set one of the multi-person interpretation mode and the self-interpretation mode through an analysis process such as analyzing the input voice of the electronic device 110. After step 890, the electronic device 110 may perform step 810 again.

As described above, in the embodiment of FIG. 8, a specific example of providing the interpreting function according to the multi-person interpretation mode and the self-interpretation mode is described. Such an embodiment of FIG. 8 shows an example of operation by the self-interpretation mode and the multi-person interpretation mode, but depending on the embodiment, only the self-interpretation mode may be provided or the multi-person interpretation may be provided. Only modes may be provided.

Communication between the electronic device 110 and the first earset device and the second earset device may be performed by direct / indirect communication as described with reference to FIGS. 3 and 4. For example, the electronic device 110 directly communicates with any one of the first earset device and the second earset device, and wireless communication between the earset device that directly communicates with the remaining one earset device. Indirect communication may be performed with the remaining one earset device. As another example, the electronic device 110 can also communicate directly with both the first earset device and the second earset device at the same time.

Further, the user (for example, the first user 140 in FIG. 1) listens to music or makes a telephone call while wearing at least one of the first earset device and the second earset device. Interpretation function may be required in some situations.

FIG. 9 is a flowchart showing an example of switching the music listening mode or the telephone call mode to the interpreting mode in one embodiment of the present invention.

At step 910, the electronic device 110 utilizes at least one of the first earset device and the second earset device based on the received application execution signal or the interpreter mode start signal input by the application being executed. You may switch to the music listening mode or the telephone call mode as the interpreter mode. For example, in the music listening mode by Bluetooth connection, the electronic device 110 receives A2DP (Advanced Audio Distribution Profile) for music reproduction while changing the music listening mode to the interpreting mode, and HFP for receiving the input of the user's voice. It may be changed to (Hands Free profile). Further, in the telephone call mode, the electronic device 110 may switch to the interpreter mode after ending the telephone call, but may switch to the interpreter mode after changing the telephone call to the standby state for a while.

Such step 910 may be performed after step 710 or step 720 described with reference to FIG. 7 has been performed. For example, when switching the music listening mode or the telephone call mode to the interpreting mode based on the received application execution signal, the step 910 may be executed after the step 710. As another example, if the music listening mode or the telephone call mode is switched to the interpreting mode based on the interpreting mode start signal input by the application to be executed, the step 910 may be executed after the step 720. Step 730 may be performed after step 910 is performed, and as an example, an interpreter function may be provided as shown in FIG.

At step 920, the electronic device 110 terminates the application termination signal generated by any one earset device of the earset pair including the first earset device and the second earset device or the interpretation mode input by the application. Based on the signal, the interpreter mode may be switched back to music listening mode or telephone call mode. In this case, the user will be able to continue listening to music and making a telephone call again.

FIG. 10 is a flowchart showing an example of providing an interpreter function during a telephone call according to an embodiment of the present invention.

At step 1010, the electronic device 110 utilizes at least one of the first earset device and the second earset device based on the received application execution signal or the interpreter mode start signal input by the application being executed. Then, you may switch the running telephone call mode to the interpreter mode. Similar to step 910 of FIG. 9, step 1010 may be performed after step 710 or step 720 described with reference to FIG. 7 has been performed.

At step 1020, the electronic device 110 receives the voice of the first language input through at least one of the first earset device and the second earset device, and uses the interpreter function to receive the second language voice. It may be converted to the voice of the language. Such step 1020 may be included and performed in step 730 of FIG.

At step 1030, the electronic device 110 switches the interpreter mode back to the telephone call mode based on the application end signal generated by the first earset device or the second earset device or the interpreter mode end signal input by the application. good. The converted second language voice may be first transmitted to at least one of the first earset device and the second earset device and confirmed by the user. Further, the converted voice of the second language may be temporarily stored in the memory 511 or the like.

At step 1040, the electronic device 110 may provide the converted second language voice to the other party of the telephone call through the telephone call mode. For example, the voice of the second language temporarily stored in the memory 511 may be transmitted to the other party of the telephone call through the telephone call mode. As a more specific example, when the first user who uses the first language is in a telephone call with the second user who uses the second language, the first user is translated into the second language by using the interpreter function. The voice can be provided to the second user. At this time, the telephone call may include an image call.

FIG. 11 is a flowchart showing an example of a method of providing an interpreter function of an earset device according to an embodiment of the present invention.

In step 1110, the first earset device 120 converts the voice of the first language input to the first earset device 120 into the voice of the second language by the interpreter function of the application executed by the electronic device 110. 1 The voice of the first language may be transmitted to the electronic device 110 as provided to the second earset device 130 that forms an earset pair with the earset device 120. As described above, the electronic device 110 may convert the received voice of the first language into the voice of the second language by the interpreting function of the executed application and provide it to the second earset device 130. The earset device 130 outputs the converted voice of the second language so that the second user wearing the second earset device 130 can refer to the first language of the first user wearing the first earset device 120. You will be able to receive the provision of an interpreter function.

In step 1120, the first earset device 120 first inputs the second language voice input to the second earset device 130 and received by the electronic device 110 by the interpreter function of the application in which the electronic device 110 is executed. In response to the conversion to the voice of the language, the voice of the first language converted from the electronic device 110 may be received. As described above, the electronic device 110 may convert the voice of the second language received from the second earset device 130 into the voice of the first language and provide it to the first earset device 120.

At step 1130, the first earset device 120 may output the received first language audio. Therefore, the first user of the first earset device 120 will be able to receive the interpretation function for the second language of the second user.

As described above, the first earset device 120 may correspond to any one of the two earset devices of the earset pair. Therefore, in the embodiment of FIG. 11, only the operation of the first ear set device 120 in the multi-person interpretation mode has been described, but based on the above-mentioned contents, the operation of the first ear set device 120 in the self-interpretation mode and the operation of the first ear set device 120 are described. 2 Those skilled in the art will be able to easily understand the operation of the ear set device 130.

Further, in the above-described embodiment, only the embodiment in which two earset devices form an earset pair has been described, but three or more earset devices are directly or indirectly connected to the electronic device 110. Those skilled in the art will be able to easily understand that the multi-person interpretation mode is feasible by referring to the above-mentioned contents. For example, it is possible that the voice of the first language of the earset device 1 is translated into the voices of the second language and the voice of the third language, respectively, and transmitted to the earset device 2 and the earset device 3, respectively. Further, the voice of the second language of the earset device 2 is translated into the voice of the first language and the voice of the third language, respectively, and transmitted to the earset device 1 and the earset device 3, respectively, or the third language of the earset device 3 is used. It is also possible that the voice of the above is translated into the voice of the first language and the voice of the second language, respectively, and transmitted to the ear set device 1 and the ear set device 2, respectively. As an example of communication, the electronic device 110 may directly communicate with the ear set device 1 and the ear set device 2, and may indirectly communicate with the ear set device 3 via the ear set device 1 and the ear set device 2.

In the above-described embodiment, the voice of the first language is converted into the voice of the second language, but the present invention is not limited to this example. For example, the voice of the first language may be converted into the characters of the second language and output, or conversely, the characters of the first language may be converted into the voice of the second language and output. Both text and voice may be output.

For example, in providing the user with an interpreter function using the first earset and the second earset, the voice of the first language input through the first earset is automatically the second. The second language voice that is converted to the characters of the language and displayed through the display of the electronic device and input through the second earset is automatically converted to the characters of the first language and electronically. It can be displayed via the display of the device. At this time, the first user of the first earset and the second user of the second earset are looking at the characters of the second language and the characters of the first language displayed through the electronic device together. You can get an interpreter function.

As described above, according to the present embodiment, the interpreter function can be provided by using the ear set, and different languages are used by using the ear set pair including the first ear set device and the second ear set device. It is possible to provide an interpreter function between two users. Further, the interpreting function can provide the voice translated from the first language to the second language to the other party of the telephone call.

note that,

The devices described above may be implemented by hardware components, software components, and / or combinations of hardware components and software components. For example, the apparatus and components described in the embodiments include, for example, a processor, a controller, an ALU (arithmetic logic unit), a digital signal processor, a microprocessor, an FPGA (field programgable gate array), a PLU (programmable log unit), and a micro. It may be implemented utilizing one or more general purpose computers or special purpose computers, such as a processor or various devices capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on said OS. The processing device may also respond to software execution, access data, and store, manipulate, process, and generate data. For convenience of understanding, one processing device may be described as being used, but one of ordinary skill in the art may include a plurality of processing elements and / or a plurality of types of processing elements. You can understand. For example, the processing device may include multiple processors or one processor and one controller. Also, other processing configurations such as parallel processors are possible.

The software may include computer programs, codes, instructions, or a combination of one or more of these, configuring the processing equipment to operate at will, or instructing the processing equipment independently or collectively. You may do it. The software and / or data is embodied in any kind of machine, component, physical device, computer storage medium or device to be interpreted based on the processing device or to provide instructions or data to the processing device. good. The software is distributed on a computer system connected to a network and may be stored or executed in a distributed state. The software and data may be stored on a recording medium readable by one or more computers.

The method according to the embodiment may be realized in the form of program instructions that can be executed by various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be those specially designed and configured for embodiments, or may be usable, known to those skilled in the art of computer software. Examples of computer-readable recording media include hard disks, floppy (registered trademark) disks, magnetic media such as magnetic tapes, optical media such as CD-ROMs and DVDs, and floptic discs. Optical magnetic media such as, and hardware devices specially configured to store and execute program instructions such as ROM, RAM, flash memory, etc. are included. Examples of program instructions include machine language code, such as those generated by a compiler, as well as high-level language code, which is executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

As described above, the embodiments have been described based on the limited embodiments and drawings, but those skilled in the art will be able to make various modifications and modifications from the above description. For example, the techniques described may be performed in a different order than the methods described, and / or components such as systems, structures, devices, circuits, etc. described may be in a different form than the methods described. Appropriate results can be achieved even if they are combined or combined, and confronted or replaced by other components or equivalents.

Therefore, even if the embodiments are different, they belong to the attached claims as long as they are equivalent to the claims.

110: Electronic device 120: First ear set device 130: Second ear set device

Claims (15)

It is a method of providing an interpreter function executed by an electronic device.
The stage of receiving the application execution signal generated by the first earset device or the second earset device that forms an earset pair with the first earset device.
The stage of executing the application corresponding to the received application execution signal,
At the stage of providing an interpreter function for voice received through one of the first earset device and the second earset device according to the control of the application to be executed, and interpreting to the other language. The result is transmitted to the other of the first earset device and the second earset device, and the first earset device and the second earset device are worn on the left and right ears. configure the ear set pairs,
The stage of providing the interpreter function is
The voice of the first language input through the first earset device and received by the electronic device is converted into a text of the second language and displayed on the display of the electronic device, and is input through the second earset device and the electronic device is used. A method for providing an interpreter function, which comprises converting a second language voice received by a device into a text in the first language and displaying it on the display of the electronic device. The stage of providing the interpreter function is
The voice of the first language input through the first earset device and received by the electronic device is converted into the voice of the second language and provided to the second earset device, and the voice is input through the second earset device and described. The method for providing an interpreter function according to claim 1, wherein the voice of the second language received by the electronic device is converted into the voice of the first language and provided to the first earset device. The electronic device directly communicates with any one of the first earset device and the second earset device, and the wireless communication between the directly communicating earset device and the remaining one earset device. The method for providing an interpreter function according to claim 1, wherein indirect communication is performed with the remaining one earset device by communication. The method for providing an interpreter function according to claim 1, wherein the electronic device directly communicates with both the first earset device and the second earset device at the same time. A music listening mode or telephone using at least one of the first earset device and the second earset device based on the received application execution signal or the interpreter mode start signal input by the application to be executed. The stage of switching the call mode to the interpreter mode, and the application end signal generated by any one of the earset devices including the first earset device and the second earset device, or the interpreter input by the application. The method for providing an interpreter function according to claim 1, further comprising a step of switching the interpreter mode to the music listening mode or the telephone call mode based on the mode end signal. A telephone running using at least one of the first earset device and the second earset device based on the received application execution signal or the translation mode start signal input by the application to be executed. The stage of switching the call mode to the interpreter mode,
The voice of the first language input through at least one of the first earset device and the second earset device and received by the electronic device is converted into the voice of the second language by using the interpreter function. step,
The step of switching the interpretation mode to the telephone call mode based on the application end signal generated by the first earset device or the second earset device or the translation mode end signal input by the application, and the conversion. The method for providing an interpreter function according to claim 1, further comprising a step of providing a second language voice to a telephone call partner through the telephone call mode. The application execution signal is input by a physical button included in at least one of the first earset device and the second earset device, or the first earset device and the second earset device. The method for providing an interpreter function according to claim 1, which is generated by an output value of a sensor included in at least one of them. The stage of providing the interpreter function is
A step of sensing a first language of voice input through the first earset device and received by the electronic device and a second language of voice input through the second earset device and received by the electronic device.
The step of converting the sensed first language voice into the sensed second language voice and the sensed second language voice into the sensed first language voice, and the converted second language. The method for providing an interpreter function according to claim 1, wherein the voice of the language is provided to the second earset device, and the converted voice of the first language is provided to the first earset device. The stage of providing the interpreter function is
The voice input through the first earset device and received by the electronic device is converted into a second language set for the second earset device by the application, and is input through the second earset device. The stage of converting the voice received by the electronic device into the first language set for the first earset device in the application, and providing the converted voice of the second language to the second earset device. The method for providing an interpreter function according to claim 1, further comprising a step of providing the converted voice of the first language to the first earset device. A interpreters function provides a method,
The voice of the first language input to the first earset device is converted into the voice of the second language by the interpreter function of the application executed by the electronic device, and the second ear that forms an earset pair with the first earset device. The stage of transmitting the voice of the first language to the electronic device as provided to the set device,
In response to the electronic device converting the second language voice input to the second earset device and received by the electronic device into the first language voice by the interpreter function of the application to be executed. The stage in which the first earset device receives the converted first language voice from the electronic device, and
The first earset device and the second earset device constitute an earset pair to be worn on the left and right ears, including a step in which the first earset device outputs the received voice of the first language .
When the electronic device executes the interpreting function of the application, the electronic device is used.
The voice of the first language input through the first earset device and received by the electronic device is converted into a text of the second language and displayed on the display of the electronic device, and is input through the second earset device and the electronic device is used. A method for providing an interpreter function, which converts a second language voice received by a device into a text in the first language and displays it on the display of the electronic device. The method for providing an interpreter function according to claim 10, wherein the communication between the second earset device and the electronic device is relayed, or the electronic device is communicated by the relay of the second earset device. A physical button input included in at least one of the first earset device and the second earset device, or a sensor included in at least one of the first earset device and the second earset device. The method for providing an interpreter function according to claim 10, wherein the application is executed by the electronic device by an application execution signal generated by an output value and transmitted to the electronic device. The first language includes a language sensed by using the executed application for the language of the voice input to the first earset device, or the first earset device in the executed application. Including corresponding preset languages
The second language includes a language sensed by using the executed application for the language of the voice input to the second earset device, or the second earset device in the executed application. The method for providing an interpreter function according to claim 10, which includes a corresponding and preset language. A computer program stored in a computer-readable recording medium in combination with a computer to cause the computer to perform the method according to any one of claims 1 to 13. A computer-readable recording medium in which a program for causing a computer to execute the method according to any one of claims 1 to 13 is recorded.

Images