JP7437356B2 - Method of providing interpretation function of electronic device and ear-set device - Google Patents

Description

The following description describes a method and apparatus for providing an interpreting function using an ear set, and a device that is stored in a computer-readable recording medium in order to cause the computer to execute the method for providing an interpreting function according to an embodiment of the present invention in combination with a computer. related to computer programs and their recording media.

An earset is a device equipped with earphones and a microphone that can be used to talk without using your hands by connecting it to a phone.For example, by connecting it to a smartphone, you can make phone calls as well as It can receive and output audio such as music, and provide a variety of control signals to the smartphone to control it through physical buttons, sensors, or voice. For example, Patent Document 1 discloses a Bluetooth (registered trademark) ear set with a built-in ear kernel microphone.

Korean Publication Patent No. 10-2017-0030375

A method and apparatus for providing an interpretation function that can provide an interpretation function using an ear set, and a computer-readable record that can be combined with a computer to cause the computer to execute the method for providing an interpretation function according to an embodiment of the present invention. Provides a computer program stored on a medium and its recording medium.

The audio in the first language input through the first ear-set device and received by the electronic device is converted into the audio in the second language and provided to the second ear-set device, and the audio input through the second ear-set device is received by the electronic device. By converting the voice in the second language into the voice in the first language and providing it to the first earset device, the earset pair including the first earset device and the second earset device is used. An interpreter function providing method and apparatus capable of providing an interpreter function between two users using different languages, and a device for causing a computer to execute an interpreter function providing method according to an embodiment of the present invention in combination with a computer. To provide a computer program stored in a computer-readable recording medium and the recording medium.

A method for providing an interpretation function of an electronic device, the method comprising: receiving an application execution signal generated in a first earset device or a second earset device; executing an application corresponding to the received application execution signal; and the steps of: providing an interpretation function into another language for the audio input through at least one of the first earset device and the second earset device and received by the electronic device according to the control of the executed application; Provides a method for providing interpretation functions, including

A method for providing an interpretation function of a first ear-set device, the voice of a first language input to the first ear-set device being converted into a voice of a second language by an interpretation function of an application executed on an electronic device; transmitting audio in the first language to the electronic device so as to be provided to a second earset device forming an earset pair with the first earset device; In response to the electronic device converting a second language voice received by the electronic device into a first language voice using the interpretation function of the executed application, A method for providing an interpretation function is provided, which includes the steps of receiving audio and outputting the received audio in a first language.

A computer-readable recording medium is provided, on which a computer program for causing a computer to execute the interpretation function providing method is recorded.

A computer program is provided that is stored in a computer-readable recording medium and is coupled to a computer to cause the computer to execute the method for providing an interpretation function.

A first ear set device, comprising a sound signal input section for inputting sound, a sound signal output section for outputting sound, and a second ear set device forming an ear set pair with the electronic device or the first ear set device. A communication unit for communicating with set equipment, and a control unit for controlling the sound signal input unit, the sound signal output unit, and the communication unit, and the control unit includes a communication unit for controlling the sound signal input unit, the sound signal output unit, and the communication unit. Audio in one language is converted into audio in a second language according to an interpretation function of an application executed on the electronic device and provided to a second earset device forming an earset pair with the first earset device. The communication unit is controlled to transmit the audio in the first language to the electronic device, and the audio in the second language input to the second ear set device and received by the electronic device is transmitted to the electronic device. controls the communication unit to receive the converted first language audio from the electronic device in response to converting the audio into a first language audio according to the interpretation function of the executed application; The present invention provides a first earset device that controls the sound signal output unit to output sound in a first language.

The earset may be used to provide an interpretation function, and an earset pair including a first earset device and a second earset device may be used to provide an interpretation function between two users of different languages. be able to.

By using the interpretation function, it is possible to provide the voice interpreted from the first language to the second language to the other party of the telephone conversation.

FIG. 3 is a diagram illustrating an example of a process of providing an interpretation function using an ear set in an embodiment of the present invention. FIG. 3 is a diagram illustrating an example of an ear set pair in an embodiment of the present invention. FIG. 3 is a diagram illustrating an example of communication between an ear set pair and an electronic device in an embodiment of the present invention. FIG. 7 is a diagram illustrating another example of communication between an ear set pair and an electronic device in an embodiment of the present invention. FIG. 2 is a block diagram for explaining the internal configurations of an electronic device, an ear-set device, and a server in an embodiment of the present invention. 1 is a diagram illustrating an example of a cloud artificial intelligence platform in an embodiment of the present invention. FIG. 1 is a flowchart illustrating an example of a method for providing an interpretation function according to an embodiment of the present invention. 2 is a flowchart illustrating an example of providing an interpretation function for self-interpretation and multi-party interpretation in an embodiment of the present invention. 2 is a flowchart illustrating an example of switching from music listening mode or telephone conversation mode to interpretation mode in an embodiment of the present invention. 2 is a flowchart illustrating an example of providing an interpretation function during a telephone call, according to an embodiment of the present invention. 2 is a flowchart illustrating an example of a method for providing an interpretation function of an earset device according to an embodiment of the present invention.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

The interpretation function providing method according to the embodiment of the present invention may be executed by an electronic device or an ear-set device described below. For example, a computer program according to an embodiment of the present invention may be installed and driven in an electronic device or an ear-set device, and an electronic device or an ear-set device may implement an embodiment of the present invention under the control of the computer program to be driven. The interpretation function providing method according to the format may be executed. The above-mentioned computer program may be stored in a computer-readable recording medium in order to cause the computer to execute the interpreting function providing method by being combined with an electronic device or an ear-set device implemented by the computer. The computer program described herein may be in the form of an independent program package, or the form of an independent program package may be in the form of an operating system or other program package that is preinstalled on the computer and runs. It may also be in a linked form.

FIG. 1 is a diagram illustrating an example of the process of providing an interpretation function using an ear set according to an embodiment of the present invention. FIG. 1 shows an electronic device 110, a first ear-set device 120, and a second ear-set device 130, in which a first user 140 puts the first ear-set device 120 in his right ear, and a second user 150 puts the first ear-set device 120 in his right ear. An example is shown in which the ear set device 130 is worn on each left ear. Here, it is assumed that the first user 140 uses the first language and the second user 150 uses the second language.

At this time, the sound of the first language uttered by the first user 140 may be input to a sound signal input unit such as a microphone included in the first earset device 120, and the first earset device 120 The audio in the first language may be transmitted to the electronic device 110. In this case, the electronic device 110 uses the interpretation function of the application installed on the electronic device 110 to convert the received first language voice into a second language voice and provide the second language voice to the second ear set device 130. good. At this time, the second ear set device 130 may output the provided sound in the second language through a sound signal output unit such as a speaker (earphone) included in the second ear set device 130. Therefore, the second user 150 can listen to the voice in the first language uttered by the first user 140 translated into the second language.

Further, the second language voice uttered by the second user 150 may be input to a sound signal input unit such as a microphone included in the second ear set device 130, and the second ear set device 130 may input the second language voice uttered by the second user 150. Audio in two languages may be transmitted to electronic device 110. In this case, the electronic device 110 uses the interpretation function of the application installed on the electronic device 110 to convert the received second language voice into the first language voice and provide it to the first ear set device 120. good. At this time, the first ear set device 120 may output the provided sound in the first language through a sound signal output unit such as a speaker (earphone) included in the first ear set device 120 . Therefore, the first user 140 can listen to the second language voice uttered by the second user 150 as a voice translated into the first language.

As described above, according to the embodiment of FIG. 1, the first earset device 120 and the second earset device 130 forming one earset pair can be used to provide a multi-party interpretation function.

In an embodiment different from the embodiment of FIG. 1, a case may be considered in which the first user 140 is wearing the first earset device 120 and the second earset device 130 on both ears, respectively. In this case, the audio of the first language uttered by the first user 140 may be input through at least one of the first ear-set device 120 and the second ear-set device 130 and transmitted to the electronic device 110 . At this time, the electronic device 110 uses the interpretation function of the application installed on the electronic device 110 to convert the received voice in the first language into the voice in the second language, and connects the first ear set device 120 and the second ear set device 120 to the second language voice. The information may be transmitted to at least one of the set devices 130. At this time, the first earset device 120 and/or the second earset device 130 may output the audio converted into the second language, and the first user 140 may output the audio that he or she uttered in the first language. You will be able to listen to the translated second language audio. Thus, unlike the embodiment of FIG. 1, other embodiments of the invention may also provide functionality for self-interpretation.

The function for multi-party interpretation and the function for self-interpretation may be provided by changing the mode in an application installed on the electronic device 110, respectively. For example, the application may control the electronic device 110 to provide a user interface that can configure one of a multiparty interpretation mode and a self-interpretation mode, and utilize the provided user interface. , the user may directly set one of the multi-party interpretation mode and the self-interpretation mode. Alternatively, setting such a mode may be performed by inputting to a physical button included in at least one of the first earset device 120 and the second earset device 130. As another example, the mode may be set by analyzing audio input to the first ear-set device 120 and the second ear-set device 130. For example, if the pitch and volume of the voices input to the first ear-set device 120 and the second ear-set device 130 are similar, the self-interpretation mode may be set, and the first ear-set device 120 and the second ear-set device If the pitches, loudness, etc. of voices input to the two ear-set devices 130 are completely different, a multi-party interpretation mode may be set.

FIG. 2 is a diagram illustrating an example of an ear set pair in an embodiment of the present invention. The earset pair according to this embodiment includes a first earset device 120 and a second earset device 130, as shown in FIG. Although the embodiments of FIGS. 1 and 2 show the first earset device 120 being worn on the right ear and the second earset device 130 being worn on the left ear, this is for purposes of illustrating the invention. This is just one embodiment and is not limited thereto. For example, the first earset device 120 may be implemented to be worn on the left ear, and the second earset device 130 may be implemented to be worn on the right ear.

Further, the embodiment of FIG. 2 shows an example of a cradle 210 that provides functions such as storing, placing, and charging the first earset device 120 and the second earset device 130. Such a cradle 210 is also just one example for explaining an embodiment in which the first earset device 120 and the second earset device 130 form one earset pair, and its shape and function are similar to those shown in FIG. is not limited by this description.

FIG. 3 is a diagram illustrating an example of communication between an ear set pair and an electronic device in an embodiment of the present invention. Typically, smartphones communicate with peripheral devices, such as earsets and wireless speakers, via near field communication protocols such as Bluetooth, Zigbee, Wi-Fi, and the like. In the embodiment of FIG. 3 as well, an example will be described in which the electronic device 110 communicates with the ear set pair using such a short-range wireless communication protocol. However, the electronic device 110 is configured to operate one device at a time (in the embodiment of FIG. 3, the first earset device 120 ) may need to be communicated with. In other words, a case is considered in which the electronic device 110 cannot communicate with two or more devices simultaneously via one short-range wireless communication protocol. In this embodiment, the electronic device 110 may communicate with the first earset device 120 as one device using one short-range wireless communication protocol, and the second earset device 130 may communicate with the first earset device 120 as one device using one short-range wireless communication protocol. Communication may be performed by relaying the device 120. To this end, the first earset device 120 and the second earset device 130 may perform short-range wireless communication with each other. For example, the second earset device 130 may transmit the input audio in the second language to the electronic device 110 via the first earset device 120. Conversely, the audio translated into the first language and transmitted by the electronic device 110 may be transmitted to the second earset device 130 via the first earset device 120. As an example of such relay communication, a TWS (True Wireless Stereo) method may be used.

FIG. 4 is a diagram showing another example of communication between an ear set pair and an electronic device in an embodiment of the present invention. Unlike the embodiment of FIG. 3, electronic device 110 may simultaneously communicate with both first earset device 120 and second earset device 130 utilizing one near field communication protocol. For example, near field communication protocols such as Bluetooth® 5.0 provide a technology that allows a smartphone to connect and communicate with two devices at the same time. The embodiment of FIG. 4 utilizes a short-range wireless communication protocol that allows communication with multiple devices at the same time, so that it can simultaneously communicate with both the first earset device 120 and the second earset device 130. is also possible.

On the other hand, when the electronic device 110 provides an interpretation function using an application, it is possible to provide the interpretation function that the application has independently, but it is also possible to receive the interpretation function from a server linked with the application. is also possible. For example, the electronic device 110 may connect to a server under the control of the application and transmit information regarding the first language audio and the second language in which the first language audio is to be interpreted. In this case, the server may convert the first language audio into the second language audio and provide it to the electronic device 110, and the electronic device 110 converts the first language audio into the second language audio and provides the converted second language audio to the electronic device 110. You will be able to get audio.

FIG. 5 is a block diagram for explaining the internal configurations of an electronic device, an ear-set device, and a server in an embodiment of the present invention. The server 520 in FIG. 5 may be an external system that provides an interpretation function in conjunction with an application installed and run on the electronic device 110.

Electronic device 110 and server 520 may include memory 511 , 521 , processor 512 , 522 , communication interface 513 , 523 , and input/output interface 514 , 524 . The memories 511 and 521 are computer-readable recording media, including random access memory (RAM), read only memory (ROM), and permanent mass storage devices such as disk drives. That's fine. Here, a permanent mass storage device such as a ROM or a disk drive may be included in the electronic device 110 or the server 520 as a separate permanent storage device that is separate from the memories 511 and 521. The memories 511 and 521 also include an operating system and at least one program code (for example, a code for an application installed in the electrical device 110 and run on the electronic device 110 to provide a specific service). may be stored. Such software components may be loaded from a computer readable storage medium separate from the memory 511, 521. Such other computer readable recording media may include computer readable recording media such as floppy drives, disks, tapes, DVD/CD-ROM drives, memory cards, and the like. In other embodiments, the software components may be loaded into the memory 511, 521 through a communication interface 513, 523 that is not a computer readable storage medium. For example, at least one program may be installed on electronic device 110 based on a computer program (such as the application described above, by way of example) that is installed by a file provided over network 530 by a developer or a file distribution system that distributes application installation files. may be loaded into the memory 511 of.

Processors 512, 522 may be configured to process instructions of computer programs by performing basic arithmetic, logic, and input/output operations. Instructions may be provided to processors 512, 522 by memory 511, 521 or communication interfaces 513, 523. For example, processors 512, 522 may be configured to execute instructions received according to program code stored in a storage device, such as memory 511, 521.

The communication interfaces 513, 523 may provide a function for the electronic device 110 and the server 520 to communicate with each other via the network 530, or may provide a function for the electronic device 110 and/or the server 520 to communicate with other electronic devices or other devices. It may also provide functionality for communicating with the server. As an example, a request generated by processor 512 of electronic device 110 according to a program code stored in a storage device such as memory 511 may be communicated to server 520 via network 530 under the control of communication interface 513. On the contrary, control signals, instructions, contents, files, etc. provided under the control of the processor 522 of the server 520 are received by the electronic device 110 through the communication interface 513 of the electronic device 110 via the communication interface 523 and the network 530. may be done. For example, control signals, instructions, content, files, etc. of the server 520 received through the communication interface 513 may be transmitted to the processor 512 and the memory 511, and the content, files, etc. may be transferred to a storage medium that the electronic device 110 may further include. (permanent storage described above).

Input/output interface 514 may be a means for interfacing with input/output device 515. For example, input devices may include devices such as a microphone, keyboard, or mouse, and output devices may include devices such as a display and speakers. As another example, input/output interface 514 may be a means for interfacing with a device that has integrated input and output functionality, such as a touch screen. Input/output device 515 may be comprised of one device with electronic device 110. The input/output interface 524 of the server 520 may also be a means for interfacing with devices for input or output (not shown) that may be coupled to or included in the server 520.

Also, in other embodiments, electronic device 110 and server 520 may include fewer or more components than those of FIG. However, most prior art components need not be clearly illustrated. For example, the electronic device 110 may be implemented to include at least some of the input/output devices 515 described above, or may include a transceiver, a GPS (Global Positioning System) module, a camera, various sensors, a database, etc. It may further include other components. As a more specific example, when the electronic device 110 is a smartphone, the smartphone generally includes an acceleration sensor, a gyro sensor, a camera module, various physical buttons, buttons using a touch panel, input/output ports, The electronic device 110 may further include various components such as a vibrator for vibration.

The first earset device 120 includes a sound signal input section 541 for receiving sound input, a sound signal output section 542 for outputting sound, and a first earset device 120 that forms an earset pair with the electronic device 110 or the first earset device 120. It may include a communication section 543 for communicating with the two-earset device 130, a sound signal input section 541, a sound signal output section 542 for outputting sound, and a control section 544 for controlling the communication section 543. For example, the control unit 544 may control the sound signal input unit 541 to receive input of audio in the first language uttered by a user (for example, the first user 140 in FIG. 1). Further, the control unit 544 converts the input audio in the first language into audio in the second language according to the interpretation function of the application executed on the electronic device 110 and provides the audio to the second ear set device 130. , the communication unit 543 may be controlled to transmit the audio in the first language to the electronic device 110. Further, the control unit 544 converts the second language sound input to the second ear set device 120 and received by the electronic device 110 into the first language sound according to the interpretation function of the application executed on the electronic device 110. In response, the communication unit 543 may be controlled to receive the converted first language audio from the electronic device 110. Furthermore, the control unit 544 may control the sound signal output unit 542 to output the sound in the first language received from the electronic device 110.

FIG. 6 is a diagram illustrating an example of a cloud artificial intelligence platform in one embodiment of the present invention. FIG. 6 shows an electronic device 610, a cloud artificial intelligence platform 620, and a content service 630.

As an example, the electronic devices 610 may include, for example, in-home devices connected via a smart home network, and each or at least one of them may correspond to the electronic device 110 described above. Such an electronic device 610 or an application installed and run on the electronic device 610 (hereinafter referred to as an application) may be linked with the cloud artificial intelligence platform 620 via an interface connect 640. Here, the interface connect 640 may provide the developer with an SDK (Software Development Kit) and/or development documents for developing the electronic device 610 or an application installed and run on the electronic device 610. Further, the interface connect 640 may provide an API (Application Program Interface) that allows the electronic device 610 or an application installed and run on the electronic device 610 to utilize functions provided by the cloud artificial intelligence platform 620. As a specific example, a developer can use the API provided by Interface Connect 640 to develop devices and applications using the SDK (Software Development Kit) and/or development documents provided by Interface Connect 640 in the cloud. It becomes possible to utilize functions provided by the artificial intelligence platform 620 (for example, an interpretation function).

Here, the cloud artificial intelligence platform 620 may provide functions for providing various voice-based services as well as an interpretation function. For example, the cloud artificial intelligence platform 620 includes an audio processing module 621 for recognizing received audio and synthesizing output audio, a vision processing module 622 for analyzing and processing received video and video, and a vision processing module 622 for analyzing and processing received audio. a conversation processing module 623 for determining an appropriate conversation for outputting a suitable voice (for example, a voice interpreted into a second language) according to the received voice; a recommendation module for recommending a function suitable for the received voice; 624, Neural Machine Translation (NMT), which supports artificial intelligence to translate sentences on a sentence-by-sentence basis based on data learning,625 includes various modules for providing voice-based services. That's fine.

For example, the electronic device 110 may send information regarding the first language audio and the interpreted second language to the cloud artificial intelligence platform 620 using an API provided by the interface connect 640. In this case, the cloud artificial intelligence platform 620 may recognize and analyze the received first language speech by utilizing the modules 621 to 625 described above, and may interpret the first language speech into a second language speech. , an appropriate response voice may be synthesized and provided.

The expansion kit 650 may also provide a development kit that allows a third party content developer or company to implement new voice infrastructure functionality based on the cloud artificial intelligence platform 620. For example, another application and server that can utilize the interpretation function of the cloud artificial intelligence platform 620 may be developed using a provided development kit.

Such a cloud artificial intelligence platform 620 is only a specific example of the server 520 for providing an interpreting function to the electronic device 110, and any server 520 that can provide an interpreting function to the electronic device 11 can be used. It is not limited.

FIG. 7 is a flowchart illustrating an example of a method for providing an interpretation function according to an embodiment of the present invention. The interpretation function providing method according to this embodiment may be executed by a computer device such as the electronic device 110 described above. At this time, the processor 512 of the electronic device 110 may be implemented to execute control instructions based on operating system code or at least one program code included in the memory 511. Here, the processor 512 controls the electronic device 110 so that the electronic device 110 performs steps 710 to 730 included in the method for providing an interpretation function of FIG. 7 according to control instructions provided by a code stored in the electronic device 110. You may do so.

At step 710, the electronic device 110 may receive an application execution signal generated by the first earset device or the second earset device. Here, the first earset device and the second earset device may correspond to the first earset device 120 and the second earset device 130 described above. For example, the first earset device and the second earset device may be configured as an earset pair for a left ear and a right ear. The application execution signal is an input from a physical button included in at least one of the first earset device and the second earset device, or an input included in at least one of the first earset device and the second earset device. It may be generated by the output value of the sensor. For example, the first earset device (or the second earset device) senses an input signal generated by the user pressing a button included in the first earset device (or the second earset device), and An application execution signal may be generated in response to the input signal. The first earset device (or the second earset device) may then transmit the generated application execution signal to the electronic device 110. As another example, the first earset device (or the second earset device) may include an illuminance sensor. At this time, the first earset device (or the second earset device) detects the action of the user holding his or her finger over the measurement area of the illuminance sensor for a certain period of time, and generates an application execution signal by sensing such action. You may. As such, those skilled in the art will readily understand that one of a variety of methods can be utilized to sense a specific action of a user and generate an application execution signal. Dew.

In step 720, the electronic device 110 may execute an application corresponding to the received application execution signal. At this time, the application to be executed may be a computer program for providing an interpretation function independently or in cooperation with the server 520 described above.

At step 730, the electronic device 110 converts the audio input through at least one of the first ear-set device and the second ear-set device and received by the electronic device 110 into another language under the control of the executed application. May provide interpretation functionality. At this time, the interpretation function may include a function for self-interpretation and a function for multi-party interpretation, as described above.

In this way, the user (or users) can use the ear-set device to execute an application for the interpretation function of the electronic device 110, and use the ear-set device to execute other applications for the interpretation function of the electronic device 110. You will be able to receive language interpretation functions.

FIG. 8 is a flowchart illustrating an example of providing an interpretation function for self-interpretation and multi-party interpretation in one embodiment of the present invention. Steps 810 to 890 in FIG. 8 illustrate an example of a specific process for providing the interpretation function in step 730 in FIG.

At step 810, electronic device 110 may determine whether interpretation mode is in progress. For example, if the interpretation mode is not running due to termination of the interpretation mode or termination of the application, the process of FIG. 8 may end. The electronic device 110 may perform step 820 if the interpretation mode is being performed in step 810.

In step 820, the electronic device 110 may determine whether the interpretation mode is a multiparty interpretation mode. Step 830 may be performed if the interpretation mode is a multiparty interpretation mode. If the interpretation mode is not a multi-party interpretation mode, the electronic device 110 may perform step 870.

In step 830, the electronic device 110 may determine whether audio is received from the first earset device. Here, the first earset device may be any one of two earset devices included in the earset pair. At this time, if the electronic device 110 receives audio from the first earset device, the electronic device 110 may perform step 840. If no audio is received from the first earset device, the electronic device 110 may perform step 850.

In step 840, the electronic device 110 may convert the first language audio input through the first ear-set device and received by the electronic device 110 into the second language audio, and provide the second language audio to the second ear-set device. Here, the second earset device may be another earset device other than the earset device that provided the audio in step 830. The second ear-set device outputs the converted audio in the second language, so that the first user wearing the first ear-set device can communicate in the first language to the second user wearing the second ear-set device. The spoken voice may be interpreted into second language voice and provided.

At step 850, the electronic device 110 may determine whether audio is received from the second earset device. At this time, if the electronic device 110 receives audio from the second earset device, the electronic device 110 may perform step 860. If no audio is received from the second earset device, electronic device 110 may perform step 810 again.

In step 860, the electronic device 110 may receive the second language audio input through the second ear-set device, convert it into the first language audio, and provide the first language audio to the first ear-set device. The first ear set device outputs the converted audio in the first language, so that the first user wearing the first ear set device can hear the second user wearing the second ear set device speaking the second language. The uttered audio may be interpreted into audio in the first language and provided. After this, electronic device 110 may perform step 810 again.

At step 870, the electronic device 110 may determine whether the interpretation mode is a self-interpretation mode. Step 880 may be performed if the interpretation mode is a self-interpretation mode. If the interpretation mode is not the self-interpretation mode, the electronic device 110 may perform step 890.

At step 880, the electronic device 110 receives and converts the first language audio input through at least one of the first earset device and the second earset device into the second language audio, and converts the audio into the second language audio. At least one of an earset device and a second earset device may be provided. In the self-interpretation mode, at least one of the first earset device and the second earset device is determined to be one user wearing both the first earset device and the second earset device. and provide interpreted second language audio to at least one of the first earset device and the second earset device. After step 880, electronic device 110 may perform step 810 again.

At step 890, electronic device 110 may set a mode. For example, the electronic device 110 may request the user to set the interpretation mode to one of a multiparty interpretation mode and a self-interpretation mode under the control of the application. For example, the electronic device 110 may request the user to configure one of a multi-party interpretation mode and a self-interpretation mode using a user interface provided by the application, and the electronic device 110 may request the user to configure one of a multi-party interpretation mode and a self-interpretation mode, and And the user may be requested to input using a button or sensor included in at least one of the second earset devices. As another example, the electronic device 110 may automatically set one of the multi-party interpretation mode and the self-interpretation mode through an analysis process such as analyzing input audio. After step 890, electronic device 110 may perform step 810 again.

In this way, the embodiment of FIG. 8 describes a specific example of providing an interpretation function according to the multi-party interpretation mode and the self-interpretation mode. Although the embodiment of FIG. 8 shows an example of operation using the self-interpretation mode and the multi-party interpretation mode, depending on the embodiment, only the self-interpretation mode may be provided, or the multi-party interpretation mode may be provided. Only modes may be provided.

Communication between the electronic device 110 and the first earset device and the second earset device may be performed by direct/indirect communication as described with reference to FIGS. 3 and 4. For example, the electronic device 110 directly communicates with any one of the first earset device and the second earset device, and wireless communication between the directly communicating earset device and the remaining one earset device. may indirectly communicate with the remaining earset device. As another example, the electronic device 110 may be in direct communication with both the first earset device and the second earset device at the same time.

Further, when a user (for example, the first user 140 in FIG. 1) listens to music or makes a phone call while wearing at least one of the first ear-set device and the second ear-set device, In some situations, an interpreter function may be required.

FIG. 9 is a flowchart illustrating an example of switching from music listening mode or telephone conversation mode to interpretation mode in an embodiment of the present invention.

At step 910, the electronic device 110 utilizes at least one of the first earset device and the second earset device based on the received application execution signal or the interpretation mode start signal input by the executed application. You can switch from music listening mode or phone call mode to interpretation mode. For example, in a music listening mode via Bluetooth (registered trademark) connection, the electronic device 110 changes the music listening mode to an interpretation mode and changes the A2DP (Advanced Audio Distribution Profile) for music playback to the user's voice input. You may change to HFP (Hands Free Profile) in order to receive it. Further, in the telephone call mode, the electronic device 110 may switch to the interpretation mode after ending the telephone call, or may change the telephone call to a standby state for a while and then switch to the interpretation mode.

Such step 910 may be performed after step 710 or step 720 described with reference to FIG. 7 is performed. For example, step 910 may be performed after step 710 when switching from music listening mode or telephone call mode to interpretation mode based on the received application execution signal. As another example, step 910 may be performed after step 720 when switching from music listening mode or telephone call mode to interpreting mode based on an interpreting mode start signal input by an executed application. After step 910 is performed, step 730 may be performed to provide an interpretation function as shown in FIG. 8, as an example.

At step 920, the electronic device 110 receives an application termination signal generated by any one earset device of the earset pair including the first earset device and the second earset device or an interpretation mode termination signal input by the application. Based on the signal, the interpretation mode may be switched back to music listening mode or telephone conversation mode. In this case, the user can continue listening to music or talking on the phone again.

FIG. 10 is a flowchart illustrating an example of providing an interpretation function during a telephone call in one embodiment of the present invention.

At step 1010, the electronic device 110 utilizes at least one of the first earset device and the second earset device based on the received application execution signal or the interpretation mode start signal input in the executed application. You can switch the currently running telephone conversation mode to interpreter mode. Similar to step 910 of FIG. 9, step 1010 may be performed after step 710 or step 720 described with reference to FIG. 7 is performed.

At step 1020, the electronic device 110 receives the audio in the first language input through at least one of the first earset device and the second earset device and converts it into a second language using the interpretation function. May be converted to audio of the language. Such step 1020 may be included and performed in step 730 of FIG.

In step 1030, the electronic device 110 switches the interpreting mode back to the telephone call mode based on the application end signal generated by the first earset device or the second earset device or the interpreter mode end signal inputted by the application. good. The converted second language audio may be first transmitted to at least one of the first earset device and the second earset device and confirmed by the user. Further, the converted second language speech may be temporarily stored in the memory 511 or the like.

In step 1040, the electronic device 110 may provide the converted second language audio to the other party in the telephone call through a telephone call mode. For example, the second language audio temporarily stored in the memory 511 may be transmitted to the other party of the phone call through the phone call mode. As a more specific example, when a first user who uses a first language is on a phone call with a second user who uses a second language, the first user uses the interpreter function to be translated into the second language. This makes it possible to provide the second user with the voice that has been created. At this time, the telephone call may include a video call.

FIG. 11 is a flowchart illustrating an example of a method for providing an interpretation function of an ear-set device according to an embodiment of the present invention.

In step 1110, the first earset device 120 converts the first language audio input into the first earset device 120 into a second language audio by the interpretation function of the application executed on the electronic device 110. The audio in the first language may be transmitted to the electronic device 110 to be provided to the second earset device 130 forming an earset pair with the first earset device 120 . As described above, the electronic device 110 may convert the received audio in the first language into audio in the second language using the interpretation function of the executed application and provide the second language audio to the second ear set device 130. The earset device 130 outputs the converted audio in the second language so that the second user wearing the second earset device 130 can listen to the first language of the first user wearing the first earset device 120. You will be able to receive interpretation functions.

At step 1120, the first earset device 120 converts the second language audio input to the second earset device 130 and received by the electronic device 110 into a first earset device using an interpretation function of an application executed by the electronic device 110. In response to converting the first language audio, the converted first language audio may be received from the electronic device 110. As described above, the electronic device 110 may convert the second language sound received from the second ear set device 130 into the first language sound and provide the first language sound to the first ear set device 120 .

In step 1130, the first earset device 120 may output the received audio in the first language. Therefore, the first user of the first earset device 120 can receive an interpretation function for the second language of the second user.

As mentioned above, the first earset device 120 may correspond to any one of two earset devices of an earset pair. Therefore, in the embodiment of FIG. 11, only the operation of the first earset device 120 in the multiparty interpretation mode has been described, but based on the above-mentioned content, the operation of the first earset device 120 in the self-interpretation mode The operation of the two-earset device 130 will also be readily understood by those skilled in the art.

Further, in the above-described embodiment, only the embodiment in which two ear-set devices form an ear-set pair has been described, but three or more ear-set devices are connected directly or indirectly to the electronic device 110. Those skilled in the art will be able to easily understand that the multi-party interpretation mode is executable by referring to the above description. For example, it is also possible that the audio in the first language of the ear-set device 1 is interpreted into the sounds in the second language and the third language, and then transmitted to the ear-set device 2 and the ear-set device 3, respectively. In addition, the second language voice of the ear set device 2 is interpreted into the first language and the third language voice and transmitted to the ear set device 1 and the ear set device 3, respectively, or the third language voice of the ear set device 3 is interpreted. It is also possible that the voice of the user is interpreted into voices of the first language and the second language and transmitted to the ear-set device 1 and the ear-set device 2, respectively. As an example of communication, the electronic device 110 may directly communicate with the ear-set device 1 and the ear-set device 2, and indirectly communicate with the ear-set device 3 via the ear-set device 1 and the ear-set device 2.

In the embodiment described above, the first language speech is converted into the second language speech, but the present invention is not limited to this example. For example, sounds in a first language may be converted into characters in a second language and output, or conversely, characters in the first language may be converted into sounds in a second language and output. Both text and audio may be output.

For example, in providing an interpreter function to a user using a first ear set and a second ear set, audio in the first language input through the first ear set is automatically transmitted to the second ear set. The audio of the second language input through the second earset is automatically converted into the characters of the first language and displayed via the display of the electronic device. It can be displayed via the device's display. At this time, the first user of the first ear set and the second user of the second ear set are viewing the characters of the second language and the characters of the first language displayed via the electronic device together. Interpretation functions can be provided.

As described above, according to the present embodiment, the interpreting function can be provided using the earset, and the earset pair including the first earset device and the second earset device is used to use mutually different languages. It can provide an interpretation function between two users. Furthermore, the interpreting function allows the voice interpreted from the first language into the second language to be provided to the other party on the telephone call.

In addition,

The apparatus described above may be realized by hardware components, software components, and/or a combination of hardware and software components. For example, the devices and components described in the embodiments include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), and a microcomputer. The implementation may be implemented using one or more general purpose or special purpose computers, such as a processor or various devices capable of executing and responding to instructions. A processing device may execute an operating system (OS) and one or more software applications that execute on the OS. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. Although for convenience of understanding, one processing device may be described as being used, those skilled in the art will appreciate that a processing device may include multiple processing elements and/or multiple types of processing elements. You will understand. For example, a processing device may include multiple processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include computer programs, code, instructions, or a combination of one or more of these that configure a processing device or instruct a processing device, independently or collectively, to perform operations as desired. You may do so. The software and/or data may be embodied in a machine, component, physical device, computer storage medium or device of any kind for being interpreted by or providing instructions or data to a processing device. good. The software may be distributed and stored and executed in a distributed manner on computer systems connected to a network. The software and data may be stored on one or more computer readable storage media.

Methods according to embodiments may be implemented in the form of program instructions executable by various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be those specially designed and configured for the embodiments, or may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs, DVDs, and floptical disks. and hardware devices specifically configured to store and execute program instructions, such as magneto-optical media, such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that is executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules, and vice versa, to perform the operations of the embodiments.

As described above, the embodiments have been described based on limited embodiments and drawings, but those skilled in the art will be able to make various modifications and variations based on the above description. For example, the techniques described may be performed in a different order than in the manner described, and/or components of the systems, structures, devices, circuits, etc. described may be performed in a different form than in the manner described. Even when combined or combined, opposed or replaced by other components or equivalents, suitable results can be achieved.

Therefore, even if the embodiments are different, if they are equivalent to the scope of the claims, they fall within the scope of the appended claims.

Hereinafter, the means taught by the present application will be exemplarily listed.
(Additional note 1)
A method for providing an interpretation function of an electronic device, the method comprising:
receiving an application execution signal generated by the first earset device or the second earset device;
executing an application corresponding to the received application execution signal; and controlling the electronic device through at least one of the first earset device and the second earset device according to the control of the application to be executed. A method for providing an interpretation function, the method comprising the step of providing an interpretation function for voice received by the user into another language.
(Additional note 2)
The step of providing the interpretation function includes:
converting the first language audio input through the first ear-set device and received by the electronic device into a second language audio and providing the second language audio; The method for providing an interpretation function according to appendix 1, wherein the second language voice received by the electronic device is converted into the first language voice and the first language voice is provided to the first ear set device.
(Additional note 3)
The electronic device directly communicates with any one of the first earset device and the second earset device, and wireless communication between the directly communicating earset device and the remaining one earset device. The method for providing an interpretation function according to appendix 1, wherein the method indirectly communicates with the remaining one ear-set device through communication.
(Additional note 4)
The method for providing an interpretation function according to appendix 1, wherein the electronic device directly communicates with both the first ear-set device and the second ear-set device at the same time.
(Appendix 5)
A music listening mode or a phone call using at least one of the first ear-set device and the second ear-set device based on the received application execution signal or the interpretation mode start signal input by the executed application. switching a call mode to an interpretation mode; and an application end signal generated by one of the earset pairs including the first earset device and the second earset device or an interpretation input by the application. The method for providing an interpretation function according to appendix 1, further comprising the step of switching the interpretation mode to the music listening mode or the telephone conversation mode based on a mode end signal.
(Appendix 6)
A telephone is being executed using at least one of the first earset device and the second earset device based on the received application execution signal or the interpretation mode start signal input by the executed application. the stage of switching the call mode to the interpretation mode;
converting audio in a first language input through at least one of the first ear-set device and the second ear-set device and received by the electronic device into audio in a second language using the interpreting function; step,
switching the interpreting mode to the telephone call mode based on an application end signal generated by the first earset device or the second earset device or an interpreter mode end signal input by the application; and The method for providing an interpretation function according to appendix 1, further comprising the step of providing a second language voice to the other party of the telephone call through the telephone call mode.
(Appendix 7)
The method for providing an interpretation function according to appendix 1, wherein the first ear-set device and the second ear-set device constitute an ear-set pair for a left ear and a right ear.
(Appendix 8)
The application execution signal may be input by a physical button included in at least one of the first earset device and the second earset device, or one of the first earset device and the second earset device. The method for providing an interpretation function according to Supplementary Note 1, wherein the method is generated based on an output value of at least one sensor.
(Appendix 9)
The step of providing the interpretation function includes:
sensing a first language of audio input through the first ear-set device and received by the electronic device and a second language of audio input through the second ear-set device and received by the electronic device;
converting the sensed first language sound into sensed second language sound, and converting the sensed second language sound into sensed first language sound; and the transformed second language sound. The method for providing an interpretation function according to appendix 1, comprising the steps of: providing audio in the second language to the second ear-set device; and providing the converted audio in the first language to the first ear-set device.
(Appendix 10)
The step of providing the interpretation function includes:
The application converts the audio input through the first ear-set device and received by the electronic device into a second language set for the second ear-set device; converting the audio received by the electronic device into a first language set for the first earset device by the application; and providing the converted audio in the second language to the second earset device. and providing the converted first language audio to the first earset device.
(Appendix 11)
A method for providing an interpretation function of a first earset device, the method comprising:
The audio in the first language input to the first ear-set device is converted into the audio in the second language by an interpreter function of an application executed on the electronic device, and the second ear-set device is paired with the first ear-set device. transmitting audio in the first language to the electronic device to be provided to an earset device;
in response to the electronic device converting a second language audio input to the second ear-set device and received by the electronic device into a first language audio by an interpreting function of the executed application; A method for providing an interpretation function, the method comprising: receiving the converted audio in the first language from the electronic device; and outputting the received audio in the first language.
(Appendix 12)
The method for providing an interpretation function according to appendix 11, wherein communication between the second ear-set device and the electronic device is relayed, or the second ear-set device communicates with the electronic device by relaying the communication.
(Appendix 13)
input via a physical button included in at least one of the first earset device and the second earset device; or an input via a sensor included in at least one of the first earset device and the second earset device; 12. The method for providing an interpretation function according to appendix 11, wherein the application is executed in the electronic device by an application execution signal generated by the output value and sent to the electronic device.
(Appendix 14)
The first language may include a language sensed using the executed application as a language of audio input to the first earset device, or a language detected by the executed application to the first earset device. Contains corresponding preset languages,
The second language may include a language sensed by using the executed application to detect the language of audio input to the second earset device, or the second language may include a language sensed using the executed application. The method for providing an interpretation function according to appendix 11, including a corresponding preset language.
(Appendix 15)
A computer program stored on a computer-readable recording medium in combination with a computer to cause the computer to execute the method according to any one of appendices 1 to 14.
(Appendix 16)
A computer-readable recording medium on which a program for causing a computer to execute the method according to any one of Supplementary Notes 1 to 14 is recorded.
(Appendix 17)
A first earset device,
a sound signal input section for receiving sound input;
a sound signal output section for outputting sound;
a communication section for communicating with an electronic device or a second ear-set device forming an ear-set pair with the first ear-set device; and a control section for controlling the sound signal input section, the sound signal output section, and the communication section. Equipped with
The control unit includes:
The sound in the first language input to the sound signal input unit is converted into the sound in the second language by an interpretation function of an application executed on the electronic device, and controlling the communication unit to transmit audio in the first language to the electronic device so as to be provided to the earset device;
in response to the electronic device converting a second language voice input to the second ear-set device and received by the electronic device into a first language voice by an interpreting function of the executed application; controlling the communication unit to receive the converted first language audio from the electronic device;
A first ear set device that controls the sound signal output unit to output the received sound in the first language.
(Appendix 18)
The control unit includes:
The communication unit is controlled to relay communication between the second earset device and the electronic device, or the communication unit is controlled to communicate with the electronic device by relaying the second earset device. , the first earset device according to appendix 17.
(Appendix 19)
input via a physical button included in at least one of the first earset device and the second earset device; or an input via a sensor included in at least one of the first earset device and the second earset device; 18. The first earset device according to claim 17, wherein the application is executed on the electronic device by an application execution signal generated by an output value and transmitted to the electronic device.
(Additional note 20)
The first language may include a language sensed using the executed application as a language of audio input to the first earset device, or a language detected by the executed application to the first earset device. Contains corresponding preset languages,
The second language may include a language sensed by using the executed application to detect the language of audio input to the second earset device, or the second language may include a language sensed using the executed application. The first earset device according to claim 17, including a corresponding preset language.

110: Electronic device 120: First earset device 130: Second earset device

Claims (11)

A method for providing an interpretation function of an electronic device, the method comprising:
The application execution signal generated by the first earset device or the second earset device is transmitted to the first earset device and the second earset device forming one earset pair worn on the left and right ears. the stage of receiving with an electronic device;
causing the electronic device to execute an application corresponding to the received application execution signal;
setting an interpreting mode executed by the electronic device to a self-interpreting mode or a multi-party interpreting mode according to the control of the executed application; and
When the interpreting mode executed by the electronic device is set to the multi-party interpreting mode, the electronic device is set to the first ear set device worn on one ear of the first user according to the control of the executed application. and communicating with the second ear set device worn on one ear of a second user to provide an interpretation function between the first user and the second user in different languages using the multi-party interpretation mode. including ,
The setting step includes setting the interpretation mode executed by the electronic device to the self-interpretation mode or the Automatically set to multiparty interpretation mode,
When the interpretation mode executed by the electronic device is set to the self-interpretation mode, the electronic device is configured to perform at least one of the first earset device and the second earset device according to the control of the executed application. converting the voice in the first language input through the earphone into a second language, and outputting the converted voice in the second language through at least one of the first earset device and the second earset device. A method for providing an interpreting function, comprising: providing an interpreting function using the self-interpreting mode . The step of providing the interpretation function includes:
Converting the first user's voice in the first language input through the first ear-set device and received by the electronic device into the second user's voice in the second language, and providing the converted voice to the second ear-set device; Converting a second language voice input through a second ear-set device and received by the electronic device into a first language voice and providing the same to the first ear-set device, thereby providing an interpreting function into different languages. The method for providing an interpretation function as described in Section 1. The electronic device directly communicates with any one of the first earset device and the second earset device, and wireless communication between the directly communicating earset device and the remaining one earset device. 2. The method for providing an interpretation function according to claim 1, wherein the method indirectly communicates with the remaining earset device through communication. The method of claim 1, wherein the electronic device directly communicates with both the first earset device and the second earset device. A music listening mode or a phone call using at least one of the first ear-set device and the second ear-set device based on the received application execution signal or the interpretation mode start signal input by the executed application. switching a call mode to an interpretation mode; and an application end signal generated by one of the earset pairs including the first earset device and the second earset device or an interpretation input by the application. The method for providing an interpretation function according to claim 1, further comprising: switching the interpretation mode to the music listening mode or the telephone conversation mode based on a mode end signal. further comprising: providing an interpreter function during a telephone call during a telephone call with the other party using the electronic device;
The step of providing an interpreting function during a telephone call includes:
An application execution signal generated by the first ear-set device or the second ear-set device or an interpreter mode during a telephone call input by the application to be executed while the electronic device is in a telephone conversation with the other party of the telephone conversation. switching an ongoing telephone call mode to a mid-phone call interpretation mode using at least one of the first earset device and the second earset device based on a start signal;
converting audio in a language input and received through at least one of the first earset device and the second earset device into audio in another language that is the language of the other party of the telephone call; and The method for providing an interpretation function according to claim 1, further comprising: providing the converted voice in the other language to the other party of the telephone call through the telephone call mode. The application execution signal may be input by a physical button included in at least one of the first earset device and the second earset device, or one of the first earset device and the second earset device. The method for providing an interpretation function according to claim 1, wherein the interpretation function is generated based on an output value of at least one sensor. The step of providing the interpretation function includes:
sensing a first language of the first user's voice input and received through the first earset device and a second language of the second user's voice input and received through the second earset device;
converting the sensed first language sound into sensed second language sound, and converting the sensed second language sound into sensed first language sound; and the transformed second language sound. 3. The method of claim 2, further comprising the steps of: providing audio in the second language to the second earset device; and providing the converted audio in the first language to the first earset device. A computer program stored on a computer-readable recording medium in combination with a computer to cause the computer to execute the method according to any one of claims 1 to 8. A computer-readable recording medium on which a program for causing a computer to execute the method according to any one of claims 1 to 8 is recorded. An electronic device that provides an interpreting function,
at least one processor configured to execute computer-readable instructions contained in the memory;
The at least one processor includes:
Receive an application execution signal generated by the first earset device or the second earset device with respect to a first earset device and a second earset device forming one earset pair worn on the left and right ears. death,
executing an application corresponding to the received application execution signal;
setting the interpretation mode executed by the electronic device to a self-interpretation mode or a multi-party interpretation mode according to the control of the executed application;
When the interpreting mode executed by the electronic device is set to the multi-party interpreting mode, the first ear set device worn on one ear of the first user and the second user's communicating with the second ear set device worn on one ear to provide an interpretation function between the first user and the second user in mutually different languages in the multi-party interpretation mode ;
When setting the interpretation mode executed by the electronic device to a self-interpretation mode or a multi-party interpretation mode, analysis of audio input through at least one of the first earset device and the second earset device is performed. automatically setting an interpreting mode executed by the electronic device to the self-interpreting mode or the multi-party interpreting mode based on the method;
When the interpretation mode executed by the electronic device is set to the self-interpretation mode, the electronic device is configured to perform at least one of the first earset device and the second earset device according to the control of the executed application. converting the voice in the first language input through the earphone into a second language, and outputting the converted voice in the second language through at least one of the first earset device and the second earset device. An electronic device that provides an interpretation function according to the self-interpretation mode .