KR102510958B1 - Mobile terminal and operation method thereof, mobile communication system - Google Patents

Abstract

A method of driving a mobile terminal is disclosed. The method includes the steps of setting a mobile terminal to STT (Speech To Text) mode or TTS (Text To Speech) mode before starting a call with an external terminal, setting sounds other than the user's voice to be muted, and the mobile terminal When operating in the STT mode, a user's voice may be received, converted into text, and the converted text may be transmitted to an external terminal. Accordingly, user convenience is improved.

Description

Mobile terminal and its driving method, mobile communication system

The present invention relates to a mobile communication system, and more particularly, to a mobile terminal equipped with a Text To Speech (TTS) function, a method for driving the same, and a mobile communication system.

Terminals can be divided into mobile/portable terminals and stationary terminals according to whether or not they can move. Again, the mobile terminal may be divided into a handheld terminal and a vehicle mounted terminal depending on whether the user can directly carry the mobile terminal.

Functions of mobile terminals are diversifying. For example, there are functions of data and voice communication, photographing and video recording through a camera, voice recording, music file reproduction through a speaker system, and outputting an image or video to a display unit. Some terminals have an electronic game play function added or a multimedia player function. In particular, recent mobile terminals can receive multicast signals providing visual content such as broadcasting and video or television programs.

Currently, mobile terminals provide various services, such as SMS, in addition to mobile phone communication services, and provide a text to speech (TTS) service that converts and outputs text into voice. The mobile terminal can also be equipped with STT (Speech to Text) technology (a technology that converts voice into text and outputs it).

When TTS/STT technology is used in a mobile terminal, it is of great help to users with visual or hearing impairments.

In the prior art, a mobile terminal transmits a rejection message (eg, text message) or transmits a message in another method (eg, chatting) in a situation where voice transmission is difficult during a voice call. However, the above message has a limitation in accurately conveying the current situation.

Accordingly, a mobile terminal technology in which user convenience is considered is required.

Publication No. 10-2009-0028151 (published on March 18, 2009)

The present invention has been devised to solve the above-mentioned limitations of the prior art, and an object of an embodiment of the present invention is to provide a mobile terminal and a driving method thereof in consideration of user convenience.

Another object of one embodiment of the present invention is to provide a mobile terminal and a method of driving the same capable of enhancing voice recognition.

Another object of one embodiment of the present invention is to provide a mobile communication terminal and a mobile communication system providing translation services.

The technical problems to be achieved in the present invention are not limited to the above-mentioned technical problems, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below. You will be able to.

A mobile communication system according to various embodiments of the present invention includes a first terminal; a second terminal; a server mediating communication between the first terminal and the second terminal; The server converts the voice received from the first terminal into text and transmits it to the second terminal, and when receiving a translation request for the converted text from the second terminal, the server converts the text into a language set in the second terminal. The converted text may be translated and provided to the second terminal.

The above aspects of the present invention are only some of the preferred embodiments of the present invention, and various embodiments in which the technical features of the present invention are reflected are detailed descriptions of the present invention to be detailed below by those skilled in the art. It can be derived and understood based on.

According to various embodiments of the present invention, there are the following effects.

First, user convenience can be improved by providing a mobile terminal equipped with STT/TTS technology.

Second, user convenience can be improved by providing a mobile terminal equipped with a language translation function.

Third, when using the STT technology, the voice recognition rate is increased, so device efficiency and user convenience can be improved.

The effects obtainable in the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below. will be.

1 is a block diagram of a mobile terminal according to an embodiment.
2 is a diagram illustrating a method of driving the mobile terminal according to an embodiment.
3 and 4 are diagrams illustrating a mobile terminal providing a translation service according to an embodiment.
5 is a diagram illustrating a mobile communication system equipped with a TTS/STT function and a translation function according to an embodiment.

Hereinafter, an apparatus and various methods to which embodiments of the present invention are applied will be described in more detail with reference to the drawings. The suffixes "module" and "unit" for components used in the following description are given or used together in consideration of ease of writing the specification, and do not have meanings or roles that are distinct from each other by themselves.

In the description of the embodiment, in the case of being described as being formed in "upper (above) or lower (below)", "before (front) or after (rear)" of each component, "upper (above) or lower (below)" (below)" and "before (front) or after (rear)" include both formed by direct contact between two components or by placing one or more other components between the two components.

In addition, terms such as "include", "comprise" or "have" described above mean that the corresponding component may be inherent unless otherwise stated, excluding other components. It should be construed as being able to further include other components. All terms, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs, unless defined otherwise. Commonly used terms, such as terms defined in a dictionary, should be interpreted as consistent with the meaning in the context of the related art, and unless explicitly defined in the present invention, they are not interpreted in an ideal or excessively formal meaning.

Also, terms such as first, second, A, B, (a), and (b) may be used in describing the components of the present invention. These terms are only used to distinguish the component from other components, and the nature, order, or order of the corresponding component is not limited by the term. When an element is described as being “connected,” “coupled to,” or “connected” to another element, that element is or may be directly connected to the other element, but there is another element between the elements. It will be understood that elements may be “connected”, “coupled” or “connected”.

1 is a block diagram of a mobile terminal according to an embodiment.

The mobile terminal 100 includes a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, personal digital assistants (PDA), a portable multimedia player (PMP), a navigation device, a slate PC, and a tablet. A PC (tablet PC), an ultrabook, a wearable device (eg, a watch type terminal (smartwatch), a glass type terminal (smart glass), a head mounted display (HMD)), and the like may be included.

However, it is easy for those skilled in the art that the configuration according to the embodiment described in this specification may be applied to fixed terminals such as digital TVs, desktop computers, digital signage, etc., except when applicable only to the mobile terminal 100. You will know.

The mobile terminal 100 may include a communication unit 110, a memory 120, a controller 130, and a trans module 140. In addition to the components described above, the mobile terminal 100 generally includes various modules, but components not related to the present invention will be omitted.

The communication unit 110 includes a module for communicating with the external terminal 200 . The communication unit 110 includes a mobile communication module. The mobile communication module complies with technical standards or communication methods for mobile communication (eg, GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), CDMA2000 (Code Division Multi Access 2000), EV-DO ( Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), WCDMA (Wideband CDMA), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), etc.) to transmit and receive radio signals with at least one of a base station, an external terminal, and a server on a mobile communication network.

The communication unit 110 may include a module for wireless Internet communication. Wireless Internet technologies include, for example, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), etc., and the wireless Internet module Data is transmitted and received according to at least one wireless Internet technology within a range including Internet technologies not listed above.

The communication unit 110 may include a short-range communication module, and the short-range communication module is for short range communication, and includes Bluetooth™, Radio Frequency Identification (RFID), and Infrared Data Association (IrDA). ), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technology. can support Such a short-range communication module may be used between the mobile terminal 100 and a wireless communication system, between the mobile terminal 100 and another mobile terminal 100, or between the mobile terminal 100 and another mobile terminal 100 through wireless area networks. Wireless communication between networks in which the mobile terminal 100 (or an external server) is located may be supported. The local area wireless communication network may be a local area wireless personal area network (Wireless Personal Area Networks).

The memory 120 may store programs for operation of the controller 130 and may temporarily store input/output data (eg, phonebook, message, still image, video, etc.). The memory 120 may store data related to vibration and sound of various patterns output when a touch is input on the touch screen.

The memory 120 may be a flash memory type, a hard disk type, a solid state disk type, a silicon disk drive type, or a multimedia card micro type. ), card-type memory (eg SD or XD memory, etc.), RAM (random access memory; RAM), SRAM (static random access memory), ROM (read-only memory; ROM), EEPROM (electrically erasable programmable read -only memory), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. The mobile terminal 100 may be operated in relation to a web storage that performs a storage function of the memory 120 on the Internet.

The trans module 140 includes a TTS module 140a and an STT module 140b. The TTS module 140a is an abbreviation of Text to Speech and is a module that generates voice corresponding to the text when text is input by the user. The STT module 140b is an abbreviation of Speech to Text, opposite to the TTS module 140a, and is a module that generates text corresponding to the voice when a voice is input by the user.

The controller 130 may control the trans module so that the terminal 100 is set to STT (Speech To Text) mode or TTS (Text To Speech) mode before starting a call with the external terminal 200 .

Here, the controller 130 may ask the user for a mode setting intention when a call is received from the external terminal 200 . For example, the user's intention may be inquired through a pop-up.

Also, the controller 130 may mute sounds other than the user's voice. For example, when the terminal 100 is driven in the STT mode, the controller 130 may increase the voice recognition rate by filtering noise other than voice. In addition, the control unit 130 may negotiate sound to be filtered through consultation with the external terminal 200 in advance.

In addition, when the terminal 100 operates in the TTS mode, the controller 130 may receive text, convert it into a pre-registered user voice, and transmit the converted user voice to the external terminal 200 . The controller 130 may select a user voice to be converted in advance.

In general, the controller 130 may be set to the STT mode when receiving a call, and may be set to the TTS mode when sending a call, but is not limited thereto.

The controller 130 may store various voices in the memory 120 in advance and transmit voices to the external terminal 200 using the corresponding voices. Various voices may be composed of different mood states and highlight states (eg, a charming voice, a loud voice, etc.).

When the terminal 100 is operated in the STT mode, the controller 130 may determine whether the input user voice is a pre-registered user voice. In this case, in the case of a pre-registered user voice, the control unit 130 may change the voice into text corresponding thereto and transmit the same to the external terminal 200 .

The control unit 130 may increase security by requiring the user to input a security code such as pattern recognition or password recognition.

In addition, the controller 130 may perform the TTS/STT mode selection through a pre-registered key input provided in the terminal 100, a touch input, or a mode selection voice input.

In addition, the terminal 100 may further include a translation unit (not shown) for translating a specific language into a preset language in addition to the above-described components. Detailed embodiments will be described later.

Hereinafter, a method of driving a mobile terminal will be described.

2 is a flowchart illustrating a method of driving a mobile terminal according to an embodiment of the present invention.

First, the first mobile terminal 100 communicates with the second mobile terminal 200 .

Before starting a call with the first mobile terminal, the first mobile terminal is set to STT (Speech To Text) mode or TTS (Text To Speech) mode (S210).

The first mobile terminal 100 may set the mode when sending or receiving a call to the second mobile terminal 200 .

In addition, the STT mode or TTS mode may be determined by inputting a key button provided in the first mobile terminal 100 or inputting a preset user voice command.

Next, the first mobile terminal 100 mutes sounds other than the user's voice (S220). For example, when the first mobile terminal 100 is set to the STT mode, the user voice recognition rate may be further increased by muting sounds other than the user voice. At this time, the first mobile terminal 100 may perform negotiation with the second mobile terminal 200 to filter a specific sound and transmit the specific sound. Also, when the first mobile terminal 100 is set to the TTS mode, it can receive text input. Accordingly, necessary sounds can be filtered through negotiation between the mobile terminals 100 and 200 while increasing the voice recognition rate.

When the first mobile terminal 100 is set to the STT mode (S230), the user's voice is received and converted into text (S240), and the converted text is transmitted to the second terminal 200 (S270).

At this time, the controller 130 of the first mobile terminal 100 may determine whether the user voice is a pre-registered user voice. When the user voice is not a pre-registered user voice, the controller 130 may generate an appropriate event (eg, warning sound, vibration, etc.).

If the first mobile terminal 100 is set to the TTS mode (S230), it receives text and converts a pre-registered user voice (S250).

The pre-registered user voice may be the voice of the owner or user of the first mobile terminal 100, but is not necessarily limited thereto and may be a voice having various characters.

After step S250, the first terminal 100 transmits the converted user voice to the second terminal 200 (S200).

Meanwhile, when the voice received from the second terminal 200 is not in a preset language, the controller 130 may convert the received voice into a preset language.

In this case, the converted voice may reflect gender, pronunciation, tone, intonation, place of origin, and emotional characteristics of the received voice.

Also, when the text received from the second terminal 200 is not in a preset language, the controller 130 may convert the text into a preset language and provide the text to the user.

According to FIG. 3 , the second terminal 200 may transmit "Hello, how are you" to the first terminal 100 to a Japanese person through the STT module.

As shown in FIG. 4 , when "Hello, how are you?" received from the second terminal 200 is different from a preset language (Japanese), the first terminal 100 translates the corresponding text and provides it to the user. can do.

Hereinafter, a case in which a server is included in a system will be described with reference to FIG. 5 .

According to FIG. 5 , the server 300 may include a communication unit 310, a memory 320, a control unit 330, a trans module 340, and a translation unit 350.

Here, since the communication unit 310, the memory 320, the control unit 330, the trans module 340, and the translation unit 350 are similar to the above-described components, detailed descriptions thereof will be omitted.

Here, it is assumed that the first terminal 100 and the second terminal 200 do not have components corresponding to the trans module 340 and the translation unit 350 .

The server 300 serves to mediate communication between the first terminal 100 and the second terminal 200 .

The server 300 may convert the voice received from the first terminal 100 into text and transmit it to the second terminal 200 . The server 300 may convert voice to text and text to voice through the trans module 340 .

When receiving a translation request for the converted text from the second terminal 200, the server 300 may translate the converted text into a language set in the second terminal 200 and provide the translated text to the second terminal 200. there is. The server 300 may also translate text received from the second terminal 200 into a language set in the first terminal 100 and provide the text to the first terminal 100 .

The method according to the above-described embodiment may be produced as a program to be executed on a computer, and the program may be stored in a computer-readable recording medium. Examples of the computer-readable recording medium include ROM, RAM, There are CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, and the like, and also include those implemented in the form of carrier waves (for example, transmission through the Internet).

The computer-readable recording medium is distributed to computer systems connected through a network, so that computer-readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the above-described method can be easily inferred by programmers in the technical field to which the embodiment belongs.

It is apparent to those skilled in the art that the present invention can be embodied in other specific forms without departing from the spirit and essential characteristics of the present invention.

Accordingly, the above detailed description should not be construed as limiting in all respects and should be considered illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

100: first terminal
200: second terminal
300; server

Claims (16)

In the driving method of the mobile terminal,
Setting a call conversion mode and a language to be used in STT mode when receiving a call and TTS mode when outgoing a call;
When a call is received from an external terminal while the call conversion mode is set to the STT mode, converting the user's voice into text in the set language and displaying the converted text; and
When a call is made to the external terminal in a state where the call conversion mode is set to the TTS mode, the text input through the mobile terminal is stored in the memory of the mobile terminal in advance and one of various voices according to the user's mood state or highlight state. A method of driving a mobile terminal comprising: converting a voice into a voice and transmitting the sound to the external terminal. delete delete According to claim 1,
The voice according to the highlight state includes a charming voice or a loud voice. delete delete delete In the mobile terminal,
display;
A trans module for setting a call conversion mode to STT mode when receiving a call and to TTS mode when sending a call;
a communication unit that communicates with an external terminal; and
A control unit configured to set the currency conversion mode and the language used according to a user's input signal;
The control unit,
When a call is received from the external terminal while the call conversion mode is set to the STT mode, the user's voice is converted into text in a set language and displayed on the display;
When a call is made to the external terminal while the call conversion mode is set to the TTS mode, the text is received and converted into one of various voices according to the user's mood state or highlight state stored in the memory of the mobile terminal in advance, A mobile terminal transmitting to the external terminal. delete delete According to claim 8,
The voice according to the highlight state includes a charming voice or a loud voice. delete delete delete delete delete

Images