KR20140061134A - Mobile terminal and control method thereof - Google Patents

Abstract

The present invention relates to a mobile terminal and a control method thereof, capable of providing voice recognition and cognition interface. The mobile terminal includes a microphone which receives a voice command if a voice recognition mode is executed, a voice command processing unit which generates result information in response to the voice command, a display unit which outputs a first object which shows the voice command, a second object which shows a part of result information in order of time, a user input unit which senses a touch input for the display unit, and a control unit which transmits the result information to an external terminal based on a message transmission command if the message transmission command for the second object is sensed.

Description

[0001] MOBILE TERMINAL AND CONTROL METHOD THEREOF [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a mobile terminal, and more particularly, to a mobile terminal providing a voice recognition and a recognition interface and a control method thereof.

A terminal can be divided into a mobile or portable terminal and a stationary terminal depending on whether the terminal is movable or not. The mobile terminal can be divided into a handheld terminal and a vehicle mount terminal according to whether the user can directly carry the mobile terminal.

As the functions of such terminals are diversified, they are implemented in the form of a multimedia player having complex functions such as photographing and photographing of movies, reproduction of music and video files, games, and broadcasting reception . Further, in order to support and enhance the function of the terminal, it may be considered to improve the structural and software parts of the terminal.

Thanks to these improvements, the terminal has been developed to recognize speech and interpret the recognized language. The terminal can identify a word or word sequence from the voice waveform received at the microphone and extract its meaning. Such speech recognition can be classified into, for example, five steps of speech analysis, phoneme recognition, word recognition, sentence analysis, and semantic extraction. By using such voice recognition, voice can be detected as an execution command and a function corresponding to the command can be executed. According to the conventional art, when a certain function is executed by the voice recognition mode, the application corresponding to the certain function is executed and the voice recognition mode is ended.

The present invention provides a mobile terminal that outputs at least a part of result information according to the voice command and the voice command when a voice command is detected, and a control method thereof.

The present invention also provides a mobile terminal for displaying voice recognition performance histories performed during execution of a voice recognition mode and transmitting specific result information to an external terminal, and a control method thereof.

One embodiment of the invention relates to a mobile terminal. A mobile terminal according to the present invention includes a microphone for receiving a voice command when the voice recognition mode is executed, a voice command processing unit for generating result information in response to the voice command, a first object representing the voice command, A display unit for outputting a second object representing at least a part of the result information in chronological order, a user input unit configured to detect a touch input to the display unit, and a controller for, when a message transmission command for the second object is sensed, And a controller for transmitting the result information to an external terminal based on a transmission command.

As an example related to the present invention, the control unit may output a message received from the external terminal during the execution of the voice recognition mode in chronological order together with the first and second objects.

In another embodiment of the present invention, the controller transmits a reply to the message based on at least one of the voice command and the touch input to the external terminal, and the first object, And may output at least one of the message and the reply in chronological order.

As another example related to the present invention, a memory for storing a speech recognition performance history including at least one of the first object, the second object, the message, and the reply may be further included. In addition, at least one of the second object, the message, and the reply may be transmitted / received by a predetermined messenger. The memory stores a plurality of messengers, and the controller can set any one of the plurality of messengers as the predetermined messenger.

As another example related to the present invention, the voice command processing unit may select an associated application based on the voice command, and generate the result information using the application. At this time, the control unit may output the second object together with information on the application. The controller may output the result information using the application instead of the first and second objects when a touch input to the second object is sensed.

As another example associated with the present invention, the message transmission command may be sensed by at least one of the microphone and the user input.

In addition, an embodiment of the present invention relates to a control method of a mobile terminal. According to another aspect of the present invention, there is provided a method for controlling a mobile terminal, the method comprising: executing a voice recognition mode for providing a voice recognition function; receiving a voice command using a microphone when the voice recognition mode is executed; Outputting a first object representing the voice command and a second object representing at least a part of the result information to the display unit in chronological order; Detecting a transmission command, and transmitting the result information to an external terminal based on the message transmission command.

In one embodiment of the present invention, the step of receiving a message from the external terminal during execution of the voice recognition mode and outputting the message to the display unit in time order with the first and second objects .

According to another aspect of the present invention, there is provided a method of transmitting a message, the method comprising: generating a reply to the message based on at least one of the voice command and the touch input; transmitting the reply to an external terminal; And outputting at least one of the two objects, the message, and the reply in chronological order to the display unit.

As another example related to the present invention, the method may further include storing a speech recognition performance history including at least one of the first object, the second object, the message, and the reply in a memory.

As another example related to the present invention, generating the result information may include selecting an associated application based on the voice command and generating the result information using the application.

According to the present invention, since at least a part of the result information generated in response to the voice command is output together with the voice command, the result information corresponding to the voice command can be confirmed on the display unit without executing the application related to the voice command .

In addition, according to the present invention, since result information generated in response to a voice command can be transmitted to an external terminal, result information can be shared by an easy method.

According to the present invention, since the voice command, the result information, and the message transmitted / received to / from the external terminal during the voice recognition mode are stored in the memory as the voice recognition performance history, the user can check the voice recognition performance records whenever necessary.

1 is a block diagram illustrating a mobile terminal according to an embodiment of the present invention;
2A and 2B are perspective views showing the appearance of a mobile terminal according to the present invention.
3 is a conceptual diagram for explaining an embodiment of the present invention.
4 is an exemplary diagram illustrating a mobile terminal according to an embodiment of the present invention.
5 is an exemplary diagram illustrating a mobile terminal for transmitting result information to an external terminal according to an embodiment of the present invention.
6 to 8 are flowcharts showing a control method of a mobile terminal according to an embodiment of the present invention.
9 and 10 are views illustrating an example of a mobile terminal that outputs a voice recognition performance history according to an embodiment of the present invention.
11 is an exemplary diagram illustrating a mobile terminal according to an embodiment of the present invention.
12 is a view for explaining a mobile terminal for changing a predetermined messenger according to an embodiment of the present invention;
13 is a diagram illustrating an example of a mobile terminal that outputs a voice recognition performance history according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings so that those skilled in the art can easily carry out the technical idea of the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly explain the present invention, parts not related to the description are omitted, and like parts are denoted by similar reference numerals throughout the specification.

The mobile terminal described herein may include a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a navigator, have. However, it should be understood that the configuration according to the embodiments described herein may be applied to a fixed terminal such as a digital TV, a desktop computer, and the like, unless it is applicable only to a mobile terminal. It will be easy to see.

1 is a block diagram illustrating a mobile terminal 100 according to an embodiment of the present invention. 1, a mobile terminal 100 includes a wireless communication unit 110, an A / V input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, A controller 160, an interface unit 170, a controller 180, and a power supply unit 190. The components shown in Fig. 1 are not essential, so that a mobile terminal having more or fewer components can be implemented.

Hereinafter, the components 110 to 190 of the mobile terminal 100 will be described in order.

The wireless communication unit 110 may include one or more modules for enabling wireless communication between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and the network in which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short distance communication module 114, and a location information module 115.

The broadcast receiving module 111 receives broadcast signals and broadcast related information from an external broadcast management server through a broadcast channel. Here, the broadcast-related information means information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast-related information can also be provided through a mobile communication network. In this case, the broadcast-related information may be received by the mobile communication module 112. The broadcast signal and the broadcast related information sensed through the broadcast reception module 111 may be stored in the memory 160. [

The mobile communication module 112 transmits and receives radio signals to at least one of a base station, an external terminal, and a server on a mobile communication network. Such wireless signals may include various types of data depending on a voice call signal, a video call signal, a text message, or a multimedia message transmission / reception.

The wireless Internet module 113 is a module for wireless Internet access, and may be embedded in the mobile terminal 100 or externally. WLAN (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access) and the like can be used as wireless Internet technologies.

The short-range communication module 114 refers to a module for short-range communication. As a short range communication technology, Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and the like can be used.

The position information module 115 is a module for acquiring the position of the mobile terminal 100, and a representative example thereof is a Global Position System (GPS) module.

1, an A / V (Audio / Video) input unit 120 is for inputting an audio signal and a video signal, and may include a camera 121, a microphone 122, and the like. The camera 121 processes image frames such as still images and moving images obtained by the image sensor in the video communication mode or the photographing mode. The image frame processed by the camera 121 can be displayed on the display unit 151. [ The image frame may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. [ More than two cameras 121 may be provided depending on the use environment.

The microphone 122 processes the sound signal input from the outside in the communication mode, the recording mode, the voice selection mode, and the like as electrical voice data. The voice data processed by the microphone 122 in the communication mode can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 and output. The microphone 122 may be implemented with various noise reduction algorithms for eliminating noise generated when an external sound signal is input.

The user input unit 130 generates input data for controlling the operation of the mobile terminal 100 by a user. The user input unit 130 may include a key pad, a dome switch, a touch pad (static pressure and static electricity), a jog wheel, a jog switch, and the like.

The sensing unit 140 senses the current state of the mobile terminal 100 such as presence or absence of a user, the open / close state of the mobile terminal 100, position, orientation, acceleration, deceleration, And generates a sensing signal. For example, when the mobile terminal 100 is in the form of a slide phone, the sensing unit 140 may detect whether the slide phone is opened or closed. The sensing unit 140 may sense whether the power supply unit 190 is powered on, whether the interface unit 170 is connected to an external device, and the like.

The sensing unit 140 may include a proximity sensor 141. The sensing unit 140 may include a touch sensor (not shown) for sensing a touch operation with respect to the display unit 151.

The touch sensor may have the form of a touch film, a touch sheet, a touch pad, or the like. The touch sensor may be configured to convert a pressure applied to a specific portion of the display portion 151 or a change in capacitance generated at a specific portion of the display portion 151 into an electrical input signal. The touch sensor can be configured to detect not only the position and area to be touched but also the touch pressure.

When the touch sensor and the display unit 151 have a mutual layer structure, the display unit 151 can be used as an input device in addition to the output device. The display unit 151 may be referred to as a 'touch screen'.

If there is a touch input via the touch screen, corresponding signals are sent to a touch controller (not shown). The touch controller processes the signals transmitted from the touch sensor, and then transmits data corresponding to the processed signals to the controller 180. Thus, the control unit 180 can know which area of the display unit 151 is touched.

In the case where the touch screen is electrostatic, it can be configured to detect the proximity of the sensing object by a change of the electric field along the proximity of the sensing target. Such a touch screen may be classified as proximity sensor 141. [

The proximity sensor 141 refers to a sensor that detects the presence or absence of an object to be sensed without mechanical contact using an electromagnetic force or infrared rays. The proximity sensor 141 has a longer life than the contact type sensor and its utilization is also high. Examples of the proximity sensor 141 include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor.

Hereinafter, for convenience of explanation, a proximity action is referred to as " proximity touch " while an object to be sensed does not touch the touch screen, and an action of touching the sensing object on the touch screen is called & touch ".

The proximity sensor 141 detects the presence or absence of a proximity touch and a proximity touch pattern (for example, a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, The information corresponding to the presence / absence of proximity touch and the proximity touch pattern can be output to the touch screen.

The output unit 150 generates an output related to visual, auditory, tactile, and the like. The output unit 150 may include a display unit 151, an audio output module 152, an alarm unit 153, and a haptic module 154.

The display unit 151 displays (outputs) information processed by the mobile terminal 100. For example, when the mobile terminal 100 operates in the call mode, the display unit 151 displays a UI (User Interface) or a GUI (Graphic User Interface) related to the call. When the mobile terminal 100 operates in the video communication mode or the photographing mode, the display unit 151 displays the photographed image, the received image, the UI, the GUI, and the like.

The display unit 151 may be a liquid crystal display (LCD), a thin film transistor liquid crystal display (TFT-LCD), an organic light emitting diode (OLED), a flexible display display, a 3D display, and an e-ink display.

At least one display (or display element) included in the display unit 151 may be configured to be transparent or light transmissive so that the display can see the outside through it. This can be referred to as a transparent display. A typical example of such a transparent display is TOLED (Transparent OLED). The rear structure of the display unit 151 may also be of a light transmission type. That is, the display unit 151 may include first and second surfaces to be overlapped, and the first and second surfaces may be configured as a transparent or light transmissive structure. With this structure, the user can see an object located behind the terminal body through the area occupied by the display unit 151 in the terminal body. Such a display unit 151 may be referred to as a transparent display unit 155.

There may be two or more display units 151 according to the embodiment of the mobile terminal 100. [ For example, in the mobile terminal 100, a plurality of display portions may be spaced apart from each other or positioned integrally with one another, and may be located on different surfaces.

The audio output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception mode, a call mode or a recording mode, a voice selection mode, a broadcast reception mode, The sound output module 152 also outputs sound signals related to functions (e.g., call signal reception sound, message reception sound, etc.) performed in the mobile terminal 100. [ The audio output module 152 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event of the mobile terminal 100. Examples of events generated in the mobile terminal 100 include call signal reception, message reception, key signal input, touch input, and the like. The alarm unit 153 may output a signal for informing occurrence of an event in a form other than a video signal or an audio signal, for example, vibration. The display unit 151 and the sound output module 152 may be classified as a part of the alarm unit 153 because the video signal or the audio signal can also be output through the display unit 151 or the sound output module 152. [

The haptic module 154 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 154 is vibration. The intensity, pattern, and the like of the vibration generated by the hit module 154 are controllable. For example, different vibrations may be synthesized and output or sequentially output.

In addition to the vibration, the haptic module 154 may include a pin arrangement vertically moving with respect to the contact skin surface, a spraying force or a suction force of the air through the injection port or the suction port, a touch on the skin surface, contact with an electrode, Various effects can be generated such as the effect of heat generation and the effect of reproducing the cold sensation using the heat absorbing or heatable element.

The haptic module 154 can be configured to not only transmit the tactile effect through the direct contact but also to allow the user to feel the tactile effect through the muscular sense of the finger or arm. At least two haptic modules 154 may be provided according to the configuration of the mobile terminal 100.

The memory 160 may store a program for the operation of the controller 180 and temporarily store input and output data (e.g., a phone book, a message, a still image, a moving picture, etc.). The memory 160 may store data related to vibrations and sounds of various patterns that are output upon touch input on the touch screen.

The memory 160 may be a flash memory, a hard disk, a multimedia card micro type, a card type memory (e.g., SD or XD memory), a random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM) And may include at least one storage medium. The mobile terminal 100 may operate in association with a web storage that performs storage functions of the memory 160 on the Internet.

The interface unit 170 serves as a path for communication with all external devices connected to the mobile terminal 100. The interface unit 170 receives data from an external device or supplies power to each component in the mobile terminal 100 or transmits data to the external device. For example, the interface unit 170 may include a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, Input / output ports, video I / O ports, earphone ports, and the like.

The identification module is a chip for storing various information for authenticating the usage right of the mobile terminal 100 and includes a user identification module (UIM), a subscriber identity module (SIM), a general user authentication module Universal Subscriber Identity Module (USIM)). A device equipped with an identification module (hereinafter referred to as "identification device") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the port.

The interface unit 170 may be a path through which the power from the cradle is supplied to the mobile terminal 100 when the mobile terminal 100 is connected to an external cradle or various command signals input from the cradle by the user, (Not shown). The various command signals or power supplied from the cradle may also act as a signal for recognizing that the mobile terminal 100 is correctly mounted in the cradle.

The controller 180 controls the overall operation of the mobile terminal 100. For example, control and processing related to voice communication, data communication, video communication, and the like. The control unit 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180. [ The control unit 180 may perform pattern selection processing for selecting handwriting input and drawing input on the touch screen as characters and images, respectively.

In addition, the control unit 180 may include a voice command processing unit 182. The voice command processing unit 182 can identify a word or a word string from the voice waveform received by the microphone 122 and extract the meaning thereof. The voice command processing unit 182 can perform speech recognition according to at least one of voice analysis, phoneme recognition, word recognition, sentence analysis, and semantic extraction, for example. It is possible to extract the execution command from the voice by using the voice recognition and execute the function corresponding to the command.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components.

The various embodiments described herein may be implemented in a recording medium readable by a computer or similar device using software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein may be applied to various types of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays Microprocessors, microprocessors, and other electronic units for performing other functions, as will be appreciated by those skilled in the art. In some cases, the embodiments described herein may be implemented by the controller 180 itself.

According to a software implementation, embodiments such as the procedures and functions described herein may be implemented with separate software modules. Each of the software modules may perform one or more of the functions and operations described herein. Software code may be implemented in a software application written in a suitable programming language. Such software codes may be stored in the memory 160 and executed by the control unit 180. [

Hereinafter, a method of processing user input to the mobile terminal 100 will be described.

The user input unit 130 is operated to receive a command for controlling the operation of the mobile terminal 100, and may include a plurality of operation units. The operating units may also be referred to as manipulating portions and may be employed in any manner as long as they are operated in a tactile manner by the user's tactile sense.

Various kinds of time information can be displayed on the display unit 151. [ Such visual information can be displayed in the form of letters, numbers, symbols, graphics, icons, and the like, and can be formed as a three-dimensional stereoscopic image. At least one of a character, a number, a symbol, a graphic, and an icon may be displayed in a predetermined arrangement for inputting time information, thereby being implemented as a keypad. Such a keypad may be referred to as a so-called " soft key ".

The display unit 151 may operate as an entire area or may be divided into a plurality of areas and operated. In the latter case, the plurality of areas can be configured to operate in association with one another. For example, an output window and an input window may be displayed on the upper and lower portions of the display unit 151, respectively. The output window and the input window are areas allocated for outputting or inputting information, respectively. In the input window, a soft key with a number for inputting a telephone number may be output. When the soft key is touched, the number corresponding to the touched soft key is displayed in the output window. When the operating unit is operated, a call connection to the telephone number displayed in the output window may be attempted, or the text displayed in the output window may be entered into the application.

The display unit 151 or the touch pad may be configured to detect touch scrolling. The user can move the cursor or the pointer positioned on the displayed object, for example, the icon, on the display unit 151 by scrolling the display unit 151 or the touch pad. Further, when the finger is moved on the display unit 151 or the touch pad, the path along which the finger moves may be visually displayed on the display unit 151. [ This will be useful for editing the image displayed on the display unit 151. [

One function of the mobile terminal 100 may be executed in response to a case where the display unit 151 and the touch pad are touched together within a predetermined time range. In the case of being touched together, there may be a case where the user clamps the main body of the mobile terminal 100 using the thumb and index finger. At this time, one function of the mobile terminal 100 to be executed may be activation or deactivation for the display unit 151 or the touch pad, for example.

The control unit 180 of the present invention will be described again. The control unit 180 can execute the voice recognition mode based on the user input. The voice recognition mode may be a mode for analyzing micro-received voice, checking whether the analyzed voice contains a command, and performing a function corresponding to the command.

At this time, the voice recognition mode includes, for example, a button related to the voice recognition mode in the terminal body, and can be turned ON or OFF according to a user input to the button. As another example, when the preset touch input is input to the touch pad, the voice recognition mode can be executed. The speech recognition mode can be performed in various ways, and is not limited to description of the present invention.

When the voice recognition mode is executed, the control unit 180 regards the voice sensed by the microphone as a voice command, and can control the voice command processor 182 to perform voice recognition of the voice command. Referring to FIG. 1, the controller 180 and the voice command processor 182 are shown to be distinguished from each other. However, the controller 180 may function as the voice command processor 182.

2A and 2B are perspective views showing an appearance of a mobile terminal 100 according to the present invention. FIG. 2A shows a front side and one side of the mobile terminal 100, and FIG. 2B shows a rear side and the other side of the mobile terminal 100. FIG.

Referring to FIG. 2A, the mobile terminal 100 includes a bar-shaped terminal body. However, the mobile terminal 100 is not limited thereto, and may be realized in various forms such as a slide type, a folder type, a swing type, and a swivel type in which two or more bodies are movably coupled to each other.

The terminal body includes a case (a casing, a housing, a cover, and the like) that forms an appearance. In the embodiment, the case may be divided into a front case 101 and a rear case 102. A variety of electronic components are embedded in the space formed between the front case 101 and the rear case 102. At least one intermediate case may be additionally disposed between the front case 101 and the rear case 102. [

The cases may be formed by injection molding of a synthetic resin or may be formed to have a metal material such as stainless steel (STS), titanium (Ti) or the like.

The display unit 151, the sound output unit 152, the camera 121, the user input unit 130 (see FIG. 1), the microphone 122, the interface 170, can do.

The display unit 151 occupies a main portion of the front case 101. [ The audio output unit 152 and the camera 121 are located in a region adjacent to one end of the display unit 151 and the first user input unit 131 and the microphone 122 are located in a region adjacent to the other end. The second user input unit 132 and the interface 170 may be located on the sides of the front case 101 and the rear case 102. [

The user input unit 130 is operated to receive a command for controlling the operation of the mobile terminal 100. [ The user input unit 130 may include a plurality of operation units 131 and 132.

The first or second operating units 131 and 132 can receive various commands. For example, the first operation unit 131 can receive commands such as start, end, scroll, and the like. The second operation unit 132 can receive commands such as adjusting the size of the sound output from the sound output unit 152, switching to the touch selection mode of the display unit 151, and the like.

Referring to FIG. 2B, a rear camera 121 'may be additionally mounted on the rear surface of the terminal body, that is, the rear case 102. The rear camera 121 'has a photographing direction opposite to that of the front camera 121 (see FIG. 2A), and may be configured to have pixels different from those of the front camera 121.

For example, the front camera 121 may be configured to have a low pixel, and the rear camera 121 'may be configured to have a high pixel. Accordingly, when the front camera 121 is used during a video call, when the user's face is photographed and the photographed image is transmitted to the other party in real time, the size of the transmitted data can be reduced. On the other hand, the rear camera 121 'can be used for the purpose of storing a high-quality image.

Meanwhile, the cameras 121 and 121 'may be installed in the terminal body so as to be rotated or popped up.

The flash 123 and the mirror 124 may be further positioned adjacent to the rear camera 121 '. The flash 123 emits light toward the subject when the user photographs the subject with the rear camera 121 '. The mirror 124 illuminates the user's face when the user photographs himself (self-photographing) using the rear camera 121 '.

A rear sound output unit 152 'may be additionally disposed on the rear surface of the terminal body. The rear sound output unit 152 'may perform a stereo function together with the front sound output unit 152 (see FIG. 2A), and may perform a speakerphone function during a call.

In addition to the antenna for communication, an antenna 116 for receiving broadcast signals may be additionally provided on the side surface of the terminal body. The antenna 116 constituting a part of the broadcast receiving module 111 (see FIG. 1) can be installed to be able to be drawn out from the terminal body.

A power supply unit 190 for supplying power to the mobile terminal 100 is mounted on the terminal body. The power supply unit 190 may be built in the main body of the terminal or may be detachable from the outside of the main body of the terminal.

The rear case 102 may further include a touch pad 135 for sensing a touch. The touch pad 135 may be of a light transmission type like the display unit 151 (see FIG. 2A). Further, a back display unit for outputting time information can be additionally mounted on the touch pad 135. [ At this time, information output from both sides of the front display unit 151 and the rear display unit can be controlled by the touch pad 135. [

The touch pad 135 operates in correlation with the display unit 151. [ The touch pad 135 may be positioned parallel to the rear of the display unit 151. The touch pad 135 may have a size equal to or smaller than that of the display unit 151.

3 is a conceptual diagram for explaining an embodiment of the present invention.

Referring to FIG. 3, the present invention may include a voice command 200 and a mobile terminal 100. The voice refers to a person's voice or speech, but the voice command 200 according to the present invention may include not only a voice uttered by a person, but also a voice generated by a machine, such as recorded or reproduced by various other methods.

The mobile terminal 100 can execute the voice recognition mode by user input, as described above with reference to Fig. At this time, the control unit 180 may output at least one of the pop-up window 310, the icon, and the image to the display unit 151 to indicate that the voice recognition mode is being executed.

When the voice recognition mode is executed, the microphone can receive the voice command 200. [ For example, the analog signal of the voice received by the microphone may be converted into a digital signal and transmitted to the voice command processing unit 182.

The voice command processing unit 182 can analyze the voice command 200 received from the microphone 122. For example, the voice command processing unit 182 may perform voice communication, text message creation, music reproduction, contact search, route search, web search, application execution, voice recording, memo, , A timer setting, a weather search, and the voice command 200, and execute the corresponding function according to the comparison result.

The voice command processing unit 182 may generate the result information in response to the voice command (200). The result information may have various methods according to voice commands. For example, it may be location information using a map-related application, or information obtained by searching a search word included in a voice command using a web browser. Another example may be text information retrieved using a dictionary-related application. As another example, the image information may be the same as the most recently photographed image.

4 is an exemplary diagram illustrating a mobile terminal according to an embodiment of the present invention.

Referring to FIG. 4, the mobile terminal 100 according to the present invention may include a display unit 151 for outputting at least one of the first object 320 and the second object 330.

The control unit 180 can convert the voice command 200 processed by the voice command processing unit 182 into text. The voice command 200 converted into text may be output to the display unit 155 as a first object. At this time, an object may mean a graphic element such as an application text, an application icon, a widget, a thumbnail image, and the like.

The result information generated by the voice command processing unit 182 in response to the voice command 200 may be output to the display unit 155. [ At this time, the controller 180 may output a second object representing at least a part of the result information instead of the result information. The first and second objects 320 and 330 may be output in chronological order.

The control unit 180 can output the histories performed during the execution of the voice recognition mode to the display unit 151 as the voice recognition performance history. The voice recognition history may include a first object 320 and a second object 330. The first object 320 may be a voice command 200 converted to text and the second object 330 may represent at least a portion of the result information generated in response to the voice command 200.

The result information can be generated in various manners, such as text information, image information, and information generated using an associated application, as described with reference to FIG. In particular, according to the related art, when the result information is generated using the related application, the mobile terminal 100 has to exit the voice recognition mode and execute the related application.

In order to output the result information as a voice recognition performance history, it is necessary to convert it into an object such as a text and a thumbnail image. Accordingly, the mobile terminal 100 according to the present invention can convert the result information into the second object, and output the second object to the display unit 151 as the voice recognition performance history.

For example, referring to FIG. 4, a voice command 200, "Find the Star Trek Starbucks" may be received at the microphone 122. Accordingly, the first object 32 corresponding to the voice command 200 can be output to the display unit 151 in a text format.

In response to the voice command 200, the voice command processing unit 182 searches the Starbucks located in the tree-lined streets using the map application "NAVER MAP ", and generates the corresponding location information as the result information. In the related art, the control unit 180 has to execute an application called "NAVER MAP" in order to output position information to the display unit 151. [

However, the controller 180 according to the present invention can output the thumbnail image including the position information to the display unit 151 as the second object 330. For example, the voice command processing unit 182 may utilize multitasking to execute an application and generate a thumbnail image of a screen to be output by the application. The control unit 180 can output the thumbnail image to the display unit 151 as a voice recognition performance history.

Also, the control unit 180 may output information about the application used to generate the result information together with the second object 330. [ For example, at least one of the name of the application, the trademark associated with the application, and the image provided by the application may be output with the second object 300. For example, referring to FIG. 4, an application name such as "NAVER MAP" may be output with the second object 330.

5 is an exemplary diagram illustrating a mobile terminal for transmitting result information to an external terminal according to an embodiment of the present invention.

Referring to FIG. 5, the mobile terminal 100 according to the present invention may include a display unit 151 for outputting first and second objects 320 and 330.

At this time, a message transmission command for the second object 300 may be detected by the mobile terminal 100. For example, the message transmission command may be sensed in the user input unit 130 as a touch input to the second object 300. As another example, a voice command such as "message transfer"

Although not shown in the figure, the message transmission command for the second object 330 may include at least one contact information. At this time, the contact information may be a telephone number, a messenger ID, an e-mail address, and the like. Also, the contact information may be selected from at least one of the contacts stored in the memory, or may be directly input from the user.

In response to the message transmission command, the control unit 180 may transmit the result information corresponding to the second object 330 to the external terminal. For example, the information transmitted to the external terminal may include a second object, and may further include link information that allows the application to be used to display the result information. Accordingly, the external terminal can not only output the second object, but also output the result information using the link information and the application.

Although not shown in the figure, the control unit 180 may output the message transmission result to the display unit 180 as a third object. For example, text information such as "Send the information of the roadside to Starbucks " 340 may be output as a third object along with the first and second objects 320 and 330. Since the first to third objects are outputted in chronological order, they can be utilized as a history of performing speech recognition.

Thus, according to the present invention, since the first and second objects are output together with the voice recognition execution history, it is possible to easily confirm the voice recognition mode execution history. Also, the output result information can be transmitted to the external terminal, and the external terminal can use the second object and the link information included in the result information, so that the external terminal can share the voice recognition performance history. Therefore, user convenience can be increased.

6 to 8 are flowcharts showing a control method of a mobile terminal according to an embodiment of the present invention.

Referring to FIG. 6, a method of controlling a mobile terminal according to the present invention includes executing a voice recognition mode (S110). The voice recognition mode can be turned on or off by the user. That is, when a command for executing the voice recognition mode is detected, the control unit 180 can execute the voice recognition mode in response to the detected command.

Next, the step of receiving a voice command using the microphone (S120) may be performed. Next, the step of generating the result information (S130) may proceed. When the voice recognition mode is executed, the microphone 122 can sense a sound generated in the vicinity of the main body. For example, the microphone 122 can only sense sounds including meaningful words and filter other sounds. Whether or not the sound detected by the microphone 122 is a voice command can be distinguished by the voice command processor 182.

The voice command processing unit 182 can discriminate whether or not the sound sensed by the microphone 122 corresponds to a voice command using the command and the command pattern stored in the memory 160. [ For example, if the detected sound includes the word "search ", it may correspond to a voice command that performs a searching function using a web browser or a map application. At this time, the voice command processing unit 182 analyzes a word located in front of "search ", and when the analyzed word corresponds to a place name, the map application uses the map application, and if not, uses the web browser.

The voice command processing unit 182 can obtain the result information by using various applications or by searching information stored in the memory 160. [ For example, when a voice command "contact Hong Kil Dong" is received, information such as a telephone number, e-mail, and address corresponding to "Hong Kil Dong" among the contacts stored in the memory 160 can be obtained as the result information. As another example, if a voice command of "every morning, nine o'clock alarm" is received, the alarm application is used to set up a wake-up call for each morning at nine o'clock and a result such as "every morning morning o'clock alarm set" Information can be generated. In this way, the voice command processing unit 182 can generate the result information in various ways.

Next, a step of outputting a first object representing a voice command and a second object representing at least a part of the result information in a time order (S140) may be performed. It can be said that the step S 140 is a step of outputting the voice recognition performance history to the display unit 151. The voice recognition performance history may mean the history of speech recognition performed from the start point to the end point of the voice recognition mode.

The control unit 180 may convert a voice command sensed by the microphone 122 into text (or characters), wherein the converted text may correspond to a first object. Also, the controller 180 may generate the second object from the result information. If the result information is composed of at least one of a text and an image, it can be used as the second object. However, if the result information is information that can be output to the display unit 151 using an application, the control unit 180 captures a screen to be output to the display unit 151, As a second object.

That is, the second object may be a combination of at least one of text and images representing at least a part of the result information. Such a second object may include link information. The control unit 180 can output the entire contents of the result information according to the user's input using the link information. For example, if the second object includes link information linking to the map application, and the touch input to the second object is detected, the result information may be output using the map application instead of the speech recognition execution history.

Next, a message transmission command for the second object may be sensed (S150). Next, the result information may be transmitted to the external terminal (S160). The message transfer command for the second object may be sensed in various ways. For example, a message transmission menu for the second object may be provided, and a user input for the transmission menu may be sensed. As another example, since the speech recognition mode is being executed, a message transmission command may be detected as a voice command.

The control unit 180 may transmit the result information corresponding to the second object to the external terminal based on the message transmission command. At this time, not only the second object but also the link information included in the second object may be transmitted to the external terminal. The external terminal can execute the related application using the link information, and can display the same result information using the application. In this way, the user can share the result information with other users.

At this time, the controller 180 can transmit the result information to the external messenger using the preset messenger. For example, the messenger may be an application that sends and receives messages. As another example, a messenger may be an application that sends and receives mail. The predetermined messenger can be changed by user input.

7 is a flowchart illustrating a method of controlling a mobile terminal according to an embodiment of the present invention.

Referring to FIG. 7, the method of controlling a mobile terminal according to the present invention may further include receiving a message from an external terminal (S210). During execution of the voice recognition mode, the mobile terminal 100 that has transmitted the result information to the external terminal can receive the message from the external terminal. At this time, the message will be received using a predetermined messenger, but may be received by another messenger.

The control unit 180 can output the message received from the external terminal to the display unit 155 as a voice recognition performance history. That is, the messages received from the external terminal together with the first and second objects can be outputted in chronological order. At this time, due to the physical limitations of the display unit 155, at least one of the first and second objects and messages received from the external terminal may be output in time order. In this case, by flicking or dragging the screen, the remaining data that have not been output to the display unit 151 can be output to the display unit 155.

8 is a flowchart illustrating a method of controlling a mobile terminal according to an embodiment of the present invention.

Referring to FIG. 8, the method of controlling a mobile terminal according to the present invention may further include generating a reply to a message (S310). This assumes that the message is received from the external terminal, and the received message is output to the display unit by the voice recognition performance history. Next, the step of transmitting the reply to the external terminal (S320) may proceed.

In general, in order for a user to create a reply to a message, an application associated with the message may be executed and a reply may be created using the application. However, the control method of a mobile terminal according to the present invention can generate a reply to a message in a state in which the voice recognition execution history is outputted.

For example, the first and second objects, and messages received from the external terminal, may be output through the user input unit. As another example, the user can input the content of the reply "Answer" by using a voice command "Send Answer ".

Next, the step of outputting at least one of the first object, the second object, the message, and the reply in chronological order may be performed. That is, the voice recognition performance history may include at least one of a first object, a second object, a message, and a reply. Various functions using the voice recognition mode can be executed in a state in which the voice recognition performance history is outputted. In addition, since at least one of the first object, the second object, the message, and the reply is output as the speech recognition performance history, user convenience can be increased.

FIG. 9 and FIG. 10 are views illustrating an example of a mobile terminal that outputs a voice recognition performance history according to an embodiment of the present invention.

9, a mobile terminal 100 according to the present invention includes a first object 410, a second object 420, and a display unit 155 for outputting a message 430 received from an external terminal . Quot; exhaustion "means a user of an external terminal, and" discipline "may mean a user of the mobile terminal 100 according to the present invention. It is possible to output information related to the user by using the information stored in the contact or the like.

Although not shown in the figure, it is assumed that the result information generated in response to a voice command is transmitted to an external terminal by a message transmission command. When the transmission of the message is completed in accordance with the message transmission command, a confirmation message such as "the map information has been transmitted to the exhaustion"

If a message is received from an external terminal during execution of the voice recognition mode, the message can be output to the display unit 155 as a voice recognition performance history. At this time, the messenger information used to receive the message may be output together with the message. For example, referring to FIG. 9, "exhaustion, how long? And at least one of the name of the icon and the name of the messenger provided from the messenger may be output. Such a messenger may be stored in the memory 160 and may be a predetermined messenger.

Thereafter, a voice command of "Reply 5 minutes question mark" The voice command processing unit 182 may generate a reply in response to the voice command, and may transmit the generated reply to the external terminal. In another example, a virtual keyboard is output to the display unit 155, and the controller 180 generates a reply based on the touch input corresponding to the keyboard, and transmits the reply to the external terminal.

The controller 180 may output at least one of the first object 410, the second object 420, the message 430 received from the external terminal, and the reply 440 in chronological order .

10 is an exemplary view illustrating a mobile terminal according to an embodiment of the present invention.

10, a mobile terminal 100 according to the present invention includes a display unit 155 that outputs a first object 510 representing a voice command and a second object 520 generated in response to a voice command can do.

While the voice recognition performance history is being output, the microphone 122 can sense a new voice command 200. [ The voice command processing unit 182 can transmit the result information corresponding to the second object 520 to the external terminal in response to the voice command "Send to family group" (200). At this time, the external terminal may be at least one.

After transmitting the result information corresponding to the second object 520, the control unit 180 may output the fact to the display unit 155 as a voice recognition performance history. Referring to FIG. 10, the family group includes contacts "Dad", "Mom", and "Sister", and the fact that the result information related to the location is transmitted to the contacts is output to the second object 520 . Also, it can be confirmed that the name of the preset messenger is "text message ".

11 is a diagram illustrating an example of a mobile terminal according to an embodiment of the present invention. Referring to FIG. 11, the mobile terminal 100 may include a display unit 155 for outputting a first object 610 indicating a voice command.

The voice command processing unit 182 can analyze the voice command sensed by using the microphone 122. At this time, if the result information generated in response to the voice command needs to use the map-related application, the map-related applications stored in the memory 160 can be searched.

The voice command processing unit 182 may generate the result information using the preset application if there is a preset application. However, if a predetermined application does not exist and a plurality of applications related to the result information exist, the information related to the plurality of pieces of information 620 may be output to the display unit 155. Then, based on the user input, at least one of the plurality can be selected and the result information can be generated using the selected application.

Referring to FIG. 11, there are a total of three map-related applications, and the user can confirm that he selected a map application called "NAVER ". Accordingly, the voice command processing unit 182 can generate the result information using the map application "NAVER ". The control unit 180 may output the second object 630 indicating at least a part of the result information to the display unit 155 together with the information about the application. At this time, the second object 630 may include link information for the application that generated the result information.

In addition, a touch input to the second object 630 may be sensed by the user input unit 130. In response to the touch input, the control unit 180 may output the result information using the application to the display unit 155 instead of the voice recognition execution history. The link information may include location information to be output using the application as well as information about the related application. Accordingly, the control unit 180 can output the result information in more detail than the second object 630 using the related application.

The mobile terminal 100 according to the present invention may output an icon 650 including link information on the history of voice recognition performance together with the result information in displaying the result information using the related application. At this time, when the user input to the icon 650 is detected, the controller 180 may output the voice recognition performance history again.

If the voice recognition mode is being executed, the icon 650 can be continuously output to the display unit 155. [ This icon 650 may be repositioned by a user input such as dragging and the voice recognition mode may be terminated using the icon 650. [ If the voice command is detected by the microphone 122 while the icon 650 is being output, the control unit 180 may output the voice recognition performance history to the display unit 155. [

Accordingly, the mobile terminal 100 according to the present invention can multitask various applications using the icon 650 during the voice recognition mode. Further, the icon 650 can be used to control the display of the voice recognition performance history whenever necessary. Thus, user convenience can be increased.

12 is a diagram for explaining a mobile terminal for changing a predetermined messenger according to an embodiment of the present invention

Referring to FIG. 12, the mobile terminal 100 according to the present invention may include a display unit 155 for outputting a menu related to setting of a voice recognition mode. In particular, the menu 710 related to whether or not the messenger is interlocked may be output. For example, if it is not checked whether the messenger is connected, the messenger application can not be executed in order to send and receive a message in a state in which the speech recognition execution history is outputted. Alternatively, when the presence or absence of the messenger is checked, the message can be transmitted and received using the predetermined messenger in a state in which the voice recognition execution history is output.

In order to change the preset messenger, the control unit 180 may first search the messengers stored in the memory 160. [ The retrieved messengers can be output to the display unit 155 in the form of a list. Thereafter, at least one of the plurality of messengers may be selected based on the user input, and the selected messenger may be reset to the predetermined messenger.

13 is a diagram illustrating an example of a mobile terminal for outputting a voice recognition performance history according to an embodiment of the present invention.

Referring to FIG. 13, the mobile terminal 100 according to the present invention may include a display unit 155 for outputting voice recognition performance histories. The voice recognition performance history may be stored in the memory 160 in order of time from the start time of the voice recognition mode to the end time. At least one of the voice recognition performance histories may be stored in the memory 160 and output to the display unit 155 in a list format.

The control unit 180 outputs a list of the voice recognition performance histories and may output the details of the voice recognition performance history to the display unit 155 when the user input is detected. The details may be the contents in which the histories performed from the start time to the end time of the voice recognition mode are outputted in chronological order.

For example, referring to FIG. 13, a touch input for "2012.10.4 ~ 5" may be sensed. In this case, the control unit 180 can output the voice recognition performance histories according to the execution of the voice recognition mode from the "October 4, 2012" to the "October 5, 2012" to the display unit 155 in chronological order have. At this time, the controller 180 can use the information stored in the memory 160. [

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention.

Claims (15)

A microphone for receiving a voice command when the voice recognition mode is executed;
A voice command processing unit for generating result information in response to the voice command;
A display unit for outputting a first object representing the voice command and a second object representing at least a part of the result information in chronological order;
A user input unit configured to detect a touch input to the display unit; And
And a controller for transmitting the result information to an external terminal based on the message transmission command when a message transmission command for the second object is sensed. The apparatus of claim 1,
And outputs a message received from the external terminal in time sequence with the first and second objects while the voice recognition mode is being executed. 3. The apparatus of claim 2,
A response to the message based on at least one of the voice command and the touch input to the external terminal and outputs at least one of the first object, the second object, the message, and the reply in chronological order Wherein the mobile terminal is a mobile terminal. The method of claim 3,
Further comprising a memory for storing a voice recognition performance history including at least one of the first object, the second object, the message, and the reply. 5. The method of claim 4, wherein at least one of the second object,
And is transmitted / received by a predetermined messenger. 6. The method of claim 5,
The memory stores a plurality of messengers,
Wherein the controller sets any one of the plurality of messengers as the predetermined messenger when the one of the plurality of messengers is selected. The voice communication device according to claim 1,
Selects an associated application based on the voice command, and generates the result information using the application. 8. The apparatus of claim 7,
And outputs the second object together with information on the application. 9. The apparatus according to claim 8,
And outputs the result information using the application instead of the first and second objects when a touch input to the second object is detected. The method of claim 1,
The microphone, and the user input unit. Executing a voice recognition mode for providing a voice recognition function;
Receiving a voice command using a microphone when the voice recognition mode is executed;
In response to the voice command, generating result information;
Outputting a first object representing the voice command and a second object representing at least a part of the result information to the display unit in chronological order;
Detecting a message transmission command for the second object; And
And transmitting the result information to an external terminal based on the message transmission command. 12. The method of claim 11,
Receiving a message from the external terminal during execution of the voice recognition mode; And
And outputting the message to the display unit in time order with the first and second objects. 13. The method of claim 12,
Generating a reply to the message based on at least one of the voice command and the touch input;
Transmitting the reply to an external terminal; And
And outputting at least one of the first object, the second object, the message, and the reply in chronological order to the display unit. 14. The method of claim 13,
Further comprising: storing, in a memory, a voice recognition performance history including at least one of the first object, the second object, the message, and the reply. 12. The method of claim 11, wherein generating the result information comprises:
Selecting an associated application based on the voice command; And
And generating the result information by using the application.

Images