KR20140055302A - Mobile terminal and method for controlling the same - Google Patents

Abstract

A mobile terminal according to the present invention includes: a display unit; and a control unit which recognizes a voice on a call, extracts a preset word among the words of the recognized voice, automatically searches information corresponding to the keyword on the call or after the call is finished, and displays the searched information on the display unit.

Description

[0001] MOBILE TERMINAL AND METHOD FOR CONTROLLING THE SAME [0002]

The present invention relates to a mobile terminal and a control method thereof for allowing an application preset by a user at the time of unlocking to be executed immediately.

A terminal can be divided into a mobile terminal (mobile / portable terminal) and a stationary terminal according to whether the terminal can be moved. The mobile terminal can be divided into a handheld terminal and a vehicle mount terminal according to whether the user can directly carry the mobile terminal.

Such a terminal has various functions, for example, in the form of a multimedia device having multiple functions such as photographing and photographing of a moving picture, reproduction of a music or video file, reception of a game and broadcasting, etc. .

In order to support and enhance the functionality of such terminals, it may be considered to improve the structural and / or software parts of the terminal.

When the mobile terminal requires information on the contents of the conversation during the conversation and needs to search for the conversation, the user must memorize the contents of the conversation after the conversation is terminated and search for the words requiring the information separately. In order to do this, the user must separately memorize the contents of the conversation, or execute a wireless Internet of the mobile terminal separately or input specific words using another terminal.

SUMMARY OF THE INVENTION The present invention has been proposed in order to satisfy the above-mentioned need, and it is an object of the present invention to enable speech recognition during a call and automatic extraction of extracted keywords.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, unless further departing from the spirit and scope of the invention as defined by the appended claims. It will be possible.

According to an aspect of the present invention, there is provided a mobile terminal including: a display; And extracting a predetermined word from the recognized vocabulary words as a keyword, automatically searching for information corresponding to the keyword during or after a call is terminated, and transmitting the retrieved information to the display unit And a control unit for controlling the display unit.

In addition, the mobile terminal control method according to the present invention includes: recognizing a voice during a call; Extracting a predetermined word from among the words of the recognized voice as a keyword; Automatically searching for information corresponding to the keyword during or after a call; And displaying the searched information.

In the mobile terminal and the control method thereof according to the present invention, keywords are automatically extracted from the conversation contents without requiring the user to separately memorize the contents of the conversation, and the retrieval can be performed on the extracted keywords.

The effects obtained by the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art from the following description will be.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.
2 is a front perspective view of a mobile terminal according to an embodiment of the present invention.
3 is a rear perspective view of a mobile terminal according to an embodiment of the present invention.
4A is a flowchart illustrating a method of controlling a mobile terminal according to an embodiment of the present invention.
4B to 4D are views showing a display unit of a mobile terminal according to an embodiment of the present invention.
FIGS. 5 to 9 are flowcharts illustrating a method of extracting a predetermined word among keywords of voice recognized during a call in a mobile terminal according to an exemplary embodiment.
10 is a flowchart illustrating a method of searching for extracted keywords among words of voice recognized during a call in a mobile terminal.
FIG. 11 is a flowchart illustrating a method of calculating a location of a mobile terminal in a mobile terminal according to an exemplary embodiment of the present invention, and searching for extracted keywords among words of voice recognized during a call.
12 is a diagram illustrating a keyword search result according to a location of a mobile terminal.
13A and 13B illustrate outputting search results for keywords extracted during a video call in a mobile terminal according to an exemplary embodiment of the present invention.

Hereinafter, a mobile terminal related to the present invention will be described in detail with reference to the drawings. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

The mobile terminal described in this specification may include a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), and navigation. However, it will be understood by those skilled in the art that the configuration according to the embodiments described herein may be applied to a fixed terminal such as a digital TV, a desktop computer, and the like, unless the configuration is applicable only to a mobile terminal.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.

The mobile terminal 100 includes a wireless communication unit 110, an audio / video input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, A controller 170, a controller 180, a power supply 190, and the like. The components shown in FIG. 1 are not essential, and a mobile terminal having more or fewer components may be implemented.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more modules for enabling wireless communication between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and the network in which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives broadcast signals and / or broadcast-related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. Two or more broadcast receiving modules may be provided to the mobile terminal 100 for simultaneous broadcast reception or broadcast channel switching for at least two broadcast channels.

The broadcast management server may refer to a server for generating and transmitting broadcast signals and / or broadcast related information, or a server for receiving broadcast signals and / or broadcast related information generated by the broadcast management server and transmitting the generated broadcast signals and / or broadcast related information. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

The broadcast-related information means information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast-related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast-related information may exist in various forms. For example, an EPG (Electronic Program Guide) of DMB (Digital Multimedia Broadcasting) or an ESG (Electronic Service Guide) of Digital Video Broadcast-Handheld (DVB-H).

For example, the broadcast receiving module 111 may be a Digital Multimedia Broadcasting-Terrestrial (DMB-T), a Digital Multimedia Broadcasting-Satellite (DMB-S), a Media Forward Link Only (Mobile Broadcasting Business Management System), ISDB-T (Integrated Services Digital Broadcasting (ISDB-T)), Digital Multimedia Broadcasting (MBMS) Digital Broadcast-Terrestrial) or the like. Of course, the broadcast receiving module 111 may be adapted to other broadcasting systems as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 may be coupled to a base station, an external terminal, or a server on a mobile communication network, such as, but not limited to, Gobal System for Mobile communications (GSM), Code Division Multiple Access (CDMA), Wideband CDMA Transmits and receives wireless signals with one. The wireless signal may include various types of data depending on a voice call signal, a video call signal or a text / multimedia message transmission / reception.

The wireless Internet module 113 is a module for wireless Internet access, and may be built in or externally attached to the mobile terminal 100. Wireless Internet technologies include WLAN (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), GSM, CDMA, WCDMA, LTE Evolution (but not limited to) may be used.

The wireless Internet module 113, which performs wireless Internet access through the mobile communication network, is connected to the mobile communication module 110 through the mobile communication network, for example, from the viewpoint that the wireless Internet access by Wibro, HSDPA, GSM, CDMA, WCDMA, LTE, (112). ≪ / RTI >

The short-range communication module 114 refers to a module for short-range communication. Bluetooth, Radio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), ZigBee, and the like can be used as a short range communication technology.

The position information module 115 is a module for obtaining the position of the mobile terminal, and a representative example thereof is a Global Position System (GPS) module. According to the current technology, the GPS module 115 calculates distance information and accurate time information from three or more satellites, and then applies trigonometry to the calculated information to obtain a three-dimensional string of latitude, longitude, The location information can be accurately calculated. At present, a method of calculating position and time information using three satellites and correcting an error of the calculated position and time information using another satellite is widely used. In addition, the GPS module 115 can calculate speed information by continuously calculating the current position in real time.

Referring to FIG. 1, an A / V (Audio / Video) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video communication mode or the photographing mode. The processed image frame can be displayed on the display unit 151. [

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. [ Two or more cameras 121 may be provided depending on the use environment.

The microphone 122 receives an external sound signal through a microphone in a communication mode, a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 when the voice data is in the call mode, and output. Various noise reduction algorithms may be implemented in the microphone 122 to remove noise generated in receiving an external sound signal.

The user input unit 130 generates input data for a user to control the operation of the terminal. The user input unit 130 may include a button 136 located on the front, rear, or side of the mobile terminal 100, and a touch sensor (static / static) 137. Although not shown, ), A dome switch, a jog wheel, a jog switch, and the like.

The sensing unit 140 senses the current state of the mobile terminal 100 such as the open / close state of the mobile terminal 100, the position of the mobile terminal 100, the presence or absence of user contact, the orientation of the mobile terminal, And generates a sensing signal for controlling the operation of the mobile terminal 100. For example, when the mobile terminal 100 is in the form of a slide phone, it is possible to sense whether the slide phone is opened or closed. It is also possible to sense whether the power supply unit 190 is powered on, whether the interface unit 170 is connected to an external device, and the like. Meanwhile, the sensing unit 140 may include a proximity sensor 141. (The proximity sensor will be described later).

The output unit 150 is for generating output related to the visual, auditory or tactile sense and includes a display unit 151, an audio output module 152, an alarm unit 153, and a haptic module 154 .

The display unit 151 displays (outputs) information processed by the mobile terminal 100. For example, when the mobile terminal is in the call mode, a UI (User Interface) or a GUI (Graphic User Interface) associated with a call is displayed. When the mobile terminal 100 is in the video communication mode or the photographing mode, the photographed and / or received video or UI and GUI are displayed.

The display unit 151 may be a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display display, and a 3D display.

Some of these displays may be transparent or light transmissive so that they can be seen through. This can be referred to as a transparent display, and a typical example of the transparent display is TOLED (Transparent OLED) and the like. The rear structure of the display unit 151 may also be of a light transmission type. With this structure, the user can see an object located behind the terminal body through the area occupied by the display unit 151 of the terminal body.

There may be two or more display units 151 according to the embodiment of the mobile terminal 100. For example, in the mobile terminal 100, a plurality of display portions may be spaced apart from one another, or may be disposed integrally with one another, and may be disposed on different surfaces, respectively.

In the case where the display unit 151 and the touch sensor 137 have a mutual layer structure or are integrally formed (hereinafter referred to as a 'touch screen'), the display unit 151 may be used as an input device . The touch sensor may be stacked on the display unit 151 to form a layer structure when the touch sensor has a form of a touch film, a touch sheet, a touch pad, or the like. ≪ / RTI >

The touch sensor may be configured to convert a change in a pressure applied to a specific portion of the display unit 151 or a capacitance generated in a specific portion of the display unit 151 into an electrical input signal. The touch sensor can be configured to detect not only the position and area to be touched but also the pressure at the time of touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller (not shown). The touch controller processes the signal (s) and transmits the corresponding data to the controller 180. Thus, the control unit 180 can know which area of the display unit 151 is touched or the like.

The proximity sensor 141 may be disposed in the inner area of the mobile terminal 100 or in the vicinity of the touch screen, which is enclosed by the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or a nearby object without mechanical contact using the force of an electromagnetic field or infrared rays. The proximity sensor has a longer life span than the contact sensor and its utilization is also high.

Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. And to detect the proximity of the pointer by the change of the electric field along the proximity of the pointer when the touch screen is electrostatic. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of recognizing that the pointer is positioned on the touch screen while the pointer is not in contact with the touch screen is referred to as "proximity touch" The act of actually contacting the pointer on the touch screen may be referred to as "contact touch. &Quot; The location where the pointer is proximately touched on the touch screen may refer to a position where the pointer corresponds vertically to the touch screen when the pointer is touched.

The proximity sensor detects a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, a proximity touch movement state, and the like). Information corresponding to the detected proximity touch operation and the proximity touch pattern may be output on the touch screen.

The audio output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output module 152 also outputs sound signals related to functions (e.g., call signal reception sound, message reception sound, etc.) performed in the mobile terminal 100. [ The audio output module 152 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event of the mobile terminal 100. Examples of events that occur in the mobile terminal include call signal reception, message reception, key signal input, touch input, and the like. The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than the video signal or the audio signal, for example, vibration. In this case, the display unit 151 and the audio output module 152 may be a type of the alarm unit 153. The display unit 151 and the audio output module 152 may be connected to the display unit 151 or the audio output module 152, .

The haptic module 154 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 154 is vibration. The intensity and pattern of the vibration generated by the hit module 154 can be controlled. For example, different vibrations may be synthesized and output or sequentially output.

In addition to the vibration, the haptic module 154 may include a pin arrangement vertically moving with respect to the contact skin surface, a spraying force or suction force of the air through the injection port or the suction port, a touch on the skin surface, contact with an electrode, And various tactile effects such as an effect of reproducing a cold sensation using an endothermic or exothermic element can be generated.

The haptic module 154 can be implemented not only to transmit the tactile effect through the direct contact but also to allow the user to feel the tactile effect through the muscular sensation of the finger or arm. At least two haptic modules 154 may be provided according to the configuration of the mobile terminal 100.

The memory unit 160 may store a program for processing and controlling the control unit 180 and temporarily store the input / output data (e.g., telephone directory, message, audio, For example. The memory unit 160 may store the frequency of use of each of the data (for example, each telephone number, each message, and frequency of use for each multimedia).

In addition, the memory unit 160 may store data on vibration and sound of various patterns output when the touch is input on the touch screen.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), a RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM A disk, and / or an optical disk. The mobile terminal 100 may operate in association with a web storage that performs a storage function of the memory 160 on the Internet.

The interface unit 170 serves as a path for communication with all external devices connected to the mobile terminal 100. The interface unit 170 receives data from an external device or supplies power to each component in the mobile terminal 100 or transmits data to the external device. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, A video input / output (I / O) port, an earphone port, and the like may be included in the interface unit 170.

The identification module is a chip for storing various information for authenticating the use right of the mobile terminal 100 and includes a user identification module (UIM), a subscriber identity module (SIM), a general user authentication module A Universal Subscriber Identity Module (USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the port.

When the mobile terminal 100 is connected to an external cradle, the interface unit may be a path through which power from the cradle is supplied to the mobile terminal 100, or various command signals input by the user to the cradle may be transmitted It can be a passage to be transmitted to the terminal. The various command signals or the power source input from the cradle may be operated as a signal for recognizing that the mobile terminal is correctly mounted on the cradle.

The controller 180 typically controls the overall operation of the mobile terminal. For example, voice communication, data communication, video communication, and the like. The control unit 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180. [

The controller 180 may perform a pattern recognition process for recognizing handwriting input or drawing input performed on the touch screen as characters and images, respectively.

The voice recognition unit 182 drives the voice recognition engine to which the voice recognition algorithm is applied to recognize the voice of the user input through the microphone 122 or the voice of the other party received through the wireless communication unit 110. [

That is, the voice recognition unit 182 converts the voice of the user input through the microphone 122 or the voice of the other party received through the wireless communication unit 110 into digital data, pre-emphasizes the digital data, A start point and an end point of the digitally converted voice are detected.

Then, the voice recognition unit 182 extracts voice feature values for the voice between the detected start point and end point, and matches the extracted voice feature values in the voice recognition database provided in the memory 160 To the control unit 180. [0051] At this time, the voice feature values may include at least one of a waveform of the input voice, a format of voice, and a pitch of voice.

The voice recognition unit 182 may be provided in the mobile terminal 100 in the form of one "module" or "unit", or may be provided in the memory 160 in the form of software. The speech recognition unit 182 may be included in the control unit 180 and the control unit 180 may perform the operation of the speech recognition unit 182 in the same manner.

The converting unit 183 can convert the voice of the user recognized by the voice recognizing unit 182 into a character and output it.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components. The power supply unit 190 may include, for example, a battery, a connection port, a power supply control unit, and a charge monitoring unit.

The battery may be an internal battery configured to be chargeable, and may be detachably coupled to the terminal body for charging or the like. The connection port may be configured as an example of an interface 170 through which an external charger for supplying power for charging the battery is electrically connected.

The various embodiments described herein may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein may be implemented as application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays , Microprocessors, microprocessors, microprocessors, and other electronic units for carrying out other functions. In some cases, the embodiments described herein may be implemented by the controller 180 itself.

According to a software implementation, embodiments such as the procedures and functions described herein may be implemented with separate software modules. Each of the software modules may perform one or more of the functions and operations described herein. Software code can be implemented in a software application written in a suitable programming language. The software code is stored in the memory 160 and can be executed by the control unit 180. [

FIG. 2 is a perspective view of a mobile terminal or a portable terminal according to the present invention, as viewed from the front.

The disclosed mobile terminal 100 includes a bar-shaped terminal body. However, the present invention is not limited thereto, and can be applied to various structures such as a slide type, a folder type, a swing type, and a swivel type in which two or more bodies are relatively movably coupled.

The body of the mobile terminal 100 includes a casing 101, 102, In this embodiment, the case may be divided into a front case 101 and a rear case 102. [ A variety of electronic components are embedded in the space formed between the front case 101 and the rear case 102.

A part such as a USIM card and a memory card, which is required to be detached, such as an auxiliary storage medium 165, may include an opening in the case so that the part can be inserted into the case from the outside. Slots may be formed on the side of the case so that the auxiliary storage medium 165 may be inserted from the side of the mobile terminal 100 and card slots 166 and 167 may be formed on the surface of the rear case 102 It is possible.

The cases 101, 102, and 103 may be formed by injection molding synthetic resin or may be formed of a metal material such as stainless steel (STS) or titanium (Ti).

The display unit 151, the sound output unit 152, the camera 121, the user input units 130/131 and 132, the microphone 122, and the interface 170 may be disposed in the terminal cases 101 and 102.

The display unit 151 occupies most of the main surface of the front case 101. A sound output unit 152 and a camera 121 are disposed in an area adjacent to one end of both ends of the display unit 151 and a user input unit 131 and a microphone 122 are disposed in an area adjacent to the other end. The user input unit 132 and the interface 170 may be disposed on the side surfaces of the front case 101 and the rear case 102. [

The user input unit 130 is operated to receive a command for controlling the operation of the mobile terminal 100 and may include a plurality of operation units 131, 132, and 133. The operation units 131, 132, and 133 may be collectively referred to as a manipulating portion.

The contents inputted by the first or second operation unit 131 or 132 may be variously set. For example, the first operation unit 131 receives a command such as start, end, scroll, and the like, and the second operation unit 132 inputs a command such as adjustment of the sound output from the sound output unit 152 And the third operation unit 133 can receive commands such as activation / deactivation of the touch recognition mode of the display unit 151 and the like.

In addition to the display unit 151, the operation units 131, 132, and 133 may be provided with touch sensors, It is also possible to receive a command of the user by touch alone.

3 is a rear perspective view of the mobile terminal shown in FIG.

Referring to FIG. 3, a camera 121 'may be further mounted on the rear surface of the terminal body, that is, the rear case 102. The camera 121 'may have a photographing direction substantially opposite to the camera 121 (see FIG. 2), and may be a camera having the same or different pixels as the camera 121.

For example, the camera 121 may have a low pixel so that the face of the user can be photographed and transmitted to the other party in case of a video call or the like, and the camera 121 ' It is preferable to have a large number of pixels. The cameras 121 and 121 'may be installed in the terminal body such that they can be rotated or pop-up.

A flash 123 and a mirror 124 may be additionally disposed adjacent to the camera 121 '. The flash 123 illuminates the subject when the subject is photographed by the camera 121 '. The mirror 124 allows the user to illuminate the user's own face or the like when the user intends to shoot himself / herself (self-photographing) using the camera 121 '.

An acoustic output 152 'may be additionally disposed on the rear surface of the terminal body. The sound output 152 'on the rear side of the body may implement the stereo function together with the sound output 152 (see FIG. 2) of the front side of the body, and may be used for the implementation of the speakerphone mode during a call.

In addition to the antenna for talking and the like, a broadcast signal reception antenna 116 may be additionally disposed on the side of the terminal body. The antenna 116, which forms part of the broadcast receiving module 111 (see FIG. 1), can be installed to be able to be drawn out from the terminal body.

For convenience of explanation, it is assumed that the mobile terminal 100 mentioned below may include at least one of the components shown in FIG.

Hereinafter, with reference to FIG. 4A to FIG. 13, a description will be given of a method for extracting a predetermined keyword among recognized words of a voice, do.

4A is a flowchart illustrating a method of controlling a mobile terminal according to an exemplary embodiment of the present invention, and FIGS. 4B through 4D illustrate a display unit of a mobile terminal according to an exemplary embodiment of the present invention.

Referring to FIG. 4A, the mobile terminal 100 related to an embodiment of the present invention may be connected to the counterpart terminal (S400). For example, when the mobile terminal 100 receives the call origination signal from the counterpart terminal, the controller 180 may control the call connection with the counterpart terminal in response to a user command. Alternatively, the control unit 180 may control the call connection with the counterpart terminal by transmitting a call origination signal to the counterpart terminal in response to a user command.

The control unit 180 controls the voice recognition unit 182 to control the voice recognition unit 182 so that the voice of the user input through the microphone 122 during the call or the voice of the other party received through the wireless communication unit 110 (S410).

According to the embodiment, the control unit 180 may start voice recognition by receiving a voice recognition start command of the user. The voice recognition start command may be generated by operating a specific hardware button provided on the mobile terminal 100 or a software button displayed on the display unit 151, a touch input, or a specific menu displayed on the display unit 151. [ When terminating the speech recognition, a speech recognition termination command may be input. For example, the voice recognition end command may be input by operating a specific hardware button, a software button, or a specific menu provided in the mobile terminal 100 during the voice recognition.

The voice recognition unit 182 receives voice through the microphone 122 and performs voice recognition. The voice recognition unit 182 may receive voice received during a call and perform voice recognition according to the voice when the call is performed with the other mobile terminal 200 through the wireless communication unit 110. [

At this time, the speech recognition unit 182 is equipped with a speech recognition algorithm, and extracts characteristics of the input speech and analyzes the speech to recognize the speech. Since the speech recognition algorithm itself recognizing speech in the present embodiment is outside the scope of the present invention, a detailed description thereof will be omitted.

According to the embodiment, the voice recognition unit 182 can convert the recognized voice into character data.

The control unit 180 may extract a predetermined word from among the words of the recognized voice (S420). The recognized voice includes a plurality of words, and may include not only words for which the user wishes to perform a voice search but also other words. Therefore, the control unit 180 can extract, from the recognized words, a predetermined word to be used for voice search in accordance with a predetermined method.

Thereafter, the controller 180 may retrieve information corresponding to the extracted keyword (S430). The search may proceed during a call or after a call is terminated. The information may be an advertisement for a product corresponding to a keyword, or may be location information for an area corresponding to a keyword.

Next, the control unit 180 may control to display the searched information on the display unit 151 (S440). 4B to 4D, if a keyword is extracted from the voice recognized during the call progress with the partner terminal as shown in FIG. 4B, as shown in FIG. 4C, The retrieved information corresponding to the keyword may be displayed on the display 151. At this time, the retrieved information may be displayed in the form of a separate pop-up window 152 as shown in FIG. 4D, the retrieved information may be displayed on the display unit 151 of the user's mobile terminal 100 after the call is terminated. In the illustrated example, a "pizza" during a call is extracted as a keyword and the search is progressed and a search result for "pizza" That is, information on an advertisement for a "pizza", a location of a selling restaurant, a telephone number, and the like may be provided to the user of the mobile terminal 100 during a call or after a call is terminated.

Hereinafter, a method of extracting a predetermined word among words of voice recognized during a call will be described in detail.

FIGS. 5 to 9 are flowcharts illustrating a method of extracting a predetermined word among keywords of voice recognized during a call in a mobile terminal according to an exemplary embodiment. The method of extracting the keywords illustrated in FIGS. 5 to 9 is not limited to each of them, and keyword extraction may be performed by a combination of these methods.

Referring to FIG. 5, the control unit 180 of the mobile terminal 100 according to an embodiment can recognize the voice by controlling the voice recognition unit 182 in operation S510. Thereafter, the control unit 180 can extract a predetermined word from the recognized voice words as a keyword. At this time, the extraction of the keyword may be performed according to a command of a predetermined user. That is, when a keyword extraction command preset by the user is input during the telephone conversation (S520), the control unit 180 determines whether the voice of the user input through the microphone 122 or the wireless communication unit 110 A word of the voice of the other party that is received through the keyword may be extracted as a keyword (S530).

For example, the predetermined user's keyword extraction command may be performed by a specific hardware button provided on the mobile terminal 100 during a call, a software button displayed on the display unit 151, a touch input, or the like.

Or the predetermined user's keyword extraction command may be a specific sound or sound. Certain sounds or sounds may include a kind of impact sound with a certain level or more, such as applause. Sounds or sounds above the specified level can be detected using a simple sound level detection algorithm. Alternatively, the predetermined user's keyword extraction command may be a voice command preset by the user. For example, the user can previously store an input of a voice of "keyword input" in the memory 160 as a keyword extraction command for voice search. A method of detecting a sound of a specific level or higher through the sound level detection algorithm and a method of storing a voice command in a memory are out of the scope of the present invention and a detailed description thereof will be omitted.

In the above description, a method of extracting keywords by a user's command has been described. However, the present embodiment is not limited to this, and a keyword can be extracted according to a predetermined keyword extraction condition even if there is no command from the user.

Hereinafter, a method of extracting keywords in accordance with a predetermined keyword extraction condition will be described.

Referring to FIG. 6, the control unit 180 of the mobile terminal 100 according to another embodiment can recognize the voice by controlling the voice recognition unit 182 during communication (S610). Thereafter, the control unit 180 may count the words of the voice recognized through the voice recognition unit 182 by the word how many times they are recognized during the call (S620). Thereafter, the controller 180 determines which word among the counted words is most frequently referred to during conversation, and extracts the most-referred word as the keyword of the recognized voice (S630).

According to an embodiment of the present invention, the recognized speech words are stored in the memory 160, and the controller 180 counts the words stored in the memory 160 by words to extract the words most frequently referred to as keywords .

At this time, the control unit 180 may control to display a pop-up window for checking whether a word extracted as a keyword corresponds to a keyword desired by the user, referring to the highest frequency. If the user corresponds to a word to be displayed in the pop-up window, the user can press the OK button to proceed to the next step. However, if the word displayed in the popup window is not a word to be searched by the user, the user can press the cancel button. In this case, the control unit 180 may extract a word most frequently referred to next to the word displayed in the pop-up window. Then, the extracted keyword is displayed on the pop-up window and the user can wait for confirmation.

In the above, the method of counting the number of recognized voices and extracting the most frequently used words as keywords is described.

Hereinafter, a method of extracting keywords by comparing the words of the recognized speech and the words registered in the database will be described.

Referring to FIG. 7, the mobile terminal 100 according to another embodiment may include a memory 160 storing a database in which keywords are pre-registered. That is, the user can pre-register the keywords that can be searched by speech recognition during the call in the database (S710).

For example, the user may pre-register the words of the product or the place of interest of interest to the database as keywords, and the database may be stored in the memory unit 160 of the mobile terminal 100 . That is, the user can register his / her favorite places, frequent foods, words related to his / her hobbies, items of interest, etc. as keywords in the database.

Thereafter, the control unit 180 recognizes the voice during the conversation with the other party (S720), and searches whether there is a word matching the keyword registered in the database among the recognized voice words S730). If there is a word matching the pre-registered keyword in the search result database, the control unit 180 can extract the word as a keyword (S740).

At this time, the control unit 180 may search the database and display a pop-up window for confirming whether the extracted word corresponds to a keyword desired by the user.

In the above, a method of extracting keywords by comparing words recognized in speech with words registered in a database has been described. However, according to the embodiment, the speech recognition words may include two or more words matched with words pre-registered as keywords in the database. Hereinafter, a method of extracting keywords when there are two or more words matched with words pre-registered as keywords in the database is described.

Referring to FIG. 8, similar to the embodiment shown in FIG. 7, the mobile terminal 100 may include a memory 160 in which a keyword stores a pre-registered database. That is, the user can pre-register the keywords that can be searched by speech recognition during the call in the database (S810). The control unit 180 recognizes the voice (S820) and searches whether there is a word matched to the keyword pre-registered in the database among the recognized voice words (S830) .

Thereafter, the control unit 180 may count the number of times the speech is recognized during the conversation, for each word, which is matched to the keyword pre-registered in the database among the words recognized through the speech recognition unit 182 (S840).

Thereafter, the control unit 180 may search for the word having the highest frequency among the words matching the keyword stored in the data base, and extract the word as a keyword (S850). At this time, the control unit 180 may control to display a pop-up window for checking whether a word extracted as a keyword corresponds to a keyword desired by the user, referring to the highest frequency.

According to the embodiment, the control unit 180 first counts how many times to mention the words in a call, then searches the database for words that match the words registered as keywords in the order of the most frequently mentioned words, May be extracted.

Although not shown, the database may be registered with priorities in the database according to an embodiment. That is, while the user registers the keyword in the database, it is possible to pre-register the word frequently searched by setting the priority to a high level in the database. In this case, when there are a plurality of words matching the words registered as keywords in the database among the words of the voice recognized during the conversation, the control unit 180 extracts words having a high priority in the database as keywords .

At this time, the controller 180 may control to display a pop-up window for confirming whether a word extracted as a keyword corresponds to a keyword desired by the user with the highest priority set. If the user corresponds to the keyword to be searched for in the pop-up window, the user can press the OK button to proceed to the next step. However, if the word displayed in the pop-up window is not a keyword to be searched by the user, the user can press the cancel button. In this case, the control unit 180 may extract a word having a priority level next to the word displayed in the pop-up window from among the words recognized as a keyword. Then, the extracted keyword is displayed on the pop-up window and the user can wait for confirmation.

The priority of the keyword registered in the database can be continuously changed and stored while searching through speech recognition. That is, if there is a word that has been searched for, the priority of the word may increase. The change of the priority may be made according to a command of the user.

In the above, a method of extracting keywords when there are two or more words that match the words pre-registered as keywords among the words recognized in the database is described.

However, according to the embodiment, any one of the words recognized as speech may not be matched with a keyword registered as a keyword in the database. Hereinafter, a method of extracting keywords when any one of the recognized words does not match a word registered as a keyword in the database will be described.

Referring to FIG. 9, when any one of the recognized words does not match a word registered as a keyword in the database, the controller 180 determines whether the word most recognized Words can be extracted as keywords. That is, the user may pre-register the keywords that are to be searched by speech recognition during the call and store them in the memory 160 (S910). The control unit 180 recognizes the voice (S920) and searches whether there is a word matching the keyword registered in the database among the recognized voice words (S930) . If there is a word that matches the keyword pre-registered in the database among the speech-recognized words, the control unit 180 can extract the matching word as a keyword (S940, S950).

However, if there is no word that matches the keyword pre-registered in the database among the voice-recognized words, the control unit 180 notifies the voice recognition unit 182 of the voice words recognized several times It is possible to count by word or word (S960). Thereafter, the control unit 180 determines which word among the counted words is most frequently referred to during conversation, and extracts the most-mentioned word as a keyword of the recognized voice (S970).

Although not shown, in this case, a word extracted by the keyword according to a command of the user can be registered as a keyword in a database and stored in a memory. By updating the database, when a user proceeds to a call in the future, the number of keywords that can be extracted by searching the database is increased, so that the voice search can be performed more quickly and conveniently.

In the above, a method of extracting keywords from recognized speech words was discussed.

Hereinafter, a method of searching for extracted keywords among recognized words of speech will be described in detail.

10 is a flowchart illustrating a method of searching for extracted keywords among words of voice recognized during a call in a mobile terminal.

The mobile terminal 100 according to the embodiment recognizes a voice during a call and extracts a predetermined word from the recognized voice words as a keyword as described above (S1010). Thereafter, the control unit 180 can automatically search for the information corresponding to the extracted keyword, either during a call or after the end of a call.

According to an embodiment, the mobile terminal 100 may include a database in which a keyword pre-registered by a user is stored in the memory 160. [ In this case, the linked information may be registered together with the keyword in the database of the memory 160 (S1020). That is, the keyword is pre-registered by the user in the database, and when the user registers the keyword, necessary information corresponding to the keyword may be registered together in the database and stored in the memory 160. For example, if a specific place is registered as a keyword, the user can link information on public transportation, a restaurant, a tourist spot, a historical place, and the like, which can go to the place, in advance in the database. Then, the control unit 180 may search the database during or after the call to obtain information corresponding to the keyword extracted from the recognized voice (S1030).

If the information corresponding to the extracted keyword is searched as a result of searching the database (S 1040), the controller 180 may output the searched result to the mobile terminal 100 (S 1060).

According to an embodiment, the database may not store information corresponding to the keyword. In this case, the controller 180 may search the information corresponding to the keyword by connecting the mobile terminal to the wireless Internet (S1050).

Specifically, the control unit 180 accesses a specific web site by using the wireless Internet module 113. Then, the retrieved keyword is retrieved from the retrieved web site through the connected web site. The particular web site may be preset by the user or may be further displayed with a window asking the user what web site to access before proceeding with the search. The mobile terminal 100 connects to the Internet server through the wireless Internet module 113 and wirelessly transmits the extracted keyword as a search word. The keyword transmitted from the user's mobile terminal 100 is transmitted to the Internet server, and the Internet server performs a predetermined information search process using the keyword as a search word. The Internet server transmits the information search result to the user's mobile terminal (100). The search result transmitted from the Internet server may be output through the display unit 151 of the mobile terminal 100 (S1060).

If the Internet search is being executed for the extracted keyword according to the embodiment, the controller 180 may display an indicator indicating that the Internet search is being executed in one area of the display unit 151. [ When the Internet search is completed, the control unit 180 may display an indicator indicating that the Internet search is completed in one area of the display unit 151. The user can view the indicator and know that the Internet search is currently in progress, and if he does not want to search the internet, he can stop the search.

In the above, a method of searching for extracted keywords among recognized words of speech has been described.

Hereinafter, a method of searching for extracted keywords from recognized voice words using position information of the mobile terminal will be described.

FIG. 11 is a flowchart illustrating a method of calculating a location of a mobile terminal in a mobile terminal according to an exemplary embodiment of the present invention, and searching for extracted keywords among words of voice recognized during a call. FIG. And FIG.

According to an embodiment, the mobile terminal 100 may further include a location information module 115. At this time, the controller 180 may calculate the location of the mobile terminal 100 during the call from the location information module 115, and may search for information corresponding to the extracted keyword and the calculated location.

Referring to FIG. 11, the mobile terminal 100 according to the embodiment recognizes a voice during a call and extracts a predetermined word from the recognized voice words as described above (S1110). The control unit 180 may calculate the position of the mobile terminal 100 during the call through the position information module 115 and store the calculated position in the memory 160 of the mobile terminal 100. [

At this time, the control unit 180 may utilize the position of the mobile terminal 100 calculated from the position information module 115 together with the extracted keyword for information search (S1130).

That is, the controller 180 can search for the extracted keyword and the calculated location together with a search word. For example, when a specific product is extracted as a keyword and a search is performed using a product name that is the extracted keyword, the control unit 180 can search for the calculated position of the current mobile terminal 100 together with a search term. In this case, information on where the merchandise is to be sold may be searched and provided to the user in the vicinity of the area where the current user is located.

12, when the mobile terminal 100 is located in area A, the controller 180 calculates the location of the mobile terminal 100 from the location information module 115 of the mobile terminal 100. In the case where the word "pizza" is extracted from the words of the recognized voice in the course of the call, the control unit 180 transmits information about the area A, which is the calculated position of the mobile terminal 100, And the search can proceed. Thereafter, the control unit 180 may display the search result on the display unit 151 as shown in FIG. In the figure, it is illustrated that information on a pizza outlet located in area A is displayed. When the mobile terminal 100 is located in area B, the controller 180 calculates the location of the mobile terminal 100, searches for information about the extracted keywords "pizza" and area B, On the display unit 151. [0064] FIG. In the figure, it is illustrated that information about a pizza outlet located in the area B is displayed.

By doing so, the user of the mobile terminal 100 can obtain the information corresponding to the extracted keyword in the vicinity of the user of the mobile terminal 100, thus making it possible to utilize the searched information more effectively.

In the above, a method of searching for extracted keywords among recognized words of speech has been described.

Hereinafter, a method of outputting search results for extracted keywords will be described.

The control unit 180 of the mobile terminal 100 according to the embodiment automatically searches for a predetermined keyword extracted from a word recognized in speech during a call or after the call ends. The control unit 180 may then control the display unit 151 of the mobile terminal 100 to display information corresponding to the searched keywords. For example, as shown in FIG. 4C or FIG. 4D, a search result for a keyword may be displayed on the display 151 during a call or displayed on the display 151 after a call is terminated, .

Alternatively, the control unit 180 may control the retrieved information to be stored in the memory 160. In this case, the user can search for and search the memory 160 at any time for necessary information at any time.

The retrieved information may be registered in the database of the memory 160 according to a user's command. For example, when a search for a keyword is performed through a wireless Internet search in FIG. 11, the search result may be linked to the database of the memory 160 together with the keyword and stored. In this case, if the same keyword is extracted while the user proceeds another call, desired information can be quickly obtained by searching the memory 160 without going through the wireless Internet search.

The retrieved information may be transmitted through the wireless communication unit 110 to the mobile terminal of the other party according to the embodiment. By doing so, the user and the other party can share information about the keywords extracted during the call.

According to the embodiment, when the user of the mobile terminal 100 proceeds with the video call with the other party, the user can share the busy information with the other party with respect to the search result.

13A and 13B illustrate outputting search results for keywords extracted during a video call in a mobile terminal according to an exemplary embodiment of the present invention.

13A, when the user proceeds with the video communication with the other party, the control unit 180 recognizes the voice of the user or the other party input through the microphone 122 or the wireless communication unit 110 during the video call, A predetermined word among the words of the recognized voice can be extracted as a keyword. Thereafter, the control unit 180 can search for information corresponding to the extracted keyword while proceeding with the video call.

13B, the control unit 180 of the mobile terminal 100 may transmit the searched information to the other party of the video call instead of the user's video. The mobile terminal 200 of the other party can receive the retrieved information instead of the user image and display the information on the display unit. In the figure, it is shown that "pizza" is extracted as a keyword, information about "pizza" is searched, and the search result is transmitted to the counterpart mobile terminal 200 instead of the user image.

By doing so, the user can share the retrieved information with the other party while continuing the video call so that the user can continue the conversation desired by the user without further communication.

Further, according to an embodiment of the present invention, the above-described method can be implemented as computer-readable code on a medium on which a program is recorded. The computer readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of the computer-readable medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like, and may be implemented in the form of a carrier wave (for example, transmission over the Internet) . In addition, the computer may include a control unit 180 of the terminal.

It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.

Accordingly, the above description should not be construed in a limiting sense in all respects and should be considered illustrative. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the scope of equivalents of the present invention are included in the scope of the present invention.

100: mobile terminal 110: wireless communication unit
120: A / V input / output unit 130: user input unit
140: sensing unit 150: output unit
160: memory 170: interface section
180: control unit 190: power supply unit

Claims (6)

A display unit; And
The method comprising the steps of: recognizing a voice in a call; extracting a predetermined word from the recognized voice words as a keyword; automatically searching for information corresponding to the keyword during or after a call is terminated; And a control unit for controlling the mobile terminal. The information processing apparatus according to claim 1, further comprising a memory unit for storing a database in which keywords are pre-registered, wherein the control unit searches the database and extracts a word matched to the pre-registered keyword among the words as the keyword . The mobile terminal of claim 1, wherein the controller extracts a word having the highest frequency among the words as a keyword. The mobile terminal according to claim 1, further comprising a location information module, wherein the controller calculates a location of the mobile terminal from the location information module, and searches the information corresponding to the keyword and the calculated location terminal. The mobile terminal of claim 1, wherein the call is a video call, and when the information is searched during the call, the control unit transmits the searched information instead of the user image to the other party of the video call. Recognizing a voice in a call;
Extracting a predetermined word from among the words of the recognized voice as a keyword;
Automatically searching for information corresponding to the keyword during or after a call; And
And displaying the searched information.

Images