KR20110130751A - Mobile terminal and method for controlling the same - Google Patents

Abstract

PURPOSE: A mobile terminal and a control method thereof are provided to output information which is registered in the voice of a inputted receiver if the voice of a receiver is inputted to a mobile terminal. CONSTITUTION: A memory(160) stores data which is mapped with a voice information. If the voice of a specific receiver is recognized, a control unit(180) searches the voice information corresponding to the recognized voice from the memory. The control unit output data which is mapped to the searched voice information through at least one between a display unit and sound output unit. The voice information and the data is stored in the address information corresponding to the receiver in the phonebook which is included in the memory.

Description

MOBILE TERMINAL AND METHOD FOR CONTROLLING THE SAME

The present invention relates to a mobile terminal and a method of controlling the same so that user's convenience can be further considered to enable the use of the terminal.

The terminal can move And can be divided into a mobile / portable terminal and a stationary terminal depending on whether the mobile terminal is a mobile terminal or a mobile terminal. The mobile terminal may be further classified into a handheld terminal and a vehicle mount terminal according to whether a user can directly carry it.

Such a terminal has various functions, for example, in the form of a multimedia device having multiple functions such as photographing and photographing of a moving picture, reproduction of a music or video file, reception of a game and broadcasting, etc. .

In order to support and increase the function of such a terminal, it may be considered to improve the structural part and / or the software part of the terminal.

An object of the present invention, when the voice information of the other party and the information to be delivered to the other party is registered in the mobile terminal, when the voice of the other party is input to the mobile terminal, to output information registered to the voice of the input party A mobile terminal and a control method thereof are provided.

A mobile terminal according to the present invention for achieving the above object, a microphone; A wireless communication unit; A display unit; A sound output unit; A memory in which at least one voice information and data mapped with the voice information are stored; A controller for retrieving voice information corresponding to the recognized voice from the memory and outputting data mapped to the retrieved voice information through one of the display unit and the audio output unit when a voice of a specific counterpart is recognized; It is made to include.

In this case, the voice information and the data may be stored in contact information corresponding to the counterpart in the phone book provided in the memory.

The control unit may transmit the data to the contact point of the counterpart through the wireless communication unit.

The voice information may be at least one of a voice of a counterpart received during a call, a voice input through the microphone, a voice registered on a web page, and a voice included in a received message.

In addition, the data may be memo content associated with the counterpart.

The recognized counterpart voice may be a voice received from the terminal of the counterpart via the wireless communication unit during a call, and the controller may output the data while maintaining a call with the counterpart terminal. In this case, the data is data including sound, and the controller controls the sound output unit to output the sound while maintaining the call, and transmits the output sound to the counterpart terminal by controlling the wireless communication unit. Can be blocked. In addition, the call is a video call, the data is data containing an image, the control unit displays the image on the video call screen of the display unit, and controls the wireless communication unit of the displayed image to the counterpart terminal You can block the transmission.

Also, the recognized counterpart's voice may be a voice input from the microphone.

In addition, the recognized voice of the other party may be a voice included in a message received from the wireless communication unit.

In addition, the recognized other party's voice may be the other party's voice registered in the web page connected through the wireless communication unit.

In addition, the control method of the mobile terminal according to the present invention includes the steps of obtaining voice information of the other party; If data to be mapped to the obtained voice information is set, mapping the voice information and the data and storing the data in a memory; If the voice of the other party is input, retrieving data mapped to the voice of the other party from the memory; And outputting the retrieved data.

In the mobile terminal and its control method according to the present invention, after registering the voice of the other party and the information to be transmitted to the other party in advance, if the voice of the other party is recognized by the mobile terminal, by notifying the user of the information registered in the other party's voice, There is an effect that the user can inform the other party the information that he wants to deliver.

1 is a block diagram illustrating a mobile terminal according to an embodiment of the present invention.
2A is a front perspective view of an example of a mobile terminal according to the present invention;
FIG. 2B is a rear perspective view of the mobile terminal shown in FIG. 2A.
3 is a flowchart illustrating a process of setting the voice of the other party and data to be mapped to the voice in the mobile terminal according to the present invention.
4 to 7 are screen state diagrams illustrating a process of setting the voice of the other party and data to be mapped to the voice in the mobile terminal according to the present invention.
8 is a flowchart illustrating a process of outputting the voice recognition of the other party and data mapped to the recognized voice in the mobile terminal according to the present invention.
9 to 12 are screen state diagrams illustrating a process of outputting, by the mobile terminal, voice recognition of the other party and data mapped to the recognized voice.

Hereinafter, a mobile terminal according to the present invention will be described in more detail with reference to the accompanying drawings. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

The mobile terminal described herein may include a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), navigation, and the like. However, it will be readily apparent to those skilled in the art that the configuration according to the embodiments described herein may also be applied to fixed terminals such as digital TVs, desktop computers, etc., except when applicable only to mobile terminals.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.

The mobile terminal 100 includes a wireless communication unit 110, an A / V input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, and an interface. The unit 170, the controller 180, and the power supply unit 190 may be included. The components shown in FIG. 1 are not essential, so that a mobile terminal having more or fewer components may be implemented.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more modules that enable wireless communication between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and a network in which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives a broadcast signal and / or broadcast related information from an external broadcast management server through a broadcast channel.

The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may mean a server that generates and transmits a broadcast signal and / or broadcast related information or a server that receives a previously generated broadcast signal and / or broadcast related information and transmits the same to a terminal. The broadcast signal may include not only a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, but also a broadcast signal having a data broadcast signal combined with a TV broadcast signal or a radio broadcast signal.

The broadcast related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast related information may exist in various forms. For example, it may exist in the form of Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld (DVB-H).

For example, the broadcast receiving module 111 may be a Digital Multimedia Broadcasting-Terrestrial (DMB-T), a Digital Multimedia Broadcasting-Satellite (DMB-S), a Media Forward Link Only A digital broadcasting system such as DVB-CB, OMA-BCAST, or Integrated Services Digital Broadcast-Terrestrial (ISDB-T). Of course, the broadcast receiving module 111 may be configured to be suitable for not only the above-described digital broadcasting system but also other broadcasting systems.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 transmits and receives a wireless signal with at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call call signal, or a text / multimedia message.

The wireless internet module 113 refers to a module for wireless internet access and may be embedded or external to the mobile terminal 100. Wireless Internet technologies may include Wireless LAN (Wi-Fi), Wireless Broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet Access (HSDPA), and the like.

The short range communication module 114 refers to a module for short range communication. As a short range communication technology, Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and the like may be used.

The location information module 115 is a module for obtaining a location of a mobile terminal, and a representative example thereof is a GPS (Global Position System) module. According to the current technology, the GPS module 115 calculates distance information and accurate time information from three or more satellites, and then applies trigonometry to the calculated information to obtain a three-dimensional string of latitude, longitude, The location information can be accurately calculated. At present, a method of calculating position and time information using three satellites and correcting an error of the calculated position and time information using another satellite is widely used. In addition, the GPS module 115 may calculate speed information by continuously calculating the current position in real time.

Referring to FIG. 1, the A / V input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video call mode or the photographing mode. The processed image frame may be displayed on the display unit 151.

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. Two or more cameras 121 may be provided according to the use environment.

The microphone 122 receives an external sound signal by a microphone in a call mode, a recording mode, a voice recognition mode, etc., and processes the external sound signal into electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 when the voice data is in the call mode, and output. The microphone 122 may implement various noise removing algorithms for removing noise generated in the process of receiving an external sound signal.

The user input unit 130 generates input data for the user to control the operation of the terminal. The user input unit 130 may include a key pad dome switch, a touch pad (static pressure / capacitance), a jog wheel, a jog switch, and the like.

The sensing unit 140 detects a current state of the mobile terminal 100 such as an open / closed state of the mobile terminal 100, a location of the mobile terminal 100, presence or absence of a user contact, orientation of the mobile terminal, acceleration / deceleration of the mobile terminal, and the like. To generate a sensing signal for controlling the operation of the mobile terminal 100. For example, when the mobile terminal 100 is in the form of a slide phone, it may sense whether the slide phone is opened or closed. In addition, whether the power supply unit 190 is supplied with power, whether the interface unit 170 is coupled to the external device may be sensed. The sensing unit 140 may include a proximity sensor 141. This will be described later with reference to the touch screen.

The output unit 150 is used to generate an output related to visual, auditory or tactile senses, which includes a display unit 151, an audio output module 152, an alarm unit 153, a haptic module 154, and a projector module ( 155) may be included.

The display unit 151 displays (outputs) information processed by the mobile terminal 100. For example, when the mobile terminal is in a call mode, the mobile terminal displays a user interface (UI) or a graphic user interface (GUI) related to the call. When the mobile terminal 100 is in a video call mode or a photographing mode, the mobile terminal 100 displays photographed and / or received images, a UI, and a GUI.

The display unit 151 includes a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display (flexible). and at least one of a 3D display.

Some of these displays can be configured to be transparent or light transmissive so that they can be seen from the outside. This may be referred to as a transparent display. A representative example of the transparent display is TOLED (Transparant OLED). The rear structure of the display unit 151 may also be configured as a light transmissive structure. With this structure, the user can see the object located behind the terminal body through the area occupied by the display unit 151 of the terminal body.

There may be two or more display units 151 according to the implementation form of the mobile terminal 100. For example, in the mobile terminal 100, a plurality of display portions may be spaced apart from one another, or may be disposed integrally with one another, and may be disposed on different surfaces, respectively.

When the display unit 151 and a sensor for detecting a touch operation (hereinafter, referred to as a touch sensor) form a mutual layer structure (hereinafter referred to as a touch screen), the display unit 151 may be configured in addition to an output device. Can also be used as an input device. The touch sensor may have, for example, a form of a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in pressure applied to a specific portion of the display unit 151 or capacitance generated in a specific portion of the display unit 151 into an electrical input signal. The touch sensor may be configured to detect not only the position and area of the touch but also the pressure at the touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to a touch controller (not shown). The touch controller processes the signal (s) and then transmits the corresponding data to the controller 180. As a result, the controller 180 can know which area of the display unit 151 is touched.

The proximity sensor 141 may be disposed in an inner region of the mobile terminal surrounded by the touch screen or near the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object present in the vicinity without using a mechanical contact by using an electromagnetic force or infrared rays. Proximity sensors have a longer life and higher utilization than touch sensors.

Examples of the proximity sensor include a transmission photoelectric sensor, a direct reflection photoelectric sensor, a mirror reflection photoelectric sensor, a high frequency oscillation proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. When the touch screen is capacitive, the touch screen is configured to detect the proximity of the pointer by the change of the electric field according to the proximity of the pointer. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of allowing the pointer to be recognized without being in contact with the touch screen so that the pointer is located on the touch screen is referred to as a "proximity touch", and the touch The act of actually touching the pointer on the screen is called "contact touch." The position where the proximity touch is performed by the pointer on the touch screen refers to a position where the pointer is perpendicular to the touch screen when the pointer is in proximity proximity.

The proximity sensor detects a proximity touch and a proximity touch pattern (for example, a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, and a proximity touch movement state). Information corresponding to the sensed proximity touch operation and proximity touch pattern may be output on the touch screen.

The sound output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. The sound output module 152 may also output a sound signal related to a function (eg, a call signal reception sound, a message reception sound, etc.) performed in the mobile terminal 100. The sound output module 152 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying occurrence of an event of the mobile terminal 100. Examples of events occurring in the mobile terminal include call signal reception, message reception, key signal input, and touch input. The alarm unit 153 may output a signal for notifying occurrence of an event in a form other than a video signal or an audio signal, for example, vibration. Since the video signal or the audio signal may be output through the display unit 151 or the audio output module 152, in this case, the display unit 151 and the audio output module 152 are a kind of alarm unit 153. It may be classified as.

The haptic module 154 generates various tactile effects that the user can feel. Vibration is a representative example of the haptic effect generated by the haptic module 154. The intensity and pattern of vibration generated by the haptic module 154 can be controlled. For example, different vibrations may be synthesized and output or may be sequentially output.

In addition to vibration, the haptic module 154 may be configured to provide a pin array that vertically moves with respect to the contact skin surface, a jetting force or suction force of air through an injection or inlet port, grazing to the skin surface, contact of an electrode, electrostatic force, and the like. Various tactile effects can be generated, such as effects by the endothermic and the reproduction of a sense of cold using the elements capable of endotherm or heat generation.

The haptic module 154 may not only deliver the haptic effect through direct contact, but also may implement the user to feel the haptic effect through a muscle sense such as a finger or an arm. Two or more haptic modules 154 may be provided according to configuration aspects of the mobile terminal 100.

The projector module 155 is a component for performing an image project function using the mobile terminal 100 and is similar to the image displayed on the display unit 151 in accordance with a control signal of the controller 180 Or at least partly display another image on an external screen or wall.

Specifically, the projector module 155 includes a light source (not shown) that generates light (for example, laser light) for outputting an image to the outside, a light source And a lens (not shown) for enlarging and outputting the image at a predetermined focal distance to the outside. Further, the projector module 155 may include a device (not shown) capable of mechanically moving the lens or the entire module to adjust the image projection direction.

The projector module 155 can be divided into a CRT (Cathode Ray Tube) module, an LCD (Liquid Crystal Display) module and a DLP (Digital Light Processing) module according to the type of the display means. In particular, the DLP module may be advantageous for miniaturization of the projector module 151 by enlarging and projecting an image generated by reflecting light generated from a light source on a DMD (Digital Micromirror Device) chip.

Preferably, the projector module 155 may be provided longitudinally on the side, front or back side of the mobile terminal 100. It goes without saying that the projector module 155 may be provided at any position of the mobile terminal 100 as occasion demands.

The memory 160 may store a program for processing and controlling the controller 180, and stores input / output data (for example, a phone book, a message, an audio, a still image, an e-book, a video, etc.). You can also perform a function for temporary storage. The memory unit 160 may also store the frequency of use of each of the data (for example, each telephone number, each message, and frequency of use for each multimedia). In addition, the memory unit 160 may store data regarding vibration and sound of various patterns output when a touch input on the touch screen is performed.

In addition, the memory 160 includes a phone book including a plurality of contact information including voice information of the other party and data mapped with the voice information.

The contact information is composed of voice information of the other party, an image of the other party, a telephone number, a fax number, an e-mail address, a blog address, and a homepage address.

In this case, the data may be content related to the counterpart set by the user, and the data may be stored in the counterpart contact information.

Meanwhile, the entire voice file in which the voice information of the other party is recorded may be stored in the contact information. Also, in order to increase the storage capacity of the memory 160, the voice recognition module 182 may be stored without storing the entire voice file. Only the voice feature vector value of the counterpart extracted through) may be stored.

As described above, the memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, an SD or XD memory). Etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EPEROM), programmable read-only memory (PROM), At least one type of storage medium may include a magnetic memory, a magnetic disk, and an optical disk. The mobile terminal 100 may operate in connection with a web storage that performs a storage function of the memory 160 on the Internet.

The interface unit 170 serves as a path with all external devices connected to the mobile terminal 100. The interface unit 170 receives data from an external device, receives power, transfers the power to each component inside the mobile terminal 100, or transmits data inside the mobile terminal 100 to an external device. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, A video input / output (I / O) port, an earphone port, and the like may be included in the interface unit 170.

The identification module is a chip that stores various types of information for authenticating the usage rights of the mobile terminal 100, and includes a user identification module (UIM), a subscriber identify module (SIM), and a universal user authentication module ( Universal Subscriber Identity Module (USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the port.

The interface unit may be a passage through which power from the cradle is supplied to the mobile terminal 100 when the mobile terminal 100 is connected to an external cradle, or various command signals input from the cradle by a user may be transferred. It may be a passage that is delivered to the terminal. Various command signals or power input from the cradle may be operated as signals for recognizing that the mobile terminal is correctly mounted on the cradle.

The controller 180 typically controls the overall operation of the mobile terminal. For example, perform related control and processing for voice calls, data communications, video calls, and the like. The controller 180 may include a multimedia module 181 for playing multimedia. The multimedia module 181 may be implemented in the controller 180 or may be implemented separately from the controller 180.

The controller 180 may perform a pattern recognition process for recognizing a writing input or a drawing input performed on the touch screen as text and an image, respectively.

The voice recognition module 182 recognizes the voice of the other party by driving the voice recognition engine to which the voice recognition algorithm is applied under the control of the controller 180.

At this time, the voice of the other party, the voice of the other party received through the wireless communication unit 110 during the call, the voice input through the microphone 122, registered on the web page connected through the wireless communication unit 110 The voice and the voice included in the message received through the wireless communication unit 110 may be at least one voice.

As described above, the voice recognition module 182 converts the other party's voice into digital data and amplifies the digital data.

Subsequently, the voice recognition module 182 extracts voice feature vector values for the digitally converted voice, and retrieves voice information corresponding to the extracted voice feature vector value from the counterpart voice information stored in the memory 160 to control the controller. (180).

In this case, the voice feature vector values may include at least one of a waveform of the voice, a format of the voice, and a pitch of the voice.

That is, the voice recognition module 182 recognizes the voice of the other party input through the microphone 122 by driving the voice recognition engine to which the voice recognition algorithm is applied under the control of the controller 180.

In addition, when the voice recognition command of the counterpart is input during the call with the counterpart, the voice recognition module 182 recognizes the call voice of the counterpart received through the wireless communication unit 110 under the control of the controller 180.

In addition, when the voice recognition command of the counterpart is input during the call with the counterpart, the voice recognition module 182 may recognize the call voice of the counterpart output from the sound output module 152 under the control of the controller 180. .

In addition, when the voice recognition command of the counterpart included in the currently playing music file is input, the voice recognition module 182 controls the counterpart in the music file output to the sound output module 152 under the control of the controller 180. You can also recognize voice.

In addition, when the voice recognition command of the other party included in the currently playing video file is input, the voice recognition module 182 controls the other party's voice in the video file output to the sound output module 152 under the control of the controller 180. Recognize your voice.

The voice recognition module 182 extracts the voice feature vector values of the voice of the other party recognized as described above under the control of the controller 180 and, according to a user's selection, extracts the extracted voice feature vector values from the memory. Attached within each contact information provided at 160.

In addition, the voice recognition module 182 recognizes the audio information included in the current screen, extracts a feature vector value of the recognized audio information, and is stored in the memory 160 under the control of the controller 180. The speech information corresponding to the extracted feature vector value may be searched among the speech information.

In this case, the screen may be any one of a website screen, a music playback screen, and a video playback screen.

Also, the audio information included in the screen of the website may be any one of a user's created contents (UCC) video including a voice post, voice comment, and voice of the other party.

As described above, the voice recognition module 182 may be provided in the mobile terminal 100 in the form of one "module" or "unit", or may be provided in the memory 160 in the form of software.

In addition, the voice recognition module 182 may be included in the controller 180, and in this case, the controller 180 may perform the same operation of the voice recognition unit 182.

In the following description, it is described that the voice recognition module 182 is included in the controller 180 and the controller 180 performs the same operation as the voice recognition module 182.

On the other hand, the power supply unit 190 receives the external power and the internal power under the control of the controller 180 to supply power required for the operation of each component.

Various embodiments described herein may be implemented in a recording medium readable by a computer or similar device using, for example, software, hardware or a combination thereof.

According to a hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), and the like. It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing other functions. The described embodiments may be implemented by the controller 180 itself.

According to the software implementation, embodiments such as the procedures and functions described herein may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein. Software code can be implemented in a software application written in a suitable programming language. The software code may be stored in the memory 160 and executed by the controller 180.

2A is a front perspective view of an example of a mobile terminal according to the present invention;

The disclosed mobile terminal 100 has a bar-shaped body. However, the present invention is not limited thereto and may be applied to various structures such as a slide type, a folder type, a swing type, a swivel type, and two or more bodies are coupled to be relatively movable.

The body includes a casing (casing, housing, cover, etc.) that forms an exterior. In this embodiment, the case may be divided into a front case 101 and a rear case 102. Various electronic components are built in the space formed between the front case 101 and the rear case 102. At least one intermediate case may be further disposed between the front case 101 and the rear case 102.

The cases may be formed by injecting synthetic resin or may be formed of a metal material, for example, a metal material such as stainless steel (STS) or titanium (Ti).

The display unit 151, the audio output unit 152, the camera 121, the user input units 130/131 and 132, the microphone 122, and the interface 170 may be disposed in the terminal body, mainly the front case 101. have.

The display unit 151 occupies most of the main surface of the front case 101. A sound output unit 151 and a camera 121 are disposed in an area adjacent to one end of both ends of the display unit 151 and a user input unit 131 and a microphone 122 are disposed in an area adjacent to the other end. The user input unit 132 and the interface 170 may be disposed on side surfaces of the front case 101 and the rear case 102.

The user input unit 130 is operated to receive a command for controlling the operation of the mobile terminal 100 and may include a plurality of operation units 131 and 132. The operation units 131 and 132 may be collectively referred to as a manipulating portion.

The content input by the first or second manipulation units 131 and 132 may be variously set. For example, the first manipulation unit 131 receives a command such as start, end, scroll, and the like, and the second manipulation unit 132 adjusts the volume of the sound output from the sound output unit 152 or displays 151. Command, such as activating / deactivating the touch recognition mode.

FIG. 2B is a rear perspective view of the mobile terminal shown in FIG. 2A.

Referring to FIG. 2B, a camera 121 ′ may be additionally mounted on the rear of the terminal body, that is, the rear case 102. The camera 121 'may have a photographing direction substantially opposite to that of the camera 121 (see FIG. 2A), and may be a camera having the same or different pixels as the camera 121.

For example, the camera 121 has a low pixel so that the user's face is photographed and transmitted to the counterpart in case of a video call, and the camera 121 'photographs a general subject and does not transmit it immediately. It is desirable to have a high pixel because there are many. The cameras 121 and 121 'may be installed in the terminal body so as to be rotatable or pop-upable.

A flash 123 and a mirror 124 may be additionally disposed adjacent to the camera 121 '. The flash 123 shines light toward the subject when the subject is photographed by the camera 121 '. The mirror 124 allows the user to see his / her own face or the like when photographing (self-photographing) the user using the camera 121 '.

An acoustic output module 152 'may be additionally disposed on the rear side of the terminal body. The sound output unit 152 ′ may implement a stereo function together with the sound output module 152 (see FIG. 2A), and may be used to implement a speakerphone mode during a call.

In addition to the antenna for communication, a broadcast signal receiving antenna 116 may be additionally disposed on the side of the terminal body. The antenna 116 constituting a part of the broadcast receiving unit 111 (refer to FIG. 1) may be installed to be pulled out from the terminal body.

The terminal body is equipped with a power supply unit 190 for supplying power to the mobile terminal 100. The power supply unit 190 may be built in the terminal body or may be directly detachable from the outside of the terminal body.

The rear case 102 may be further equipped with a touch pad 135 for sensing a touch. The touch pad 135 may be of a light transmission type for the display unit 151. [ In this case, if the display unit 151 is configured to output visual information on both sides (i.e., in the direction of both the front and rear surfaces of the mobile terminal), the visual information may be recognized through the touch pad 135. do. The information output on both surfaces may be controlled by the touch pad 135.

Meanwhile, since a display dedicated to the touch pad 135 is separately installed, the touch screen may be disposed on the rear case 102.

The touch pad 135 operates in association with the display unit 151 of the front case 101. The touch pad 135 may be disposed in parallel to the rear of the display unit 151. The touch pad 135 may have the same or smaller size as the display unit 151.

Hereinafter, the voice recognition of the other party and the data output process set in the recognized voice according to the present invention will be described in detail with reference to FIGS. 3 to 12.

First, a process of setting a voice of the other party and data to be mapped to the voice will be described with reference to FIGS. 3 to 7.

3 is a flowchart illustrating a process of setting the voice of the other party and data to be mapped to the voice in the mobile terminal according to the present invention.

4 to 7 are screen state diagrams illustrating a process of setting the voice of the other party and data to be mapped to the voice in the mobile terminal according to the present invention.

3 to 7, the controller 180 of the mobile terminal 100 activates a voice recognition mode for recognizing a voice of the other party and drives the voice recognition module 182 [S111].

When the voice of the other party is input [S152], the controller 180 controls the voice recognition module 182 to recognize the input voice of the other party [S153].

At this time, the input voice, the voice of the other party received through the wireless communication unit 110 during the call, the voice input through the microphone 122, and registered on the web page connected through the wireless communication unit 110 The voice and the voice included in the message received through the wireless communication unit 110 may be at least one voice.

Meanwhile, when the voice of the input counterpart is recognized through the voice recognition module 182, the controller 180 displays a window for setting data to be mapped with information of the recognized voice on the display unit 151. [S114].

In this case, the data is content related to the counterpart, and may be a memo, an image, a video, a sound, a text, etc. to be delivered to the counterpart, and in some cases, all content included in the memory 160 may be included.

If data to be mapped with the recognized counterpart's voice information is set through the data setting window [S115], the controller 180 maps the recognized counterpart's voice information and the set data to each other and stores them in the memory 160. [S116].

In this case, although not shown in the drawing, the controller 180 displays the contact information list to the user after step S115, and if contact information corresponding to the counterpart is selected from the user, the controller 180 displays the voice information of the recognized counterpart and the set data. It may be stored in the contact information of the other party.

4 is a screen state diagram illustrating a process of setting data related to the counterpart in the voice of the counterpart input from the microphone.

As shown in FIG. 4A, when the microphone 122 is driven and a command 11 for recognizing a voice input from the microphone 122 is input, the controller 180 may recognize a voice recognition module ( 182 is driven to recognize the voice input from the microphone 122.

In this case, as shown in (b) of FIG. 4, when the command 12 for data setting to be mapped to the information about the recognized voice is input, the controller 180 is shown in (c) of FIG. 4. As described above, the setting window of the data is displayed on the display unit 151.

In this case, the data setting window may include a memo input window 13 and a file attachment window 14 to be transmitted to the other party.

In addition, the memo input window 13 may include a first icon 13a for text memo input and a second icon 13b for voice memo input.

That is, when the first icon 13a is selected, the controller 180 may display a text memo input window on the screen and receive text memo contents from the user through the text memo input window.

In addition, when the second icon 13b is selected, the controller 180 may drive the microphone 122 and receive voice memo contents from the user through the microphone 122.

In addition, when the text memo is input through the memo input window 13, the controller 180 converts the input text memo into a voice memo by driving a TTS (Text to Speech) application provided in the memory 160. The memory 160 may be stored in the memory 160.

In addition, the file attachment window 14 includes a third icon 14a for searching contents stored in the memory 160, so that the user selects the third icon 14a to display the third icon 14a in the memory 160. The stored content can be searched and attached.

If data including at least one of the memo 13c and the content 14b is set from the user through the data setting window, the controller 180 may recognize the recognized voice information as shown in (d) of FIG. 4. The data is mapped and stored in the memory 160.

5 is a screen state diagram illustrating a process of setting data related to the counterpart to a voice of the counterpart received during a call.

As shown in (a) of FIG. 5, if the controller 180 receives a command 21 for recognizing a call voice of the other party, which is received through the wireless communication unit 110, during a call with the other party's terminal, The voice recognition module 182 is driven to recognize the received voice of the called party.

In this case, as shown in (b) of FIG. 5, when the command 22 for data setting to be mapped to the information on the recognized voice is input, the controller 180 is shown in (c) of FIG. 5. As described above, the setting window of the data is displayed on the display unit 151.

When the data including at least one of the memo 13c and the content 14b is set from the user through the data setting window, the controller 180 may recognize the recognized voice information as shown in FIG. The data is mapped and stored in the memory 160.

6 is a screen state diagram illustrating a process of setting data related to the counterpart in a voice of the counterpart included in a received message.

As shown in FIG. 6A, the controller 180 recognizes the voice of the other party included in the message in a state in which a message including the voice of the other party is received from the other terminal through the wireless communication unit 110. When the command 31 is inputted, the voice recognition module 182 is driven to recognize the voice of the other party included in the message.

At this time, as shown in (b) of FIG. 6, the controller 180 receives a command 32 for data setting to be mapped to the recognized voice information, as shown in (c) of FIG. 6. The setting window of the data is displayed on the display unit 151.

If data including at least one of the memo 13c and the content 14b is set up from the user through the data setting window, the controller 180 may recognize the recognized voice information as shown in FIG. The data is mapped and stored in the memory 160.

7 is a screen state diagram illustrating a process of setting data related to the counterpart in the voice of the counterpart registered in the web page.

As shown in (a) of FIG. 7, the controller 180 outputs the voice of the counterpart in a state of accessing a web page including the voice of the counterpart via the wireless communication unit 110. If is input, the other party's voice is output.

Next, as shown in (b) of FIG. 7, when the command 42 for recognizing the voice of the other party is input, the controller 180 drives the voice recognition module 182 to output the command. Recognize the other party's voice.

Next, as shown in (c) of FIG. 7, the controller 180 receives a command 43 for data setting to be mapped to the recognized voice information. As described above, the setting window of the data is displayed on the display unit 151.

When the data including at least one of the memo 13c and the content 14b is set from the user through the data setting window, the controller 180 may display the recognized voice information as shown in FIG. The data is mapped and stored in the memory 160.

In the above, the process of setting the voice of the other party and data to be mapped to the voice according to the present invention will be described with reference to FIGS. 3 to 7.

Hereinafter, a process of recognizing a voice of the other party and outputting data mapped to the recognized voice will be described in detail with reference to FIGS. 8 to 12.

8 is a flowchart illustrating a process of outputting the voice recognition of the other party and data mapped to the recognized voice in the mobile terminal according to the present invention.

9 to 12 are screen state diagrams illustrating a process of outputting, by the mobile terminal, voice recognition of the other party and data mapped to the recognized voice.

8 to 12, in a state in which the controller 180 maps and stores voice information and data of the counterpart according to the processes of FIGS. 3 to 7, a command for recognizing the voice of a specific counterpart is input from the user. If the voice of the specific counterpart is input [S211], the voice of the input counterpart is recognized through the voice recognition module 182 [S212].

In this case, the voice of the other party input in step S211 may be a voice input through the microphone 122, a voice received during a call, a voice included in a received message, or a voice registered on a web page.

Next, the controller 180 searches for voice information corresponding to the recognized voice of the other party in the memory 160 [S213].

That is, the controller 180 extracts a feature vector value of the voice of the other party input in step S211, and retrieves voice information matching the extracted voice feature vector value from the memory 160.

Subsequently, the controller 180 retrieves the data mapped to the voice information searched in the memory 160 again [S214], and outputs the retrieved data through the output unit 150 [S215].

That is, the controller 180 displays the data on the display unit 151 when the searched data is text or an image, and outputs the data through the sound output module 152 when the searched data is sound.

In addition, the controller 180 may search for contact information in which the data found in step S214 is stored on the phone book provided in the memory 160 [S216], and transmit the searched data to the searched contact information through the wireless communication unit 110. It may be [S217].

As shown in FIG. 9A, when the microphone 122 is driven and the voice 50 of the other party is input from the microphone 122, the controller 180 may enter the memory 160 in the memory 160. The voice information corresponding to the input voice 50 is searched for, and as illustrated in FIG. 9B, data 13c and 14b mapped to the retrieved voice information are output.

9 (b) shows the text or sound data 13c of the memo content of "birthday April 20" (13c) and the image data 14b of "gift.jpg" 14b.

Next, as shown in FIG. 10A, when the voice 60 of the other party is received during a call with the other terminal through the wireless communication unit 110, the controller 180 may store the voice in the memory 160. The voice information corresponding to the received voice 60 is searched for, and as illustrated in FIG. 10B, data 13c and 14b mapped to the retrieved voice information are output.

In this case, the call may be a voice call.

That is, if the "birthday April 20" data 13c is text data, the controller 180 displays the text data 13c and "gift." On the screen during the voice call while maintaining the voice call. "Image data 14b is popped up.

In addition, if the call is a voice call and the " birthday April 20 " data 13c is sound data, the controller 180 outputs the sound data 13c in the state of maintaining the voice call. 152, the wireless communication unit 110 may be controlled to block transmission of the output sound data 13c to the counterpart terminal.

That is, the user may hear the sound data 13c mapped to the voice information of the counterpart in the mobile terminal 100 during the voice call with the current counterpart, and directly transmit the contents of the sound data 13c to the counterpart via voice. .

In addition, the call may be a video call.

That is, the controller 180 displays the data 13c and 14b on the screen during the video call while maintaining the video call.

In this case, the controller 180 displays the data 13c and 14b in an area where a user's image is displayed or an area where an image of the other party is displayed, and controls the wireless communication unit 110 to display the data to the other terminal. Transmission of data 13c and 14b can be interrupted.

That is, the user may report data 13c and 14b displayed on the video call screen and transmit the contents of the data 13c and 14b directly to the other party in a video call with the other party.

Next, as shown in FIG. 11A, when the message including the voice 70 of the other party is received through the wireless communication unit 110, the controller 180 receives the message in the memory 160. The voice information corresponding to the voice 70 included in the retrieved message is retrieved, and as illustrated in FIG. 11B, data 13c and 14b mapped to the retrieved voice information are output.

Lastly, as shown in (a) of FIG. 12, the controller 180 is connected to a website where the voice 80 of the other party is registered through the wireless communication unit 110. When the other party's voice 80 is reproduced, the voice information corresponding to the reproduced voice 80 is retrieved from the memory 160 and mapped to the retrieved voice information as shown in FIG. The generated data 13c and 14b are outputted.

It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.

The present invention described above can be embodied as computer-readable codes on a medium on which a program is recorded. The computer-readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include ROM, RAM, CD-ROM, magnetic tape, floppy disks, optical data storage devices, and the like, which are also implemented in the form of carrier waves (eg, transmission over the Internet). It also includes. Also, the computer may include a control unit 180 of the terminal.

Accordingly, the above detailed description should not be construed as limiting in all aspects and should be considered as illustrative. The scope of the invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the invention are included in the scope of the invention.

The above-described mobile terminal and its control method are not limited to the configuration and method of the above-described embodiments, but the embodiments may be modified such that all or some of the embodiments are selectively And may be configured in combination.

100: mobile terminal 110: wireless communication unit
111: broadcast receiver 112: mobile communication module
113 wireless Internet module 114 short-range communication module
115: Position information module 120: A / V input section
121: camera 122: microphone
130: user input unit 140: sensing unit
141: proximity sensor 150: output section
151: display unit 152: sound output module
153: Alarm module 154: Haptic module
155: Projector Module 160: Memory
170: interface unit 180: control unit
181: multimedia module 182: speech recognition module
190: power supply

Claims (12)

MIC;
A wireless communication unit;
A display unit;
Sound output unit;
A memory in which at least one voice information and data mapped with the voice information are stored; And
A controller for retrieving voice information corresponding to the recognized voice from the memory and outputting data mapped to the retrieved voice information through any one of the display unit and the audio output unit when a voice of a specific counterpart is recognized; Mobile terminal comprising a. The method according to claim 1,
And the voice information and the data are stored in contact information corresponding to the counterpart in a phone book provided in the memory. The method according to claim 1,
And the control unit transmits the data to the contact information of the counterpart through the wireless communication unit. The method according to claim 1,
The voice information may include at least one of a voice of a counterpart received during a call, a voice input through the microphone, a voice registered on a web page, and a voice included in a received message. The method according to claim 1,
And the data is memo content associated with the counterpart. The method according to claim 1,
The recognized other party's voice is a voice received from the terminal of the other party through the wireless communication unit during a call,
The controller outputs the data while maintaining a call with the counterpart terminal. The method of claim 6,
The data is data containing sound,
And the controller controls the sound output unit to output the sound while maintaining the call, and controls the wireless communication unit to block transmission of the output sound to the counterpart terminal. The method of claim 6,
The call is a video call,
The data is data containing an image,
The control unit displays the image on a video call screen of the display unit, and controls the wireless communication unit to block transmission of the displayed image to the counterpart terminal. The method according to claim 1,
The recognized other party's voice is a voice input from the microphone. The method according to claim 1,
The recognized other party's voice is a voice included in a message received from the wireless communication unit. The method according to claim 1,
And the recognized other party's voice is the other party's voice registered in the web page connected through the wireless communication unit. Obtaining voice information of the other party;
If data to be mapped to the obtained voice information is set, mapping the voice information and the data and storing the data in a memory;
Retrieving data mapped to the other party's voice from the memory when the other party's voice is input; And
And outputting the retrieved data.

Images