KR101940530B1 - Terminal and method for controlling the same - Google Patents

Abstract

The present invention relates to a portable terminal and a control method thereof, in which, when a sound is output to an earphone of a portable terminal and a keyword to be received, which is set by the user, is included in the announcement broadcast broadcasted in the vicinity, .

Description

TECHNICAL FIELD [0001] The present invention relates to a portable terminal,

The present invention relates to a portable terminal and a control method thereof, in which the use of the terminal can be realized by further considering the convenience of the user.

The terminal can move And can be divided into a mobile / portable terminal and a stationary terminal depending on whether the mobile terminal is a mobile terminal or a mobile terminal. The mobile terminal can be divided into a handheld terminal and a vehicle mount terminal according to whether the user can directly carry the mobile terminal.

Such a terminal has various functions, for example, in the form of a multimedia device having multiple functions such as photographing and photographing of a moving picture, reproduction of a music or video file, reception of a game and broadcasting, etc. .

In order to support and enhance the functionality of such terminals, it may be considered to improve the structural and / or software parts of the terminal.

2. Description of the Related Art [0002] In recent years, a portable terminal has provided various multimedia functions, so that a user can listen to sounds, music, and broadcasts of moving pictures played on a portable terminal through an earphone after attaching earphones to the portable terminal.

However, when the user is listening to music through the earphone of the portable terminal while moving to a desired destination by using public transportation (bus, subway), it is difficult to hear the destination announcement broadcast by public transportation.

In addition, when a user is listening to music through an earphone of a portable terminal while waiting for his / her turn at a bank, a government office, etc., his / her turn is broadcasted, but he can not recognize it.

An object of the present invention is to provide a portable terminal capable of notifying a user of a keyword, which is set by a user, to be notified when a sound is output to the earphone of the portable terminal, .

According to an aspect of the present invention, there is provided a portable terminal including: a display unit; An earphone for outputting a sound reproduced from the portable terminal; A microphone for receiving a voice around the portable terminal; A voice recognition unit for recognizing a voice received through the microphone; And a controller for outputting the sound to the earphone when the voice recognized by the voice recognition unit corresponds to a keyword for informing the user.

According to another aspect of the present invention, there is provided a method of controlling a portable terminal, including: outputting sounds reproduced by the portable terminal with an earphone connected to the portable terminal; Receiving the sound around the portable terminal through the microphone when the sound is output to the earphone; Recognizing the received voice; And if the recognized voice corresponds to a keyword for informing a user, performing an operation of informing the user of the voice.

The portable terminal according to the present invention and the control method thereof can be applied to a case in which even if a user does not recognize announcements broadcast in the vicinity as the user listens to music through the earphone of the portable terminal, The user can be notified of the desired destination or desired announcement while listening to music through the earphone.

1 is a block diagram illustrating a portable terminal according to an embodiment of the present invention.
2 is a front perspective view of a portable terminal according to an embodiment of the present invention.
3 is a flowchart illustrating a process of changing a theme of a function item display style of the portable terminal according to the present invention.
FIG. 4 is a flowchart illustrating a process of informing a user of an external announcement broadcast required by a user while sound is output to an earphone of the portable terminal, according to the present invention.
5 to 19 are explanatory diagrams illustrating a process of informing a user of an external announcement broadcast required by a user while outputting sound to the earphone of the portable terminal according to the present invention.

Hereinafter, a terminal related to the present invention will be described in detail with reference to the drawings. The suffix " module " and " part " for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

The terminal described in this specification may be a portable terminal such as a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a PDA (Personal Digital Assistant), a PMP (Portable Multimedia Player) , And may also be a fixed terminal such as a digital TV, a smart TV, a desktop computer, and the like.

In the following description, it is assumed that the terminal according to the present invention is a mobile terminal. However, it will be understood by those skilled in the art that the configuration of the mobile terminal according to the present invention may be applied to a fixed terminal.

1 is a block diagram of a portable terminal according to an embodiment of the present invention.

The portable terminal 100 includes a wireless communication unit 110, an A / V input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, A controller 170, a controller 180, a power supply 190, and the like. The components shown in Fig. 1 are not essential, and a portable terminal having more or fewer components may be implemented.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more modules that enable wireless communication between the wireless terminal 100 and the wireless communication system or between the wireless terminal 100 and a network in which the wireless terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives broadcast signals and / or broadcast-related information from an external broadcast management server through a broadcast channel.

The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

The broadcast-related information may refer to a broadcast channel, a broadcast program, or information related to a broadcast service provider. The broadcast related information may also be provided through a mobile communication network through the mobile communication module 112. [ The broadcast-related information may be in the form of an EPG (Electronic Program Guide) of DMB (Digital Multimedia Broadcasting) or an ESG (Electronic Service Guide) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast receiving module 111 may be a DMB-T (Digital Multimedia Broadcasting-Terrestrial), a DMB-S (Digital Multimedia Broadcasting-Satellite), a MediaFLO (Media Forward Link Only), a DVB- A digital broadcasting system such as DVB-CBMS, OMA-BCAST, or Integrated Services Digital Broadcast-Terrestrial (ISDB-T).

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 transmits and receives radio signals to at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data depending on a voice call signal, a video call signal or a text / multimedia message transmission / reception.

In addition, the mobile communication module 112 can download an application, a widget, and the like provided in an external web site or web store to the mobile terminal 100.

The wireless Internet module 113 refers to a module for wireless Internet access, and may be built in or externally mounted in the mobile terminal 100. WLAN (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access) and the like can be used as wireless Internet technologies.

In addition, the wireless Internet module 113 can download an application, a widget, and the like provided in an external web site or web store to the portable terminal 100.

The short-range communication module 114 refers to a module for short-range communication. Bluetooth, Radio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), ZigBee, and the like can be used as a short range communication technology.

The position information module 115 is a module for acquiring the position of the portable terminal, and a representative example thereof is a GPS (Global Position System) module. According to the current technology, the GPS module 115 calculates distance information and accurate time information from three or more satellites, and then applies trigonometry to the calculated information to obtain a three-dimensional string of latitude, longitude, The location information can be accurately calculated. At present, a method of calculating position and time information using three satellites and correcting an error of the calculated position and time information using another satellite is widely used. In addition, the GPS module 115 can calculate speed information by continuously calculating the current position in real time.

Referring to FIG. 1, an A / V (Audio / Video) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video communication mode or the photographing mode. The processed image frame can be displayed on the display unit 151. [

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. [ Two or more cameras 121 may be provided depending on the use environment.

The microphone 122 receives an external sound signal through a microphone in a communication mode, a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 when the voice data is in the call mode, and output. Various noise reduction algorithms may be implemented in the microphone 122 to remove noise generated in receiving an external sound signal.

The user input unit 130 generates input data for a user to control the operation of the terminal.

The user input unit 130 may include a directional keypad, a keypad, a dome switch, a touchpad (static / static), a jog wheel, a jog switch, and the like.

The sensing unit 140 senses the current state of the portable terminal 100 such as the open / close state of the portable terminal 100, the position of the portable terminal 100, the presence of the user, the orientation of the portable terminal, And generates a sensing signal for controlling the operation of the portable terminal 100. For example, when the portable terminal 100 is in the form of a slide phone, it is possible to sense whether the slide phone is opened or closed. It is also possible to sense whether the power supply unit 190 is powered on, whether the interface unit 170 is connected to an external device, and the like. Meanwhile, the sensing unit 140 may include a proximity sensor 141 and a light intensity sensor 142. The proximity sensor 141 will be described later in relation to the touch screen.

The illuminance sensor 142 senses the illuminance around the portable terminal 100 and outputs the sensed result to the controller 180.

The output unit 150 is for generating an output relating to visual, auditory or tactile sense and includes a display unit 151, an acoustic output module 152, an alarm unit 153, a haptic module 154, 155, and the like.

The display unit 151 displays (outputs) the information processed in the portable terminal 100. For example, when the portable terminal is in the call mode, a UI (User Interface) or a GUI (Graphic User Interface) associated with a call is displayed. When the portable terminal 100 is in the video communication mode or the photographing mode, the photographed and / or received image, UI, or GUI is displayed.

The display unit 151 may be a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED) A flexible display, and a three-dimensional display (3D display).

Some of these displays may be transparent or light transmissive so that they can be seen through. This can be referred to as a transparent display, and a typical example of the transparent display is TOLED (Transparent OLED) and the like. The rear structure of the display unit 151 may also be of a light transmission type. With this structure, the user can see an object located behind the terminal body through the area occupied by the display unit 151 of the terminal body.

There may be two or more display units 151 according to the embodiment of the portable terminal 100. [ For example, in the portable terminal 100, a plurality of display units may be spaced apart from one another or may be disposed integrally with each other, or may be disposed on different surfaces.

(Hereinafter, referred to as a 'touch screen') in which a display unit 151 and a sensor for sensing a touch operation (hereinafter, referred to as 'touch sensor') form a mutual layer structure, It can also be used as an input device. The touch sensor may have the form of, for example, a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in a pressure applied to a specific portion of the display unit 151 or a capacitance generated in a specific portion of the display unit 151 into an electrical input signal. The touch sensor can be configured to detect not only the position and area to be touched but also the pressure at the time of touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller (not shown). The touch controller processes the signal (s) and transmits the corresponding data to the controller 180. Thus, the control unit 180 can know which area of the display unit 151 is touched or the like.

The proximity sensor 141 may be disposed in an inner area of the portable terminal or in the vicinity of the touch screen, which is enclosed by the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or a nearby object without mechanical contact using the force of an electromagnetic field or infrared rays. The proximity sensor has a longer life span than the contact sensor and its utilization is also high.

Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. And to detect the proximity of the pointer by the change of the electric field along the proximity of the pointer when the touch screen is electrostatic. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of recognizing that the pointer is positioned on the touch screen while the pointer is not in contact with the touch screen is referred to as " proximity touch & The act of actually touching the pointer on the screen is called " contact touch. &Quot; The position where the pointer is proximately touched on the touch screen means a position where the pointer is vertically corresponding to the touch screen when the pointer is touched.

The proximity sensor detects a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, a proximity touch movement state, and the like). Information corresponding to the detected proximity touch operation and the proximity touch pattern may be output on the touch screen.

The audio output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output module 152 also outputs sound signals related to functions (e.g., call signal reception tones, message reception tones, etc.) performed in the portable terminal 100. The audio output module 152 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event of the portable terminal 100. Examples of events occurring in the portable terminal include receiving a call signal, receiving a message, inputting a key signal, and touch input. The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than the video signal or the audio signal, for example, vibration. In this case, the display unit 151 and the audio output module 152 may be a type of the alarm unit 153. The display unit 151 and the audio output module 152 may be connected to the display unit 151 or the audio output module 152, .

The haptic module 154 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 154 is vibration. The intensity and pattern of the vibration generated by the hit module 154 can be controlled. For example, different vibrations may be synthesized and output or sequentially output.

In addition to the vibration, the haptic module 154 may include a pin arrangement vertically moving with respect to the contact skin surface, a spraying force or a suction force of the air through the injection port or the suction port, a touch on the skin surface, contact with an electrode, And various tactile effects such as an effect of reproducing a cold sensation using an endothermic or exothermic element can be generated.

The haptic module 154 can be implemented not only to transmit the tactile effect through the direct contact but also to allow the user to feel the tactile effect through the muscular sensation of the finger or arm. The haptic module 154 may include two or more haptic modules 154 according to the configuration of the portable terminal 100.

The projector module 155 is a component for performing an image project function using the portable terminal 100 and is similar to the image displayed on the display unit 151 in accordance with a control signal of the controller 180 Or at least partly display another image on an external screen or wall.

Specifically, the projector module 155 includes a light source (not shown) that generates light (for example, laser light) for outputting an image to the outside, a light source And a lens (not shown) for enlarging and outputting the image at a predetermined focal distance to the outside. Further, the projector module 155 may include a device (not shown) capable of mechanically moving the lens or the entire module to adjust the image projection direction.

The projector module 155 can be divided into a CRT (Cathode Ray Tube) module, an LCD (Liquid Crystal Display) module and a DLP (Digital Light Processing) module according to the type of the display means. In particular, the DLP module may be advantageous for miniaturization of the projector module 151 by enlarging and projecting an image generated by reflecting light generated from a light source on a DMD (Digital Micromirror Device) chip.

Preferably, the projector module 155 may be provided on the side surface, the front surface, or the back surface of the portable terminal 100 in the longitudinal direction. It goes without saying that the projector module 155 may be provided at any position of the portable terminal 100 as needed.

The memory 160 may store a program for processing and controlling the controller 180 and may store the input / output data (e.g., a telephone directory, a message, an audio, a still image, an electronic book, History, map data for each region, and the like). The memory 160 may also store the frequency of use of each of the data (for example, each telephone number, each message, and frequency of use for each multimedia). In addition, the memory 160 may store data on vibration and sound of various patterns outputted when a touch is input on the touch screen.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory, etc.) ), A random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read- A magnetic disk, an optical disk, a memory, a magnetic disk, or an optical disk. The portable terminal 100 may operate in association with a web storage that performs a storage function of the memory 160 on the Internet.

The interface unit 170 serves as a path for communication with all external devices connected to the portable terminal 100. The interface unit 170 receives data from an external device or receives power from the external device and transmits the data to each component in the portable terminal 100 or allows data in the portable terminal 100 to be transmitted to an external device. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, A video input / output (I / O) port, an earphone port, and the like may be included in the interface unit 170.

The identification module is a chip for storing various information for authenticating the usage right of the mobile terminal 100 and includes a user identification module (UIM), a subscriber identity module (SIM), a general user authentication module A Universal Subscriber Identity Module (USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the port.

When the portable terminal 100 is connected to an external cradle, the interface unit may be a path through which power from the cradle is supplied to the portable terminal 100, or various command signals input from the cradle by the user It can be a passage to be transmitted to the terminal. The various command signals input from the cradle or the power source may be operated as a signal for recognizing that the portable terminal is correctly mounted on the cradle.

The controller 180 typically controls the overall operation of the portable terminal. For example, voice communication, data communication, video communication, and the like. The control unit 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180. [

The controller 180 may perform a pattern recognition process for recognizing handwriting input or drawing input performed on the touch screen as characters and images, respectively.

Under the control of the controller 180, the voice recognition unit 182 drives the voice recognition engine to which the voice recognition algorithm is applied, and recognizes the voice of the announcement broadcasted in the vicinity of the portable terminal 100 according to the present invention.

That is, when the sound of the specific media (moving picture or music) is output to the earphone according to the present invention, the control unit 180 drives the microphone 122 and the voice recognition unit 182, And recognizes the sound of the received announcement broadcast through the voice recognition unit 182. [

At this time, the announcement broadcast may be a voice corresponding to all or a part of the announcement related to the specific place or destination.

That is, the announcement may include announcement of the previous / current / next history of the subway, announcement of the previous / current / next station of the bus, and announcement of order information of a restaurant / bank.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components.

2 is a perspective view of a portable terminal according to an embodiment of the present invention.

The disclosed mobile terminal 100 has a bar-shaped terminal body. However, the present invention is not limited thereto, and can be applied to various structures such as a slide type, a folder type, a swing type, and a swivel type in which two or more bodies are relatively movably coupled.

The body includes a case (a casing, a housing, a cover, and the like) which forms an appearance. In this embodiment, the case may be divided into a front case 101 and a rear case 102. [ A variety of electronic components are embedded in the space formed between the front case 101 and the rear case 102. At least one intermediate case may be additionally disposed between the front case 101 and the rear case 102. [

The cases may be formed by injection molding a synthetic resin, or may be formed to have a metal material such as stainless steel (STS) or titanium (Ti) or the like.

The front body 101 mainly includes a display unit 151, an audio output unit 152, a camera 121, user inputs 130/131 and 132, a microphone 122, an interface 170, a proximity sensor 141 And a contact sensor 143 may be disposed.

The display unit 151 occupies most of the main surface of the front case 101. A sound output unit 151 and a camera 121 are disposed in an area adjacent to one end of both ends of the display unit 151 and a user input unit 131 and a microphone 122 are disposed in an area adjacent to the other end. The user input unit 132 and the interface 170 may be disposed on the side surfaces of the front case 101 and the rear case 102. [

The user input unit 130 is operated to receive a command for controlling the operation of the portable terminal 100 and may include a plurality of operation units 131 and 132. The operation units 131 and 132 may be collectively referred to as a manipulating portion and may be employed in any manner as long as the user operates in a tactile manner.

The contents inputted by the first or second operation unit 131 or 132 may be variously set. For example, the first operation unit 131 receives commands such as start, end, scroll, and the like, and the second operation unit 132 controls the size of the sound output from the sound output unit 152 or the size of the sound output from the display unit 151 To the touch recognition mode of the touch screen.

In addition to the antenna for talking and the like, a broadcast signal receiving antenna 116 may be further disposed on the side of the terminal body. The antenna 116, which forms part of the broadcast receiving module 111 (see FIG. 1), can be installed to be able to be drawn out from the terminal body.

A power supply unit 190 for supplying power to the portable terminal 100 is mounted on the terminal body. The power supply unit 190 may be built in the terminal body or may be detachable from the outside of the terminal body.

Hereinafter, a related operation of the display unit 151 and the touch pad 135 will be described with reference to FIG.

3 is a front view of a portable terminal for explaining an operation state of the portable terminal according to the present invention.

Various types of time information can be displayed on the display unit 151. [ These pieces of information can be displayed in the form of letters, numbers, symbols, graphics, or icons.

At least one of the letters, numbers, symbols, graphics, or icons may be displayed in a predetermined arrangement for inputting such information, thereby being implemented as a keypad. Such a keypad may be referred to as a so-called " virtual keypad ".

3 illustrates inputting of a touch applied to the virtual keypad through the front surface of the terminal body.

The display unit 151 may operate as an entire area or may be divided into a plurality of areas and operated. In the latter case, the plurality of areas can be configured to operate in association with each other.

For example, an output window 151a and an input window 151b are displayed on the upper and lower portions of the display unit 151, respectively. The output window 151a and the input window 151b are areas allocated for outputting or inputting information, respectively. In the input window 151b, a virtual keypad 151c is displayed in which a number for inputting a telephone number or the like is displayed. When the virtual keypad 151c is touched, numbers and the like corresponding to the touched virtual keypad are displayed on the output window 151a. When the first operation unit 131 is operated, call connection to the telephone number displayed on the output window 151a is attempted.

In addition, the display unit 151 or the touch pad 135 may be configured to receive a touch input by scrolling. The user can move a cursor or a pointer located on an object displayed on the display unit 151, for example, an icon or the like, by scrolling the display unit 151 or the touch pad 135. [ Further, when the finger is moved on the display portion 151 or the touch pad 135, the path along which the finger moves may be visually displayed on the display portion 151. [ This will be useful for editing the image displayed on the display unit 151. [

One function of the terminal may be executed in response to a case where the display unit 151 (touch screen) and the touch pad 135 are touched together within a predetermined time range. In the case of being touched together, there may be a case where the user clamps the terminal body using the thumb and index finger. The one function may be activation or deactivation of the display unit 151 or the touch pad 135, for example.

For convenience of explanation, it is assumed that the portable terminal 100 mentioned below includes at least one of the components shown in FIG.

Also, an arrow or a finger-shaped graphic for pointing a specific object on the display unit 151 or selecting a menu on the display unit 151 is called a pointer or a cursor.

However, in the case of a pointer, it often means a finger or a stylus pen for a touch operation or the like. Therefore, in this specification, a graphic displayed on the display unit is referred to as a cursor, and a physical means for performing a touch, a proximity touch, and a gesture, such as a finger or a stylus pen, is referred to as a pointer.

3 shows a state in which the earphone 200 is connected to the portable terminal 100. FIG. At this time, the earphone 200 includes a receiver for outputting a sound reproduced from the portable terminal 100 and a microphone 210 for receiving an external voice.

4 to 19, in a state in which sound is output to the earphone 200 of the mobile terminal 100 according to the present invention, a keyword that the user wants to receive, which is set by the user, The process of informing the user, if any, is described in detail.

FIG. 4 is a flowchart illustrating a process of informing a user of an external announcement broadcast required by a user while sound is output to an earphone of the portable terminal, according to the present invention.

5 to 19 are explanatory diagrams illustrating a process of informing a user of an external announcement broadcast required by a user while outputting sound to the earphone of the portable terminal according to the present invention.

Referring to FIG. 4, the control unit 180 of the portable terminal 100 receives the earphone 200 at the earphone port of the interface unit 170 (S110). When the media selected by the user is reproduced, To the earphone 200 (S120).

The medium is a medium including sound such as sound, voice, sound effect, etc., for example, moving picture, music, broadcasting, and the like. That is, the media according to the present invention includes all contents including sound.

The control unit 180 drives the microphone 122 and the voice recognition unit 182 according to the present invention when the sound of the currently playing media is output to the earphone 200, 100) at step S130.

At this time, the controller 180 drives the microphone 122 and the voice recognition unit 182 when the sound volume of the playing medium is equal to or greater than the predetermined volume size, and outputs the voice of the external announcement through the microphone 122 You can masturbate.

The predetermined volume size may be such that the user can not hear the external announcement due to the sound of the media when the user listens to the sound of the media through the earphone.

Next, the control unit 180 recognizes the sound of the announcement broadcast received from the microphone 122 through the speech recognition unit 182 (S140).

At this time, the voice of the external announcement usually has an effect sound notifying the start and end of the broadcast, or has a unique tone (broadcast announcer voice) or a frequency band. The voice recognition unit 182 recognizes the voice itself of the announcement announcement received from the microphone 122 or the voice received by the current microphone 122 based on the effect tone, It is possible to confirm whether or not it is a voice for broadcasting and recognize it.

On the other hand, if the user holds the portable terminal 100 in a bag or a pocket and the voice for announcement broadcasting is not well received through the microphone 122 of the portable terminal 100, the voice recognition unit 182 receives the voice from the microphone 122 The sound for the received announcement may not be well recognized. In this case, the controller 180 may further drive the microphone 210 of the earphone 200 to improve the sound reception capability of the announcement sound.

That is, if the sound level of the microphone 122 of the portable terminal 100 is below the reference value, the controller 180 further drives the microphone 210 of the earphone 200 to be broadcasted around the portable terminal 100 Thereby improving the sound receiving ability of the voice for announcement.

When the driving state of the display unit 151 of the portable terminal 100 is off, the user usually holds the portable terminal 100 in a bag or a pocket, The audio for the announcement may not be received well through the microphone 122 of the receiver. Accordingly, when the display unit 151 is in the OFF state, the control unit 180 can further drive the microphone 210 of the earphone 200 to improve the sound reception capability of the announcement voice.

If the illuminance level around the portable terminal 100 sensed by the illuminance sensor 142 of the portable terminal 100 is lower than the reference value, the user usually holds the portable terminal 100 in a bag or a pocket , So that the announcement voice may not be received well through the microphone 122 of the portable terminal 100. Accordingly, the controller 180 may further drive the microphone 210 of the earphone 200 to improve the sound reception capability of the announcement voice when the display unit 151 is in the OFF state.

The control unit 180 checks whether or not the notification keyword set by the user is included in the recognized announcement voice [S150].

If the recognized announcement keyword is included in the recognized announcement voice, the control unit 180 performs an operation for informing the user of the recognized announcement voice (S170).

At this time, at least one voice for informing the user is set as the notification keyword in the memory 160, and if the recognized voice for announcement includes at least one voice set in the memory, And performs an operation for informing the user of the recognized announcement voice.

In addition, in the memory 160, one or more texts for notifying the user are set as the keyword for notification, the control unit 180 converts the recognized voice for announcement into text, If the one or more texts are included in the list, the user is notified of the recognized announcement voice. At this time, the memory 160 is provided with text to speech (TTS) software, and the controller 180 can convert the recognized voice for announcement into text using the TTS software have. Also, the TTS software may be provided in the controller 180 in a module form.

Hereinafter, the notification keyword setting process will be described in detail with reference to FIGs. 5 and 6. FIG.

5, the control unit 180 displays the menu 310 for setting the keyword for notification, and the user can listen to the sound of the medium through the earphone 200 while setting the keyword for setting 310 can set a keyword for notifications used for searching for an announcement to receive a notification.

At this time, the notification keyword setting menu 310 includes a voice keyword setting window 311 and a text keyword input window 312 as a notification keyword.

5 (a), when the voice keyword setting window 311 is selected, the controller 180 drives the microphone 122 and the voice recognition unit 182, and the microphone 122, ("Kangnam station") 311A is recognized through the voice recognition unit 182 when the user's voice keyword ("Kangnam station") 311A is received through the voice recognition unit 182, As shown in the figure, the recognized voice keyword (" Gangnam Station ") 311A is stored in the memory 160 as a keyword for notification.

5 (a), when the user's text keyword (" Gangnam station ") 312A is inputted through the text keyword input window 312, , The input text keyword (" Gangnam Station ") 311A is stored in the memory 160 as a keyword for notification.

Next, as shown in FIG. 6, the control unit 180 may provide an application 320 that can set a notification keyword for each place. At this time, each of the places may be a government office, a bank, and public transportation.

The application 320 may be provided in the memory 160 of the portable terminal 100 as a default or downloaded from the external application server through the wireless communication unit 110.

6, the application 320 may include a notification keyword setting function for notifying the user of a notification broadcast provided by the bank 321, the hospital 322, the subway 323, and the bus 324, .

That is, if the user sets a desired public transportation and a destination through the application 320, the portable terminal 100 may transmit the sound of the media to the earphone 200, If the voice corresponding to the voice is included, it informs the user.

6 (a), when the subway 323 is selected in the application 320, the controller 180 displays the subway line list 323, and the list 323 ) 323A is selected in the list 323A, a list 323A of the histories in the selected specific line ("Line 2") 323A is displayed, and in the list 323A, When the subway history (" Gangnam station ") 323B is selected, the selected subway, the subway line 2 and the Gangnam station are stored in the memory 160 as keywords for notifications.

Thereafter, the control unit 180 transmits a voice for announcement including "subway", "line 2" and "Gangnam station" through the microphone 122 and voice recognition unit 182 during the sound output of the media to the earphone 200 If it is recognized, it informs the user.

Hereinafter, the notification keyword setting process has been described in detail with reference to FIGS. 5 and 6. FIG.

Hereinafter, the process of performing the notification operation in step S170 of FIG. 4 will be described in detail with reference to FIG. 7 to FIG.

7 to 16 illustrate a process of performing the notification operation in step S170 when the display unit 151 is in the ON state.

7 to 9, the control unit 180 controls the microphone 122 and the voice recognition unit 182 to broadcast the sound of the currently playing media to the earphone 200 It is determined whether the operation state of the display unit 151 is on or off if the announcement voice is recognized and the recognized announcement voice corresponds to the notification keyword set in the memory 160. [

If the display unit 151 is turned on, the control unit 180 displays notification information 410, 420, and 430 indicating that the voice corresponding to the notification keyword is recognized on the display unit 151, The user can be notified.

At this time, if the recognized voice for guidance is the same as or related to the notification keyword set in the memory 160, the control unit 180 generates notification information 410, 420 indicating that a voice identical or related to the notification keyword is recognized And 430 can be displayed on the display unit 151.

7 (a), when the user boarded the subway and listened to the sound of the medium through the earphone 200 of the portable terminal 100, the control unit 180 controls the microphone 122 and the voice recognition And receives and broadcasts a voice for announcement broadcasted in the subway through the voice unit 182. [

The control unit 180 receives the announcement voice 330 informing of the arrival of the "Gangnam Station" through the microphone 122 and the voice recognition unit 182 and receives a notification corresponding to "Gangnam Station" in the memory 160 The notification information 410 indicating the "Gangnam Station" is displayed on the display unit 151 as shown in FIG. 7 (b).

That is, the user can listen to the music through the earphone 200, report the notification information 410 indicating " Gangnam Station " displayed on the screen of the portable terminal 100, and recognize that the user should get off the subway .

8 (a), when the user boarded the bus and listened to the sound of the medium through the earphone 200 of the portable terminal 100, the control unit 180 controls the microphone 122 and / And recognizes and recognizes a voice for announcement broadcasted on the bus through the voice recognition unit 182. [

The control unit 180 receives the announcement sound 340 informing the arrival of the " Seochon Lake " through the microphone 122 and the voice recognition unit 182, On the display unit 151, the notification information 420 indicating the " seacoast lake " as shown in Fig. 8 (b).

9 (a), when the user listens to the sound of the media to the earphone 200 of the portable terminal 100 while the user is in the bank, the controller 180 controls the microphone 122, And a voice recognition unit 182. The voice recognition unit 182 receives the voice for announcement broadcasted by the bank.

The control unit 180 then receives the announcement voice 350 that informs the user of the call "155" through the microphone 122 and the voice recognition unit 182 to be received and recognized, Quot; 155 " is displayed on the display unit 151 as shown in FIG. 8 (b).

That is, while listening to the music through the earphone 200, the user can see the notification information 430 indicating " 155 " displayed on the screen of the portable terminal 100,

Next, as shown in FIG. 10, the control unit 180 emphasizes and displays the notification information according to the procedures of FIGS. 7 to 9, thereby displaying a notification broadcast related to the current situation of the user It can be quickly recognized.

10 (a), when the voice for announcement recognized through the microphone 122 and voice recognition unit 182 corresponds to the notification keyword set in the memory 160, , The recognized voice for announcement may be converted into text and displayed, or the notification keyword 410 may be displayed as the notification information.

10 (b), the control unit 180 converts the announcement information 410 into a 3D form having a specific depth, and displays the announcement information 410 in the 3D form. Can be easily recognized.

10 (c), the controller 180 blinks the notification information 410 to easily recognize that the user has arrived at the current "Gangnam Station" .

Next, as shown in FIGS. 11A to 11D, when the voice corresponding to the notification keyword is recognized by the process of FIG. 7 to FIG. 9, The user can quickly recognize that the announcement broadcast currently being broadcast is the announcement broadcast related to his / her situation by changing the brightness of the screen of the display device (gradually increasing and then brightening).

Next, FIG. 12 shows a process of providing detailed information of the notification information shown in FIG. 7 to FIG.

12 (a), when the notification information 410 is selected, the control unit 180 transmits detailed information 411 (hereinafter, referred to as " And downloads and displays the detailed information 411 of the voice for announcement of the "Gangnam Station" from the server, as shown in FIG. 12 (b).

12, when the notification information 410 is selected, the controller 180 stops displaying the notification information 410 and controls the operation of the microphone 122 and the voice recognition unit 182 Can be turned off.

 Next, FIG. 13 shows a process of informing the other party, such as a friend or acquaintance, who is boarding the public transportation together with the user, that the other party should also get off the current public transportation will be.

13 (a), when the notification information 410 is selected, the control unit 180 determines whether the notification information 410 is to be transmitted, as shown in FIG. 13 (b) The contact setting window 510 is displayed.

At this time, the counterpart contact setting window 510 includes a contact input window 511 to send the notification information 410 and a comment input window 512 to be transmitted to the counterpart.

The control unit 180 receives the contact information 511A and the comment 512B to which the notification information 410 is to be transmitted through the contact input window 511 and the comment input window 512, 512B and notification information 410 to the other party's contact 511A.

13C shows that the comment 512B and the notification information 410 transmitted by the portable terminal 100 are displayed on the other party's terminal.

Next, FIG. 14 shows that when the external announcement voice corresponding to the notification keyword is recognized, corresponding announcement information and current position information are provided.

14 (a), the control unit 180 receives and sounds the announcement voice 330 informing the arrival of the " Gangnam Station " through the microphone 122 and the voice recognition unit 182, When the notification keyword corresponding to " Kangnam station " is set in the memory 160, the notification information 410 indicating the above " Kangnam station " is displayed on the display unit 151 as shown in Fig. 14 (b) Display.

At this time, the control unit 180 may display the current position information 410A together with the notification information 410, thereby informing the user of the current accurate position.

That is, when the announcement voice 330 informing the arrival of the " Gangnam Station " is received and recognized through the microphone 122 and the voice recognition unit 182, 100), and display the information 410A indicating the obtained current position.

At this time, the controller 180 may display the GPS coordinate value, the address of the current position, or the map image corresponding to the current position as the information 410A indicating the current position.

Next, FIG. 15 shows that when the external announcement voice corresponding to the notification keyword is recognized, information indicating the corresponding announcement information and the travel path from the current position to the position corresponding to the notification keyword is provided Respectively.

15A, the control unit 180 receives and sounds the announcement voice 330 informing the arrival of the " Gangnam station " through the microphone 122 and the voice recognition unit 182, When the notification keyword corresponding to " Kangnam station " is set in the memory 160, the notification information 410 indicating the " Kangnam station " is displayed on the display unit 151 as shown in Display.

At this time, the controller 180 acquires the current position of the portable terminal 100 through the position information module 115, grasps the movement path from the obtained current position to the " Gangnam Station " position, the route information 410B indicating the identified travel route can be displayed as shown in FIG.

That is, the control unit 180 retrieves and acquires the location information of the " Gangnam Station " from the web site through the wireless communication unit 110, grasps the travel route from the current location to the obtained & . Further, the memory unit 160 further sets the position information for the " Gangnam station ", and the controller 180 sets the current position and the position information of the " Gangnam station " Gangnam Station "and provide it to the user.

Next, FIG. 16 shows that the notification information is highlighted and displayed as the current position of the portable terminal 100 approaches the position corresponding to the notification keyword.

16 (a), the control unit 180 controls the "sound field station" associated with the keyword for the "Kangnam station" notification preset in the memory 160 via the microphone 122 and the speech recognition unit 182, The announcement information 410 indicating the " Kangnam station " is displayed on the display unit 151 for the first time.

Next, as shown in FIG. 16 (b), the control unit 180 controls the microphone 122 and the voice recognition unit 182 to select the " alternate station " If the announcement announcement sound is received and recognized, the announcement information 410 indicating the "Gangnam Station" is displayed at a size larger than the announcement information 410 shown in FIG. 16 (a).

Next, as shown in Fig. 16C, the control unit 180 instructs the microphone 122 and the voice recognition unit 182 to notify the arrival of the "Gangnam Station" as the final destination after the "Seocho Station" When the broadcast voice is sounded and recognized, the notification information 410 indicating the "Gangnam Station" is displayed at a size larger than the notification information 410 shown in FIG. 16B.

That is, the user can report that the notification information 410 is displayed in a larger size and recognize that the user is getting closer to a desired destination.

7 to 16, the process of performing the notification operation in step S170 when the display unit 151 is on has been described in detail.

Hereinafter, the process of performing the notification operation in step S170 when the display unit 151 is off will be described in detail with reference to FIG. 17 to FIG.

17 through 19, the control unit 180 controls the microphone 122 and the voice recognition unit 182 to output the sound of the currently playing media to the earphone 200, It is determined whether the operation state of the display unit 151 is ON or OFF when the recognized voice is recognized and the recognized voice for announcement corresponds to the keyword for notification set in the memory 160. [

If the display unit 151 is in the off state, the controller 180 can notify the user by outputting a voice, an effect sound, or a vibration sound indicating that the voice corresponding to the notification keyword is recognized.

17 (a), when the user board the subway and listens to the sound of the medium through the earphone 200 of the portable terminal 100, the control unit 180 controls the microphone 122 and the voice recognition And receives and broadcasts a voice for announcement broadcasted in the subway through the voice unit 182. [

The control unit 180 receives the announcement voice 330 informing of the arrival of the "Gangnam Station" through the microphone 122 and the voice recognition unit 182 and receives a notification corresponding to "Gangnam Station" in the memory 160 17B, the recognized announcement voice 330 is output to the earphone 200 or the notification keyword is converted into voice and the earphone 200 200).

At this time, the control unit 180 mutes the sound of the media being output to the earphone 200 and then outputs the recognized announcement voice 330 to the earphone 200 or converts the notification keyword into voice And output to the earphone 200.

The control unit 180 synthesizes the sound of the media with the recognized announcement speech 330 and outputs the combined result to the earphone 200 or synthesizes the sound of the media and the keyword for notification converted into the voice, And output it to the earphone 200.

Next, as shown in FIG. 18A, the control unit 180 controls the microphone 122 and the voice recognition unit 182 so that the announcement voice 330 informing the arrival of the " Gangnam Station " And the notification keyword corresponding to the "Gangnam Station" is set in the memory 160, as shown in FIG. 18 (b), a voice for announcement broadcasting corresponding to the notification keyword corresponding to the "Gangnam Station" (An effect sound or a warning sound) indicating that the portable terminal 200 is recognized.

Finally, as shown in FIG. 19 (a), the control unit 180 receives the announcement voice 330 informing the arrival of "Gangnam Station" through the microphone 122 and the voice recognition unit 182, , If the notification keyword corresponding to "Gangnam Station" is set in the memory 160, as shown in FIG. 19 (b), a voice for announcement 330 corresponding to the notification keyword corresponding to the "Gangnam Station" ) Can be output.

It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.

The present invention described above can be embodied as computer-readable codes on a medium on which a program is recorded. The computer readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of the computer-readable medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like, and also implemented in the form of a carrier wave (for example, transmission over the Internet) . Also, the computer may include a control unit 180 of the terminal.

Accordingly, the above description should not be construed in a limiting sense in all respects and should be considered illustrative. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the scope of equivalents of the present invention are included in the scope of the present invention.

The above-described portable terminal and its control method are not limited to the configuration and method of the embodiments described above, but the embodiments may be modified so that all or some of the embodiments are selectively And may be configured in combination.

100: mobile terminal 110: wireless communication unit
111: broadcast receiver 112: mobile communication module
113 wireless Internet module 114 short-range communication module
115: Position information module 120: A / V input section
121: camera 122: microphone
130: user input unit 140: sensing unit
141: proximity sensor 142: illuminance sensor
150: output unit 151: display unit
152: Audio output module 153: Alarm module
154: Haptic module 155: Projector module
160: memory 170: interface section
180: control unit 181: multimedia module
182: Voice recognition unit 190: Power supply unit

Claims (20)

A display unit;
An earphone for outputting a sound reproduced from the portable terminal;
A microphone for receiving a voice around the portable terminal;
A voice recognition unit for recognizing a voice received through the microphone; And
And a controller for outputting the sound to the earphone and notifying the user of the voice when the voice recognized by the voice recognition unit corresponds to a keyword for informing the user,
Wherein,
And if the voice corresponding to the keyword corresponds to all or a part of the announcement related to the specific place or destination to be received in the vicinity of the portable terminal, the voice corresponding to the keyword is notified to the user. delete The method according to claim 1,
Further comprising a memory in which one or more voices for notifying a user are set as the keyword,
Wherein the control unit performs an operation of informing the user of the recognized voice if the recognized voice corresponds to one or more voice set in the memory. The method according to claim 1,
Wherein the keyword further comprises a memory in which one or more texts for notifying a user are set,
Wherein the control unit converts the recognized voice into text and notifies the user of the recognized voice if the converted text corresponds to one or more texts set in the memory. The method according to claim 1,
Wherein the control unit controls the microphone to receive the voice when the volume of the sound being output to the earphone is greater than a predetermined size. delete The method according to claim 1,
The earphone further comprises a microphone,
Wherein the controller further drives a microphone of the earphone to receive a voice around the portable terminal when the display unit is in an OFF state. The method according to claim 1,
The earphone further comprises a microphone,
The portable terminal may further include an illuminance sensor for measuring ambient illuminance,
Wherein the controller further drives the microphone of the earphone to receive the voice around the portable terminal when the illuminance of the surroundings of the portable terminal measured by the illuminance sensor is below a reference value. The method according to claim 1,
Wherein the control unit displays information indicating the recognized voice on the display unit when the voice corresponds to the keyword and the display unit is in the ON state. delete delete delete delete delete 10. The method of claim 9,
A position information module for obtaining a current position of the portable terminal; And
And a memory in which position information corresponding to the keyword is set,
Wherein the control unit emphasizes and displays the information as the current position obtained through the position information module approaches a position corresponding to the keyword. 10. The method of claim 9,
A position information module for obtaining a current position of the portable terminal; And
And a memory in which position information corresponding to the keyword is set,
Wherein the control unit recognizes a movement route to a position corresponding to the keyword based on a current position obtained through the position information module and displays information indicating the detected movement route. delete delete delete delete

Images