KR20120085589A - Method for providing user interface and mobile communications terminal thereof - Google Patents

Abstract

PURPOSE: A method for providing a user interface of a portable terminal and portable terminal thereof are provided to offer a keypad control handle and tilted keypad by sensing the tilting of the portable terminal and by providing various three dimensional gestures to a user. CONSTITUTION: A display unit(151) includes a touch pad. Function keys are included for sensing three-dimensional gestures. A control unit(180) senses the activation of the function keys. When a portable terminal(100) is tilted, the control unit displays a handle for controlling a keypad on the display unit. When a specific gesture is sensed by the handle, the control unit controls the display unit to remove the tilted keypad from the display unit.

Description

Method of providing a user interface of a mobile communication terminal and the mobile communication terminal {METHOD FOR PROVIDING USER INTERFACE AND MOBILE COMMUNICATIONS TERMINAL THEREOF}

The present invention relates to a method for providing a user interface to a terminal and a terminal thereof.

The terminal can move And can be divided into a mobile / portable terminal and a stationary terminal depending on whether the mobile terminal is a mobile terminal or a mobile terminal. The mobile terminal can be divided into a handheld terminal and a vehicle mount terminal according to whether the user can directly carry the mobile terminal.

Such a terminal has various functions, for example, in the form of a multimedia device having multiple functions such as photographing and photographing of a moving picture, reproduction of a music or video file, reception of a game and broadcasting, etc. .

Recently, as mobile terminals provide complex and various functions, nulls are being used as real-time chat applications such as SMS chat or Google talk.

The conventional chat screen using the terminal includes a chat screen on which a chat history is displayed, a chat input window in which letters are input, and a keypad for creating a chat. By the way, while the user is inputting letters on the keypad, there is a case where the active area of the chat screen on which the chat history is displayed is enlarged. At this time, the user must remove the keypad screen.

The present invention provides a method of activating a specific area (aka handle) that can hide a keypad while inputting letters using the keypad through a 3D gesture.

In addition, the present invention provides a user interface method for detecting a tilt of a terminal and displaying the tilted keypad and a handle for controlling the keypad.

Mobile communication terminal according to an embodiment of the present invention for realizing the above object is

A display unit having a touch pad; A function key for performing a 3D gesture; Detects activation of the function key, detects whether the terminal is tilted, displays a handle for controlling a keypad on the display unit when a tilt of the terminal is detected, and tilts when a specific gesture is detected on the handle. And a controller for controlling the keypad not to appear on the display unit.

When the controller detects a specific gesture on the handle, the controller hides the tilted keypad on the display unit and enlarges and displays the active area of the conversation screen.

In addition, through the configuration of the mobile communication terminal as described above, the user interface providing method of the mobile communication terminal according to the present invention,

A method of providing a user interface in a mobile communication terminal that provides a screen including a chat screen on which a chat history is displayed and a keypad for inputting a chat, the controller comprising: detecting a 3D gesture; A specific area for controlling the keypad is displayed on the screen of the terminal; In addition, when an input of a specific gesture is detected in the displayed specific area, the keypad displayed on the screen of the terminal is hidden and the active area of the conversation screen is enlarged and displayed.

Meanwhile, when the specific area is displayed, a tilted keypad may be displayed on the screen of the terminal together with the specific area. In addition, when a specific gesture is input while the keypad is hidden and the conversation screen is enlarged and displayed, the keypad and the conversation screen are displayed together.

Meanwhile, the 3D gesture may have two meanings. That is, the tilt of the terminal or the tilt of the terminal when the function key is activated.

In addition, from the aspect of the screen configuration of the present invention, the user interface providing method according to the present invention,

Detect activation of a function key in a first screen of a conversation mode;

Detecting the tilt of the terminal when the function key is activated;

When a tilt of the terminal is detected while the function key is activated, a second screen including a specific area for controlling the tilted keypad and the tilted keypad is displayed on the first screen;

When a specific gesture is detected in the specific area, displaying the third screen on which the tilted keypad is hidden and an active area of the conversation screen is enlarged;

When a specific gesture is detected on the third screen, the display switches to the first screen or the second screen.

The mobile terminal according to at least one embodiment of the present invention configured as described above provides a user with various and fun 3D gestures (aka, 3D gesture UX (user experience), and also detects the tilt of the terminal and is tilted. By providing a keypad and a 'handle' to control the keypad, it is possible to hide the keypad by dragging the handle even while typing characters, it is possible to provide a convenient dialog screen to the user.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.
Figure 2a is a front perspective view of a mobile terminal according to an embodiment of the present invention.
2B is a rear perspective view of a mobile terminal according to an embodiment of the present invention.
3A and 3B are front views of a portable terminal for explaining an operation state of the portable terminal according to the present invention.
4 is a diagram illustrating a screen of a chat window of a terminal according to one embodiment of the present invention.
5 is an embodiment of the present invention and is a screen of a user interface using a 3D gesture according to the present invention.
6 is a diagram illustrating an operation of a 3D gesture according to an embodiment of the present invention.
7 illustrates another screen of a user interface using a 3D gesture according to an embodiment of the present invention.
FIG. 8 illustrates an example in which a tap gesture is input to a conversation input window.
FIG. 9 illustrates an example of returning to the screen of FIG. 4 or 6 by inputting a 'touch and hold' gesture to a tournament input window.
FIG. 10 illustrates an example of returning to the screen of FIG. 4 or 6 by inputting a 3D gesture in the screen state of FIG. 7.
FIG. 11 is a flowchart illustrating a user interface method of detecting a 3D gesture and providing a screen including a handle and a tilted keypad according to one embodiment of the present invention.

Hereinafter, a mobile terminal related to the present invention will be described in detail with reference to the drawings. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

The mobile terminal described in this specification may include a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), navigation and the like. However, it will be readily apparent to those skilled in the art that the configuration according to the embodiments described herein may also be applied to fixed terminals such as digital TVs, desktop computers, etc., except when applicable only to mobile terminals.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.

The mobile terminal 100 includes a wireless communication unit 110, an A / V input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, and an interface. The unit 170, the controller 180, and the power supply unit 190 may be included. The components shown in FIG. 1 are not essential, so that a mobile terminal having more or fewer components may be implemented.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more modules for enabling wireless communication between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and the network in which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives a broadcast signal and / or broadcast related information from an external broadcast management server through a broadcast channel.

The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may refer to a server for generating and transmitting broadcast signals and / or broadcast related information, or a server for receiving broadcast signals and / or broadcast related information generated by the broadcast management server and transmitting the generated broadcast signals and / or broadcast related information. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

The broadcast-related information may refer to a broadcast channel, a broadcast program, or information related to a broadcast service provider. The broadcast related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast related information may exist in various forms. For example, it may exist in the form of Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast receiving module 111 may include, for example, Digital Multimedia Broadcasting-Terrestrial (DMB-T), Digital Multimedia Broadcasting-Satellite (DMB-S), Media Forward Link Only (MediaFLO), and Digital Video Broadcast (DVB-H). Digital broadcast signals can be received using digital broadcasting systems such as Handheld and Integrated Services Digital Broadcast-Terrestrial (ISDB-T). Of course, the broadcast receiving module 111 may be adapted to other broadcasting systems as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 transmits and receives radio signals to at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data depending on a voice call signal, a video call signal or a text / multimedia message transmission / reception.

The wireless Internet module 113 is a module for wireless Internet access, and may be built in or externally attached to the mobile terminal 100. Wireless Internet technologies may include Wireless LAN (Wi-Fi), Wireless Broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet Access (HSDPA), and the like.

The short range communication module 114 refers to a module for short range communication. Bluetooth, Radio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), ZigBee, and the like can be used as a short range communication technology.

The position information module 115 is a module for obtaining the position of the mobile terminal, and a representative example thereof is a Global Position System (GPS) module.

Referring to FIG. 1, the A / V input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video call mode or the photographing mode. The processed image frame can be displayed on the display unit 151. [

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. Two or more cameras 121 may be provided depending on the use environment.

The microphone 122 receives an external sound signal through a microphone in a communication mode, a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 when the voice data is in the call mode, and output. Various noise reduction algorithms may be implemented in the microphone 122 to remove noise generated in receiving an external sound signal.

The user input unit 130 generates input data for a user to control the operation of the terminal. The user input unit 130 may include a key pad dome switch, a touch pad (static pressure / capacitance), a jog wheel, a jog switch, and the like.

The sensing unit 140 senses the current state of the mobile terminal 100 such as the open / close state of the mobile terminal 100, the position of the mobile terminal 100, the presence or absence of user contact, the orientation of the mobile terminal, And generates a sensing signal for controlling the operation of the mobile terminal 100. For example, when the mobile terminal 100 is in the form of a slide phone, it may sense whether the slide phone is opened or closed. In addition, whether the power supply unit 190 is supplied with power, whether the interface unit 170 is coupled to the external device may be sensed. The sensing unit 140 may include a proximity sensor 141.

The output unit 150 is for generating output related to the visual, auditory or tactile sense and includes a display unit 151, an audio output module 152, an alarm unit 153, and a haptic module 154 .

The display unit 151 displays (outputs) information processed by the mobile terminal 100. For example, when the mobile terminal is in the call mode, a UI (User Interface) or a GUI (Graphic User Interface) associated with a call is displayed. When the mobile terminal 100 is in a video call mode or a photographing mode, the mobile terminal 100 displays photographed and / or received images, a UI, and a GUI.

The display unit 151 includes a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display (flexible). and at least one of a 3D display.

Some of these displays may be transparent or light transmissive so that they can be seen through. This can be referred to as a transparent display, and a typical example of the transparent display is TOLED (Transparent OLED) and the like. The rear structure of the display unit 151 may also be configured as a light transmissive structure. With this structure, the user can see an object located behind the terminal body through the area occupied by the display unit 151 of the terminal body.

There may be two or more display units 151 according to the embodiment of the mobile terminal 100. For example, in the mobile terminal 100, a plurality of display portions may be spaced apart from one another, or may be disposed integrally with one another, and may be disposed on different surfaces, respectively.

When the display unit 151 and a sensor for detecting a touch operation (hereinafter, referred to as a touch sensor) form a mutual layer structure (hereinafter referred to as a touch screen), the display unit 151 may be configured in addition to an output device. Can also be used as an input device. The touch sensor may have the form of, for example, a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in a pressure applied to a specific portion of the display unit 151 or a capacitance generated in a specific portion of the display unit 151 into an electrical input signal. The touch sensor can be configured to detect not only the position and area to be touched but also the pressure at the time of touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and transmits the corresponding data to the controller 180. As a result, the controller 180 can know which area of the display unit 151 is touched.

Referring to FIG. 1, a proximity sensor 141 may be disposed in an inner region of the mobile terminal or in the vicinity of the touch screen, which is surrounded by the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or a nearby object without mechanical contact using the force of an electromagnetic field or infrared rays. The proximity sensor has a longer life span than the contact sensor and its utilization is also high.

Examples of the proximity sensor include a transmission photoelectric sensor, a direct reflection photoelectric sensor, a mirror reflection photoelectric sensor, a high frequency oscillation proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. And to detect the proximity of the pointer by the change of the electric field along the proximity of the pointer when the touch screen is electrostatic. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of allowing the pointer to be recognized without being in contact with the touch screen so that the pointer is located on the touch screen is referred to as a "proximity touch", and the touch The act of actually touching the pointer on the screen is called "contact touch." The position where the pointer is proximately touched on the touch screen means a position where the pointer is vertically corresponding to the touch screen when the pointer is touched.

The proximity sensor detects a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, a proximity touch movement state, and the like). Information corresponding to the detected proximity touch operation and the proximity touch pattern may be output on the touch screen.

The sound output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. The sound output module 152 also outputs sound signals related to functions (e.g., call signal reception sound, message reception sound, etc.) performed in the mobile terminal 100. [ The audio output module 152 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event of the mobile terminal 100. Examples of events that occur in the mobile terminal include call signal reception, message reception, key signal input, touch input, and the like. The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than the video signal or the audio signal, for example, vibration. The video signal or the audio signal may be output through the display unit 151 or the audio output module 152 so that they may be classified as a part of the alarm unit 153.

The haptic module 154 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 154 is vibration. The intensity and pattern of vibration generated by the haptic module 154 can be controlled. For example, different vibrations may be synthesized and output or sequentially output.

In addition to vibration, the haptic module 154 may be configured to provide a pin array that vertically moves with respect to the contact skin surface, a jetting force or suction force of air through an injection or inlet port, grazing to the skin surface, contact of an electrode, electrostatic force, and the like. Various tactile effects can be generated, such as effects by the endothermic and the reproduction of a sense of cold using the elements capable of endotherm or heat generation.

The haptic module 154 may not only deliver a tactile effect through direct contact, but may also be implemented so that a user may feel the tactile effect through a muscle sense such as a finger or an arm. The haptic module 154 may include two or more haptic modules 154 according to the configuration of the portable terminal 100.

The memory 160 may store a program for the operation of the controller 180 and temporarily store input / output data (e.g., a phone book, a message, a still image, a moving picture, etc.). The memory 160 may store data relating to various patterns of vibration and sound output when a touch input on the touch screen is performed.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), a RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM A disk, and / or an optical disk. The mobile terminal 100 may operate in association with a web storage that performs a storage function of the memory 160 on the Internet.

The interface unit 170 serves as a path for communication with all external devices connected to the mobile terminal 100. The interface unit 170 receives data from an external device or supplies power to each component in the mobile terminal 100 or transmits data to the external device. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, A video input / output (I / O) port, an earphone port, and the like may be included in the interface unit 170.

The identification module is a chip for storing various information for authenticating the use right of the mobile terminal 100 and includes a user identification module (UIM), a subscriber identity module (SIM), a general user authentication module A Universal Subscriber Identity Module (USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the port.

The interface unit may be a passage through which power from the cradle is supplied to the mobile terminal 100 when the mobile terminal 100 is connected to an external cradle, or various command signals input from the cradle by a user may be transferred. It may be a passage that is delivered to the terminal. The various command signals or the power source input from the cradle may be operated as a signal for recognizing that the mobile terminal is correctly mounted on the cradle.

The controller 180 typically controls the overall operation of the mobile terminal. For example, voice communication, data communication, video communication, and the like. The controller 180 may include a multimedia module 181 for playing multimedia. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180. [

The controller 180 may perform a pattern recognition process for recognizing handwriting input or drawing input performed on the touch screen as characters and images, respectively.

The power supply unit 190 receives an external power source and an internal power source under the control of the controller 180 to supply power for operation of each component.

Various embodiments described herein may be implemented in a recording medium readable by a computer or similar device using, for example, software, hardware or a combination thereof.

According to a hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), and the like. It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing other functions. The described embodiments may be implemented by the controller 180 itself.

According to the software implementation, embodiments such as the procedures and functions described herein may be implemented as separate software modules. Each of the software modules may perform one or more of the functions and operations described herein. Software code may be implemented in software applications written in a suitable programming language. The software code is stored in the memory 160 and can be executed by the control unit 180. [

2A is a perspective view of an example of a mobile terminal or a mobile terminal according to the present invention.

The disclosed portable terminal 100 has a terminal body in the form of a bar. However, the present invention is not limited thereto, and can be applied to various structures such as a slide type, a folder type, a swing type, and a swivel type in which two or more bodies are relatively movably coupled.

The body includes a case (a casing, a housing, a cover, and the like) which forms an appearance. In this embodiment, the case may be divided into a front case 101 and a rear case 102. [ A variety of electronic components are embedded in the space formed between the front case 101 and the rear case 102. At least one intermediate case may be additionally disposed between the front case 101 and the rear case 102. [

The cases may be formed by injecting synthetic resin or may be formed of a metal material, for example, a metal material such as stainless steel (STS) or titanium (Ti).

The display unit 151, the audio output unit 152, the camera 121, the user input units 130/131 and 132, the microphone 122, and the interface 170 may be disposed in the terminal body, mainly the front case 101. have.

The display unit 151 occupies most of the main surface of the front case 101. A sound output unit 151 and a camera 121 are disposed in an area adjacent to one end of both ends of the display unit 151 and a user input unit 131 and a microphone 122 are disposed in an area adjacent to the other end. The user input unit 132 and the interface 170 may be disposed on the side surfaces of the front case 101 and the rear case 102. [

The user input unit 130 is operated to receive a command for controlling the operation of the portable terminal 100 and may include a plurality of operation units 131 and 132. The manipulation units 131 and 132 may also be collectively referred to as manipulating portions, and may be employed in any manner as long as the user operates the tactile manner with a tactile feeling.

Content input by the first or second manipulation units 131 and 132 may be variously set. For example, the first manipulation unit 131 receives a command such as start, end, scroll, and the like, and the second manipulation unit 132 adjusts the volume of the sound output from the sound output unit 152 or displays 151. Command), such as switching to the touch recognition mode.

FIG. 2B is a rear perspective view of the portable terminal shown in FIG. 2A.

Referring to FIG. 2B, a camera 121 'may be further mounted on the rear surface of the terminal body, that is, the rear case 102. The camera 121 'may have a photographing direction substantially opposite to the camera 121 (see FIG. 2A), and may be a camera having different pixels from the camera 121.

For example, the camera 121 may have a low pixel so that the face of the user can be photographed and transmitted to the other party in case of a video call or the like, and the camera 121 ' It is preferable to have a large number of pixels. The cameras 121 and 121 'may be installed in the terminal body so as to be rotatable or pop-upable.

A flash 123 and a mirror 124 are further disposed adjacent to the camera 121 '. The flash 123 illuminates the subject when the subject is photographed by the camera 121 '. The mirror 124 allows the user to see his / her own face or the like when photographing (self-photographing) the user using the camera 121 '.

An acoustic output 152 'may be additionally disposed on the rear surface of the terminal body. The sound output unit 152 'may implement the stereo function together with the sound output unit 152 (see FIG. 2A), and may be used for the implementation of the speakerphone mode during a call.

In addition to the antenna for talking and the like, a broadcast signal reception antenna 116 may be additionally disposed on the side of the terminal body. The antenna 116, which forms part of the broadcast receiving module 111 (see FIG. 1), can be installed to be able to be drawn out from the terminal body.

A power supply unit 190 for supplying power to the portable terminal 100 is mounted on the terminal body. The power supply unit 190 may be embedded in the terminal body or may be directly detachable from the outside of the terminal body.

The rear case 102 may further include a touch pad 135 for sensing a touch. The touch pad 135 may also be of a light transmission type like the display unit 151. [ In this case, if the display unit 151 is configured to output time information on both sides, the time information can be recognized through the touch pad 135 as well. The information output on both sides may be all controlled by the touch pad 135. [ Alternatively, a display may be additionally mounted on the touch pad 135, and a touch screen may be disposed on the rear case 102 as well.

The touch pad 135 operates in association with the display unit 151 of the front case 101. The touch pad 135 may be disposed parallel to the rear of the display unit 151. The touch pad 135 may have a size equal to or smaller than that of the display unit 151. However, for convenience of description of the embodiments of the present disclosure below, the touch pad 135 is assumed to have a layer structure on the display unit 151.

Hereinafter, the operation of the display unit 151 and the touch pad 135 will be described with reference to FIGS. 3A and 3B.

3A and 3B are front views of a portable terminal for explaining an operation state of the portable terminal according to the present invention.

Various types of time information can be displayed on the display unit 151. [ These pieces of information can be displayed in the form of letters, numbers, symbols, graphics, or icons.

At least one of the letters, numbers, symbols, graphics, or icons may be displayed in a predetermined arrangement for inputting such information, thereby being implemented as a keypad. Such a keypad may be called a so-called " soft key ".

3A illustrates receiving a touch applied to a softkey through the front of the terminal body.

The display unit 151 may operate as an entire area or may be divided into a plurality of areas and operated. In the latter case, the plurality of areas can be configured to operate in association with each other.

For example, an output window 151a and an input window 151b are displayed on the upper and lower portions of the display unit 151, respectively. The output window 151a and the input window 151b are areas allocated for output or input of information, respectively. In the input window 151b, a soft key 151c displaying a number for inputting a telephone number or the like is output. When the softkey 151c is touched, a number or the like corresponding to the touched softkey is displayed on the output window 151a. When the first manipulation unit 131 is operated, a call connection to the telephone number displayed on the output window 151a is attempted.

3B illustrates receiving a touch applied to a softkey through the rear of the terminal body. If FIG. 3A is a portrait in which the terminal body is arranged vertically, FIG. 3B illustrates a landscape in which the terminal body is arranged horizontally. The display unit 151 may be configured to convert the output screen according to the arrangement direction of the terminal body.

3B shows that the text input mode is activated in the mobile terminal. The display unit 151 displays an output window 151a 'and an input window 151b'. A plurality of soft keys 151c 'displaying at least one of letters, symbols, and numbers may be arranged in the input window 151b'. The softkeys 151c 'may be arranged in the form of a QWERTY key.

When the soft keys 151c 'are touched through the touch pad 135 (refer to FIG. 2B), letters, numbers, symbols, etc. corresponding to the touched soft keys are displayed on the output window 151a'. As described above, the touch input through the touch pad 135 has an advantage of preventing the softkey 151c 'from being covered by the finger when touched, as compared with the touch input through the display unit 151. When the display unit 151 and the touch pad 135 are transparent, the fingers located at the rear of the terminal body can be visually checked, and thus more accurate touch input is possible.

The display unit 151 or the touch pad 135 may be configured to receive a touch input by scrolling, as well as the input method described in the above embodiments. The user can move a cursor or a pointer located on an object displayed on the display unit 151, for example, an icon or the like, by scrolling the display unit 151 or the touch pad 135. [ Further, when the finger is moved on the display portion 151 or the touch pad 135, the path along which the finger moves may be visually displayed on the display portion 151. [ This will be useful for editing the image displayed on the display unit 151. [

One function of the terminal may be executed in response to a case where the display unit 151 (touch screen) and the touch pad 135 are touched together within a predetermined time range. In the case of being touched together, there may be a case where the user clamps the terminal body using the thumb and index finger. The one function may be activation or deactivation of the display unit 151 or the touch pad 135, for example.

Hereinafter, embodiments related to a control method that can be implemented in a terminal configured as described above will be described with reference to the accompanying drawings. Embodiments described later may be used alone or in combination with each other. In addition, embodiments described below may be used in combination with the above-described user interface (UI).

Explanation of the term

Hereinafter, for convenience of explanation of the present invention, terms used in the present invention will be described.

Tilt means that the terminal is inclined in one direction and is not limited to the meaning of the word itself. Accordingly, the tilt is a comprehensive concept including not only tilting but also moving up, down, left, and right, twisting, twisting, impacting, moving, vibration, and the like in a specific direction.

- a handle (handle) is when a particular gesture is detected, it refers to the activation of a specific area on the screen. In particular, when the controller of the terminal detects a specific gesture (for example, when the terminal is tilted while pressing a specific function key), a specific region (ie, a handle) appears on the screen, or a specific region on the screen is displayed. Is activated. When a user inputs a gesture such as touching, dragging, or tapping the handle, the controller of the terminal may control a specific operation related to the handle and the gesture.

3D gesture is a term defined for convenience of description in the present invention and refers to an operation of activating a specific key and tilting the terminal while the specific key is activated. That is, when the controller 180 of the terminal 100 detects the 3D gesture, the controller 180 performs a corresponding operation. On the other hand, the specific key, as a physical key, when the terminal is held in hand (that is, grip), ergonomically comfortable to use (for example, one side of the terminal, the back of the terminal, etc.) It may be assigned among the keys shown in FIG. 2A or 2B. Alternatively, the specific key may be allocated to a part of the screen and implemented as a user interface. However, for convenience of description of the present invention, the specific key is assumed as a function key and a physical key configured at one side of the terminal.

Embodiments disclosed in this document may be implemented in the mobile terminal 100 described with reference to FIGS. 1 to 3B.

Gesture  Kinds

The controller 180 may detect various patterns input to the user input unit 130 (for example, a touch pad). In other words, these various patterns are input by a so-called gesture. The user may draw various patterns on the screen by using various gestures. Hereinafter, examples of various gestures will be described.

The user input unit 130 may detect various gestures using a touch sensor. For example, the gesture may be 'tap', 'drag', 'flick', 'swipe', 'double tap' and the like using a finger or a stylus. Gestures such as touch and hold and slicing can be included.

The 'tap' gesture is the most basic gesture of touching a user interface element of a screen, and refers to a motion of touching a touch screen or a finger or stylus for a short time. It means the same motion as the single click of the mouse. Touch motion, long press motion, etc. may mean the same motion.

'Drag' gesture refers to a motion of scrolling in a specific direction while touching the screen. For example, drag gestures are used to scroll up and down the screen in scrollable user interface elements.

'Flick' gesture refers to a gesture that takes a drag gesture but releases a finger or stylus from the touch screen in a short time. For example, flick gestures are used to quickly scroll up and down the screen in scrollable user interface elements.

'Swipe' gesture refers to a gesture that takes a drag gesture but strongly drags a finger or a star illustration on the touch screen. For example, drag gestures are used to display a hidden delete menu corresponding to each item in a list.

'Double tap' gesture refers to two consecutive tap motions, for example, to zoom in or center content or an image, or use it for a zoom out function. Alternatively, when the multimedia mapped to the drawn pattern is executed (played back), the multimedia mapped to the pattern may be executed by double tapping a part of the pattern.

The 'touch and hold' gesture refers to a motion of maintaining a predetermined time while touching a finger or a stylus on a touch screen. Touch and hold gestures are also called long presses. The user may draw a desired pattern on the screen using the touch and hold gesture. It is also used for the function of displaying an enlarged view of the position of the cursor in editable text. It may also be used to overlap a specific pattern with another pattern or to concatenate a specific pattern with another pattern.

Meanwhile, the touch screen of the mobile terminal 100 may recognize a multi touch. In this case, the gesture may further include 'pinch open' and 'pinch close' gestures.

'Pinch open' gesture refers to a gesture with two fingers touching each other on the touch screen as if stretching the screen. On the other hand, when editing the pattern according to the present invention, the pinch open gesture is used to increase the length of the specific pattern or to enlarge or expand the size of the pattern. Pinch open gestures are used to zoom in on the map screen in the map view. The double tap gesture is used to automatically zoom in by a predetermined magnification, but the pinch open gesture is used to adjust the zoom in level based on the degree of the user's finger spreading.

'Pinch close' gesture is the opposite of the pinch open gesture, meaning that the gesture is to close two fingers while touching the touch screen as if to narrow the screen. For example, a pinch open gesture is used to zoom in on a map screen in map view. Meanwhile, when editing an object according to the present invention, a pinch open gesture may be used when reducing the length of a pattern or reducing the size of a pattern.

The 'slicing' gesture, on the other hand, is used to cut the pattern when editing the pattern according to the invention.

Hereinafter, embodiments of generating objects by mapping patterns drawn using the above-described various gestures with multimedia data and executing the generated objects will be described.

3D Gesture  Hide keypad using embodiment

The basic concept of the present invention is: a keypad is displayed on a display unit (screen) of a terminal; When a specific key is activated and a gesture (eg, 3D gesture) is detected that tilts (tilts in a specific direction) the terminal; On the terminal screen, an area for controlling the tilted keypad (aka handle) is activated; the keypad is displayed with the keypad tilted in a three-dimensional shape; thus, the user can use the tilted keypad to display letters (characters). While still writing, the tilted keypad can be hidden by touching or tapping the activated area (ie, hand).

The basic concept of the present invention can be implemented through various embodiments. Hereinafter, an embodiment in which the basic concept of the present invention is implemented in a chat window will be described with reference to FIGS. 4 to 11.

4 is a diagram illustrating a screen of a chat window of a terminal according to one embodiment of the present invention.

There is a case in which the user talks with the other party on the chat screen by using a chat program or related application (eg, SMS chat, KakaoTalk or Google Talk).

As shown in FIG. 4, conversation contents of the user and the counterpart are separately displayed on the screen of the display unit 151 of the terminal 100, and a conversation input window and a keypad may be displayed. The history of the conversation contents between the user and the counterpart may be displayed as shown in FIG. 4. However, the user may want to largely switch the screen on which the contents of the conversation between the user and the counterpart are displayed. In this case, the user may remove a keypad activated on the screen by pressing a key (for example, a cancel key) assigned to remove the keypad. On the other hand, the user may want to control the keypad to be activated (ie, displayed on the surface) or deactivated (ie, disappeared from the screen) while the conversation is being input using the keypad.

In this case, if the 3D gesture defined in the present invention is used, the keypad can be activated or deactivated even while the keypad is in use.

That is, using the 3D gesture according to the present invention, as shown in FIGS. 5 and 7, the conversation screen may be controlled.

5 is an embodiment of the present invention and is a screen of a user interface using a 3D gesture according to the present invention. 6 is a diagram illustrating an operation of a 3D gesture according to an embodiment of the present invention. 7 illustrates another screen of a user interface using a 3D gesture according to an embodiment of the present invention.

As shown in FIG. 4, while the user presses a specific key (namely, a function key) while inputting a character (for example, “when” in FIG. 4) during a conversation with the counterpart, As described above, when the user tilts the terminal 100, in other words, when the user makes a 3D gesture, the corresponding area of the handle is activated and displayed on some screens of the display unit 151 of the terminal 100 as shown in FIG. 6.

That is, the controller 180 of the terminal 100 detects an activation state of the specific key and monitors whether the terminal 100 is tilted while the specific key is activated. If the control unit 180 detects that the terminal 100 is tilted in the state where the specific key is pressed and activated, the control unit 180 displays a specific area of the screen of the display unit 151 in FIG. 6. Mark and activate the handle as shown. For example, as illustrated in FIG. 6, the controller 180 may display a specific gesture (eg, “Drag down the handle” in FIG. 6) in an area where a handle is displayed.

As described above, as illustrated in FIG. 6, the user may continue to input a conversation to be sent to the other party (ie, “when” in FIG. 6) while being provided with a user interface displaying a handle for controlling the keypad at the same time. have.

Meanwhile, when the controller 180 detects a 3D gesture (that is, tilts the terminal while a specific key is activated in FIG. 5), the controller 180 tilts the screen of the keypad as illustrated in FIG. 6. (Aka tilted keypad) may be provided. That is, FIG. 6 corresponds to an embodiment in which a handle is displayed on one surface of the tilted keypad by the 3D gesture operation of FIG. 5.

In order to view the conversation history progressed with the other party, the user may largely switch the conversation screen. In this case, the user may hide the keypad from the screen by inputting a specific gesture (for example, dragging down) the “handle”. That is, when a specific gesture (that is, drag down) is input to the “handle”, the controller 180 of the terminal 100 performs an operation of removing (or hiding) the tilted keypad from the screen. A user interface of a screen as shown in FIG. 7 is provided. The gesture for hiding the tilted keypad may be dragging up, left or right, in addition to dragging down the “handle” (ie, “touching and holding” the handle area). In addition, an operation (or gesture) of hiding the tilted keypad may be variously implemented, for example, a method of pressing the specific key (ie, a function key), another tilt gesture, a gesture of tapping a “handle”. Or the like.

FIG. 7 includes a chat screen on which a chat history is displayed and a chat input window for inputting contents to be sent to a counterpart. Meanwhile, unlike FIG. 7, the conversation input window may not be displayed, and the conversation screen may be configured to check only the conversation history. From the perspective of the user, the user can conveniently check the conversation history through the conversation screen of FIG. 7 which is larger than the conversation screens shown in FIGS. 4 and 6. At this time, the user may need a keypad screen to send a text (or message) to the other party. In this case, the user may switch to the screen on which the keypad is displayed as shown in FIG. 4 or 6, for example, by inputting a specific gesture or 3D gesture as shown in FIGS. 8 to 10. That is, FIG. 8 illustrates an embodiment in which a tap (or click and touch) gesture is input to a conversation input window. FIG. 9 illustrates an example of returning to the screen of FIG. 4 or 6 by inputting a 'touch and hold' gesture (or 'drag up' gesture) into a competition input window. FIG. 10 illustrates an embodiment of the present invention by inputting a 3D gesture (eg, tilting a terminal or pressing a specific key (ie, activating) in the screen state of FIG. 7) to input a gesture. 4 or 6 returns to the screen of FIG.

On the other hand, through the gesture of Figs. 8 to 10, as to which screen of the keypad is displayed as shown in Figs. 4 and 6, it is determined by the user's preference and operator's policy. Can be set.

In the screen state of FIG. 7, when the gesture of FIG. 8 to FIG. 10 is input, the controller 108 detects this and calls the preset return screen (ie, FIG. 4 or FIG. 6).

FIG. 11 is a flowchart illustrating a user interface method of detecting a 3D gesture and providing a screen including a handle and a tilted keypad according to one embodiment of the present invention.

Referring to FIG. 11, a user is talking to a counterpart on a chat screen (eg, a first screen as shown in FIG. 4) including a keypad using a chat application (for example, an SMS chat) stored in a terminal. (S111). That is, it is assumed that the terminal is in a chat mode (SMS Chat view). In this case, when the controller 180 detects the 3D gesture (S112), the controller 180 displays a specific area (eg, a handle) for controlling the keypad (eg, controlling to hide the keypad). (Or activate a specific area) and display a tilted keypad having a three-dimensional shape (for example, a second screen as shown in FIG. 6) (S113). In this case, the user may continuously input a conversation with the counterpart by using the tilted keypad. The controller 180 monitors whether a gesture is input to the specific area (ie, the handle). When the user inputs the 'handle' gesture to hide the tilted keypad (S114), the controller 180 detects the gesture and removes (hides) the tilted keypad from the screen and displays only the conversation screen of the conversation history, or A chat screen and a chat input window are displayed (for example, a third screen as shown in FIG. 7) (S115). Therefore, since the tilted keypad is not displayed on the screen, the user can check the conversation history with the counterpart as shown in FIG. 7.

There is a case in which the user wants to return to the screen including the keypad (for example, the screen of FIG. 4 or 6). In this case, the controller 180 monitors a gesture or a 3D gesture. Then, the controller 180 detects this when a gesture or a 3D gesture for returning to the screen including the keypad is input (S116). In this case, the controller 180 checks whether the return screen is set to one of FIG. 4 or FIG. 6, and displays the set screen on the display unit 151 (S117). The user can create a conversation content (that is, a message or a text) to be sent to the other party by using the keypad on the screen including the keypad.

According to an embodiment of the present invention, the above-described method may be implemented as code that can be read by a processor in a medium in which a program is recorded. Examples of the medium that can be read by the processor include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, etc., and may be implemented in the form of a carrier wave (e.g., transmission over the Internet) .

(Mobile terminal) having the above-described (name) is not limited to the configuration and method of the embodiments described above, the embodiments are all or all of the embodiments so that various modifications can be made Some may be optionally combined.

In the above, preferred embodiments of the present invention have been described with reference to the accompanying drawings.

Here, the terms and words used in the present specification and claims should not be construed as limited to ordinary or dictionary meanings, but should be construed as meaning and concept consistent with the technical idea of the present invention.

Therefore, the embodiments described in the specification and the drawings shown in the drawings are only the most preferred embodiment of the present invention, and do not represent all of the technical idea of the present invention, which can be replaced at the time of the present application It should be understood that there may be various equivalents and variations.

Claims (9)

A display unit having a touch pad;
A function key for performing a 3D gesture;
Detects activation of the function key, detects whether the terminal is tilted, displays a handle for controlling a keypad on the display unit when a tilt of the terminal is detected, and tilts when a specific gesture is detected on the handle. And a controller for controlling the keypad not to appear on the display unit. The apparatus of claim 1, wherein the control unit
If the 3D gesture is detected,
And displaying a tilted keypad together with the handle on the display unit. The method of claim 2, wherein the 3D gesture is
Tilt of the terminal, or
And a tilt of the terminal when the function key is activated. The apparatus of claim 1, wherein the control unit
And detecting a specific gesture on the handle to hide the tilted keypad on the display and to enlarge and display an active area of a chat screen. A method of providing a user interface in a mobile communication terminal providing a screen including a chat screen on which a chat history is displayed and a keypad for inputting a chat.
Detecting a 3D gesture;
Displaying a specific area for controlling a keypad on a screen of the terminal when the 3D gesture is detected;
And when an input of a specific gesture is detected in the displayed specific area, hiding a keypad displayed on a screen of the terminal and displaying an enlarged area of the active area of the conversation screen. The method of claim 5, wherein the displaying of the specific area is performed.
And displaying the tilted keypad on the screen of the terminal together with the specific area. The method of claim 5, wherein the 3D gesture is
Tilt of the terminal, or
And a tilt of the terminal when the function key is activated. The method of claim 5, wherein when a specific gesture is input while the keypad is hidden and the conversation screen is enlarged and displayed,
A method for providing a user interface of a mobile communication terminal, the method comprising: switching to a screen on which a keypad and a conversation screen are displayed together. A method of providing a user interface including a chat screen, a chat input window and a keypad,
Detecting activation of a function key on a first screen of an interactive mode;
Detecting the tilt of the terminal when the function key is activated;
If a tilt of the terminal is detected while the function key is activated, displaying a second screen including a specific area for controlling the tilted keypad and the tilted keypad on the first screen;
Displaying a third screen in which the tilted keypad is hidden and the active area of the conversation screen is enlarged when a specific gesture is detected in the specific area;
Switching to the first screen or the second screen when a specific gesture is detected on the third screen.

Images