KR20180079052A - Mobile terminal and method for controlling the same - Google Patents

Abstract

Disclosed are a mobile terminal and a control method thereof. According to the present invention, the mobile terminal comprises: a touch screen configured to display a first page; and a control unit generating a panel area with a predetermined transparency in a predetermined area of the touch screen, and providing a result of applying a function associated with the first page to the panel area when a predetermined gesture is applied to the touch screen while the first page is displayed. In addition, the control unit can differently control the application of the function associated with the first page in response to the panel area being variable based on a touch input. According to the present invention, it is possible to provide UI and UX capable of controlling information output to a display unit having a bezel-less shape or a thin bezel with one hand.

Description

[0001] MOBILE TERMINAL AND METHOD FOR CONTROLLING THE SAME [0002]

The present invention relates to a mobile terminal having a bezel-shaped or bezel-thin touch screen and a control method thereof.

A terminal can be divided into a mobile terminal (mobile / portable terminal) and a stationary terminal according to whether the terminal can be moved. The mobile terminal can be divided into a handheld terminal and a vehicle mounted terminal according to whether the user can directly carry the mobile terminal.

The functions of mobile terminals are diversified. For example, there are data and voice communication, photographing and video shooting through a camera, voice recording, music file playback through a speaker system, and outputting an image or video on a display unit. Some terminals are equipped with an electronic game play function or a multimedia player function. In particular, modern mobile terminals can receive multicast signals that provide visual content such as broadcast and video or television programs.

Such a terminal has various functions, for example, in the form of a multimedia device having multiple functions such as photographing and photographing of a moving picture, reproduction of a music or video file, reception of a game and broadcasting, etc. .

Meanwhile, a mobile terminal having a display unit structure of a bezel type which does not have a hard key on the front side of the terminal is implemented in order to provide a design element of the mobile terminal and a larger screen. In this case, it often happens that it is inconvenient to perform touch input for controlling information output at the upper or lower end of the display unit. Particularly, when information is controlled on the upper and lower sides of the display unit while holding the terminal with one hand or if the user controls the upper and lower sides alternately even when the terminal is held by both hands, the operation is accompanied by a very unstable and inconvenient operation . In addition, there is an inconvenience in selectively interacting with some of the displayed information.

Accordingly, it is an object of the present invention to provide a mobile terminal and a control method thereof that can provide a UI and UX capable of one-handed control of information output to a display unit having a bezel shape or a thin bezel.

It is another object of the present invention to provide a mobile terminal and a control method thereof that can provide various UXs of a panel area that can multi-interact with all or a part of a currently executing screen.

Accordingly, a mobile terminal according to an embodiment of the present invention includes a touch screen configured to display a first page; Wherein when a predetermined touch gesture is applied to the touch screen while the first page is displayed, a panel area having a predetermined transparency is created in a predetermined area of the touch screen, and the application result of the function associated with the first page Wherein the control unit controls the application of the function associated with the first page in response to the size of the panel area being changed based on the touch input.

In one embodiment, when a drag input is applied to the panel area, an application range of a function associated with the first page is varied along a drag path of the drag input.

In one embodiment, when a drag input is applied to the panel area, an image whose size gradually changes in the panel area along the drag path of the drag input is generated, and the application result of the function associated with the first page is changed Is displayed in the image.

In one embodiment, the image is differently processed according to the point at which the drag input is released.

In one embodiment, if the drag input is continued from one point of the panel area to the top of the display part, the panel area is fixed to the entire display part, and when the reference time elapses, And the panel region disappears while being converted into information displayed in the panel region.

In one embodiment, responsive to a variation of a portion of the first page covered by the panel region along a drag path of the drag input, the function associated with the first page is progressively Is applied or released.

In one embodiment, a plurality of objects are included in the first page, and at least a part of the objects are displayed with a notification object indicating that an event has occurred, and the control unit displays, in accordance with a drag path of the drag input, When a part of the first page is changed so that the notification object includes the object, the event information corresponding to the notification object is provided to the panel area.

In one embodiment, when the drag input reaches the top of the touch screen, the panel area disappears and all notification objects displayed on the first page disappear.

In one embodiment, when the predetermined touch gesture is applied to the touch screen while the virtual keyboard area for inputting information is displayed on the first page, the control unit displays the execution result of the first function associated with the information input, In response to a change in the size of the panel area based on a touch input, providing a result of execution of the first function as a result of execution of a second function different from the first function .

In one embodiment, when the drag-touch input predetermined to the touch screen is applied while the virtual keyboard area for inputting information on the first page is displayed, the controller moves the position of the virtual keyboard area, And controls the degree of scrolling of the keys displayed in the virtual keyboard area and the panel area according to the degree to which the drag-touch input is dragged.

In one embodiment, the first page corresponds to a list including a plurality of items, and the control unit moves the panel region up and down based on a drag input applied to the panel region, Recognizes a specific item in the list in contact with the line, and provides detailed information of the recognized specific item to the panel area.

In one embodiment, the first page corresponds to a screen during a call, and the controller executes a voice recognition mode when a predetermined touch gesture is applied to the touch screen while the during-call screen is being displayed, And at least a part of the conversation contents is converted into text and provided to the panel area according to the execution of the mode.

In one embodiment, in the speech recognition mode, in response to the touch input being applied to one point of the panel area being dragged to the top of the touch screen, the speech recognition mode is terminated and the text provided in the panel area is automatically saved .

In one embodiment, the control unit detects a lower function of the function associated with the first page, in response to the size of the panel area being changed based on the touch input, and displays the detected lower function on the first page And weighted and applied results to the panel area.

In one embodiment, the control unit is configured to determine a function associated with the first page based on the repeated usage pattern in the first page, and to determine, based on the processing on the information displayed in the panel region, Is updated.

In one embodiment, the apparatus further includes an illumination sensing unit for sensing illuminance around the main body to acquire illuminance information, wherein the controller displays a corresponding screen filter value based on illuminance information obtained at the time when the panel region is created To the first page.

In one embodiment, when the predetermined touch gesture is applied to the touch screen while the first page is displayed, the controller extracts first type information from the first page, and then, based on the first type information, The notification information informing that the first page has been reconstructed is output to the panel area.

In one embodiment, when a specific object is selected in the first page, a function associated with the first page is limited to the specific object and is provided to the panel region.

In one embodiment, the type of function associated with the first page is different depending on the type of information included in the specific object.

As described above, according to the embodiment of the present invention, in a display unit having a bezel shape or a thin bezel, information output to a display unit and an auxiliary screen area interact with each other and multitasking related to information output to the display unit It can be done more easily. In addition, by sliding the sub screen area up and down, it is possible to rapidly change the application range and application target of a specific function. Accordingly, it is possible to immediately compare before and after application of the function. Furthermore, there is an advantage in that, when the sub screen area is called, the functions frequently used in the past can be executed directly without going through a few steps.

1A is a block diagram illustrating a mobile terminal according to the present invention.
FIGS. 1B and 1C are conceptual diagrams illustrating an example of a mobile terminal according to the present invention in different directions.
FIGS. 2A, 2B, 2C, 2D, 3A, 3B and 3C are various illustrations showing how panel regions are created, removed and modified in a touch screen according to an embodiment of the present invention.
FIG. 4 is a flowchart illustrating an operation of a mobile terminal according to an embodiment of the present invention. Referring to FIG.
5 is a conceptual diagram for explaining the process of the flowchart of FIG.
6A, 6B, 7A, 7B, 8A, 8B, 9A, 9B, 9C, 10, and 11, Lt; RTI ID = 0.0 > and / or < / RTI >
FIGS. 12, 13, 14, 15A, 15B and 15C are diagrams illustrating a variety of methods for quickly executing appropriate functions based on usage patterns in a page using a panel area according to an embodiment of the present invention. Illustrations showing examples.
Figures 16A, 16B, 17A, 17B, 17C, 18A, 18B illustrate various embodiments relating to providing functions associated with selected objects in a page through a panel region, according to an embodiment of the present invention. These are the drawings.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like or similar components are denoted by the same reference numerals, and redundant explanations thereof will be omitted. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role. In the following description of the embodiments of the present invention, a detailed description of related arts will be omitted when it is determined that the gist of the embodiments disclosed herein may be blurred. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. , ≪ / RTI > equivalents, and alternatives.

Terms including ordinals, such as first, second, etc., may be used to describe various elements, but the elements are not limited to these terms. The terms are used only for the purpose of distinguishing one component from another.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The singular expressions include plural expressions unless the context clearly dictates otherwise.

In the present application, the terms "comprises", "having", and the like are used to specify that a feature, a number, a step, an operation, an element, a component, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

The mobile terminal described in this specification includes a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, a slate PC A tablet PC, an ultrabook, a wearable device such as a smartwatch, a smart glass, and a head mounted display (HMD). have.

The mobile terminal described in this specification includes a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, a slate PC A tablet PC, an ultrabook, a wearable device such as a smartwatch, a smart glass, and a head mounted display (HMD). have.

However, it will be appreciated by those skilled in the art that the configuration according to the embodiments described herein may be applied to fixed terminals such as a digital TV, a desktop computer, a digital signage, and the like, will be.

1A to 1C are block diagrams for explaining a mobile terminal according to the present invention, and FIGS. 1B and 1C are conceptual diagrams showing an example of a mobile terminal according to the present invention in different directions.

The mobile terminal 100 includes a wireless communication unit 110, an input unit 120, a sensing unit 140, an output unit 150, an interface unit 160, a memory 170, a control unit 180, ), And the like. The components shown in FIG. 1A are not essential for implementing a mobile terminal, so that the mobile terminal described herein may have more or fewer components than the components listed above.

The wireless communication unit 110 may be connected between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and another mobile terminal 100 or between the mobile terminal 100 and the external server 100. [ Lt; RTI ID = 0.0 > wireless < / RTI > In addition, the wireless communication unit 110 may include one or more modules for connecting the mobile terminal 100 to one or more networks.

The wireless communication unit 110 may include at least one of a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short distance communication module 114, and a location information module 115 .

The input unit 120 includes a camera 121 or an image input unit for inputting a video signal, a microphone 122 for inputting an audio signal, an audio input unit, a user input unit 123 for receiving information from a user A touch key, a mechanical key, and the like). The voice data or image data collected by the input unit 120 may be analyzed and processed by a user's control command.

The sensing unit 140 may include at least one sensor for sensing at least one of information in the mobile terminal, surrounding environment information surrounding the mobile terminal, and user information. For example, the sensing unit 140 may include a proximity sensor 141, an illumination sensor 142, a touch sensor, an acceleration sensor, a magnetic sensor, A G-sensor, a gyroscope sensor, a motion sensor, an RGB sensor, an infrared sensor, a finger scan sensor, an ultrasonic sensor, A microphone 226, a battery gauge, an environmental sensor (for example, a barometer, a hygrometer, a thermometer, a radiation detection sensor, A thermal sensor, a gas sensor, etc.), a chemical sensor (e.g., an electronic nose, a healthcare sensor, a biometric sensor, etc.). Meanwhile, the mobile terminal disclosed in the present specification can combine and utilize information sensed by at least two of the sensors.

The output unit 150 includes at least one of a display unit 151, an acoustic output unit 152, a haptic tip module 153, and a light output unit 154 to generate an output related to visual, auditory, can do. The display unit 151 may have a mutual layer structure with the touch sensor or may be integrally formed to realize a touch screen. The touch screen may function as a user input unit 123 that provides an input interface between the mobile terminal 100 and a user and may provide an output interface between the mobile terminal 100 and a user.

The interface unit 160 serves as a path to various types of external devices connected to the mobile terminal 100. The interface unit 160 is connected to a device having a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, And may include at least one of a port, an audio I / O port, a video I / O port, and an earphone port. In the mobile terminal 100, corresponding to the connection of the external device to the interface unit 160, it is possible to perform appropriate control related to the connected external device.

In addition, the memory 170 stores data supporting various functions of the mobile terminal 100. The memory 170 may store a plurality of application programs or applications running on the mobile terminal 100, data for operation of the mobile terminal 100, and commands. At least some of these applications may be downloaded from an external server via wireless communication. Also, at least a part of these application programs may exist on the mobile terminal 100 from the time of shipment for the basic functions (e.g., telephone call receiving function, message receiving function, and calling function) of the mobile terminal 100. Meanwhile, the application program may be stored in the memory 170, installed on the mobile terminal 100, and may be operated by the control unit 180 to perform the operation (or function) of the mobile terminal.

In addition to the operations related to the application program, the control unit 180 typically controls the overall operation of the mobile terminal 100. The control unit 180 may process or process signals, data, information, and the like input or output through the above-mentioned components, or may drive an application program stored in the memory 170 to provide or process appropriate information or functions to the user.

In addition, the controller 180 may control at least some of the components illustrated in FIG. 1A in order to drive an application program stored in the memory 170. FIG. In addition, the controller 180 may operate at least two of the components included in the mobile terminal 100 in combination with each other for driving the application program.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power to the components included in the mobile terminal 100. The power supply unit 190 includes a battery, which may be an internal battery or a replaceable battery.

At least some of the components may operate in cooperation with one another to implement a method of operation, control, or control of a mobile terminal according to various embodiments described below. In addition, the operation, control, or control method of the mobile terminal may be implemented on the mobile terminal by driving at least one application program stored in the memory 170. [

Hereinafter, the various components of the mobile terminal 100 will be described in detail with reference to FIG. 1A.

First, referring to the wireless communication unit 110, the broadcast receiving module 111 of the wireless communication unit 110 receives broadcast signals and / or broadcast-related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. Two or more broadcast receiving modules may be provided to the mobile terminal 100 for simultaneous broadcast reception or broadcast channel switching for at least two broadcast channels.

The mobile communication module 112 may be a mobile communication module or a mobile communication module such as a mobile communication module or a mobile communication module that uses technology standards or a communication method (e.g., Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), Wideband CDMA (WCDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution And an external terminal, or a server on a mobile communication network established according to a long term evolution (AR), a long term evolution (AR), or the like.

The wireless signal may include various types of data depending on a voice call signal, a video call signal or a text / multimedia message transmission / reception.

The wireless Internet module 113 is a module for wireless Internet access, and may be built in or externally attached to the mobile terminal 100. The wireless Internet module 113 is configured to transmit and receive a wireless signal in a communication network according to wireless Internet technologies.

Wireless Internet technologies include, for example, wireless LAN (WLAN), wireless fidelity (Wi-Fi), wireless fidelity (Wi-Fi) Direct, DLNA (Digital Living Network Alliance), WiBro Interoperability for Microwave Access, High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE) and Long Term Evolution-Advanced (LTE-A) 113 transmit and receive data according to at least one wireless Internet technology, including Internet technologies not listed above.

The wireless Internet module 113 for performing a wireless Internet connection through the mobile communication network can be used for wireless Internet access by WiBro, HSDPA, HSUPA, GSM, CDMA, WCDMA, LTE or LTE- May be understood as a kind of the mobile communication module 112.

The short-range communication module 114 is for short-range communication, and includes Bluetooth ™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB) (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technology. The short-range communication module 114 is connected to the mobile terminal 100 and the wireless communication system through the wireless area networks, between the mobile terminal 100 and another mobile terminal 100, or between the mobile terminal 100 ) And the other mobile terminal 100 (or the external server). The short-range wireless communication network may be a short-range wireless personal area network.

Here, the other mobile terminal 100 may be a wearable device (e.g., a smartwatch, a smart glass, etc.) capable of interchanging data with the mobile terminal 100 according to the present invention (smart glass), HMD (head mounted display)). The short range communication module 114 may detect (or recognize) a wearable device capable of communicating with the mobile terminal 100 around the mobile terminal 100. [ If the detected wearable device is a device authenticated to communicate with the mobile terminal 100 according to the present invention, the control unit 180 may transmit at least a part of the data processed by the mobile terminal 100 to the short- 114 to the wearable device. Therefore, the user of the wearable device can use the data processed by the mobile terminal 100 through the wearable device. For example, according to this, when a telephone is received in the mobile terminal 100, the user performs a telephone conversation via the wearable device, or when a message is received in the mobile terminal 100, It is possible to check the message.

The position information module 115 is a module for obtaining the position (or current position) of the mobile terminal, and a representative example thereof is a Global Positioning System (GPS) module or a Wireless Fidelity (WiFi) module. For example, when the mobile terminal utilizes the GPS module, it can acquire the position of the mobile terminal by using a signal transmitted from the GPS satellite. As another example, when the mobile terminal utilizes the Wi-Fi module, it can acquire the position of the mobile terminal based on information of a wireless access point (AP) that transmits or receives the wireless signal with the Wi-Fi module. Optionally, the location information module 115 may perform any of the other functions of the wireless communication unit 110 to obtain data relating to the location of the mobile terminal, in addition or alternatively. The location information module 115 is a module used to obtain the location (or current location) of the mobile terminal, and is not limited to a module that directly calculates or obtains the location of the mobile terminal.

Next, the input unit 120 is for inputting image information (or signal), audio information (or signal), data, or information input from a user. For inputting image information, Or a plurality of cameras 121 may be provided. The camera 121 processes an image frame such as a still image or moving image obtained by the image sensor in the video communication mode or the photographing mode. The processed image frame may be displayed on the display unit 151 or stored in the memory 170. [ A plurality of cameras 121 provided in the mobile terminal 100 may be arranged to have a matrix structure and various angles or foci may be provided to the mobile terminal 100 through the camera 121 having the matrix structure A plurality of pieces of image information can be input. In addition, the plurality of cameras 121 may be arranged in a stereo structure to acquire a left image and a right image for realizing a stereoscopic image.

The microphone 122 processes the external acoustic signal into electrical voice data. The processed voice data can be utilized variously according to a function (or a running application program) being executed in the mobile terminal 100. Meanwhile, the microphone 122 may be implemented with various noise reduction algorithms for eliminating noise generated in receiving an external sound signal.

The user input unit 123 is for receiving information from a user and when the information is inputted through the user input unit 123, the control unit 180 can control the operation of the mobile terminal 100 to correspond to the input information . The user input unit 123 may include a mechanical input means (or a mechanical key such as a button located on the front, rear or side of the mobile terminal 100, a dome switch, a jog wheel, Jog switches, etc.) and touch-type input means. For example, the touch-type input means may comprise a virtual key, a soft key or a visual key displayed on the touch screen through software processing, And a touch key disposed on the touch panel. Meanwhile, the virtual key or the visual key can be displayed on a touch screen having various forms, for example, a graphic, a text, an icon, a video, As shown in FIG.

Meanwhile, the sensing unit 140 senses at least one of information in the mobile terminal, surrounding environment information surrounding the mobile terminal, and user information, and generates a corresponding sensing signal. The control unit 180 may control the driving or operation of the mobile terminal 100 or may perform data processing, function or operation related to the application program installed in the mobile terminal 100 based on the sensing signal. Representative sensors among various sensors that may be included in the sensing unit 140 will be described in more detail.

First, the proximity sensor 141 refers to a sensor that detects the presence of an object approaching a predetermined detection surface, or the presence of an object in the vicinity of the detection surface, without mechanical contact by using electromagnetic force or infrared rays. The proximity sensor 141 may be disposed in the inner area of the mobile terminal or in proximity to the touch screen, which is covered by the touch screen.

Examples of the proximity sensor 141 include a transmission type photoelectric sensor, a direct reflection fluorescent light sensor, a mirror reflection fluorescent light sensor, a high frequency oscillation proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. In the case where the touch screen is electrostatic, the proximity sensor 141 can be configured to detect the proximity of the object with a change of the electric field along the proximity of the object having conductivity. In this case, the touch screen (or touch sensor) itself may be classified as a proximity sensor.

On the other hand, for convenience of explanation, the act of recognizing that the object is located on the touch screen in proximity with no object touching the touch screen is referred to as "proximity touch & The act of actually touching an object on the screen is called a "contact touch. &Quot; The position at which the object is closely touched on the touch screen means a position where the object corresponds to the touch screen vertically when the object is touched. The proximity sensor 141 can detect a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, have. Meanwhile, the control unit 180 processes data (or information) corresponding to the proximity touch operation and the proximity touch pattern sensed through the proximity sensor 141 as described above, and further provides visual information corresponding to the processed data It can be output on the touch screen. Furthermore, the control unit 180 can control the mobile terminal 100 such that different operations or data (or information) are processed according to whether the touch to the same point on the touch screen is a proximity touch or a touch touch .

The touch sensor senses a touch (or touch input) applied to the touch screen (or the display unit 151) by using at least one of various touch methods such as a resistance film type, a capacitive type, an infrared type, an ultrasonic type, do.

For example, the touch sensor may be configured to convert a change in a pressure applied to a specific portion of the touch screen or a capacitance generated in a specific portion to an electrical input signal. The touch sensor may be configured to detect a position, an area, a pressure at the time of touch, a capacitance at the time of touch, and the like where a touch object touching the touch screen is touched on the touch sensor. Here, the touch object may be a finger, a touch pen, a stylus pen, a pointer, or the like as an object to which a touch is applied to the touch sensor.

Thus, when there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and transmits the corresponding data to the controller 180. Thus, the control unit 180 can know which area of the display unit 151 is touched or the like. Here, the touch controller may be a separate component from the control unit 180, and may be the control unit 180 itself.

On the other hand, the control unit 180 may perform different controls or perform the same control according to the type of the touch object touching the touch screen (or a touch key provided on the touch screen). Whether to perform different controls or to perform the same control according to the type of the touch object may be determined according to the current state of the mobile terminal 100 or an application program being executed.

On the other hand, the touch sensors and the proximity sensors discussed above can be used independently or in combination to provide a short touch (touch), a long touch, a multi touch, a drag touch ), Flick touch, pinch-in touch, pinch-out touch, swipe touch, hovering touch, and the like. Touch can be sensed.

The ultrasonic sensor can recognize the position information of the object to be sensed by using ultrasonic waves. Meanwhile, the controller 180 can calculate the position of the wave generating source through the information sensed by the optical sensor and the plurality of ultrasonic sensors. The position of the wave source can be calculated using the fact that the light is much faster than the ultrasonic wave, that is, the time when the light reaches the optical sensor is much faster than the time the ultrasonic wave reaches the ultrasonic sensor. More specifically, the position of the wave generating source can be calculated using the time difference with the time when the ultrasonic wave reaches the reference signal.

The camera 121 includes at least one of a camera sensor (for example, a CCD, a CMOS, etc.), a photo sensor (or an image sensor), and a laser sensor.

The camera 121 and the laser sensor may be combined with each other to sense a touch of the sensing object with respect to the three-dimensional stereoscopic image. The photosensor can be laminated to the display element, which is adapted to scan the movement of the object to be detected proximate to the touch screen. More specifically, the photosensor mounts photo diodes and TRs (Transistors) in a row / column and scans the contents loaded on the photosensor using an electrical signal that varies according to the amount of light applied to the photo diode. That is, the photo sensor performs coordinate calculation of the object to be sensed according to the amount of change of light, and position information of the object to be sensed can be obtained through the calculation.

The display unit 151 displays (outputs) information processed by the mobile terminal 100. For example, the display unit 151 may display execution screen information of an application program driven by the mobile terminal 100 or UI (User Interface) and GUI (Graphic User Interface) information according to the execution screen information .

In addition, the display unit 151 may be configured as a stereoscopic display unit for displaying a stereoscopic image.

In the stereoscopic display unit, a three-dimensional display system such as a stereoscopic system (glasses system), an autostereoscopic system (no-glasses system), and a projection system (holographic system) can be applied.

The sound output unit 152 may output audio data received from the wireless communication unit 110 or stored in the memory 170 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output unit 152 also outputs sound signals related to functions (e.g., call signal reception sound, message reception sound, etc.) performed in the mobile terminal 100. [ The audio output unit 152 may include a receiver, a speaker, a buzzer, and the like.

The haptic module 153 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 153 may be vibration. The intensity and pattern of the vibration generated in the haptic module 153 can be controlled by the user's selection or the setting of the control unit. For example, the haptic module 153 may synthesize and output different vibrations or sequentially output the vibrations.

In addition to vibration, the haptic module 153 may be configured to perform various functions such as a pin arrangement vertically moving with respect to the contact skin surface, a spraying force or suction force of the air through the injection port or the suction port, a touch on the skin surface, And various tactile effects such as an effect of reproducing a cold sensation using an endothermic or exothermic element can be generated.

The haptic module 153 can transmit the tactile effect through the direct contact, and the tactile effect can be felt by the user through the muscles of the finger or arm. The haptic module 153 may include two or more haptic modules 153 according to the configuration of the mobile terminal 100.

The light output unit 154 outputs a signal for notifying the occurrence of an event using the light of the light source of the mobile terminal 100. Examples of events that occur in the mobile terminal 100 may include message reception, call signal reception, missed call, alarm, schedule notification, email reception, information reception through an application, and the like.

The signal output from the light output unit 154 is implemented as the mobile terminal emits light of a single color or a plurality of colors to the front or rear surface. The signal output may be terminated by the mobile terminal detecting the event confirmation of the user.

The interface unit 160 serves as a path for communication with all external devices connected to the mobile terminal 100. The interface unit 160 receives data from an external device or supplies power to each component in the mobile terminal 100 or transmits data in the mobile terminal 100 to an external device. For example, a port for connecting a device equipped with a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, an audio I / O port, a video I / O port, an earphone port, and the like may be included in the interface unit 160.

The identification module is a chip for storing various information for authenticating the use right of the mobile terminal 100 and includes a user identification module (UIM), a subscriber identity module (SIM) A universal subscriber identity module (USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the interface unit 160. [

The interface unit 160 may be a path through which power from the cradle is supplied to the mobile terminal 100 when the mobile terminal 100 is connected to an external cradle, And various command signals may be transmitted to the mobile terminal 100. The various command signals or the power source input from the cradle may be operated as a signal for recognizing that the mobile terminal 100 is correctly mounted on the cradle.

The memory 170 may store a program for the operation of the controller 180 and temporarily store input / output data (e.g., a phone book, a message, a still image, a moving picture, etc.). The memory 170 may store data related to vibration and sound of various patterns outputted when a touch is input on the touch screen.

The memory 170 may be a flash memory type, a hard disk type, a solid state disk type, an SDD type (Silicon Disk Drive type), a multimedia card micro type ), Card type memory (e.g., SD or XD memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read memory, a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and / or an optical disk. The mobile terminal 100 may operate in association with a web storage that performs the storage function of the memory 170 on the Internet.

Meanwhile, as described above, the control unit 180 controls the operations related to the application program and the general operation of the mobile terminal 100. [ For example, when the state of the mobile terminal meets a set condition, the control unit 180 can execute or release a lock state for restricting input of a user's control command to applications.

In addition, the control unit 180 performs control and processing related to voice communication, data communication, video call, or the like, or performs pattern recognition processing to recognize handwriting input or drawing input performed on the touch screen as characters and images, respectively . Further, the controller 180 may control any one or a plurality of the above-described components in order to implement various embodiments described below on the mobile terminal 100 according to the present invention.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components. The power supply unit 190 includes a battery, the battery may be an internal battery configured to be chargeable, and may be detachably coupled to the terminal body for charging or the like.

In addition, the power supply unit 190 may include a connection port, and the connection port may be configured as an example of an interface 160 through which an external charger for supplying power for charging the battery is electrically connected.

As another example, the power supply unit 190 may be configured to charge the battery in a wireless manner without using the connection port. In this case, the power supply unit 190 may use at least one of an inductive coupling method based on a magnetic induction phenomenon from an external wireless power transmission apparatus and a magnetic resonance coupling method based on an electromagnetic resonance phenomenon Power can be delivered.

In the following, various embodiments may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

Referring to FIGS. 1B and 1C, the disclosed mobile terminal 100 includes a bar-shaped terminal body. However, the present invention is not limited thereto and can be applied to various structures such as a folder type, a flip type, a slide type, a swing type, and a swivel type in which a watch type, a clip type, a glass type or two or more bodies are relatively movably coupled . A description of a particular type of mobile terminal, although relevant to a particular type of mobile terminal, is generally applicable to other types of mobile terminals.

Here, the terminal body can be understood as a concept of referring to the mobile terminal 100 as at least one aggregate.

The mobile terminal 100 includes a case (for example, a frame, a housing, a cover, and the like) that forms an appearance. As shown, the mobile terminal 100 may include a front case 101 and a rear case 102. Various electronic components are disposed in the inner space formed by the combination of the front case 101 and the rear case 102. At least one middle case may be additionally disposed between the front case 101 and the rear case 102.

A display unit 151 is disposed on a front surface of the terminal body to output information. The window 151a of the display unit 151 may be mounted on the front case 101 to form a front surface of the terminal body together with the front case 101. [

In some cases, electronic components may also be mounted on the rear case 102. Electronic parts that can be mounted on the rear case 102 include detachable batteries, an identification module, a memory card, and the like. In this case, a rear cover 103 for covering the mounted electronic components can be detachably coupled to the rear case 102. Therefore, when the rear cover 103 is separated from the rear case 102, the electronic parts mounted on the rear case 102 are exposed to the outside.

As shown, when the rear cover 103 is coupled to the rear case 102, a side portion of the rear case 102 can be exposed. In some cases, the rear case 102 may be completely covered by the rear cover 103 during the engagement. Meanwhile, the rear cover 103 may be provided with an opening for exposing the camera 121b and the sound output unit 152b to the outside.

These cases 101, 102 and 103 may be formed by injection molding of synthetic resin or may be formed of metal such as stainless steel (STS), aluminum (Al), titanium (Ti) or the like.

The mobile terminal 100 may be configured such that one case provides the internal space, unlike the above example in which a plurality of cases provide an internal space for accommodating various electronic components. In this case, a unibody mobile terminal 100 in which synthetic resin or metal is connected from the side to the rear side can be realized.

Meanwhile, the mobile terminal 100 may include a waterproof unit (not shown) for preventing water from penetrating into the terminal body. For example, the waterproof portion is provided between the window 151a and the front case 101, between the front case 101 and the rear case 102, or between the rear case 102 and the rear cover 103, And a waterproof member for sealing the inside space of the oven.

The mobile terminal 100 is provided with a display unit 151, first and second sound output units 152a and 152b, a proximity sensor 141, an illuminance sensor 142, a light output unit 154, Cameras 121a and 121b, first and second operation units 123a and 123b, a microphone 122, an interface unit 160, and the like.

1B and 1C, a display unit 151, a first sound output unit 152a, a proximity sensor 141, an illuminance sensor 142, an optical output unit (not shown) A second operation unit 123b, a microphone 122 and an interface unit 160 are disposed on a side surface of the terminal body, And a mobile terminal 100 having a second sound output unit 152b and a second camera 121b disposed on a rear surface thereof.

However, these configurations are not limited to this arrangement. These configurations may be excluded or replaced as needed, or placed on different planes. For example, the first operation unit 123a may not be provided on the front surface of the terminal body, and the second sound output unit 152b may be provided on the side surface of the terminal body rather than the rear surface of the terminal body.

The display unit 151 displays (outputs) information processed by the mobile terminal 100. For example, the display unit 151 may display execution screen information of an application program driven by the mobile terminal 100 or UI (User Interface) and GUI (Graphic User Interface) information according to the execution screen information .

The display unit 151 may be a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display display, a 3D display, and an e-ink display.

In addition, the display unit 151 may exist in two or more depending on the embodiment of the mobile terminal 100. In this case, the mobile terminal 100 may be provided with a plurality of display portions spaced apart from each other or disposed integrally with one another, or may be disposed on different surfaces, respectively.

The display unit 151 may include a touch sensor that senses a touch with respect to the display unit 151 so that a control command can be received by a touch method. When a touch is made to the display unit 151, the touch sensor senses the touch, and the control unit 180 generates a control command corresponding to the touch based on the touch. The content input by the touch method may be a letter or a number, an instruction in various modes, a menu item which can be designated, and the like.

The touch sensor may be a film having a touch pattern and disposed between the window 151a and a display (not shown) on the rear surface of the window 151a, or may be a metal wire . Alternatively, the touch sensor may be formed integrally with the display. For example, the touch sensor may be disposed on a substrate of the display or inside the display.

In this way, the display unit 151 can form a touch screen together with the touch sensor. In this case, the touch screen can function as a user input unit 123 (see FIG. 1A). In some cases, the touch screen may replace at least some functions of the first operation unit 123a.

The first sound output unit 152a may be implemented as a receiver for transmitting a call sound to a user's ear and the second sound output unit 152b may be implemented as a loud speaker for outputting various alarm sounds or multimedia playback sounds. ). ≪ / RTI >

The window 151a of the display unit 151 may be provided with an acoustic hole for emitting the sound generated from the first acoustic output unit 152a. However, the present invention is not limited to this, and the sound may be configured to be emitted along an assembly gap (for example, a gap between the window 151a and the front case 101) between the structures. In this case, the appearance of the mobile terminal 100 can be made more simple because the hole formed independently for the apparent acoustic output is hidden or hidden.

The optical output unit 154 is configured to output light for notifying the occurrence of an event. Examples of the event include a message reception, a call signal reception, a missed call, an alarm, a schedule notification, an email reception, and reception of information through an application. The control unit 180 may control the light output unit 154 to terminate the light output when the event confirmation of the user is detected.

The first camera 121a processes an image frame of a still image or a moving image obtained by the image sensor in the photographing mode or the video communication mode. The processed image frame can be displayed on the display unit 151 and can be stored in the memory 170. [

The first and second operation units 123a and 123b may be collectively referred to as a manipulating portion as an example of a user input unit 123 operated to receive a command for controlling the operation of the mobile terminal 100 have. The first and second operation units 123a and 123b can be employed in any manner as long as the user is in a tactile manner such as touch, push, scroll, or the like. In addition, the first and second operation units 123a and 123b may be employed in a manner that the user operates the apparatus without touching the user through a proximity touch, a hovering touch, or the like.

In this figure, the first operation unit 123a is a touch key, but the present invention is not limited thereto. For example, the first operation unit 123a may be a mechanical key or a combination of a touch key and a touch key.

The contents input by the first and second operation units 123a and 123b can be variously set. For example, the first operation unit 123a receives a command such as a menu, a home key, a cancellation, a search, and the like, and the second operation unit 123b receives a command from the first or second sound output unit 152a or 152b The size of the sound, and the change of the display unit 151 to the touch recognition mode.

On the other hand, a rear input unit (not shown) may be provided on the rear surface of the terminal body as another example of the user input unit 123. The rear input unit is operated to receive a command for controlling the operation of the mobile terminal 100, and input contents may be variously set. For example, commands such as power on / off, start, end, scrolling, and the like, the size adjustment of the sound output from the first and second sound output units 152a and 152b, And the like can be inputted. The rear input unit may be implemented as a touch input, a push input, or a combination thereof.

The rear input unit may be disposed so as to overlap with the front display unit 151 in the thickness direction of the terminal body. For example, the rear input unit may be disposed at the rear upper end of the terminal body such that when the user holds the terminal body with one hand, the rear input unit can be easily operated using the index finger. However, the present invention is not limited thereto, and the position of the rear input unit may be changed.

When a rear input unit is provided on the rear surface of the terminal body, a new type of user interface using the rear input unit can be realized. When the first operation unit 123a is not disposed on the front surface of the terminal body in place of at least a part of the functions of the first operation unit 123a provided on the front surface of the terminal body, The display unit 151 may be configured as a larger screen.

Meanwhile, the mobile terminal 100 may be provided with a fingerprint recognition sensor for recognizing the fingerprint of the user, and the controller 180 may use the fingerprint information sensed through the fingerprint recognition sensor as authentication means. The fingerprint recognition sensor may be embedded in the display unit 151 or the user input unit 123.

The microphone 122 is configured to receive the user's voice, other sounds, and the like. The microphone 122 may be provided at a plurality of locations to receive stereophonic sound.

The interface unit 160 is a path through which the mobile terminal 100 can be connected to an external device. For example, the interface unit 160 may include a connection terminal for connection with another device (for example, an earphone or an external speaker), a port for short-range communication (for example, an infrared port (IrDA Port), a Bluetooth port A wireless LAN port, or the like), or a power supply terminal for supplying power to the mobile terminal 100. The interface unit 160 may be implemented as a socket for receiving an external card such as a SIM (Subscriber Identification Module) or a UIM (User Identity Module) or a memory card for storing information.

And a second camera 121b may be disposed on a rear surface of the terminal body. In this case, the second camera 121b has a photographing direction which is substantially opposite to that of the first camera 121a.

The second camera 121b may include a plurality of lenses arranged along at least one line. The plurality of lenses may be arranged in a matrix form. Such a camera can be named an 'array camera'. When the second camera 121b is configured as an array camera, images can be taken in various ways using a plurality of lenses, and a better quality image can be obtained.

The flash 124 may be disposed adjacent to the second camera 121b. The flash 124 shines light toward the subject when the subject is photographed by the second camera 121b.

And a second sound output unit 152b may be additionally disposed in the terminal body. The second sound output unit 152b may implement a stereo function together with the first sound output unit 152a and may be used for implementing a speakerphone mode in a call.

The terminal body may be provided with at least one antenna for wireless communication. The antenna may be embedded in the terminal body or formed in the case. For example, an antenna constituting a part of the broadcast receiving module 111 (see FIG. 1A) may be configured to be able to be drawn out from the terminal body. Alternatively, the antenna may be formed in a film type and attached to the inner surface of the rear cover 103, or a case including a conductive material may be configured to function as an antenna.

The terminal body is provided with a power supply unit 190 (see FIG. 1A) for supplying power to the mobile terminal 100. The power supply unit 190 may include a battery 191 built in the terminal body or detachable from the outside of the terminal body.

The battery 191 may be configured to receive power through a power cable connected to the interface unit 160. In addition, the battery 191 may be configured to be wirelessly chargeable through a wireless charger. The wireless charging may be implemented by a magnetic induction method or a resonance method (magnetic resonance method).

The rear cover 103 is configured to be coupled to the rear case 102 so as to cover the battery 191 to restrict the release of the battery 191 and to protect the battery 191 from external impact and foreign matter . When the battery 191 is detachably attached to the terminal body, the rear cover 103 may be detachably coupled to the rear case 102.

The mobile terminal 100 may be provided with an accessory that protects the appearance or supports or expands the function of the mobile terminal 100. [ One example of such an accessory is a cover or pouch that covers or accommodates at least one side of the mobile terminal 100. [ The cover or pouch may be configured to interlock with the display unit 151 to expand the function of the mobile terminal 100. Another example of an accessory is a touch pen for supplementing or extending a touch input to the touch screen.

Referring back to the display unit 151, the bezel that surrounds the display unit 151 may be implemented in a bezel shape that is very thin or scarcely formed, in order to realize a sensible design while providing a larger screen. To this end, a separate area where a key such as a home key is disposed at the lower end and / or the upper end of the display unit 151 may be provided in the form of a touch key in the display unit 151. In this case, information displayed on the upper or lower end of the display unit 151 or a finger, slip, or the like for controlling the touch key may frequently occur. In addition, when the display unit 151 is implemented in a bezel shape, the length of the display unit 151 becomes longer, and the information output to the display unit 151 while controlling the terminal is controlled by one hand Became more difficult.

Accordingly, in the mobile terminal 100 according to the present invention having the structure of the above-described display unit, a panel area for more easily controlling the information output to the display unit 151 is provided in the display unit 151, And can perform various interactions with the information output to the display unit 151 using the information. Hereinafter, various operations performed in conjunction with the creation, removal, and modification of the panel region in the display unit 151 will be described in detail with reference to the drawings. Hereinafter, the display unit 151 may be referred to as a touch screen 151.

FIGS. 2A, 2B, 2C, 2D, 3A, 3B, and 3C are various examples illustrating the creation, removal, and modification of a panel region on a touch screen according to an embodiment of the present invention.

First, referring to FIG. 2A, an example of a method of creating a panel area in the touch screen 151 is shown. When a predetermined touch gesture on the touch screen 151, for example, a touch input started at the lower end of the touch screen 151 is dragged to face the upper side of the touch while the first screen information 201 is displayed in the touch screen 151 And the control unit 180 of the mobile terminal 100 may recognize the control command as an input of a control command for generating the panel area.

2B, an area having a predetermined transparency is gradually drawn out from the lower end of the touch screen 151 along the drag path of the drag input applied to the touch screen 151, so that a part of the first screen information (212). The display state of the first screen information 201 is continuously maintained while the area is drawn out. When the area is drawn from the lower end of the touch screen 151 to a predetermined line, the area is fixed to the corresponding area as shown in FIG. 2C (hereinafter referred to as a 'panel area').

Accordingly, the panel area 220 is displayed on the first part of the touch screen 151 and the first screen information 201 is continuously output on the second part. Both the first and second portions of the touch screen 151 remain activated. Accordingly, the user can control the information output to the panel area 220 based on the touch input to the first part, and can control the first screen information 201 based on the touch input to the second part .

The panel area 220 is disposed under the touch screen 151 and is formed to cover a part of the first screen information 201, as shown in FIG. 2C. The panel region 220 is formed to have a predetermined degree of transparency so that a portion C 'of the first screen information 201 displayed on the touch screen 151 passes through the panel region 220, ). That is, a part ('C') of the first screen information 201 hidden by the panel area 220 can be seen in a translucent manner.

2C, the panel area 220 may be formed to have a structure that covers the left and right sides of the touch screen 151, that is, a width corresponding to the width of the touch screen 151. [ In another example, the panel area 220 may be formed such that the touch screen 151 has a structure having a predetermined space in at least one of the left and right sides, that is, a width shorter than the width of the touch screen 151 . In this case, while the panel area 220 is being displayed, the first screen information 201 may continue to be displayed in the predetermined space.

In addition, information related to the first screen information 201 may be output to the panel area 220. In addition, the panel area 220 can receive a touch input and control information output to the panel area 220 based on the received touch input.

2D shows an example of a method of removing the panel area generated in the touch screen 151. FIG. A touch gesture defined in the panel area 220 such as a touch input initiated at any point of the panel area 220 is displayed on the touch screen 151 while the panel area 220 is displayed in the predetermined area of the touch screen 151. [ The control unit 180 of the mobile terminal 100 can recognize this as an input of a control command for removing the panel area. The panel area 220 is completely disappeared from the touch screen 151 as the displayed panel area 220 is gradually drawn in toward the lower end of the touch screen 151. [ When the panel area 220 disappears, the first screen information 201 is displayed on the entire touch screen 151.

Next, FIGS. 3A and 3B show examples in which panel regions are generated at different positions when the touch screen 151 is switched to the landscape mode. First, FIG. 3A shows a first panel area 220A drawn along a drag input that is raised from the lower end of the touch screen 151 in a state where the touch screen 151 is placed in the landscape mode. The first panel region 220A may be disposed such that the ratio of the horizontal and vertical lengths is adjusted so that the touch screen 151 corresponds to the horizontal mode. 3B shows a second panel area 220B drawn along the dragging from the left / right end of the touch screen 151 to the center with the touch screen 151 in the landscape mode. In this case, when the touch screen 151 is switched to the portrait mode, the first or second panel area 220A or 220B can be switched to the panel area 220 shown in FIG. 2C.

Next, FIG. 3C shows an example of changing the size of the panel area 220 using a drag-touch input. 3, when the touch input applied to any point or upper bar of the panel area 220 is dragged toward the top of the touch screen 151 while the panel area 220 is displayed in the touch screen 151, The position of the top bar of the area 220 moves along the drag path. As a result, the vertical length or height of the panel area is further increased (hereinafter, 'extended panel area'). At this time, the controller 180 can determine the vertical length or the height of the panel area 220L extended to correspond to the release point of the drag-touch input applied to the panel area 220. [ Alternatively, the control unit 180 may generate different control commands according to the release point of the drag-touch input applied to the panel area 220.

FIG. 4 is a flowchart illustrating an exemplary operation of the mobile terminal according to an exemplary embodiment of the present invention. Referring to FIG.

First, a first page is displayed on the touch screen 151 of the mobile terminal 100 according to the present invention (S10). Here, there is no limitation on the kind of the first page. The first page may be an initial screen of a specific application, a screen of a specific execution step, or a home screen screen. Also, the first page may be a screen corresponding to one of the plurality of tabs.

While the first page is displayed on the touch screen 151. The mobile terminal 100 may detect occurrence of an event that calls a panel area (S20).

Here, the panel area may be defined as an auxiliary screen area for providing information associated with the screen displayed on the touch screen 151. [ In addition, an event that invokes the panel area may be generated, for example, in response to the touch gesture being applied to the touch screen 151. In addition to the predetermined touch gesture, the event for calling the panel area may include a predetermined type of user input (e.g., a voice command, a hard key push input, a specific gesture input) or another kind of event occurrence Or the like). Hereinafter, all kinds of user input that generate such an event can be referred to as an 'input for calling a panel area'.

In response to the occurrence of the call event of the panel area, the control unit 180 creates a panel area having a predetermined transparency in one area of the touch screen 151 (S30).

Specifically, the panel area may be disposed under the touch screen 151 while an image gradually drawn out from the lower end of the touch screen 151 is output along the drag path of the drag touch input applied to the touch screen 151 . Once the panel area is created, a portion of the first page being output to the touch screen 151 may be translucent or almost invisible through the panel area. If a soft key such as a home key or a back key is displayed at the lower end of the touch screen 151 before the panel area is created, soft keys such as a home key and a back key may be displayed in a translucent manner at the bottom of the panel area , And can interact through touch input. On the other hand, the display state of the first page can be maintained in the portion of the touch screen 151 that is outside the panel area.

Next, the control unit 180 may provide the application result of the function associated with the first page to the generated panel region (S40). Here, the function associated with the first page may mean a specific function frequently used in the use of the application corresponding to the first page or the first page. To this end, the control unit 180 may record and learn the usage pattern of the user during execution of the application corresponding to the first page or the first page. Thereafter, when the application corresponding to the first page or the first page is executed again, the specific function frequently used can be immediately executed based on the learned usage pattern as soon as the input for calling the panel area is performed. Further, the object to which the specific function is applied may be a part of the first page, for example, information displayed in the area covered by the panel area from the first page.

If the size of the panel area is changed based on the touch input while the panel area is displayed, the controller 180 may control the application of the function associated with the first page differently (S50). Here, to control the application of the function differently may mean different kinds of functions to be applied, application range of the function, and / or application object of the function.

When the dragging touch input directed upward or downward at an arbitrary point of the panel area is applied, the controller 180 can display the variable length of the panel area along the direction and path of the drag input. When the vertical length of the panel area is changed as described above, a part of the first page covered by the panel area is increased / decreased and information displayed on the panel area is also changed. For example, as the panel area is pulled up, the coverage of the function associated with the first page can also be linearly increased. Alternatively, as long as the panel area is pulled down, the coverage of the function associated with the first page can also be linearly reduced. In such a case, an intuitive comparison of a portion of the first page to which the specific function is applied and the remaining portion of the original first page becomes possible.

Hereinafter, the process of the flowchart of FIG. 4 will be specifically described with reference to FIG.

First, a specific web page 501 displayed in a foreign language (e.g., English), which is not a native language, may be displayed on the touch screen 151 of the mobile terminal 100. In this case, the control unit 180 recognizes a function frequently used in an application corresponding to a specific web page 501 or a specific web page 501 in the background in the background, for example, a target program for executing a translation function in advance . To this end, the memory 170 of the mobile terminal 100 may store history and usage patterns of functions used in conjunction with the specific web page 501 displayed. On the other hand, if the usage pattern is not yet formed, a table matched with a specific function linked to each application or a specific step of the application may be utilized.

When a dragging touch input (or a flicking / swipe input) that is directed upward from the lower end of the touch screen 151 is applied while the specific web page 501 is displayed, the control unit 180 calls the panel area It is recognized that an event is generated. Accordingly, a panel region 520 having a predetermined transparency can be generated in a predetermined area of the touch screen 151. [ At this time, the predetermined area may be formed such that a vertical length is formed to have a predetermined height at the lower end of the touch screen 151, for example, and a horizontal length is formed to correspond to the horizontal length of the touch screen 151.

In response to the receipt of an input corresponding to the event calling the panel area 520, the control unit 180 can immediately execute a frequently used function, for example a translation program, in conjunction with the particular web page 501 displayed have. In the generated panel area, a result (502a) of translating a foreign language (e.g., English) displayed in a specific web page 501 into a native language (e.g., Korean) can be displayed. At this time, the range to which the translation function is applied may be a foreign language notation included in the web page portion which is covered by the panel region 520 in a translucent manner.

5, a portion of the web page 501 translated into the mother language is displayed in the panel area 520 below the touch screen 151, and a portion of the web page 501 translated into the mother language is displayed on the upper side of the touch screen 151, The original web page 501 portion is displayed. 5, only some translation results ('I want to call me') 502a are displayed in the panel area 520, but in one example, the corresponding original sentences ('I miss you guy. me ') may be displayed together.

Next, if the size of the panel area 520 is changed based on the touch input, the control unit 180 can control the function associated with the displayed specific web page 501, that is, the range of application of the translation function.

More specifically, when an upward drag input is applied at an arbitrary point in the panel area 520, one end of the panel area 520 is fixed to the lower end of the touch screen 151, and the other end of the panel area 520 is It is pulled up along the path. That is, the size of the panel area is expanded. More parts of the web page 501 are then obscured by the expanded panel area 520L.

The control unit 180 continuously translates the foreign language notation included in the web page 501 which is linearly hidden along the path of the drag input and further provides it in the extended panel area 520L. To this end, when a drag input is applied to the panel area 520, the controller 180 generates an image whose size gradually changes in the panel area 520 along the drag path of the drag input. Then, the application result of the function associated with the first page is displayed in the image. At this time, along with the drag path of the drag input, the portion of the original web page 501 written in a foreign language is gradually reduced, and the portion of the translated web page 501 translated into the mother language displayed in the expanded panel region 520L is gradually . On the other hand, when the upper line of the extended panel area 520L is located in the middle of the text lines of the original web page 501, only the text line contained in the expanded panel area 520L is completely translated, The line may be displayed in the expanded panel area 520L in an untranslated state. The user can slide the panel area up and down to quickly compare before and after the translation on a line-by-line basis.

In addition, the control unit 180 may perform the processing of the image differently depending on the point at which the drag input applied to the panel area 520 is released.

For example, when the drag input applied to the panel area 520 is released at the top of the touch screen 151, one end of the image is fixed to the lower end of the touch screen 151, The extended panel region 520L is displayed in the full window mode within the panel region of the full window mode and the mother language translation result 502c When the scroll operation is performed on the touch screen 151, only the text information translated into the native language will be displayed while the automatic translation function is continued. In this case, when the reference time elapses, The specific web page 501 is page-switched to the translation result displayed in the panel area of the full window mode, and the display of the panel area can be automatically terminated. The color display panel showing a predetermined region (for example, gray) may be missing.

As another example, when the drag input applied to the panel area 520 is released without exceeding the middle line of the touch screen 151, the extended panel area 520L is displayed only during the reference time (e.g., 2 to 3 seconds) And returns to the panel area 520 of the original size. At this time, the visual effect that the expanded panel area 520L is elastically reduced or gradually reduced in size like a rubber band can be outputted. In addition, the text line of the portion of the web page 501 that has been exposed as much as the reduced size can be converted back to the original foreign language notation.

Meanwhile, although not shown, when the touch input applied to the panel area is dragged downward while the panel area 520 (or the extended panel area 520L or the panel area of the full-screen mode) is displayed, The display of the panel area can be terminated regardless. Then, the translation result of the specific web page 501 disappears, and the web page 501 written in a foreign language can be displayed in the full window mode.

In response to the change of a portion of the specific web page 501 covered by the panel region along the drag path of the drag input applied to the panel region, the control unit 180 displays a translation in association with the specific web page 501 Function may be gradually applied or de-applied to a portion of a particular web page 501. [ Thus, while the drag input applied to the panel area is being dragged up and down, the translation applied to a portion of a particular web page 501 can be linearly reverted to the original foreign language notation or linearly (or rapidly) switched to native language notation have. Accordingly, it is possible to quickly compare the foreign language representation and the native language representation on a line-by-line basis. Thereafter, when dragging downward from one point of the panel area is released, the controller 180 may terminate the display of the panel area.

As described above, in the present invention, while a specific screen is being displayed, a panel area can be fetched and a frequently used function can be executed immediately in conjunction with a specific screen. Also, by sliding the panel area up and down, it is possible to control the application range of the function and the application target, and it is possible to quickly compare before and after application of the function.

6A, 6B, 7A, 7B, 8A, 8B, 9A, 9B, 9C, 10, and 11, Lt; RTI ID = 0.0 > and / or < / RTI >

6A illustrates an example in which a predetermined filter value is applied to a camera preview screen by varying the size of a panel region. 6A, when a touch gesture for calling a panel area is received while a preview screen 601 corresponding to the execution of a camera application is displayed on the touch screen 151, various filter values associated with the preview screen 601 The icons 611, 612, 613, and 614 corresponding to the icons 611, 612, 613, and 614 are displayed in the panel area 620. At this time, if there is no desired filter value, a desired filter value may be searched by applying a swipe input to the panel area from left to right.

When the icon 611 of the desired filter value is selected, the application result of the filter value is immediately displayed on the entire panel area 620. [ Accordingly, the filter value is applied to the portion 602a corresponding to the panel region in the preview screen 601, and the remaining portions remain in the original state, so that the before / after application of the filter value is immediately compared. Also, if different filter values are to be applied to the preview screen 601 differently, one of the other icons 612, 613, and 614 can be selected and executed.

On the other hand, if a drag input for pulling up the panel area 620 is applied, the application of the desired filter value can be further extended to the preview screen. Specifically, when the touch input applied to the upper line of the panel area 620 is dragged upward, the size of the panel area gradually increases (620L) along the drag path, the portion 602b to which the filter value is applied in the preview screen, Is gradually increased. At this time, if the drag input continues to the top of the touch screen 151, the corresponding filter value can be applied to the entire preview screen 602c.

Next, FIG. 6B shows an example of adjusting the panorama length of the camera by varying the size of the panel area. 6B, when the drag input is applied from the right end to the left end of the touch screen 151 while the display direction of the touch screen 151 is in the horizontal direction in the camera panorama mode, At the end, the panel area is pulled out and the panorama length adjustment function is immediately interlocked. At this time, while the specific scene 603 of the panorama is being displayed, a thumbnail image 630 informing the current position PP, the previous 631 scene and the following scene 632 based on the current position is displayed on the touch screen 151 ). ≪ / RTI > As the panel area 620 is drawn, a part of the current scene in the thumbnail image 630 is shortened by the length of the panel area drawn (632a).

Next, when the panel area is further expanded (620L) along the drag path of the drag input applied to the drawn panel area, the length after the current time of the panorama is further removed by the length of the expanded panel area (632b). On the other hand, if the panel region is pushed to the right end of the touch screen 151, the shortened panorama length will become longer again. When the drag input applied to the panel area is released, the panel area is automatically disappeared (630 ') while the length of the panorama is determined by the length.

Here, although not shown, the panel area may be drawn at the left end of the touch screen 151 to adjust the previous panorama length of the current position. In addition, the above-described operation may be similarly applied to adjusting the continuous shooting length of the camera burst shot.

FIGS. 7A and 7B illustrate examples in which the size of the panel area is changed to quickly identify desired event contents.

7A, a touch screen 151 may display a home screen including a plurality of objects. At this time, the plurality of objects may include, for example, icons, widgets, buttons, menus, files, folders, images, links, and the like of the application. Also, at least some of the plurality of objects may be displayed with a graphic object in the form of a badge informing that there is an unidentified event in the corresponding application. Here, an event may occur when an event that affects the operation of at least one application installed in the terminal occurs or a change of an item of a database (or a data file) associated with at least one application occurs, or when the at least one application Quot; means that data is transmitted or transmitted to an external terminal or an external network through the network. For example, 'call reception' or 'message reception' may be an example of the event. Accordingly, an unacknowledged event such as an update completion, an unacknowledged call reception, an unidentified message reception, or the like may be an unidentified event.

7A, a first icon 712 without an event occurrence is displayed as it is, and a first graphic object 711b indicating that there are two unanswered call reception is displayed on the telephone application icon 711, and 2 It is confirmed that a second graphic object 713b indicating that there is an arrival e-mail date is displayed in the e-mail application icon 713. [

At this time, when an input for calling the panel area is received, the controller 180 can recognize whether the object displayed in the home screen 701 hidden by the drawn panel area and an event occurred in the object. If there is an event corresponding to the object hidden by the panel area, the controller 180 can provide the event information of the object in the panel area. Accordingly, two pieces of information related to a missed call (e.g., Tom, Mike missed calls) 721 and 722 are provided in the panel area 820 as shown in FIG. 7A.

Thereafter, if the email application icon 713 is included in the panel area 720 by further pulling up the top of the panel area 720 (720l) based on the drag input, two pieces of arrival email related information 723 and 724 And is additionally provided in the panel region 7201. At this time, the event related information provided in the panel area 720 or 720l may be provided with a menu icon for instantly responding to the contents of the event or for entering the corresponding application step immediately.

When the upper end of the panel area 720 is raised to the upper end of the touch screen 151, event information corresponding to the object displayed on the displayed home screen 701 is displayed, and the panel area disappears when the reference time elapses. At this time, all the graphic objects indicating the unidentified event displayed on the home screen 701 disappear, and the objects are displayed in the confirmation processing state (713 ', 714').

7B, while the locked screen is displayed on the touch screen 151, notification icons indicating the occurrence of the event may be displayed on the lock screen 702. [ At this time, in order to protect personal information, only the minimum information related to the event, for example, the arrival of the message and the identification information of the corresponding application, may be provided on the lock screen 702. [

In this locked state, when an input for calling the panel area is received, event information (e.g., 'see') 721 corresponding to the first notification icon that is received in the drawn panel area 720 Can be displayed immediately. At this time, the display of the event information 721 corresponds to the position and the area of the corresponding notification icon on the lock screen. The event information 721 may be a preview of the generated event content (e.g., a preview of the received message content).

When the panel area 720 is gradually pulled up, the event information 722 corresponding to the second notification icon and the event information 722 corresponding to the third notification icon, together with the event information 721 corresponding to the first notification icon, The display panel 723 can be additionally displayed in the expanded panel areas 720a and 720b. At this time, in the case of an event related to the setting of a specific operation such as the second and third notification icons, command buttons (e.g., 'snooze', 'stop') for changing the set specific operation or setting the next operation Information can be provided.

Also, although not shown, when the extended panel area 720a or 720b is pulled down, the displayed event information 723, 722, and 721 are displayed in the reverse order of display, the corresponding notification icons are displayed again Maintenance) or confirmation and disappeared. Thereafter, when the panel area is pulled up again from the lock screen, it is determined that there is no event information to be displayed any more, and the application result of the other function related to the lock screen is provided to the panel area or the operation shown in FIG. 7B can be repeated have.

Hereinafter, with reference to Figs. 8A and 8B, examples of changing the size of the panel area and rapidly changing or adding means of information input will be described.

8A, while the first page 801 is displayed on the touch screen 151, a virtual keyboard 802 for inputting information on the first page may be popped up below the touch screen 151 have. At this time, in order to use the function of another application as a means of inputting information, an attaching icon was selected from a virtual keyboard, and then an additional input process was required to enter the desired application. In particular, there was a disadvantage in that, in order to override the function once selected, the user must perform a backward operation and then perform the above-described procedure from the beginning.

8A shows an example in which the size of the panel area is changed to quickly change information input means corresponding to various applications. Specifically, while the virtual keyboard 802 is being displayed, the clipboard area 803 may be immediately provided within the panel area when an input is received that calls the panel area. For example, when an input for calling the panel area is received, the clipboard area 803 can be gradually drawn out from the bottom of the touch screen 151. [

In addition, if the size of the panel area is variable, the control unit 180 can provide extended information input means. At this time, the control unit 180 can change the information input means in stages according to the degree of variation of the size of the panel area.

Specifically, when the clipboard area 803 is continuously pulled up, a thumbnail list 804 of the gallery application appears in the expanded panel area 820L. At this time, a desired image can be searched by applying a scroll operation to the thumbnail list 804, and when a specific image is selected, the corresponding image is input as information. When the expanded panel area 820L is pulled up to the upper end of the touch screen 151, the thumbnail list 804 is switched to the preview screen 805 of the camera application. When the photographing is performed through the preview screen 805, the photographed image is input as information.

On the other hand, if a downward drag input is applied from the upper end of the preview screen 805, the different information input means 804, 803, and 802 described above are displayed in reverse order according to the drag degree of the drag input. Accordingly, the user can quickly search for and select information input means of various applications by simply dragging the panel area up and down.

As another example, while a virtual keyboard 802 of a specific type (e.g., English) is displayed on the touch screen 151 as shown in FIG. 8B, other types of virtual keyboards You can add it quickly. For example, if alphabets and symbols need to be repeated frequently when inputting information into the displayed page 801, the virtual keyboard 802 may be swiped to the left or right each time, or the symbol / character keys may be toggled, There has been an inconvenience to change each time.

Thus, as shown in FIG. 8B, when a virtual keyboard 802 of a specific type (e.g., English) is displayed, a panel area 820 is called to display a virtual keyboard 802 of the displayed type (e.g., English) ) And to add another type of virtual keyboard (e.g., symbol) 802a below the virtual keyboard 802. [ As a result, the user can continuously input English text and symbols without interruption.

In addition, the control unit 180 displays a virtual keyboard area of a specific type (hereinafter referred to as a 'first type keyboard') 802 and a new keyboard area displayed in the panel area according to the degree of dragging the drag- (Hereinafter, " second type keyboard ") 802a.

At this time, the range of the keys scrolled according to the release point of the drag-touch input may be changed. For example, if the drag input is continued from the lower end of the touch screen 151 to the upper end of the first type keyboard 802 as shown in FIG. 8B, the panel area is expanded 820l, All of the keys of the keyboards of the keyboards are switched to different kinds of keys (802b). On the other hand, if the drag input started at the lower end of the touch screen 151 is released in the second type keyboard 802a or connected to the upper end of the first type keyboard 802 at the lower end of the second type keyboard 802a, Only the keys of the first or second type keyboards included in the path will be switched to different types and the keys of the other keyboards will be maintained.

Hereinafter, with reference to Figs. 9A and 9B, examples will be described in which the upper line of the panel area is moved to quickly confirm the details of the displayed plurality of items.

9A, a touch screen 151 of the mobile terminal 100 is displayed with a list (FIG. 9A) in which a plurality of message room items 911, 912, 913, 914, 915, 915 corresponding to the execution of a message application are displayed 910) may be displayed. When an input for calling the panel area is received while the list 910 is displayed, the control unit 180 can recognize an item in contact with the top line of the panel area. Then, the controller 180 may provide the detailed information of the recognized item in the retrieved panel area 920. [

A portion 921 of the dialogue screen of the 'Amy' message room item 915 in contact with the upper line TL1 of the original panel area 920 is displayed on the panel Area. At this time, when a vertical flicking input is applied at an arbitrary point except the upper end of the panel area 920, a scroll signal is generated and another part of the dialog screen is searched. On the other hand, when the touch point of the touch input applied to the upper line TL1 of the panel area 920 is moved upward, the position of the upper line TL1 is changed while the panel area is expanded 920l.

The user can move the upper line TL1 of the panel area 9201 to the position of a desired item and quickly check the corresponding detailed information without entering the item. For example, when the top line TL1 of the panel area 9201 moves to the message room item 913 using the terminal number of 010-1234-4567, the conversation corresponding to the corresponding message room item 913 A portion of the screen is provided in the expanded panel area 9201. Accordingly, the user can quickly search for and check the details of a plurality of items while maintaining a list including a plurality of items.

FIG. 9B illustrates a method of quickly moving the position of the top line of the panel area when the contact list 930 corresponding to the execution of the telephone application is displayed on the touch screen 151 to quickly check the contact information of a specific person in the contact list 930 . 9B, contact information corresponding to the 'Amy' contact item 935 is provided in the panel area 920. At this time, contact information of the same person recorded in another application associated with the telephone application (eg, , E-mail, other contact means, profile information, etc.). In addition, execution menus may be provided within the panel area 920, which may call or send a message to 'Amy'. According to the above-described embodiment, the panel area can be slid up and down to quickly search and confirm detailed information about an item that is in contact with the top line that is slid up / down.

As another example, FIG. 9C shows an example of processing information provided to the panel area differently according to the release point of the drag touch input, without changing the size of the panel area.

9C, a plurality of images 941, 942, and 943 corresponding to the execution of the gallery application may be displayed on the touch screen 151 as a list 930 in the form of a grid view . While the list 930 is being displayed, an image editing area can be used as the drawn panel area 920 when an input for calling the panel area is received. At this time, the image to be edited is selected as the image outputted at the point where the drag input started at an arbitrary point in the panel area 920 is released.

Specifically, the detail view 924 of the first image is displayed in the panel area 920 while the drag touch input is fixed to the first image 943 in the list 930. [ When the drag-touch input is released at the position, the edit mode of the detail view 924 is immediately executed. Accordingly, a variety of tool boxes corresponding to the edit mode can be provided in the panel area 920, and editing operations such as drawing, cropping, coloring, and the like can be performed on the detail view 924 using the tool box provided have.

On the other hand, if the drag-touch input started at any point in the panel area 920 is released from the second image 942 at the top of the list, a detailed view of the second image 942 is provided in the panel area. At this time, while the drag-touch input is held, the size of the panel area can be maintained. In addition, while the drag-touch input is held, the up-and-down scrolling of the list 930 can be broken.

In the case where the moving image is included in the list 930 and the drag-touch input is fixed to a specific moving image in the list 930 or released from the specific moving image, the panel region 920 displays not the still image of the moving image, Can be continuously reproduced. Alternatively, even if the touch input is fixed to the burst shot or panorama image of the list 930, or the touch input is released in the image, the panel area 920 may have all of the burst images of the burst shot, Images may all be provided.

Hereinafter, FIG. 10 shows an example of automatically recording and storing call content by calling a panel area during a call.

Referring to FIG. 10, when an input for calling a panel area is received during a voice / video call, the controller 180 can immediately enter the voice recognition mode. In the speech recognition mode (Speech To Text, STT), the controller 180 converts the voice of a user who is in a voice / video call into a text into a text, The converted text 1021 can be provided in the area 1020 in real time. At this time, in the panel region 1020, areas displayed in different colors or separated from each other (for example, one on the left side and one on the left side are displayed on the right side in the panel area 1020 so as to be visually easily distinguished from the voice uttered by the user and the voice uttered by the other party. ). ≪ / RTI > In addition, it is possible to quickly search recorded contents even during a call by applying a flicking touch input in left / right / up and down directions within the panel area 1020 or 10201. When the voice / video call is ended, the activated voice recognition mode can be automatically terminated.

On the other hand, when a drag input for pulling up the panel area 1020 is applied, more text is provided in the expanded panel area 10201 while the size of the panel area is expanded 1020l. In addition, when the panel area 10201 is pulled up more than the reference range, menus (for example, editing and storing) B1 and B2 related to the processing of the recorded conversation contents can be provided at the bottom of the panel area 10201. When the drag input applied to the panel area 10201 reaches the upper end of the touch screen 151, the voice recognition mode is terminated and all the conversation contents recorded in the panel area 10201 are immediately stored. Thereafter, if a voice call is in progress, the screen is switched to the execution screen 1002 of a specific application (e.g., Q-memo application) that can view and edit the stored call content, as shown in FIG. On the other hand, during the video call, the busy screen 1001 can be continuously displayed.

11 shows an example in which the size of the panel area is varied to gradually apply the sub-functions of the specific functions in a stepwise manner.

Referring to FIG. 11, an execution screen 1101 of an SNS application may be displayed on the touch screen 151. FIG. When an input for calling the panel area is received while the execution screen 1101 is being displayed, the friend information 1121 set in the SNS application is provided to the panel area 1120, for example, in the form of an execution icon.

At this time, the controller 180 can detect the sub-function of the function associated with the first page, in response to the variable size of the panel area, based on the drag input applied to the panel area. In addition, the control unit 180 may provide the result of weighting the detected lower function to the first page and applying the result to the panel area. For example, in FIG. 11, the latest update information aggregator function of a friend set as a favorite is detected as a sub-function of the friend information 1121 set as favorites, and the collected latest update information gather view 1122 is additionally expanded May be provided in the panel area 11201.

When the extended panel area 11201 is pulled up to the upper end of the touch screen 151, a reply or comment creation screen 1123 for the collected latest update information is displayed in the panel area of the full screen mode. According to this, the user can quickly enter the sub-functions of the currently viewed page step by step using the panel area without entering the application.

Meanwhile, while the extended panel area 11201 is dragged down on the basis of the drag input, the collected latest update information gather view 1122 and the favorites set friend information 1121 are displayed in the reverse order. When the corresponding drag input is released, the display of the panel area is terminated. According to this, the user can quickly enter or exit the lower / higher level of the currently viewed page simply by sliding the panel area up and down.

In the above, various examples of rapidly varying the size of the panel region and controlling the application of the specific function have been described. In the following, various examples related to utilizing the usage pattern of the user in determining this specific function will be described.

12 shows an example of automatically executing a specific function associated with a usage pattern on the touch screen 151 while the execution screen 1201 of the map application is displayed.

Specifically, when an input for calling the panel area is received while the execution screen 1201 of the map application is being displayed, the control unit 180 displays an associated function on the displayed execution screen 1201 based on the usage pattern repeated in the past You can navigate immediately. Then, the function searched for while the panel area is drawn is immediately executed. For example, an indicator 1210 indicating that the searched function (e.g., friend searching function) is being executed is displayed as shown in Fig. 12 while a graphic object to which the panel area is drawn is displayed (omitted in Fig. 12) do.

On the other hand, the control unit 180 can generate and store repeated usage patterns in the execution screen 1201 of the map application or the map application. Here, the usage pattern may be generated when a specific menu, function, or invocation or execution of another application is repeated a predetermined number of times (for example, five or more times) on the page. In addition, the usage pattern can be generated in consideration of environmental conditions such as a specific menu, a function, a location of the terminal in a call or execution of another application, a current time, and an operation state of the terminal in the corresponding page. For example, FIG. 12 is an example of a case where the user repeatedly uses the friend search function when executing the map application. Alternatively, if navigation functionality is primarily used at the time of execution of the map application, the navigation function that guides to a specific destination (e.g., home) will be executed immediately upon receipt of an input that calls the panel area.

Also, the control unit 180 can continuously learn and update the usage pattern based on the processing on the information displayed in the panel area.

Next, a map screen 1202 showing the results of execution of the friend searching function displays the positions (FP1, FP2, FP3) of friends. At this time, the display of the panel area may be omitted or output in the full-screen mode. In the latter case, the background color of the map screen 1202 may be different from the previous one. On the other hand, if a downward drag input is applied from the top of the touch screen 151, the execution of the friend searching function is terminated and the execution screen 1201 of the map application is displayed again.

As another example, FIG. 13 shows an example of quickly calling the security card information based on the usage pattern on the account transfer screen 1301 of the financial application.

Specifically, when an input for calling the panel area is received while the account transfer screen 1301 is being displayed, the control unit 180 executes a gallery application mainly on the screen 1301 to display a usage pattern for searching for a secure card image Can be detected. Accordingly, the process for providing the secure card image stored in the gallery application in the retrieved panel area 1320 can be executed immediately.

At this time, for example, the fingerprint recognition screen 1321 may be displayed in the panel area 1320 in order to authenticate the user in terms of personal information protection. When the fingerprint recognition is completed, the called secure card image 1322 is immediately provided in the panel area 1320. Meanwhile, when there are a plurality of called security card images, it is possible to search an appropriate security card image by applying a left-right swipe input to the panel area 1320. [

As another example, FIG. 14 shows an example in which a specific function is automatically executed in consideration of surrounding environmental conditions, for example, illuminance information, when a panel area is called.

Specifically, as shown in Fig. 14, when the illuminance value around the terminal is dark at or below the reference range at the time when the panel area is generated, or when the corresponding position and time zone are the outdoor night time zone, The screen filter value may be applied to the page (1402). For this, the controller 180 activates the illuminance sensor while the first page is being displayed, and based on the illuminance value obtained through the illuminance sensor at the time when the input for calling the panel area is received, You can search for filter values.

On the other hand, if the illuminance value around the terminal is much brighter than the reference range at the time when the panel area is generated, or if the corresponding position and time zone is the outdoor noon time zone, the touch screen 151 displays a screen filter value Can be applied to the displayed page.

Figures 15A, 15B and 15C illustrate examples of calling the panel area to quickly reconstruct the displayed page according to user preference.

15A, a web page 1501 may be displayed on a touch screen 151, and one or more advertisement contents 1511, 1512, and 1513 may be included in a web page 1501 . At this time, the user may remove the advertisement contents 1511, 1512, and 1513 from the web page 1501 by inputting a call to the panel area.

Specifically, when an input for calling a panel area is received while the web page 1501 is displayed, the control unit 180 can extract a specific type of information from the displayed web page 1501. [ Here, the specific type of information may be advertisement content, for example, and the specific type may be determined based on the usage pattern of the user. Next, the control unit 180 reconstructs the display of the displayed web page 1501 with the extracted specific type information. Accordingly, other information included in the web page 1501 may be filled in the empty space from which the advertisement contents are removed, and information displayed on the scrolled space or other page may be arranged in the space where the other information was present.

At this time, notice information (e.g., hidden advertisement) indicating that the displayed web page 1501 'is reconstructed may be displayed in the drawn-out panel area 1520. If the panel area 1520 is pulled down and terminated, the reconstructed web page 1501 'returns to its original state again. Accordingly, the web page 1501 including the advertisement contents 1511, 1512, and 1513 will be displayed.

FIG. 15B shows an example of quickly removing spam message items from the list 1502 corresponding to the execution of the message application. Specifically, while the list 1502 including a number of message items 1514, 1515, 1516, 1517, 1518 is displayed, when an input for calling the panel area is received, the control unit 180 displays the corresponding list 1502 ) To identify spam message items. Then, the identified message items may be removed from the list 1502 to display the reconstructed list 1502 '. At this time, a message 1522 informing that the spam message is hidden may be displayed in the drawn-out panel area 1520. The reconstructed list 1502 'includes only the message items 1531, 1532 and 1533 stored in the telephone number list.

FIG. 15C shows an example of quickly gathering only a specific type of information in a page 1503 made up of various types of information, that is, multimedia information. Referring to FIG. 15C, a web page 1503 including various types of multimedia information such as text, images, moving pictures, and the like may be displayed on the touch screen 151. [ When the input for calling the panel area is received while the web page 1503 is displayed, the page 1503a reconstructed from the web page 1503 with text type information is displayed. The first notification information (e.g., Text) 1523 indicating the type of the currently displayed information may be displayed in the drawn-out panel area 1520. At this time, second announcement information (e.g., Image) 1524 may be displayed on the right side of the first announcement information to guide switching to another type. When the user applies a flicking input to the panel area 1520 along the display direction of the second announcement information, the page 1503a reconstructed with the text type information is converted into the image type information corresponding to the second announcement information 1524 To the page 1503b reconstructed by the user.

At this time, the second announcement information 1524 is placed in the area where the first announcement information was displayed in the panel area 1520. Then, the first notification information 1523 moves to the left of the panel area 1520. On the right side of the panel area 1520, third notification information (e.g., Video) 1525 for guiding the switch to another type may be displayed. The user can flick the panel area 1520 left and right to quickly identify the reconstructed page only with the desired type of information.

In this state, when the panel area 1520 is pulled down, the original web page 1503 including all the multimedia information is displayed while the panel area 1520 is terminated.

Also, although not shown, when the panel area is executed in a list including a plurality of items, only the items with an unidentified event can be extracted and the reconstructed list can be displayed. In this case, it is not necessary to repeat the scroll operation to search for an item with an unidentified event. In addition, by performing the operations described with reference to FIGS. 9A and 9B in the reconfigured list, it is possible to quickly identify a specific event using the panel region without entering an item having an unidentified event.

Hereinafter, various embodiments related to providing a function associated with a specific object included in a displayed page to a panel area will be described with reference to Figs. 16A, 16B, 17A, 17B, 17C, 18A, I will explain.

When a specific object is selected on the displayed first page, the control unit 180 may apply the function associated with the first page to the selected specific object, and provide the application result of the function to the panel area.

In addition, in one embodiment, the controller 180 may determine the type of the function associated with the first page according to the type of information included in the selected specific object. For example, when an image object is selected on a particular page, and when a text object is selected, the functions performed through the panel area can be quite different.

Further, in one embodiment, the search range of the function may be changed depending on whether the object selection on the first page is before or after the panel area is generated. For example, if the object selection is made before the creation of the panel area, it is determined that the intention of the user intends to receive more detailed information related to the object, and the purpose of displaying the object on the page or the sharing intention, It is possible to search the related functions in a limited manner. That is, the search range of the function to be applied to a specific object may be limited to the overall contents of the page, the display purpose of the object, or the sharing intention. On the other hand, if the object is selected after the panel region is created, the object search is not initially intentionally made at first, and the search range of the function may not be limited specifically.

FIGS. 16A and 16B illustrate examples when a user does not intend to select a specific object from the beginning, that is, when a panel region is first created and then a specific object is selected.

First, FIG. 16A shows an example of providing a pre-search result of a specific object to a panel area. When an input for creating the panel area 1620 is received while the web page 1601 is displayed on the touch screen 151, the application result of the function associated with the web page 1601 (e.g., translated into Korean) May be provided in the panel area 1620. [ At this time, if the user performs the proximity touch to the web page 1601, the dictionary search function which is one of general functions can be searched. Accordingly, a notification icon 1611 for guiding the selection of a specific object is popped up on the top of the web page 1601.

In response to the pop-up notification icon 1611, if a specific word (e.g., an effect) 1612 is selected, a panel 1620 may be provided with a recommendation word screen 1621 for verifying that the selected word is correct . When a word to be searched for in the word selection screen 1621 is selected, the controller 180 searches the meaning of the word and provides the search result to the panel area 1620 immediately. When a touch input is successively applied to another word on the web page 1601, a dictionary search result of a word to which a touch input is applied will be provided in the panel area 1620.

In another example, FIG. 16B shows an example of providing a contact search result of a specific person corresponding to a specific object. For example, when a panel area is called while a message conversation screen 1602 with a specific person is displayed, special information may not be provided in the drawn panel area 1620 as shown in FIG. 16B.

At this time, if a predetermined touch input (e.g., long-touch input or strong touch input) is applied to the message area 1613 of the specific person in the message conversation screen 1602, the message area 1613 is transmitted The contact information 1623 of the other party may be provided in the panel area 1620. [ At this time, if the call origination icon is touched in the contact information 1623 of the other party, the message conversation screen 1602 is switched to the call screen 1603 and the call origination is immediately performed.

17A, 17B, and 17C show examples in which a user intends to apply a function only to a specific object, and a panel region is created after selecting a specific object first.

Referring to FIG. 17A, the map information 1711 shared by the message conversation screen 1701 can be selected as an object. At this time, the selection of the object can be performed, for example, by applying a touch input to the message area in which the map information 1711 is output, which exceeds the reference pressure. When the input for calling the panel area is received, the controller 180 recognizes the sharing purpose (e.g., meeting place notification) of the shared map information 1711 in the drawn-out panel area, (E.g., navigation information from the current position of the terminal to the meeting place displayed on the shared content, accommodation information around the meeting place displayed on the shared content, etc.) 1721 and 1722 can be provided to the panel area . When the desired specific information 1721 is selected in the panel area 1720, the detailed view of the information 1721 is displayed in the full-screen mode (1702).

In another example, FIG. 17B shows a case in which the panel area is called after the meal date information 1712 shared in the message conversation screen 1703 is selected as an object. Specifically, when an input for calling the panel area is received, the control unit 180 recognizes the sharing purpose (e.g., a drink date notification) of the drinking date information 1712 and displays a screen of the application associated with the sharing purpose, A calendar screen 1723 of the application can be called and provided to the panel area. The calendar display 1723 can automatically display the schedule display 1723P on the shared drinking date. The user can select the storage button provided at the top of the panel area 1720 to store the shared date of drinking as a schedule.

In another example, FIG. 17C shows a case where a panel word is called after an included word (e.g., V20) 1713 of a specific message sent from the message conversation screen 1704 is selected as an object. Specifically, when an input for calling the panel area is received, the controller 180 recognizes the selected word and then determines the purpose of the selected word in consideration of the entire conversation contents of the message conversation screen 1704. For example, if the entire conversation content of the message conversation screen 1704 is associated with a new mobile phone, the control unit 180 may limit the web search range of the selected 'V20' word by associating with the new mobile phone. Accordingly, the V20 launching news information 17240 can be provided as a web search result in the drawn-out panel area 1720. At this time, when the sharing button provided at the top of the panel area 1720 is pushed, The search result 1724 can be immediately shared with the other party of the message conversation screen 1704 (1714).

Next, FIGS. 18A and 18B show an example in which, when a specific person is selected as an object in the image stored in the gallery application, the related regions are quickly searched and sorted using the panel region.

18A, when a specific image 1801 is displayed on the touch screen 151 and a face area is selected based on the touch input in the specific image 1801, It is possible to recognize a specific person to be a person. Accordingly, an indication (e.g., 'Tom') 1811 indicating the recognized specific person may be displayed around the selected face area. Thereafter, when an input for calling the panel area is received, the control unit 180 searches the gallery application for images containing the recognized specific person and provides the images to the panel area 1820. [ Accordingly, photographs of other time / place including a specific person can be immediately confirmed. Selecting one of the images provided in the panel area 1820 terminates the panel area and switches the displayed specific image 1801 to the selected image 1821 detail view.

Next, referring to FIG. 18B, when the panel area is again called during the execution of the gallery application, images including the most recently provided images, i.e., 'Tom', are again provided to the panel area 1820. Thereafter, when a specific image 1821 of the images provided in the panel area 1820 is selected, the detail area 1802 of the selected image is displayed in the full-screen mode and the panel area 1820 is terminated. At this time, the controller 180 may store the path information of the image displayed in the full-screen mode and the display order. Thus, when the panel area 1820 is called again, then, not only the images including 'Tom', but also the image 1801 that was displayed in the full mode before the specific image 1821 was selected is also displayed in the panel area 1820 / RTI > Accordingly, the user can quickly move to the position of the last viewed image using the panel area.

Also, although not shown, if a touch input exceeding the reference pressure is applied to the background area while the specific image 1801 or 1802 is displayed in the full-screen mode, the controller 180 displays the location . Thereafter, when an input for calling the panel area is received, images photographed at the same location may be arranged in the panel area.

As described above, according to the embodiment of the present invention, in a display unit having a bezel shape or a thin bezel, information output to a display unit and an auxiliary screen area interact with each other and multitasking related to information output to the display unit It can be done more easily. In addition, by sliding the sub screen area up and down, it is possible to rapidly change the application range and application target of a specific function. Accordingly, it is possible to immediately compare before and after application of the function. Furthermore, there is an advantage in that, when the sub screen area is called, the functions frequently used in the past can be executed directly without going through a few steps.

The present invention described above can be embodied as computer-readable codes on a medium on which a program is recorded. The computer readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of the computer readable medium include a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, , And may also be implemented in the form of a carrier wave (e.g., transmission over the Internet). Also, the computer may include a control unit 180 of the terminal. Accordingly, the above description should not be construed in a limiting sense in all respects and should be considered illustrative. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the scope of equivalents of the present invention are included in the scope of the present invention.

Claims (19)

A touch screen configured to display a first page;
Wherein when a predetermined touch gesture is applied to the touch screen while the first page is displayed, a panel area having a predetermined transparency is created in a predetermined area of the touch screen, and the application result of the function associated with the first page And a control section for providing the panel area,
Wherein,
Wherein the controller controls the application of the function associated with the first page differently in response to varying the size of the panel area based on the touch input. The method according to claim 1,
Wherein the application range of the function associated with the first page is varied along a drag path of the drag input when a drag input is applied to the panel region. 3. The method of claim 2,
Wherein when a drag input is applied to the panel area, an image whose size gradually changes in the panel area along the drag path of the drag input is generated, and the application result of the function associated with the first page is displayed in the image The mobile terminal comprising: The method of claim 3,
Wherein the image is processed differently depending on a point at which the drag input is released. 5. The method of claim 4,
Wherein the panel region is fixed to the entire display unit when the drag input is continued from one point of the panel region to an upper end of the display unit,
Wherein when the reference time elapses, the information displayed on the first page is converted into information displayed on the panel area, and the panel area disappears. 3. The method of claim 2,
Responsive to a portion of the first page being covered by the panel region being varied along a drag path of the drag input, the function associated with the first page is gradually applied or unapplied to a portion of the first page, To the mobile terminal. The method according to claim 6,
Wherein the first page includes a plurality of objects and at least a part of the objects displays a notification object indicating that an event has occurred,
Wherein,
And provides the event information corresponding to the notification object to the panel area if a part of the first page is changed so that the notification object includes the object displayed along the drag path of the drag input. 8. The method of claim 7,
Wherein when the drag input reaches the upper end of the touch screen, the panel area disappears and all the notification objects displayed on the first page disappear. The method according to claim 1,
Wherein,
Providing a result of execution of a first function associated with the information input to the panel area when a predetermined touch gesture is applied to the touch screen while a virtual keyboard area for inputting information on the first page is displayed,
Wherein the execution result of the first function is switched to the execution result of the second function that is related to the information input and is different from the first function in response to the size of the panel area being changed based on the touch input Mobile terminal. The method according to claim 1,
Wherein,
When a predetermined drag-type touch input is applied to the touch screen while the virtual keyboard area for inputting information is displayed on the first page, moving the position of the virtual keyboard area, , ≪ / RTI &
Wherein the controller controls the degree of scrolling of the keys displayed in the virtual keyboard area and the panel area according to the dragged degree of the drag-touch input. The method according to claim 1,
The first page corresponds to a list comprising a plurality of items
Wherein,
A panel area is slid up and down based on a drag input applied to the panel area, a specific item in the list that contacts the top line of the panel area to be slid is recognized, and detailed information of the recognized specific item is displayed on the panel area To the mobile terminal. The method according to claim 1,
The first page corresponds to a busy screen
Wherein,
A voice recognition mode is executed when a predetermined touch gesture is applied to the touch screen while the communication screen is being displayed, and at least a part of the conversation contents is converted into text according to the execution of the voice recognition mode, Wherein the mobile terminal is a mobile terminal. 13. The method of claim 12,
Wherein in the voice recognition mode, in response to the touch input being applied to one point of the panel area being dragged to the upper end of the touch screen, the voice recognition mode is terminated and the text provided in the panel area is automatically stored. terminal. The method according to claim 1,
Wherein,
Detecting a lower function of the function associated with the first page in response to the size of the panel area being changed based on the touch input and outputting a result of weighting and applying the detected lower function to the first page, To the mobile terminal. The method according to claim 1,
Wherein,
Determining a function associated with the first page based on the usage pattern repeated in the first page, and updating the repeated usage pattern based on processing on the information displayed in the panel area, . The method according to claim 1,
Further comprising an illuminance detection unit for sensing illuminance around the main body to obtain illuminance information,
Wherein,
And applies the corresponding screen filter value to the first page based on the roughness information obtained at the time when the panel region is created. The method according to claim 1,
Wherein,
If the predetermined touch gesture is applied to the touch screen while the first page is being displayed, extracting the first type information from the first page, reconstructing the first page based on the first type information,
And notification information indicating that the first page has been reconstructed is output to the panel area. The method according to claim 1,
Wherein when the specific object is selected from the first page, the function is limited to the specific object and provides the result of applying the function associated with the first page to the specific region. 19. The method of claim 18,
Wherein the type of the function associated with the first page is changed according to the type of the information included in the specific object.

Images