KR100912310B1 - User interface controlling method by detecting user's gestures - Google Patents

Abstract

A method for detecting the gesture of a user and controlling the user interface of a mobile terminal through various methods is provided to input data by executing a command about contents displayed on a mobile terminal. A selection point(12) is set up on a display. A page is indicated on the display. The motion of a mobile terminal generated by the gesture of a user is detected. A scroll of one among the selection point or page is performed. A selectable item which is included in the page is located on surface the selection point of the display. The selectable item is selected in response to the predetermined event.

Description

User interface controlling method by detecting user's gestures

The present invention relates to a user interface control method of a mobile terminal.

More specifically, the present invention provides various methods for controlling a user interface of a mobile terminal by detecting a gesture of a user.

There are various portable mobile terminals such as mobile phones, MP3 players, PMPs, notebooks, and PDAs. Because of their portability, they place many restrictions on the user interface for input and output. It is not used at any one place, but it is often used while the user is moving and the volume of the device is very small due to the nature of the portable device, so the input / output of QWERTY keyboard, remote controller, etc. The device is often inadequate.

Therefore, a user interface suitable for a portable device is used, and a touch pad, a touch screen, a navigation pad, and the like are used.

In spite of these various user interfaces, input and output of mobile terminals are subject to many limitations, and the user is also uncomfortable.

For example, in the case of a mobile phone, various additional functions are added in addition to a call function. For example, an MP3 player, a camera function, and an internet connection function are added.

Naturally, many of these functions are configured to be controlled by the user, but it is inconvenient for the user to control the functions properly according to the use state of the mobile terminal.

Recently, as a number of additional functions have been added to a mobile phone, efforts are being made to improve a user interface provided to a user to operate the mobile phone.

Due to the portability of the mobile phone, there are many restrictions on the user interface for input and output. It is not used at any one place, but it is often used while the user is moving and the volume of the device is very small due to the nature of the portable device, so the input / output of QWERTY keyboard, remote controller, etc. The device is often inadequate.

Therefore, a user interface suitable for a portable device is used, and a touch pad, a touch screen, a navigation pad, etc. may be used.

Despite these various user interfaces, input and output of mobile terminals are subject to a lot of limitations, and the user also feels uncomfortable.

It is an object of the present invention to provide various methods for controlling a user interface of a mobile terminal.

An object of the present invention is to provide a user interface that allows a user to execute commands and input data in various ways using a mobile terminal.

According to an embodiment of the present invention, a method of controlling a user interface of a mobile terminal including a display includes: detecting a movement of the mobile terminal by a gesture of a user; Scrolling at least one of a page and a selection point displayed on the display according to the detected movement of the mobile terminal; Determining whether a selectable item displayed on the page is located on a selection point of the display; And in response to determining that the selectable item is located on a selection point of the display, selecting the selectable item in response to a predetermined event.

A user interface control method according to an embodiment of the present invention includes detecting a movement of a mobile terminal applied by a gesture of a user; Scrolling a page displayed on a display of the mobile terminal according to the detected movement of the mobile terminal; Determining whether at least a portion of an input window of the display overlaps or borders with an input window of a page displayed on the display; And activating the user interface according to the determined result.

A user interface control method according to an embodiment of the present invention includes detecting a movement of a mobile terminal applied by a gesture of a user; Scrolling a page displayed on the display according to the movement of the mobile terminal; And displaying a guide window on the page currently displayed on the display, wherein the guide window is a reduced size of the page currently displayed on the display.

A mobile terminal according to an embodiment of the present invention, a display; A detector configured to detect a movement of the mobile terminal applied by the user's gesture; And scrolling at least one of a page of the display and a selection point of the display according to the detected movement of the mobile terminal, and determining whether a selectable item displayed in the page is located on a selection point of the display, And if it is determined that the selectable item is located on a selection point of the display, the controller controls the display to select the selectable item in response to a predetermined event.

A mobile terminal according to an embodiment of the present invention, a display; User interface; A detector configured to detect a movement of the mobile terminal applied by the user's gesture; And scrolling a page displayed on the display according to the detected movement of the mobile terminal, determining whether at least a portion of an input area of the display overlaps or reaches a boundary of an input window of a page displayed on the display, and determines the determination. It includes a control for activating the user interface based on the result.

A mobile terminal according to an embodiment of the present invention, a display; A detector configured to detect a movement of the mobile terminal applied by the user's gesture; And a control unit configured to control the display to scroll a page of the display according to the movement of the mobile terminal and to display a guide window in addition to the page currently displayed on the display, wherein the guide window is currently displayed on the display. Contains miniature pages.

According to the present invention, a live user interface of a mobile terminal can be provided.

In addition, the present invention can provide a user interface that allows a user to execute commands and input data on contents displayed on a mobile terminal in various ways.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1A and 1B illustrate a method of controlling a user interface of a mobile terminal using gestures according to the present invention.

The mobile terminal according to the present invention may be equipped with a motion sensor such as a gyro sensor, an inclination sensor, an acceleration sensor, and the like. The mobile terminal according to the present invention includes all kinds of portable devices such as a mobile phone, an MP3 player, a PMP, a computer notebook, a PDA, a smart phone.

Hereinafter, it is assumed that the mobile terminal is a mobile phone. However, the present invention is not limited to mobile phones, but also includes various types of mobile terminals or portable electronic devices.

The mobile phone shown here is a so-called "full touch screen phone", which has no physical buttons for entering numbers or data, but instead occupies most of the front surface of the touch screen.

As shown in FIG. 1A, when a user applies a certain gesture in a state where a user raises a hand, a motion sensor detects the movement. The controller of the mobile terminal / mobile phone performs a function corresponding to the detected movement.

For example, suppose that you are browsing wired or wireless Internet with a mobile terminal. For example, if the user tilts the mobile terminal to the right, the motion sensor detects the tilt. If the detected movement is tilted to the right, the controller (not shown) of the mobile terminal scrolls the screen on the web page currently viewed in the direction in which the user tilts the mobile terminal.

The user can tilt the mobile terminal in various directions and angles. In the present invention, such tilting of various directions and angles causes a page (eg, a web page or a word document) displayed on the display of the mobile terminal to be scrolled. In this case, the direction of page scrolling may be determined by the direction in which the user tilts the mobile terminal, and the amount of scrolling of the page may vary depending on the amount of angle at which the user tilts the mobile terminal.

In another embodiment, the user may tilt the mobile terminal once (eg, shake the mobile phone once) and page turning may be performed according to the number of times the page scrolls to the next page and shakes the mobile terminal.

In another embodiment, the user may tilt the mobile terminal and hold it for a predetermined time or more in the tilted state. In this case, the scrolled distance may be scrolled by as much as the predetermined time.

In another embodiment, a page displayed by a particular action (eg, shake twice quickly) may be automatically scrolled continuously.

In all the above embodiments, the scrolling direction may be determined by the shaking / tilting direction. In addition, the tilting direction of the present invention includes all directions (eg, the diagonal direction of the display), and is not limited to only up, down, left, and right.

The left screen of FIG. 1B shows a portion of the web site being browsed by the mobile terminal, and the right screen shows the screen scrolled to the right. As shown in FIG. 1A, when the user tilts the mobile terminal to the right by a predetermined angle, the content displayed on the screen of the mobile terminal is scrolled to the right according to the tilting direction.

When the user tilts the mobile terminal to the right by a predetermined angle, the motion sensor mounted in the mobile terminal detects the tilt, and the control unit scrolls the screen to the right according to the detection. Likewise, tilting left can scroll left.

The mode of detecting the user's gesture as described above and performing an operation corresponding to the gesture may be activated or deactivated by the user's selection. That is, the gesture mode of the mobile terminal according to the present invention is a mode in which the mobile terminal operates according to various gestures applied to the mobile terminal and controls the user interface.

That is, in FIG. 1A, an activation button 11 may be provided on one side of the mobile terminal, and the gesture mode may be activated only when the user presses the activation button 11 or is pressed for a predetermined time or more.

Various changes may be made to activate the gesture mode. For example, depending on the embodiment, the software installed in the mobile terminal may be operated by the user's operation without using the physical activation key 11 to enter the mode for detecting the gesture.

In addition, a user's specific gesture may be designated to enter or exit the gesture mode. For example, when the user shakes the mobile terminal, the mobile terminal enters the gestured mode, and the controller of the mobile terminal may execute an operation according to the gesture measured by the motion sensor. Also, when the user shakes the mobile terminal in the gestured mode, the mobile terminal exits the gestured mode. It can also be configured to

In addition, the user may specify an angle at which the user views the mobile terminal, and the gesture mode may be activated when the user stops for a predetermined time. In other words, if the user stops, for example, 3 seconds or more while holding the mobile terminal, the motion sensor in the mobile terminal may detect that there is no movement and enter the gesturing mode. When the user does not use the mobile terminal, the display may be configured to be turned off, and when the display is turned off, the gesture mode may not be activated.

As a non-limiting example, if the user presses the activation key 11 and tilts the top of the mobile terminal close to the user, that is, the display of the mobile terminal faces downward as shown in FIG. 2A (down to the horizontal axis). Tilted), the screen may scroll down the webpage as shown in FIG. 2B. When the display of the mobile terminal is tilted upward, the screen may be configured to scroll upward of the web page.

According to an exemplary embodiment, whenever the mobile terminal is tilted more than a predetermined angle, the mobile terminal may be scrolled by a predetermined distance. That is, when the mobile terminal is tilted more than a preset threshold angle, it may scroll by a predetermined distance. Alternatively, when the mobile terminal is tilted at least slightly from its original position, scrolling may be started, and the screen may be configured to scroll the web page by a length proportional to the angle of tilting the mobile terminal.

The scrolling direction may also be variously modified according to the tilting direction of the mobile terminal. For example, it is possible to scroll to the left when tilted to the right, to the right when tilted to the left, to the right when tilted up, to the bottom when tilted up, and to the top when tilted down.

Mobile terminals such as mobile phones are usually not equipped with a keyboard or a mouse like a general PC. Therefore, when a user connects to the Internet using a mobile terminal such as a mobile phone, a command can be executed by clicking, double-clicking or dragging selectable items such as hyperlinks or various virtual buttons using a stylus pen instead of a mouse on the touch screen. Done.

Also, in the conventional mobile terminal, when typing, the cursor is moved to a window for inputting an ID and password for a search window or a login on a touch screen, a button is pressed, a virtual input window is displayed, or a handwriting recognition mode is switched. Enter it.

3A and 3B illustrate an example of a method of selecting a selectable item on a screen displayed on a mobile terminal according to an embodiment of the present invention.

When browsing the Internet using a mobile terminal, it is assumed that a hyperlink 13 is arranged on a web page as shown in FIG. 3A. Any position of the display of the mobile terminal may be set as the selection point 12. The selection point 12 is preferably located in the center of the display. However, it can be configured to allow the user to set the location of the selection point. The selection point 12 is fixed on the display of the mobile terminal, and the position of the selection point 12 on the display does not change even when the web page is scrolled. The selection point 12 may be invisible or may display a mark for the user to identify the location so that the user can visually confirm the location. In other embodiments, the selection point 12 may be a selection area.

In the state of FIG. 3A, when the user tilts the mobile terminal so that the display of the mobile terminal faces downward (ie, the upper portion of the mobile terminal is closer to the user) as shown in FIG. 2A, the web page displayed on the mobile terminal is displayed as shown in FIG. 3B. The scroll causes the hyperlink 13 to rest on the selection point 12. At this time, when the user clicks the hyperlink 13 with a finger or a stylus, the hyperlink 13 may be executed.

In another embodiment, when the hyperlink 13 is placed on the selection point 12, pressing the predetermined button on the mobile terminal may be configured to execute the hyperlink 13.

When the hyperlink 13 arrives at the location of the selection point 12, an indicator 14 can be displayed to indicate that the hyperlink can be clicked to indicate that the hyperlink is in a selectable state. When the indicator 14 is displayed, the hyperlink 13 may be configured to be executed by pressing a specific physical button disposed outside the mobile terminal.

Execution of the hyperlink 13 may include accessing and displaying a hyperlinked website on a display, performing an operation related to the hyperlink, and the like.

In another embodiment, if the hyperlink 13 stops for a predetermined time, for example, 2 seconds or more in a state where the hyperlink 13 is placed on the selection point 12, the hyperlink may be connected without a separate operation.

In another embodiment, when the user is shaking the mobile terminal when the hyperlink 13 is on the selection point 12, the hyperlink 13 may be performed. In another embodiment, the scrolling of the displayed page may be automatically stopped when the hyperlink 13 is placed on the selection point 12. The hyperlink 13 may be performed when the user shakes the mobile terminal, presses a button, or takes a predetermined operation while the scrolling is stopped.

Alternatively, if the hyperlink 13 is placed on the selection point 12, a virtual button that can be selected is displayed on a portion of the screen, and the hyperlink is executed when the user clicks the virtual button.

The above embodiment may be applied to selecting any virtual buttons displayed on a web page in addition to hyperlinks when browsing the Internet using a mobile terminal. For example, when a video playback screen is displayed on a web page, the present invention can also be applied to selecting virtual buttons for operations such as playing, posing, and stopping the video.

In another embodiment, an indicator 14 is displayed at a selection point on the display of the mobile terminal as shown in FIG. 4A and a user interface indicating that the user selects when the hyperlink 13 comes to the selection point by tilting the mobile terminal. It can be activated. For example, as shown in FIG. 4B, the indicator 14 may be changed to another form to indicate that it is in a selectable state. In the state where the changed indicator 14 is displayed as shown in FIG. 4B, when the user clicks the hyperlink 13 or presses a physical button on the mobile terminal, the hyperlink may be executed.

According to an embodiment, when the user shakes the mobile terminal as shown in FIG. 5 while the indicator 14 is displayed, the motion sensor of the mobile terminal may detect the shake gesture to execute the corresponding hyperlink. Before the user applies the shake gesture, the hyperlink may be executed in response to the shake gesture only while pressing the activation key 15 mounted on one side of the mobile terminal or another specific physical button.

The above-described embodiment of the gesture of shaking the mobile terminal may be applied to the case of clicking a virtual button displayed on the screen in addition to executing a hyperlink.

According to an embodiment of the present invention, when the mobile terminal is tilted while pressing the activation key 15 or another specific physical button, the screen is fixed and the selection point 12 may be configured to move on the screen.

That is, in a state where a web page is displayed on the display of the mobile terminal as shown in FIG. 6A, when the user presses and holds the activation key 15 and tilts the mobile terminal, the screen is fixed and does not move (the web page is fixed). And select point 12 only. If the user moves the selection point 12 appropriately (for example, by tilting the mobile terminal while pressing the activation button 15) and the selection point 12 is placed on the hyperlink 13, the selection point 12 as shown in FIG. 6B. ) Is replaced by an indicator 14 and the hyperlink 13 becomes executable. With the indicator 14 displayed, the user may be configured to execute the hyperlink by clicking on the hyperlink 13, pressing a physical button on the mobile terminal, or by any other method described above. The user can then move the selection point 12 in any direction on the display.

In another embodiment, without pressing the button 15, the user can scroll the page by tilting the mobile terminal as described above. The user may then press the button 15 and tilt the mobile terminal to move the selection point in any direction (eg, diagonal, up, down, left, right, etc.).

According to an embodiment, when the user shakes the mobile terminal as shown in FIG. 5 while the indicator 14 is displayed, the motion sensor of the mobile terminal may detect the shake gesture and configure the hyperlink to be executed. Before the user applies the shaking gesture, the hyperlink may be executed in response to the shaking gesture as described above only while releasing the activation key 15 mounted on one side of the mobile terminal or pressing another specific physical button.

When browsing a web page, a hyperlink on the web page may click a virtual button, but sometimes you need to enter letters or numbers to search or log in. Cell phones or PDAs without keyboards or mobile terminals (i.e. mobile phones) with simplified keyboards require an interface for keyboard input for typing letters or symbols.

According to an embodiment of the present invention, an interface may be provided without a separate operation in which a user displays an interface for keyboard input.

As shown in FIG. 7A, an input area 16 is set on a part of the display of the cellular phone / mobile terminal. The input area 16 is preferably set to a rectangle having the same shape as an input window of a typical Internet web page. The input area 16 may be displayed invisibly, or may be displayed in a dotted line, invisible colored lines, or in other ways. In one embodiment, the input area 16 may be displayed in shades or other colors. The page (eg, web page) displayed on the display of the mobile terminal may have an input window 17 for the user to input data or information.

In some embodiments, the input area 16 may be shaded to distinguish it from other areas of the display. In one embodiment, the input area 16 may be fixed on the display and not move.

As shown in FIG. 7A, the user inputs the input area 16 and the input window 17 so that the user inputs the input window 17 while the input area 16 and the input window 17 are located at different positions. As shown in FIGS. 1A to 2B, the user may tilt the mobile terminal to scroll the input window 17 so that the input window 17 and the input area 16 overlap each other.

In this case, the user can tilt the mobile terminal to scroll / move the page in the tilted direction, in which case the input window 17 is moved on the page. By controlling the tilting direction, the user moves the input window 17 so that the input window 17 is placed on the input area 16 (partly or entirely overlapping), or the input area 16 and the input window. The boundary of (17) can be made to contact.

The user interface can be activated on the display when overlapped or encountered. For example, the virtual keyboard 18 capable of inputting keyboard characters may be displayed at the bottom of the screen. The virtual keyboard 18 may be a keyboard mounted on a conventional mobile phone or a qwerty keyboard.

When the virtual keyboard 18 is displayed, the focus moves to the input window 17 so that the user can input characters into the input window 17 by touching keys of the keyboard 18 with a finger or a stylus, for example.

Characters can be input without using the virtual keyboard 18. For example, in the case of a general mobile terminal without a touch screen, only the focus may be moved to the input window 17 when the input area 16 and the input window 17 overlap each other. In this case, the user can input characters by pressing a physical keyboard mounted on the mobile terminal.

Optionally, a virtual handwriting input window may be displayed instead of the virtual keyboard 18 to input characters through handwriting recognition on a touch screen.

In another embodiment of the present invention, the input area 16 may be visually displayed, and when the input window is in a state in which the input window can be input, the user may be visually displayed.

By selecting a button (virtual or physical button), or performing a specific action (such as tilting the mobile terminal, shaking the mobile terminal more than once, or touching a certain area of the display), the user does not have a virtual keyboard displayed. Or the displayed page can be returned to the previous page.

In the state as shown in FIG. 8A, the user operates the mobile terminal to overlap at least a portion of the input area 16 and the input window 17 or to contact the boundary thereof. As shown in FIG. 8B, when the input area 16 and the input window 17 overlap, the keyboard 18 is displayed.

In addition, a mode in which the user can input characters by displaying a line overlapping the input area 16 and the input window 17 as a thick line 16 + 17, or by displaying the overlapping line in a bright or distinct color. You can easily recognize that

7A and 8A, the size of the input area 16 and the input window 17 may be different. This is because the input area 16 may be displayed in a constant size, but the input window 17 may be different for each web page. In this case, as shown in FIG. 9, when a portion 19 in which the input area 16 and the input window 17 overlap each other occurs, the keyboard (eg, 18 of FIGS. 7B and 8B) may be displayed. That is, as shown in FIG. 9, the keyboard may be displayed when there is a portion where the input area 16 and the input window 17 overlap at all.

According to an exemplary embodiment, the input area 16 may be configured to display a keyboard when a portion overlapped by the input window 17 becomes more than a predetermined area.

The case where one corner 16a of the input area overlaps with one corner 17a of the input window may be set to be overlapped. As shown in FIG. If (17a, 17b) overlap, it may be set to determine that the overlap.

In addition, the input area 16 may be set relatively small, and when the input area 16 is included in the input window 17, the input area 16 may be determined to overlap and the virtual keyboard may be displayed.

According to an embodiment of the present invention, when the input area 16 and the input window 17 overlap, a keyboard or a handwriting input window for inputting characters is displayed, and when the user presses a specific button in this state, the input mode state is displayed. By locking, the keyboard may be configured to maintain a text input menu such as a cursive input window even when the user moves the mobile terminal. That is, even if the input area 16 and the input window 17 do not overlap, the text input menu may be maintained as it is.

The specific button may be a physical button mounted outside the mobile terminal, or may be a virtual button displayed on a touch screen of the mobile terminal.

Optionally, the input area 16 and the input window 17 may be set to be locked in the text input mode when the input area 16 and the input window 17 stay for a predetermined time, for example, 2 seconds or more. That is, if the input area 16 and the input window 17 overlap and remain stopped for a predetermined time while the text input mode is activated, the text input menu is locked so that the text input menu is maintained even when the mobile terminal is tilted or moved. Can be.

7A to 9, the input area 16 is fixedly displayed at a preset position and the input area 16 may not move even when the input window 17 is moved by moving the mobile terminal. Here, the user can change the preset position of the input area 16 by controlling the menu. In another embodiment, similar to FIGS. 6A and 6B, the input area 16 may be moved as the user moves the mobile terminal while the input window 17 is fixed.

According to another embodiment of the present invention, when the user moves the mobile terminal while holding the activation key 11 as shown in FIG. 1B and FIG. 1B or 2B, the web page is scrolled in response to the movement. Can be configured to In this case, if the above-described selection point is placed on the selectable item or the input area overlaps with the input window, the user interface can be activated to allow the user to select an item or input information.

According to an embodiment of the present invention, when browsing a web page of the Internet using a mobile terminal, a guide may be provided that indicates where the currently browsing portion is displayed on the display.

For example, as illustrated in FIG. 11, the guide window 20 may be displayed on a portion of the display. The guide window 20 includes a miniature of the entire screen of the web page currently being browsed, and the position display unit 21 for displaying a portion currently displayed on the display of the mobile terminal is displayed in the guide window 20. .

As the web page is scrolled through the gestures in the above-described embodiments, the position display unit 21 moves in the guide window 20. The user may recognize through the location display unit 21 which part of the web page he is viewing through the mobile terminal.

According to an exemplary embodiment, when the web page is zoomed in or zoomed out through the operation of the mobile terminal, the position display unit 21 may be enlarged or reduced accordingly.

According to the embodiment, the position of the selection point 12 or the input area 16 may be displayed in the position display section 21. Accordingly, the user may select only the selected item even by looking only at the guide window 20 or scroll to input text into the input window.

According to the invention, the hyperlink 13 and the input window 17 are examples of selectable items. Other types of selectable items may also be displayed and may be selected by the user by moving the mobile terminal in the same or similar manner as the way in which the hyperlink 13 is executed in the above-described embodiment. Another type of selectable items may be one of buttons related to the function of controlling playback / recording of moving picture or audio data. In addition, the movement of the mobile terminal by the user used to control the user interface according to an embodiment of the present invention is rolling (for example, to rotate the mobile terminal to the left and right), pitching (pitching) to rotate the terminal up and down Motion), yawing / heading / pivoting (e.g., pivoting up, down, left and right) and transitions (left and right movements). In addition, according to the present invention, the scroll operation of one page may include a page turning and a zoom in / out operation.

Various embodiments of the method and operation of the present invention described above may be implemented in a mobile terminal or a portable electronic device. Hereinafter, an example of an apparatus of the mobile terminal 100 will be described with reference to FIGS. 12 and 13. The controller 180 of the mobile terminal 100 may be configured to control the display unit 151 to perform various user interface control methods of the present invention. The mobile terminal according to the present invention may include some or all of the components of the mobile terminal 100.

12 is a block diagram of a mobile terminal 100 according to an embodiment of the present invention. Any mobile terminal (eg, mobile terminals 10, 21) referred to herein may be a mobile terminal 100 equipped with components that include a distance sensor. As mentioned above, the mobile terminal of the present invention can be implemented using various types of devices. For example, the mobile terminal described herein may be a mobile phone, a smart phone, a notebook computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), navigation, or the like. have.

The illustrated mobile terminal 100 includes a wireless communication unit 110 that can be implemented with several known components. For example, the wireless communication unit 110 typically includes one or more components that enable wireless communication between the mobile terminal 100 and a wireless communication system or network in which the mobile terminal is located.

The broadcast receiving module 111 receives a broadcast signal and / or broadcast related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may mean a server that generates and transmits a broadcast signal and / or broadcast related information or a server that receives a previously generated broadcast signal and / or broadcast related information and transmits the same to a terminal. The broadcast related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast signal may include not only a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, but also a broadcast signal having a data broadcast signal combined with a TV broadcast signal or a radio broadcast signal.

Meanwhile, the broadcast related information may be provided through a mobile communication network, and in this case, may be received by the mobile communication module 112.

The broadcast related information may exist in various forms. For example, it may exist in the form of Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast receiving module 111 receives broadcast signals using various broadcasting systems, and in particular, digital multimedia broadcasting-terrestrial (DMB-T), digital multimedia broadcasting-satellite (DMB-S), and media forward link (MediaFLO). Digital broadcast signals can be received using digital broadcasting systems such as only), digital video broadcast-handheld (DVB-H), integrated services digital broadcast-terrestrial (ISDB-T), and the like. Of course, the broadcast receiving module 111 is configured to be suitable for all broadcast systems providing broadcast signals as well as the digital broadcast system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

In addition, the mobile communication module 112 transmits and receives a radio signal with at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call call signal, or a text / multimedia message.

The wireless internet module 113 refers to a module for wireless internet access, and the wireless internet module 113 may be internal or external.

The short range communication module 114 refers to a module for short range communication. As a short range communication technology, Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and the like may be used.

In addition, the location information module 115 is a module for checking or obtaining the location of the mobile terminal. One example is the Global Position System (GPS) module. The GPS module receives location information from a plurality of satellites. Here, the location information may include coordinate information represented by latitude and longitude. For example, the GPS module can measure the exact time and distance from three or more satellites and accurately calculate the current position by triangulating three different distances. A method of obtaining distance and time information from three satellites and correcting the error with one satellite may be used. In particular, the GPS module can obtain not only the location of latitude, longitude, and altitude but also accurate time together with the three-dimensional speed information from the location information received from the satellite.

Meanwhile, the A / V input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video call mode or the photographing mode. The processed image frame may be displayed on the display unit 151.

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. Two or more cameras 121 may be provided according to the configuration aspect of the terminal.

The microphone 122 receives an external sound signal by a microphone in a call mode, a recording mode, a voice recognition mode, etc., and processes the external sound signal into electrical voice data. The processed voice data may be converted into a form transmittable to the mobile communication base station through the mobile communication module 112 and output in the call mode. The microphone 122 may implement various noise removing algorithms for removing noise generated in the process of receiving an external sound signal.

The user input unit 130 generates input data for the user to control the operation of the terminal. The user input unit 130 may include a key pad dome switch, a touch pad (static pressure / capacitance), a jog wheel, a jog switch, and the like. In particular, when the touch pad has a mutual layer structure with the display unit 151 described later, this may be referred to as a touch screen.

The sensing unit 140 detects a current state of the mobile terminal 100 such as an open / closed state of the mobile terminal 100, a location of the mobile terminal 100, presence or absence of a user contact, orientation of the mobile terminal, acceleration / deceleration of the mobile terminal, and the like. To generate a sensing signal for controlling the operation of the mobile terminal 100.

The sensing unit 140 may include an inertial sensor for detecting a movement or a position of a gyro sensor, an acceleration sensor, or a distance sensor for detecting or measuring a distance relationship between a user and the mobile terminal 100.

The interface unit 170 serves as an interface with all external devices connected to the mobile terminal 100. For example, wired / wireless headset ports, external charger ports, wired / wireless data ports, memory card ports, ports for connecting devices with identification modules, audio input / output (I / O) ports, Video I / O (Input / Output) port, earphone port, etc. may be included.

Here, the identification module is a chip that stores various types of information for authenticating the use authority of the mobile terminal 100, and includes a user identification module (UIM) and a subscriber identify module (SIM). ), A Universal Subscriber Identity Module (“USIM”), and the like. In addition, the device equipped with the identification module (hereinafter, 'identification device') may be manufactured in the form of a smart card. Therefore, the identification device may be connected to the terminal 100 through a port. The interface unit 170 receives data from an external device or receives power and transmits the data to each component inside the mobile terminal 100 or transmits the data inside the mobile terminal 100 to an external device.

The output unit 150 is for outputting an audio signal, a video signal, or an alarm signal. The output unit 150 may include a display unit 151, a sound output module 152, an alarm unit 153, and the like. The display unit 151 displays and outputs information processed by the mobile terminal 100. For example, when the mobile terminal is in a call mode, a user interface (UI) or a graphic user interface (GUI) related to a call is displayed. When the mobile terminal 100 is in a video call mode or a photographing mode, the mobile terminal 100 displays a photographed and / or received image, a UI, or a GUI.

The touch screen panel may be mounted on the display unit 151. By such a configuration, the display unit 151 may serve as both an input unit and an output unit.

The display unit 151 may be a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, or a three-dimensional display. 3D display). In addition, two or more display units 151 may exist according to the implementation form of the mobile terminal 100. For example, an external display unit (not shown) and an internal display unit (not shown) may be simultaneously provided in the mobile terminal 100.

The sound output module 152 outputs audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. In addition, the sound output module 152 outputs a sound signal related to a function (eg, a call signal reception sound, a message reception sound, etc.) performed in the mobile terminal 100. The sound output module 152 may include a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying occurrence of an event of the mobile terminal 100. Examples of events occurring in the mobile terminal include call signal reception, message reception, and key signal input. The alarm unit 153 may output a signal for notifying occurrence of an event in a form other than an audio signal or a video signal. For example, the signal may be output in the form of vibration. When a call signal is received or a message is received, the alarm unit 153 may output a vibration to inform this. Alternatively, when a key signal is input, the alarm unit 153 may output a vibration in response to the key signal input. Through the vibration output as described above, the user can recognize the occurrence of the event. Of course, the signal for notification of event occurrence may be output through the display unit 151 or the voice output module 152.

The memory 160 may store a program for processing and controlling the controller 180 and may provide a function for temporarily storing input / output data (for example, a phone book, a message, a still image, a video, etc.). It can also be done.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), RAM Random Access Memory (RAM) Static Random Access Memory (SRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read-Only Memory (PROM) magnetic memory, It may include at least one type of storage medium of the optical disk. In addition, the mobile terminal 100 may operate a web storage that performs a storage function of the memory 150 on the Internet.

The controller 180 typically controls the overall operation of the mobile terminal. For example, perform related control and processing for voice calls, data communications, video calls, and the like. In addition, the controller 180 may include a multimedia module 181 for playing multimedia. The multimedia module 181 may be implemented in the controller 180 or may be implemented separately from the controller 180.

The power supply unit 190 receives an external power source and an internal power source under the control of the controller 180 to supply power for operation of each component.

The various embodiments described herein may be implemented in a computer readable recording medium using, for example, software, hardware or a combination thereof.

According to a hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), and the like. It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing the functions. It may be implemented by the controller 180.

In a software implementation, embodiments such as procedures or functions may be implemented with separate software modules that allow at least one function or operation to be performed. The software code may be implemented by a software application written in a suitable programming language. In addition, the software code may be stored in the memory 160 and executed by the controller 180.

The terminal 100 illustrated in FIG. 12 may be configured to be operable in a communication system capable of transmitting data through a frame or packet, including a wired / wireless communication system and a satellite based communication system. Can be.

Hereinafter, a communication system capable of operating a terminal according to the present invention will be described with reference to FIG. 13.

The communication system may use different air interfaces and / or physical layers. For example, a radio interface usable by a communication system may include frequency division multiple access (FDMA), time division multiple access (TDMA), and code division multiple access (Code Division). Multiple Access (CDMA), Universal Mobile Telecommunications Systems (UMTS) (especially Long Term Evolution (LTE)), Global System for Mobile Communications (GSM), etc. May be included. Hereinafter, for convenience of description, the description will be limited to CDMA. However, it is a matter of course that the present invention is applicable to all communication systems including the CDMA wireless communication system.

As shown in FIG. 13, the CDMA wireless communication system includes a plurality of terminals 100, a plurality of base stations (BS) 270, and base station controllers (BSCs) 275. ), And a mobile switching center (MSC) 280. The MSC 280 is configured to be connected to a public switched telephone network (PSTN) 290 and is also configured to be connected to BSCs 275. The BSCs 275 may be coupled to the BS 270 through a backhaul line. The backhaul line may be provided according to at least one of E1 / T1, ATM, IP, PPP, Frame Relay, HDSL, ADSL, or xDSL. Thus, a plurality of BSCs 275 may be included in the system shown in FIG.

Each BS 270 may include at least one sector, and each sector may include an omnidirectional antenna or an antenna pointing radially out of BS 270. In addition, each sector may include two or more antennas of various types. Each BS 270 may be configured to support a plurality of frequency assignments, each of which has a specific spectrum (eg, 1.25 MHz, 5 MHz, etc.).

The intersection of sectors and frequency assignments may be called a CDMA channel. BS 270 may be referred to as a base station transceiver subsystem ('BTSs'). In this case, the word "base station" may be called the sum of one BSC 275 and at least one BS 270. The base station may also indicate “cell site”. Alternatively, each of the plurality of sectors for a particular BS 270 may be called a plurality of cell sites.

As shown in FIG. 13, a broadcasting transmitter (BT) 295 transmits a broadcast signal to terminals 100 operating in the system. The broadcast module 111 shown in FIG. 12 is provided in the terminal 100 to receive a broadcast signal transmitted by the BT 295.

In addition, FIG. 13 illustrates multiple satellite positioning systems (GPS) satellites 300. The satellites 300 help to locate the at least one terminal of the plurality of terminals 100. Although two satellites are shown in FIG. 13, useful location information may be obtained by two or less or more satellites. The GPS module 115 shown in FIG. 12 cooperates with the satellites 300 to obtain desired location information. Here, the location can be tracked using all the techniques that can track the location, not just the GPS tracking technology. In addition, at least one of the GPS satellites 300 may optionally or additionally be responsible for satellite DMB transmission.

During typical operation of a wireless communication system, BS 270 receives a reverse link signal from various terminals 100. At this time, the terminals 100 are connecting a call, transmitting or receiving a message, or performing another communication operation. Each of the reverse link signals received by a particular base station 270 is processed within by the particular base station 270. The data generated as a result of the processing is transmitted to the connected BSC 275. BSC 275 provides call resource allocation and mobility management functionality, including the organization of soft handoffs between base stations 270. In addition, the BSC 275 transmits the received data to the MSC 280, and the MSC 280 provides an additional transmission service for connection with the PSTN 290. Similarly, PSTN 290 connects with MSC 280, MSC 280 connects with BSCs 275, and BSCs 275 send BSs 270 to transmit a forward link signal to terminals 100. ).

Although described above with reference to various embodiments, those skilled in the art will understand that various modifications can be made to the above embodiments, all such modifications are included in the scope of the invention.

1A and 1B illustrate a method of controlling a user interface of a mobile terminal using gestures according to the present invention.

2A and 2B illustrate a method of controlling a user interface of a mobile terminal using gestures according to the present invention.

3A and 3B illustrate an example of a method of selecting a selectable item on a screen displayed in accordance with one embodiment of the present invention.

4A and 4B illustrate an example of a method of selecting a selectable item on a screen displayed according to another embodiment of the present invention.

5 illustrates a method of executing a selectable item on a screen according to an embodiment of the present invention.

6A and 6B show an example of a method of selecting a selectable item on a screen displayed according to another embodiment of the present invention.

7A and 7B illustrate a method of providing an input menu to a user according to an embodiment of the present invention.

8A and 8B illustrate a method of providing an input menu to a user according to another embodiment of the present invention.

9A and 9B illustrate a method of determining whether to activate a user interface according to an embodiment of the present invention.

10 illustrates a gesture of a user according to an embodiment of the present invention.

11 illustrates a user interface control method of a mobile terminal according to an embodiment of the present invention.

12 is a block diagram of a mobile terminal according to an embodiment of the present invention.

FIG. 13 is a block diagram of a CDMA wireless communication system to which the mobile terminal of FIG. 12 may be applied.

Claims (32)

In the method for controlling a user interface of a mobile terminal equipped with a display, Setting a selection point on the display; Displaying a page on the display; Detecting movement of the mobile terminal caused by a gesture of a user; Scrolling one of the set selection point or a page displayed on the display in response to the detected movement; Determining whether a selectable item included in the displayed page is located on a selection point of the display; And And in response to determining that the selectable item is located on a selection point of the display, selecting the selectable item in response to a predetermined event. The method of claim 1, wherein the scrolling is performed when the mobile terminal moves by the user's gesture while the user presses a button of the mobile terminal. The method of claim 1, wherein the selectable item is a hyperlink, and wherein the selecting step includes connecting the mobile terminal to the hyperlink. The method of claim 1, wherein the selectable item is a virtual button. The method of claim 4, wherein the virtual button is associated with playing video or audio data. The method of claim 1, If it is determined in the determining step that the selectable item is located on a selection point of the display, indicating that the selectable item is in a selectable state on the display. The method of claim 1, wherein the scrolling comprises at least one of the following: Performing a zoom in or zoom out operation on a portion of the page displayed on the display according to the movement of the mobile terminal; And Skipping to another page according to the movement of the mobile terminal. The method of claim 7, wherein And displaying a guide window on a portion of the display, the guide window displaying a currently displayed area, which is an entire page or part of a plurality of pages. The method of claim 1, wherein the predetermined event is one of the following: Shaking the mobile terminal; Elapse of a predetermined time after the selectable item is located on a selection point of the display; Touching a designated virtual button displayed on the display; And Selecting the designated physical button of the mobile terminal. The method of claim 1, The movement of the mobile terminal comprises at least one of a rolling, pitching, yawing, transition. The method of claim 1, wherein the display is a touch screen. The method of claim 1, wherein, in the scrolling step, the selection point is set to a fixed position on the display, and only the page is scrollable. The method of claim 1, wherein in the scrolling step, the selection point is displayed to the user. The method of claim 1, wherein in the scrolling step, the selection point is invisible to a user. In the method for controlling a user interface of a mobile terminal equipped with a display, Setting an input area on the display; Displaying a page on a display in which the input area is set; Detecting movement of the mobile terminal caused by a gesture of a user; Scrolling a page displayed on the display according to the detected movement of the mobile terminal; Determining whether at least a portion of an input area of the display overlaps or borders an input window included in a page displayed on the display; And If it is determined that the input window overlaps or borders with the input window, displaying a virtual text input interface. The method of claim 15, wherein the scrolling is performed when the mobile terminal is moved by a gesture of a user while the user presses a predetermined button mounted on the mobile terminal. The method of claim 15, wherein in the scrolling step, the input window is scrolled according to the movement of the mobile terminal while the input area is fixed to the display. The method of claim 15, wherein the input area is visually displayed to a user. The method of claim 15, wherein the input area is invisible to the user. The method of claim 15, wherein the movement of the mobile terminal comprises at least one of rolling, pitching, yawing, and transition. The method of claim 15, If at least a portion of the input area of the display overlaps or touches a border of an input window of a page displayed on the display, displaying the overlap or contact to the user. delete The method of claim 15, And removing the displayed virtual text input interface from the display when a predetermined gesture is applied by the user. 24. The method of claim 23, wherein the predetermined gesture comprises: tilting the mobile terminal, shaking the mobile terminal at least once, pressing a virtual button on the display, pressing a physical button on the mobile terminal, or A user interface control method of a mobile terminal, which is one of operations of touching a predetermined area. The method of claim 15, wherein the virtual text input interface comprises a virtual keyboard or a handwriting recognition interface. The method of claim 15, wherein the scrolling comprises at least one of the following: Performing a zoom in / zoom out operation on a page currently displayed on the display according to the movement of the mobile terminal; And Performing the page turning according to the movement of the mobile terminal. The method of claim 26, And displaying a guide window on a portion of the display, the guide window displaying a currently displayed area, which is an entire page or part of a plurality of pages. In the method for controlling a user interface of a mobile terminal equipped with a display, Setting a selection point or input area on the display; Displaying a page on a display on which the selection point or input area is set; Detecting movement of the mobile terminal applied by a gesture of a user; Scrolling a page displayed on the display, a selection point or an input area according to the detected movement of the mobile terminal; And And displaying a guide window on a page currently displayed on the display, wherein the guide window is a reduced type of the page currently displayed on the display, and the guide window includes a reduced type of the selection point or the input area. Display control method of the terminal. 29. The method of claim 28, wherein the movement of the mobile terminal comprises at least one of rolling, pitching, yawing and transition. display; A detector configured to detect a movement of a mobile terminal generated by a gesture of a user; And In response to the detected movement, the user scrolls one of a page or a selection point displayed on the display, determines whether a selectable item included in the displayed page is located on the selection point, and selects the selection. And if it is determined that a possible item is located on a selection point of the display, controlling the display to select the selectable item in response to a predetermined event. display; User interface; A detector configured to detect a movement of a mobile terminal generated by a gesture of a user; And Scrolls a page displayed on the display according to the detected movement of the mobile terminal, determines whether at least a portion of an input area of the display overlaps or borders an input window of a page displayed on the display, and determines the result A mobile terminal comprising a control unit for displaying a virtual text input interface based on the. display; A detector configured to detect a movement of a mobile terminal generated by a gesture of a user; And And a controller for scrolling a page displayed on the display, a selection point or an input area according to the movement of the mobile terminal, and controlling the display to display a guide window in addition to the page currently displayed on the display. The guide window is a reduced type of the page currently displayed on the display, and the guide window includes a reduced type of the selection point or the input area.

Images