US9582049B2 - Method and device for controlling user interface based on user's gesture - Google Patents

Method and device for controlling user interface based on user's gesture Download PDF

Info

Publication number
US9582049B2
US9582049B2 US12/196,080 US19608008A US9582049B2 US 9582049 B2 US9582049 B2 US 9582049B2 US 19608008 A US19608008 A US 19608008A US 9582049 B2 US9582049 B2 US 9582049B2
Authority
US
United States
Prior art keywords
display
mobile device
input
user
movement
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US12/196,080
Other versions
US20090265627A1 (en
Inventor
Joo Min KIM
Youk Kwon
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
LG Electronics Inc
Original Assignee
LG Electronics Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to KR20080035556 priority Critical
Priority to KR10-2008-0035556 priority
Priority to KR1020080068312A priority patent/KR100912310B1/en
Priority to KR10-2008-0068312 priority
Application filed by LG Electronics Inc filed Critical LG Electronics Inc
Assigned to LG ELECTRONICS INC. reassignment LG ELECTRONICS INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: KIM, JOO MIN, KWON, YOUK
Publication of US20090265627A1 publication Critical patent/US20090265627A1/en
Application granted granted Critical
Publication of US9582049B2 publication Critical patent/US9582049B2/en
Application status is Active legal-status Critical
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00Details not covered by groups G06F3/00 – G06F13/00 and G06F21/00
    • G06F1/16Constructional details or arrangements
    • G06F1/1613Constructional details or arrangements for portable computers
    • G06F1/1633Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
    • G06F1/1684Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
    • G06F1/1694Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00Details not covered by groups G06F3/00 – G06F13/00 and G06F21/00
    • G06F1/16Constructional details or arrangements
    • G06F1/1613Constructional details or arrangements for portable computers
    • G06F1/1626Constructional details or arrangements for portable computers with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object or an image, setting a parameter value or selecting a range
    • G06F3/0485Scrolling or panning
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2200/00Indexing scheme relating to G06F1/04 - G06F1/32
    • G06F2200/16Indexing scheme relating to G06F1/16 - G06F1/18
    • G06F2200/163Indexing scheme relating to constructional details of the computer
    • G06F2200/1637Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer

Abstract

A mobile device and a method for controlling a user interface of the mobile device are disclosed. According to an embodiment, the method includes detecting a movement of the mobile device as applied by a user's gesture, scrolling at least one of a page displayed on the display and a selection point on the display according to the detected movement of the mobile device, determining whether a selectable item displayed within the page is positioned on the selection point of the display, and selecting the selectable item in response to a predetermined event, once the determining step determines that the selectable item is positioned on the selection point of the display.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority under 35 U.S.C. 119 to Korean Patent Application No. 10-2008-0035556 filed on Apr. 17, 2008 and Korean Patent Application No. 10-2008-0068312 filed on Jul. 14, 2008, which are hereby incorporated by reference in its entirety.

BACKGROUND

Field of the Invention

The present invention relates to a method for controlling a user interface, and more particularly to a mobile device and a computer program and method for controlling a user interface of a mobile device by detecting a user's gesture applied to the mobile device.

Description of the Related Art

There exist many kinds of mobile devices such as a mobile phone, an MP3 player, a PMP (portable multimedia player), a computer notebook, a PDA etc. These devices have limited user interfaces for users to input or output data due to the feature of portability. These mobile devices are usually not used in a fixed place but used while the user is moving with the devices. Also, the size of the mobile device is relatively small.

As such, it is improper to employ general I/O devices in these mobile devices which are devised for fixed devices such as a QWERTY key board, a mouse or a remote control which is devised for a desktop computer, large size TV, etc.

As such, different kinds of user interfaces were devised and widely used for the mobile devices such as a touch pad, a touch screen, a navigation pad, etc.

Nevertheless, the mobile devices are often still limited for users to input and output data and users feel that they can be more user-friendly and convenient to use.

Moreover, in case of mobile phones, tendency is toward adding many kinds of annexed functions to mobile phones such as an MP3 player, a camera, an internet browser, an image viewer, a video player, etc.

Mobile device users often need to use a keyboard especially when the user is browsing an internet webpage through the mobile device to input keywords for a search or password.

In case of a mobile device which substitutes a touch screen for physical buttons such as a full touch screen phone which has a large screen and does not have physical number buttons, if the user needs to input text, he needs to go through a few or more steps by manipulating the menu to have the virtual number buttons show up.

This causes much convenience to the users of the mobile devices. Advanced user interfaces for providing users with easy access to control mobile devices are required.

SUMMARY OF THE INVENTION

Accordingly, the present invention is to provide various mobile devices, and computer software and various methods for controlling a user interface of the mobile device.

The present invention is to provide a user interface which enables the user to input directions and data.

Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

To achieve these objects and other advantages and in accordance with the purpose of the invention, as embodied and broadly described herein, there is provided according to an embodiment a method for controlling a user interface comprising: detecting a user's gesture applied to a mobile device; scrolling a display over a page displayed on the mobile device according to the detected gesture; detecting that a selectable item on the page is positioned on a selection point of the display of the mobile device; and activating a user interface.

In another aspect of the present invention, there is provided a method of controlling a user interface comprising: detecting a user's gesture applied to a mobile device; scrolling a display over a page displayed on the mobile device according to the detected gesture; detecting that at least a part of an input area of the display is overlapped with at least a part of the input window of the page; and activating a user interface.

In another aspect of the present invention, there is provided a method for controlling a user interface comprising: detecting a user's gesture applied to a mobile device; scrolling a display of the mobile device over a page displayed on the mobile device; and displaying a guide window including a miniature of the page.

In another aspect of the present invention, there is provided a method for controlling a user interface of a mobile device, comprising: detecting a user's gesture applied to the mobile device; scrolling a display over a page displayed on the mobile device according to the detected gesture; detecting that a selectable item on the page is positioned on a selection point of the display of the mobile device; detecting a user's gesture for execution; and executing the selectable item.

In another aspect of the present invention, there is provided a method for controlling a user interface of a mobile device, comprising: detecting a user's gesture applied to the mobile device; moving a selection point on a display of the mobile device according to the detected gesture; detecting that the selection point on the display is positioned on a selection item of on a displayed page of the mobile device; detecting a user's gesture for execution; and executing the selectable item.

In another aspect of the present invention, there is provided a method for controlling a user interface of a mobile device including a display, the method comprising: detecting a movement of the mobile device as applied by a user's gesture; scrolling at least one of a page displayed on the display and a selection point on the display according to the detected movement of the mobile device; determining whether a selectable item displayed within the page is positioned on the selection point of the display; and selecting the selectable item in response to a predetermined event, once the determining step determines that the selectable item is positioned on the selection point of the display.

In another aspect of the present invention, there is provided a method of controlling a user interface of a mobile device, the method comprising: detecting a movement of the mobile device as applied by a user's gesture; scrolling a page displayed on a display of the mobile device according to the detected movement of the mobile device; determining when at least a part of an input area of the display is in contact with or overlapping an input window of the page displayed on the display; and activating a user interface based on the determination result.

In another aspect of the present invention, there is provided a method for controlling a display of a mobile device, the method comprising: detecting a movement of the mobile device as applied by a user's gesture; scrolling a page on the display according to the movement of the mobile device; and displaying, on the display, a guide window in addition to the page currently displayed on the display, the guide window including a miniature of the page currently displayed on the display.

In another aspect of the present invention, there is provided a mobile device comprising: a display; a detector configured to detect a movement of the mobile device as applied by a user's gesture; and a controller, by controlling the display, configured to scroll at least one of a page displayed on the display and a selection point on the display according to the detected movement of the mobile device; to determine whether a selectable item displayed within the page is positioned on the selection point of the display; and to select the selectable item in response to a predetermined event, once the controller determines that the selectable item is positioned on the selection point of the display.

In another aspect of the present invention, there is provided a mobile device comprising: a display; a user interface; a detector configured to detect a movement of the mobile device as applied by a user's gesture; and a controller, by controlling the display, configured to scroll a page displayed on the display according to the detected movement of the mobile device; to determine when at least a part of an input area of the display is in contact with or overlapping an input window of the page displayed on the display; and to activate the user interface based on this determination result.

In another aspect of the present invention, there is provided a mobile device comprising: a display; a detector configured to detect a movement of the mobile device as applied by a user's gesture; and a controller, by controlling the display, configured to scroll a page on the display according to the movement of the mobile device; and to display, on the display, a guide window in addition to the page currently displayed on the display, the guide window including a miniature of the page currently displayed on the display.

It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principle of the invention. In the drawings:

FIGS. 1A and 1B show an embodiment of a controlling method for a user interface of a mobile device using a gesture according to the present invention;

FIGS. 2A and 2B show another embodiment of a controlling method for a user interface of a mobile device using a gesture according to the present invention;

FIGS. 3A and 3B show an embodiment of a method for selecting a selectable item on the display according to the present invention;

FIGS. 4A and 4B show an embodiment of a method for selecting a selectable item on the display according to the present invention;

FIG. 5 shows a gesture of shaking the mobile phone according to an embodiment of the present invention;

FIGS. 6A and 6B show another embodiment of a method for selecting a selectable item on the display according to the present invention;

FIGS. 7A and 7B show an example of a method for providing an input menu according to an embodiment of the present invention;

FIGS. 8A and 8B show an example of a method for provided an input menu according to another embodiment of the present invention;

FIG. 9 shows an example of a method for determining whether to activate a user interface according to an embodiment of the present invention;

FIG. 10 shows an example of a user's gesture according to an embodiment of the present invention;

FIG. 11 shows a user interface controlling method of a mobile device according to an embodiment of the present invention.

FIG. 12 is a block diagram of a mobile device in accordance with an embodiment of the present invention.

FIG. 13 is a block diagram of a CDMA wireless communication system operable with the mobile device of FIG. 12.

DETAILED DESCRIPTION OF THE INVENTION

Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts.

FIG. 1A and FIG. 1B show a user interface controlling method using a gesture according to an embodiment of the present invention.

A mobile device according to the present invention may be equipped with an inertia sensor such as a gyro sensor, a tilt sensor, an acceleration sensor, etc. Examples of the mobile device according to the present invention include a mobile phone, an MP3 player, a PMP (Portable Multimedia Player), a computer notebook, a PDA, a smart phone, etc.

In the following detailed description, the embodiments of the present invention are described assuming a case where the mobile device is a mobile phone. However, it should be clearly understood that the present invention is not limited thereto, and covers various types of mobile devices or portable electronic devices.

The mobile phone in the accompanying drawings is a so-called “full touch screen phone” which has no physical number buttons on the front surface of the mobile phone and the display covers most of the front surface.

As shown in FIG. 1A, if user applies a gesture to the mobile device such as a mobile phone, the inertia sensor in the mobile phone may sense the gesture of a user as the user applies it to the mobile phone. A controller inside the mobile phone directs the mobile device/phone to perform a certain function corresponding to the applied gesture.

For example, assuming a user is browsing an internet webpage through wired or wireless connections using the mobile phone, if the user tilts the mobile phone, the inertia sensor detects the motion caused by the tilting gesture. For instance, if the user tilts the mobile phone toward the right side, the controller of the mobile phone may cause scrolling of the display over the webpage currently being viewed according to the detected motion. The user can tilt the mobile phone in various directions and angles, which according to the present invention causes the scrolling of the page (e.g., webpage or Word document page, etc.) on the display of the mobile phone to occur, according to the tilting directions and/or angles. In these cases, the direction of the page scrolling may correspond with the direction in which the mobile device is tilted by the user, and the amount of the page scrolling (e.g., speed of scrolling or number of pages being scrolled) may correspond with the angle at which the mobile device is tilted by the user. In other examples, the user may tilt the mobile device once (e.g., by flicking the mobile device back and forth once), then the page may be scrolled to a next page so that the page-to-page scrolling may occur according to the number of times the mobile device is flicked. In another example, the user may tilt the mobile device in a certain direction and hold it there for a predetermined time duration. In that case, the distance of scrolling may be proportional to the predetermined time duration. In another example, a certain action (e.g., flick twice quickly) may cause the continuous scrolling of the page to occur automatically. In all these examples, the flicking/tilting directions may control the scrolling directions as discussed above. Further, the tilting directions of the invention include any and all directions (e.g., slant angle directions, diagonal directions, etc.) and are not limited to the right, left, up and down directions.

The left picture of FIG. 1B shows a part of a webpage being browsed and the right picture shows the webpage scrolled to the right. As shown in FIG. 1A, if the user tilts the mobile phone to the right, the inertia sensor may detect this gesture and may scroll the display over the webpage to the right according to the gesture detection.

Similarly, if the user tilts the mobile phone to the left, the inertia sensor may detect this gesture and may scroll the display over the webpage to the left according to this gesture detection.

The operation mode where the inertia sensor detects the user's gesture as applied to the mobile device and the controller of the mobile device directs the mobile phone/device responsive to the detection (hereinafter referred to as “gesturing mode” may be activated or deactivated by the user's selection). That is, the gesturing mode of the mobile device is a mode in which the mobile device operates and controls the user interface according to various gestures applied to the mobile device by the user, as discussed herein according to the present invention.

In an example, in FIG. 1A, the mobile phone may be equipped with an activation button 11 on a side of a mobile device, and the gesturing mode may be activated only when the activation button 11 is pushed or is pushed down for a predetermined time duration.

The scheme to activate the gesturing mode may be varied. For example, in one embodiment, without a physical button 11, the user may manipulate software for a menu control embedded in the mobile phone to activate the gesturing mode.

In one embodiment, it may be configured that the gesturing mode is activated when a specific gesture is applied. For example, the gesturing mode may be activated when the user quickly shakes the mobile phone. The inertia sensor may detect the shaking gesture and forward the gesture detection to the controller in the mobile phone which directs the mobile phone to enter the gesturing mode. Also, it may be configured that the user may escape from the gesturing mode when the user quickly shakes the mobile phone again in the gesturing mode. In a non-gesturing mode, the mobile phone is not responsive to the user's gesture applied to the mobile device.

In one embodiment, it may be configured that the mobile phone enters the gesturing mode when the mobile phone is tilted at a specific angel for a predetermined certain time. That is, if the user grabs and holds the mobile phone at a certain angle and for 3 seconds, the inertia sensor detects this gesture and the mobile phone may enter the gesturing mode.

The display of a mobile device such as a mobile phone is often configured to be deactivated as the power to the display is cut off for power saving if the mobile phone is not in use for a certain time. In one embodiment of the present invention, the gesturing mode of the mobile phone may be deactivated when the display is deactivated, to save power and other resources. When the mobile phone is not in use for a long time, the gesturing mode would not be needed either.

In a non-limiting example, as shown in FIG. 2A, if the user holds up the phone and tilts the mobile phone down so that the top/upper part of the mobile phone gets closer to the user with holding the activation button 11 (tilting the phone in the upward direction with respect to the horizontal axis), the display of the mobile phone may be scrolled down to the webpage as shown in FIG. 2B. If the user tilts the mobile phone up so that the phone turns up, the display of the mobile phone may be scrolled up toward the top of the webpage being browsed.

In one embodiment, it may be configured that whenever the user tilts the mobile phone more than a predetermined angel, the display may be scrolled by a predetermined distance. In this case, it only matters whether the tilted angle is beyond the predetermined angel or not. If the user wants to scroll a long distance, he needs to tilt the mobile phone several times. Alternatively, the display may be scrolled by a distance in proportion to the tilted angle.

The scroll direction according to the tilted direction may be varied. For example, opposite to the above cases, if the mobile phone is tilted to the tight, the display may be scrolled to the left; if the mobile device is tiled to the left, the display is scrolled to the right; if the mobile device is tilted toward the top, the display is scrolled down; and if the mobile device is tilted down, the display is scrolled to the top of the display.

Conventionally, mobile devices such mobile phones are not equipped with QWERTY keyboard or mouse as in PC. Accordingly, when the user browses the internet web pages, stylus pen or finger is used instead of a mouse to click or double click the selectable items such as hyperlinks or to push various virtual buttons on the web pages.

Also, conventionally, when the user needs to type in characters such as ID or password, he needs to move the cursor on the input window before typing in the characters. In case of the mobile device which has a touch screen, the user needs to enable the virtual text input panel or switch to a note recognition mode to input characters.

FIGS. 3A and 3B show an embodiment of a method for selecting selectable items on a display of a mobile device according to the present invention.

Assume the user is browsing the internet web page on the mobile device, and a hyperlink 13 is positioned on the display as shown in FIG. 3A. A certain position on the display may be preset as a selection point 12. Preferably, the selection point 12 is positioned in the middle of the display, but can set at any desired position, which may be set or changed by the user or the mobile device. Preferably the selection point is fixed on the display and does not change even if the web page is scrolled. The selection point 12 may be invisible to the user or may be marked with a certain symbol or color so that it is visible to the user. In another example, the selection point 12 can be a selection area.

In the state as shown in FIG. 3A, if the user tilts the mobile phone downward (that is, the top of the mobile phone is turned toward the user) as shown in FIG. 2A, the web page displayed on the display of the mobile phone is scrolled so that the hyperlink 13 of the web page is positioned on the selection point 12. When the hyperlink 13 of the web page is positioned on the selection point 12, the user can execute the hyperlink by clicking the hyperlink 13 with his finger or stylus pen. An indicator 14 may be displayed to inform the user the hyperlink 13 is available to select. Executing the hyperlink can include, accessing the hyperlink website and displaying it on the display, performing a predetermined operation associated with the hyperlink, etc.

In one embodiment, the hyperlink may be executed when the hyperlink 13 stays on the selection point 12 for a predetermined time, for example two seconds without any input from the user. In another example, when the hyperlink 13 is on the selection point 12, if the user shakes or flicks the mobile device, then the hyperlink 13 may be executed. In another example, the scrolling of the page may stop automatically when the hyperlink 13 comes over the selection point 12. Then the user can shake, push a button, or perform a preset action to execute the hyperlink 13.

Alternatively, a virtual button may be displayed in a part of the display when the hyperlink 13 is positioned on the selection point 12. The hyperlink may be executed when the user touches the virtual button.

The above embodiment may be applied to selecting any virtual buttons other than hyperlinks provided on the web pages when the user is browsing the internet web page. For example, the above embodiment may be applied to a case where a video playing panel is displayed and the user needs to select the virtual buttons to manipulate the video file such as playback, pause, stop, etc.

In one embodiment, an indicator 14 may be displayed on the selection point 12 as shown in FIG. 4A. When the display of the mobile phone is scrolled and the hyperlink 13 is positioned on the selection point 12, it may be configured to inform the user the hyperlink 13 is available to select by changing the indicator 14 to another shape of indicator 14 as shown in FIG. 4B. The hyperlink 13 may be executed by the user touching on the display or pushing a physical button on the mobile phone or in other way as discussed above.

In one embodiment of the present invention, while the indicator 14 is displayed as shown in FIG. 4B, it may be configured that if the user shakes the mobile device as shown in FIG. 5, the hyperlink upon which the indicator 14 is positioned is executed by detecting the user's gesture of shaking. The gesture of shaking may be effective to execute the hyperlink only while the user pushes an activation button 15 or other physical button on the mobile device.

The gesture of shaking the mobile device may also be used to click a virtual button displayed on the mobile device.

In one embodiment of the present invention, if the user tilts the mobile device while pushing and holding down on the activation button 15 or other physical button on the mobile device, the selection point 12 itself may be moved on the display with the displayed page being fixed and not moved on the display.

That is, as a web page is displayed on the display of the mobile device as shown in FIG. 6A, if the user tilts the mobile device while pushing and holding down on the activation button 15, the displayed web page is not scrolled (and the web page is fixedly displayed) but the selection point 12 may move on the display. When the user moves the selection point 12 (e.g., by tilting the mobile device while the activation button 15 is pressed down) and puts the selection point 12 on the hyperlink 13, the selection point 12 may be changed to an indicator 14 and become executable as shown in FIG. 6B. While the indicator 14 is displayed, the hyperlink 13 may be executed by clicking or touching the hyperlink 13 or by pushing a physical button on the mobile device or in some other way as discussed above. Here, the user can move the selection point 12 in any direction on the display.

In another example, without pressing the button 15, the user scrolls the pages by tilting the mobile device as discussed above. Then the user can press down on the button 15 and tilt the mobile device, to move the selection point in any direction (e.g., diagonally, up, down, right, left, etc.).

In one embodiment of the present invention, if the user shakes the mobile device as shown in FIG. 5 while the indicator 14 is displayed, a motion sensor of the mobile device may detect the gesture of shaking of mobile device and the hyperlink may be executed upon detecting this gesture. The execution of hyperlink may be effectively triggered only when the activation button 15 is pushed or when other physical button is pushed.

In other aspects, the user often needs to type in characters when browsing the internet web pages other than to execute the hyperlink or click the virtual button so as to input key words for a search or ID and password for login. User interface for typing in various characters is required in a full touch screen phone, PDA or a normal mobile phone which has the number buttons not a QWERTY keyboard especially when the user is browsing the internet web page.

In one embodiment of the present invention, the user can be provided with a user interface for typing in characters, symbols, etc.

As shown in FIG. 7A, a predetermined area of the display on the mobile phone/device may be set as an input area 16. The input area 16 may be in the same shape of a rectangle as the input windows in the conventional internet web pages, or may be in other shape. The input area 16 may be invisible to the user or be visible to the user by being visualized with dot lines or inconspicuous lines or in some other manner. In one example, the input area 16 may be visualized by shadowing or in different color. A page (e.g., a webpage) displayed on the display of the mobile device may have an input window 17 for a user to input data and information.

As shown in FIG. 7B, the input area 16 and the input window 17 of the web page may be matched from the state shown in FIG. 7A by tilting the mobile phone in various directions as shown in FIGS. 1A through 2B. In one example, the input area 16 is fixed on the display and is not moved. In this case, the user can tilt the mobile device to scroll/move the page in the tilting direction, which in turn moves the input window 17 included in the page. By controlling the tilting direction, then the user can move the input window 17 to be placed over (in part or in entirety) or in contact with the input area 16.

Then, when the input window 17 matches the input area 16 or is placed over at least in part or touches the input area 16, a user interface may be provided on the display. For example, when the input area 16 and the input window 17 of the web page are overlapped at least in part or are in contact with each other, a virtual keyboard 18 may be displayed as shown in FIG. 7B. The virtual keyboard 18 may be either of number buttons used in normal mobile phones or QWERTY keyboard or some other input unit.

When the virtual keyboard 18 is displayed, the focus of the web page is moved to the input window 17 and the user can input characters into the input window 17 through the virtual keyboard 18 by, e.g., touching the keys by a finger or stylus.

Characters may be inputted without using the virtual keyboard in an embodiment of the present invention. For example, in case of a mobile phone which does not have a touch screen, when the input area 16 and the input window 17 are overlapped, it is not needed to display the virtual keyboard 18 and only focus is moved to the input area 16. The user can input characters by pushing the physical buttons provided on the mobile phone.

Alternatively, a note recognition panel may be provided instead of the virtual keyboard 18 and the user can input characters through the note recognition panel.

In one embodiment of the present invention, when the input window of a web page being browsed on the mobile device is available to input characters, it may be displayed to the user together with visualizing the input area 16.

By selecting a button (virtual or physical button) or performing a certain action (e.g., tilting the mobile device, shaking the mobile device at least once, touching an area of the display in a certain manner, etc.) on the mobile device, the user may cause the displayed virtual keyboard to disappear or the displayed page to go back to a previous page.

In a state shown in FIG. 8A, the user may apply gestures to the mobile phone to overlap at least in part (or in touch) the input area 16 with the input window 17 as discussed above. A virtual key pad 18 may be displayed as shown in FIG. 8B when the input area 16 and the input window 17 are overlapped.

In one embodiment, the lines 16+17 where the input area 16 and the input window 17 are overlapped may be made conspicuous by increasing the brightness of the lines of changing the color of lines. As such, the user may easily recognize that it is in a character-input mode where he can input characters.

In FIGS. 7A-8B, the sizes of the input area 16 and input window 17 may be different. In this case, the input area 16 and the input window 17 cannot be exactly matched. The virtual keyboard panel (e.g., 18) may be configured to be displayed when any part of the input area 16 is overlapped with the input window 17 or conversely any part of the input window 17 is overlapped with the input area 16 as shown in FIG. 9.

In one embodiment, the virtual keyboard panel may be displayed when the overlapped area of the input area and the input window is beyond a predetermined area.

In one embodiment, the virtual keyboard panel may be displayed when an edge of the input area 16 is overlapped with an edge of the input window 17 or when two edges of the input area 16 are overlapped with the edges of the input window 17.

In one embodiment, the input area 16 may be set as relatively small and the virtual keyboard panel may be displayed when the input area 16 is included in the input window 17.

In one embodiment of the present invention, when the input area 16 and the input window 17 are matched or overlapped, a note recognition panel may be displayed instead of the virtual keyboard panel. The moment the note recognition panel is displayed, the note recognition mode may be locked by a predetermined input from the user to maintain the note recognition panel. When the note recognition mode is locked, the note recognition panel is maintained without overlapping between the input area 16 and the input window 17 anymore.

The predetermined input from the user may be either a physical button on the surface of the mobile phone or a virtual button displayed on the mobile phone.

Alternatively, the virtual keyboard or the note recognition panel may be displayed and locked when the input area 16 and the input window 17 are kept overlapped for more than a predetermined time, for example, 2 seconds. That is, when the input area 16 and the input window 17 are overlapped and the state is kept for a predetermined time, the virtual keyboard or the note recognition panel may be kept being displayed despite moving or tilting of the mobile phone.

In the examples of FIGS. 7A-9, the input area 16 is preset and fixedly displayed and does not move while the input window 17 may be moved by moving the mobile device. Here, the user may vary the preset location of the input area 16 by controlling menus. In another example, similar to the example of FIGS. 6A and 6B, the input area 16 may be moved by the user as the user tilts the mobile device, while the input window 17 is fixedly displayed.

In one embodiment of the present invention, a mobile device may be equipped with an inertia sensor and the display of the mobile device may be scrolled by detecting the motion of the mobile device with an activation key 11 pushed as shown in FIG. 10. A user interface may be activated when a selection point is positioned on a selectable item or an input area is matched or overlapped with an input window. The user may select an item or input data through the user interface.

In one embodiment, a guide on the display of the mobile device may be provided which is used to locate the part currently being viewed in the whole page such a case as browsing the internet web page.

For example, as shown in FIG. 11, a guide window 20 may be displayed on a part of the display. The guide window 20 comprises a miniature of the whole web page, in which a location indicator 21 may be displayed. The location indicator 21 indicates the location of the part currently being viewed on the display.

The location indicator 21 moves as the web page is scrolled through the user's gestures as discussed above in various embodiments. The user may recognize where the part currently being shown is located on the display, from the location indicator 21.

In one embodiment, as the web page is zoomed in or out, the location indicator 21 may be zoomed in or out accordingly.

In one embodiment, the selection point 12 or input area 16 may be included in the location indicator 21. As such, the user may select a selectable item or scroll the web page by looking at only the guide window 20.

According to the present invention, the hyperlink 13 and the input window 17 are examples of selectable items. Other types of selectable items may be displayed on the display and selected by the user as the user tilts the mobile device, in the same or similar manner the hyperlink 13 is executed as discussed above. Other types of selectable items may be one or more buttons associated with controlling a playing/recording of a motion picture or audio data. etc. Further, the movement of the mobile device as applied by the user, which controls the user interface as discussed according to the embodiments of the present invention, can include at least one of rolling (e.g., rotating right and left), pitching (e.g., rotating front and back), yawing/heading/pivoting (e.g., right and left or front and back pivoted movement) and transition (e.g., lateral movement). Further, scrolling of a page as discussed above can include a page-to-page flipping and an in/out zooming operation according to the present invention.

Various embodiments and examples of methods and operations of the present invention discussed above are implemented in a mobile device or portable electronic device. An example of such a device as a mobile device 100 is discussed below referring to FIGS. 12 and 13. A controller 180 of the mobile device 100 may be configured to control a display 151 (which may be a touch screen) to perform the various user interface controlling methods of the present invention. The mobile device of the present invention may have all or part of the components of the mobile device 100.

FIG. 12 is a block diagram of the mobile device 100 in accordance with an embodiment of the present invention. The mobile device may be implemented using a variety of different types of devices. Examples of such devices include mobile phones, user equipment, smart phones, computers, digital broadcast devices, personal digital assistants, portable multimedia players (PMP) and navigators. By way of non-limiting example only, further description will be with regard to a mobile device. However, such teachings apply equally to other types of devices. FIG. 12 shows the mobile device 100 having various components, but it is understood that implementing all of the illustrated components is not a requirement. Greater or fewer components may alternatively be implemented.

FIG. 12 shows a wireless communication unit 110 configured with several commonly implemented components. For instance, the wireless communication unit 110 typically includes one or more components which permits wireless communication between the mobile device 100 and a wireless communication system or network within which the mobile device is located.

The broadcast receiving module 111 receives a broadcast signal and/or broadcast associated information from an external broadcast managing entity via a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast managing entity refers generally to a system which transmits a broadcast signal and/or broadcast associated information. Examples of broadcast associated information include information associated with a broadcast channel, a broadcast program, a broadcast service provider, etc. For instance, broadcast associated information may include an electronic program guide (EPG) of digital multimedia broadcasting (DMB) and electronic service guide (ESG) of digital video broadcast-handheld (DVB-H).

The broadcast signal may be implemented as a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, among others. If desired, the broadcast signal may further include a broadcast signal combined with a TV or radio broadcast signal.

The broadcast receiving module 111 may be configured to receive broadcast signals transmitted from various types of broadcast systems. By nonlimiting example, such broadcasting systems include digital multimedia broadcasting-terrestrial (DMB-T), digital multimedia broadcasting-satellite (DMB-S), digital video broadcast-handheld (DVB-H), the data broadcasting system known as media forward link only (MediaFLO®) and integrated services digital broadcast-terrestrial (ISDB-T). Receiving of multicast signals is also possible. If desired, data received by the broadcast receiving module 111 may be stored in a suitable device, such as memory 160.

The mobile communication module 112 transmits/receives wireless signals to/from one or more network entities (e.g., base station, Node-B). Such signals may represent audio, video, multimedia, control signaling, and data, among others.

The wireless internet module 113 supports Internet access for the mobile device. This module may be internally or externally coupled to the device.

The short-range communication module 114 facilitates relatively short-range communications. Suitable technologies for implementing this module include radio frequency identification (RFID), infrared data association (IrDA), ultra-wideband (UWB), as well at the networking technologies commonly referred to as Bluetooth and ZigBee, to name a few.

Position-location module 115 identifies or otherwise obtains the location of the mobile device. If desired, this module may be implemented using global positioning system (GPS) components which cooperate with associated satellites, network components, and combinations thereof.

Audio/video (AN) input unit 120 is configured to provide audio or video signal input to the mobile device. As shown, the AN input unit 120 includes a camera 121 and a microphone 122. The camera receives and processes image frames of still pictures or video.

The microphone 122 receives an external audio signal while the portable device is in a particular mode, such as phone call mode, recording mode and voice recognition. This audio signal is processed and converted into digital data. The portable device, and in particular, A/V input unit 120, typically includes assorted noise removing algorithms to remove noise generated in the course of receiving the external audio signal. Data generated by the A/V input unit 120 may be stored in memory 160, utilized by output unit 150, or transmitted via one or more modules of communication unit 110. If desired, two or more microphones and/or cameras may be used.

The user input unit 130 generates input data responsive to user manipulation of an associated input device or devices. Examples of such devices include a keypad, a dome switch, a touchpad (e.g., static pressure/capacitance), a touch screen panel, a jog wheel and a jog switch.

The sensing unit 140 provides status measurements of various aspects of the mobile device. For instance, the sensing unit may detect an open/close status of the mobile device, relative positioning of components (e.g., a display and keypad) of the mobile device, a change of position of the mobile device or a component of the mobile device, a presence or absence of user contact with the mobile device, orientation or acceleration/deceleration of the mobile device.

The sensing unit 140 may comprise an inertia sensor for detecting movement or position of the mobile device such as a gyro sensor, an acceleration sensor etc. or a distance sensor for detecting or measuring the distance relationship between the user's body and the mobile device.

The interface unit 170 is often implemented to couple the mobile device with external devices. Typical external devices include wired/wireless headphones, external chargers, power supplies, storage devices configured to store data (e.g., audio, video, pictures, etc.), earphones, and microphones, among others. The interface unit 170 may be configured using a wired/wireless data port, a card socket (e.g., for coupling to a memory card, subscriber identity module (SIM) card, user identity module (UIM) card, removable user identity module (RUIM) card), audio input/output ports and video input/output ports.

The output unit 150 generally includes various components which support the output requirements of the mobile device. Display 151 is typically implemented to visually display information associated with the mobile device 100. For instance, if the mobile device is operating in a phone call mode, the display will generally provide a user interface or graphical user interface which includes information associated with placing, conducting, and terminating a phone call. As another example, if the mobile device 100 is in a video call mode or a photographing mode, the display 151 may additionally or alternatively display images which are associated with these modes.

A touch screen panel may be mounted upon the display 151. This configuration permits the display to function both as an output device and an input device.

The display 151 may be implemented using known display technologies including, for example, a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT-LCD), an organic light-emitting diode display (OLED), a flexible display and a three-dimensional display. The mobile device may include one or more of such displays.

FIG. 12 further shows an output unit 150 having an audio output module 152 which supports the audio output requirements of the mobile device 100. The audio output module is often implemented using one or more speakers, buzzers, other audio producing devices, and combinations thereof. The audio output module functions in various modes including call-receiving mode, call-placing mode, recording mode, voice recognition mode and broadcast reception mode. During operation, the audio output module 152 outputs audio relating to a particular function (e.g., call received, message received, and errors).

The output unit 150 is further shown having an alarm 153, which is commonly used to signal or otherwise identify the occurrence of a particular event associated with the mobile device. Typical events include call received, message received and user input received. An example of such output includes the providing of tactile sensations (e.g., vibration) to a user. For instance, the alarm 153 may be configured to vibrate responsive to the mobile device receiving a call or message. As another example, vibration is provided by alarm 153 as a feedback responsive to receiving user input at the mobile device, thus providing a tactile feedback mechanism. It is understood that the various output provided by the components of output unit 150 may be separately performed, or such output may be performed using any combination of such components.

The memory 160 is generally used to store various types of data to support the processing, control, and storage requirements of the mobile device. Examples of such data include program instructions for applications operating on the mobile device, contact data, phonebook data, messages, pictures, video, etc. The memory 160 shown in FIG. 12 may be implemented using any type (or combination) of suitable volatile and non-volatile memory or storage devices including random access memory (RAM), static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk, card-type memory, or other similar memory or data storage device.

The controller 180 typically controls the overall operations of the mobile device. For instance, the controller performs the control and processing associated with voice calls, data communications, video calls, camera operations and recording operations. If desired, the controller may include a multimedia module 181 which provides multimedia playback. The multimedia module may be configured as part of the controller 180, or this module may be implemented as a separate component.

The power supply 190 provides power required by the various components for the portable device. The provided power may be internal power, external power, or combinations thereof.

Various embodiments of the present invention described herein may be implemented in a computer-readable medium using, for example, computer software, hardware, or some combination thereof. For a hardware implementation, the embodiments described herein may be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described herein, or a selective combination thereof. In some cases, such embodiments are implemented by controller 180.

For a software implementation, the embodiments described herein may be implemented with separate software modules, such as procedures and functions, each of which perform one or more of the functions and operations described herein. The software codes can be implemented with a software application written in any suitable programming language and may be stored in memory (for example, memory 160), and executed by a controller or processor (for example, controller 180).

The mobile device 100 may be implemented in a variety of different configurations. Examples of such configurations include folder-type, slide-type, bar-type, rotational-type, swing-type and combinations thereof. For clarity, further disclosure will primarily relate to a slide-type mobile device. However such teachings apply equally to other types of devices.

The mobile device 100 of FIG. 12 may be configured to operate within a communication system which transmits data via frames or packets, including both wireless and wireline communication systems, and satellite-based communication systems. Such communication systems utilize different air interfaces and/or physical layers.

Examples of such air interfaces utilized by the communication systems include example, frequency division multiple access (FDMA), time division multiple access (TDMA), code division multiple access (CDMA), and universal mobile telecommunications system (UMTS), the long term evolution (LTE) of the UMTS, and the global system for mobile communications (GSM). By way of non-limiting example only, further description will relate to a CDMA communication system, but such teachings apply equally to other system types.

Referring now to FIG. 13, a CDMA wireless communication system is shown having a plurality of mobile devices 100, a plurality of base stations 270, base station controllers (BSCs) 275, and a mobile switching center (MSC) 280. The MSC 280 is configured to interface with a conventional public switch telephone network (PSTN) 290. The MSC 280 is also configured to interface with the BSCs 275. The BSCs 275 are coupled to the base stations 270 via backhaul lines. The backhaul lines may be configured in accordance with any of several known interfaces including, for example, E1/T1, ATM, IP, PPP, Frame Relay, HDSL, ADSL, or xDSL. It is to be understood that the system may include more than two BSCs 275.

Each base station 270 may include one or more sectors, each sector having an omnidirectional antenna or an antenna pointed in a particular direction radially away from the base station 270. Alternatively, each sector may include two antennas for diversity reception. Each base station 270 may be configured to support a plurality of frequency assignments, with each frequency assignment having a particular spectrum (e.g., 1.25 MHz, 5 MHz).

The intersection of a sector and frequency assignment may be referred to as a CDMA channel. The base stations 270 may also be referred to as base station transceiver subsystems (BTSs). In some cases, the term “base station” may be used to refer collectively to a BSC 275, and one or more base stations 270. The base stations may also be denoted “cell sites.” Alternatively, individual sectors of a given base station 270 may be referred to as cell sites.

A terrestrial digital multimedia broadcasting (DMB) transmitter 295 is shown broadcasting to portable devices 100 operating within the system. The broadcast receiving module 111 (FIG. 12) of the portable device is typically configured to receive broadcast signals transmitted by the DMB transmitter 295. Similar arrangements may be implemented for other types of broadcast and multicast signaling (as discussed above).

FIG. 13 further depicts several global positioning system (GPS) satellites 300. Such satellites facilitate locating the position of some or all of the portable/mobile devices 100. Two satellites are depicted, but it is understood that useful positioning information may be obtained with greater or fewer satellites. The position-location module 115 (FIG. 12) of the portable device 100 is typically configured to cooperate with the satellites 300 to obtain desired position information. It is to be appreciated that other types of position detection technology, (i.e., location technology that may be used in addition to or instead of GPS location technology) may alternatively be implemented. If desired, some or all of the GPS satellites 300 may alternatively or additionally be configured to provide satellite DMB transmissions.

During typical operation of the wireless communication system, the base stations 270 receive sets of reverse-link signals from various mobile devices 100. The mobile devices 100 are engaging in calls, messaging, and other communications. Each reverse-link signal received by a given base station 270 is processed within that base station. The resulting data is forwarded to an associated BSC 275. The BSC provides call resource allocation and mobility management functionality including the orchestration of soft handoffs between base stations 270. The BSCs 275 also route the received data to the MSC 280, which provides additional routing services for interfacing with the PSTN 290. Similarly, the PSTN interfaces with the MSC 280, and the MSC interfaces with the BSCs 275, which in turn control the base stations 270 to transmit sets of forward-link signals to the mobile devices 100.

Although embodiments have been described with reference to a number of illustrative embodiments thereof, it should be understood that numerous other modifications and embodiments can be devised by those skilled in the art that will fall within the spirit and scope of the principles of this disclosure.

Claims (22)

What is claimed is:
1. A method for controlling a user interface of a mobile device including a display, the method comprising:
detecting a movement of the mobile device as applied by a user's gesture;
scrolling a page displayed on the display according to the detected movement of the mobile device only while a button for activating the scrolling is pressed;
displaying, on the display, a guide window in addition to the page currently displayed on the display while the scrolling is performed, wherein the guide window includes a miniature of the page currently displayed on the display and a location indicator indicating the location of the part currently being viewed on the display; and
if at least part of an input area of the display overlaps an input window of the page, displaying a virtual text input panel for inputting text into the overlapped input window
wherein the input window is an area for inputting and displaying text,
wherein if the input area and the input window are overlapped for a predetermined time, the virtual text input panel continues to be displayed despite movement of the mobile terminal, and
wherein if the input area and the input window are not overlapped for the predetermined time due to the movement of the mobile terminal, the virtual text input panel disappears.
2. The method according to claim 1, further comprising:
indicating, on the display, that the overlapped input window is ready so that a user can input text into the overlapped input window.
3. The method according to claim 1, wherein the scrolling step includes at least one of the following:
performing a zooming in/out operation on a portion of the page currently displayed on the display, according to a movement of the mobile device; and
flipping through other pages according to a movement of the mobile device.
4. The method according to claim 1, wherein the movement of the mobile device includes at least one of rolling, pitching, yawing, and transition.
5. The method according to claim 1, wherein the display includes a touch screen.
6. The method according to claim 1, wherein in the scrolling step, only the page is scrollable while the input area is fixed on the display.
7. The method according to claim 1, wherein in the scrolling step, the input area is visible to a user.
8. The method according to claim 1, wherein in the scrolling step, the input area is invisible to a user.
9. A method of controlling a user interface of a mobile device, the method comprising:
detecting a movement of the mobile device as applied by a user's gesture;
scrolling a page displayed on a display of the mobile device according to the detected movement of the mobile device; and
if at least part of an input area of the display overlaps an input window of the page, displaying a virtual text input panel for inputting text into the overlapped input window
wherein the input window is an area for inputting and displaying text,
wherein, if the input area and the input window are overlapped for a predetermined time, the virtual text input panel continues to be displayed despite movement of the mobile terminal, and
wherein if the input area and the input window are not overlapped for the predetermined time due to the movement of the mobile terminal, the virtual text input panel disappears.
10. The method according to claim 9, wherein the scrolling is performed, when the mobile device is moved by the user's gesture made while the user presses a button on the mobile device.
11. The method according to claim 9, wherein in the scrolling step, the input window is scrolled according the movement of the mobile device while the input area is fixed on the display.
12. The method according to claim 9, wherein the input area is visible to a user.
13. The method according to claim 9, wherein the input area is invisible to a user.
14. The method according to claim 9, wherein the movement of the mobile device includes at least one of rolling, pitching, yawing, and transition.
15. The method according to claim 9, further comprising:
indicating to a user when at least the part of the input area of the display overlaps the input window of the page displayed on the display.
16. The method according to claim 9, further comprising:
removing the displayed virtual text input panel in response to a movement of the mobile device as applied by a predetermined gesture of the user.
17. The method according to claim 16, wherein the predetermined gesture includes tilting the mobile device, shaking the mobile device at least once, pressing a virtual button on the display, pressing a physical button on the mobile device, or touching an area of the display in a certain manner.
18. The method according to claim 9, wherein the virtual text input panel comprises a virtual keyboard or a note recognition panel.
19. The method according to claim 9, wherein the scrolling step includes at least one of the following:
performing a zooming in/out operation on a portion of the page currently displayed on the display, according to a movement of the mobile device; and
flipping through other pages according to a movement of the mobile device.
20. The method according to claim 19, further comprising:
displaying, on a portion of the display, a guide window for indicating the actual area currently displayed on the display with respect to the entire page or a plurality of pages.
21. A mobile device comprising:
a display configured to display a guide window in addition to a page currently displayed on the display while a scrolling is performed, wherein the guide window includes a miniature of the page currently displayed on the display and a location indicator indicating the location of the part currently being viewed on the display;
a detector configured to detect a movement of the mobile device as applied by a user's gesture; and
a controller, by controlling the display, configured to:
perform the scroll of the page displayed on the display according to the detected movement of the mobile device only while a button for activating the scroll is pressed, and
if at least part of an input area of the display overlaps an input window of the page, display a virtual text input panel for inputting text into the overlapped input window
wherein the input window is an area for inputting and displaying text,
wherein if the input area and the input window are overlapped for a predetermined time, the virtual text input panel continues to be displayed despite movement of the mobile terminal, and
wherein if the input area and the input window are not overlapped for the predetermined time due to the movement of the mobile terminal, the virtual text input panel disappears.
22. A mobile device comprising:
a display;
a detector configured to detect a movement of the mobile device as applied by a user's gesture; and
a controller, by controlling the display, configured to:
scroll a page displayed on the display according to the detected movement of the mobile device; and
if at least part of an input area of the display overlaps an input window of the page, display the virtual text input panel for inputting text into the overlapped input window
wherein the input window is an area for inputting and displaying text,
wherein if the input area and the input window are overlapped for a predetermined time, the virtual text input panel continues to be displayed despite movement of the mobile terminal, and
wherein if the input area and the input window are not overlapped for the predetermined time due to the movement of the mobile terminal, the virtual text input panel disappears.
US12/196,080 2008-04-17 2008-08-21 Method and device for controlling user interface based on user's gesture Active 2031-03-16 US9582049B2 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
KR20080035556 2008-04-17
KR10-2008-0035556 2008-04-17
KR1020080068312A KR100912310B1 (en) 2008-04-17 2008-07-14 User interface controlling method by detecting user's gestures
KR10-2008-0068312 2008-07-14

Publications (2)

Publication Number Publication Date
US20090265627A1 US20090265627A1 (en) 2009-10-22
US9582049B2 true US9582049B2 (en) 2017-02-28

Family

ID=41199270

Family Applications (1)

Application Number Title Priority Date Filing Date
US12/196,080 Active 2031-03-16 US9582049B2 (en) 2008-04-17 2008-08-21 Method and device for controlling user interface based on user's gesture

Country Status (2)

Country Link
US (1) US9582049B2 (en)
WO (1) WO2009128593A1 (en)

Families Citing this family (73)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8617048B2 (en) 2001-03-09 2013-12-31 Boston Scientific Scimed, Inc. System for implanting an implant and method thereof
US9152309B1 (en) * 2008-03-28 2015-10-06 Google Inc. Touch screen locking and unlocking
US8723812B2 (en) * 2008-09-08 2014-05-13 Lg Electronics Inc. Mobile terminal and method of controlling the mobile terminal
GB2464094A (en) * 2008-09-30 2010-04-07 Rapid Mobile Media Ltd Method and apparatus for displaying content at a mobile device
US20100097322A1 (en) * 2008-10-16 2010-04-22 Motorola, Inc. Apparatus and method for switching touch screen operation
US8788977B2 (en) * 2008-11-20 2014-07-22 Amazon Technologies, Inc. Movement recognition as input mechanism
US8645871B2 (en) * 2008-11-21 2014-02-04 Microsoft Corporation Tiltable user interface
US10013135B2 (en) * 2009-03-20 2018-07-03 Microsoft Technology Licensing, Llc Adjustable user interfaces with movable separators
JP5489572B2 (en) * 2009-07-29 2014-05-14 キヤノン株式会社 Acceleration detecting device and a control method thereof, a program
KR20110037657A (en) 2009-10-07 2011-04-13 삼성전자주식회사 Method for providing gui by using motion and display apparatus applying the same
US8621380B2 (en) 2010-01-06 2013-12-31 Apple Inc. Apparatus and method for conditionally enabling or disabling soft buttons
US20110187651A1 (en) * 2010-02-03 2011-08-04 Honeywell International Inc. Touch screen having adaptive input parameter
US8878773B1 (en) 2010-05-24 2014-11-04 Amazon Technologies, Inc. Determining relative motion as input
WO2011149515A1 (en) * 2010-05-24 2011-12-01 Will John Temple Multidirectional button, key, and keyboard
US9007304B2 (en) * 2010-09-02 2015-04-14 Qualcomm Incorporated Methods and apparatuses for gesture-based user input detection in a mobile device
KR101743632B1 (en) 2010-10-01 2017-06-07 삼성전자주식회사 Apparatus and method for turning e-book pages in portable terminal
US9678572B2 (en) * 2010-10-01 2017-06-13 Samsung Electronics Co., Ltd. Apparatus and method for turning e-book pages in portable terminal
EP2437151A3 (en) 2010-10-01 2017-03-15 Samsung Electronics Co., Ltd. Apparatus and method for turning e-book pages in portable terminal
EP2437153A3 (en) * 2010-10-01 2016-10-05 Samsung Electronics Co., Ltd. Apparatus and method for turning e-book pages in portable terminal
US8253684B1 (en) * 2010-11-02 2012-08-28 Google Inc. Position and orientation determination for a mobile computing device
US8547354B2 (en) 2010-11-05 2013-10-01 Apple Inc. Device, method, and graphical user interface for manipulating soft keyboards
US8587547B2 (en) 2010-11-05 2013-11-19 Apple Inc. Device, method, and graphical user interface for manipulating soft keyboards
US9092132B2 (en) 2011-01-24 2015-07-28 Apple Inc. Device, method, and graphical user interface with a dynamic gesture disambiguation threshold
US9436381B2 (en) 2011-01-24 2016-09-06 Apple Inc. Device, method, and graphical user interface for navigating and annotating an electronic document
US10152951B2 (en) * 2011-02-28 2018-12-11 Varian Medical Systems International Ag Method and system for interactive control of window/level parameters of multi-image displays
US9619035B2 (en) 2011-03-04 2017-04-11 Microsoft Technology Licensing, Llc Gesture detection and recognition
US9035940B2 (en) * 2011-03-08 2015-05-19 Nokia Corporation Apparatus and associated methods
US9094813B2 (en) * 2011-04-02 2015-07-28 Open Invention Network, Llc System and method for redirecting content based on gestures
US9041733B2 (en) 2011-05-04 2015-05-26 Blackberry Limited Methods for adjusting a presentation of graphical data displayed on a graphical user interface
US9123272B1 (en) 2011-05-13 2015-09-01 Amazon Technologies, Inc. Realistic image lighting and shading
KR101878141B1 (en) * 2011-05-30 2018-07-13 엘지전자 주식회사 Mobile terminal and method for controlling thereof
TW201301079A (en) * 2011-06-30 2013-01-01 Hon Hai Prec Ind Co Ltd Handheld device and multimedia terminal device controlling method
US9041734B2 (en) 2011-07-12 2015-05-26 Amazon Technologies, Inc. Simulating three-dimensional features
US10088924B1 (en) 2011-08-04 2018-10-02 Amazon Technologies, Inc. Overcoming motion effects in gesture recognition
US8891868B1 (en) 2011-08-04 2014-11-18 Amazon Technologies, Inc. Recognizing gestures captured by video
FR2979722A1 (en) * 2011-09-01 2013-03-08 Myriad France Portable electronic device i.e. mobile phone, has activation unit activating processing rule application unit upon detection of movement of phone by motion sensor, where activation unit is inhibited in absence of selection of graphic object
US8947351B1 (en) 2011-09-27 2015-02-03 Amazon Technologies, Inc. Point of view determinations for finger tracking
US8902180B2 (en) * 2011-12-16 2014-12-02 Nokia Corporation Methods, apparatuses, and computer program products for enabling use of remote devices with pre-defined gestures
US9223415B1 (en) 2012-01-17 2015-12-29 Amazon Technologies, Inc. Managing resource usage for task performance
US8884928B1 (en) 2012-01-26 2014-11-11 Amazon Technologies, Inc. Correcting for parallax in electronic displays
US9063574B1 (en) 2012-03-14 2015-06-23 Amazon Technologies, Inc. Motion detection systems for electronic devices
US8504842B1 (en) 2012-03-23 2013-08-06 Google Inc. Alternative unlocking patterns
US9285895B1 (en) 2012-03-28 2016-03-15 Amazon Technologies, Inc. Integrated near field sensor for display devices
US9075872B2 (en) 2012-04-25 2015-07-07 International Business Machines Corporation Content-based navigation for electronic devices
KR101680059B1 (en) * 2012-05-02 2016-11-28 삼성전자주식회사 Method for moving screen and an electronic device thereof
US8830171B2 (en) 2012-05-22 2014-09-09 Eminent Electronic Technology Corporation Apparatus for non-contact 3D hand gesture recognition with code-based light sensing
US9798457B2 (en) 2012-06-01 2017-10-24 Microsoft Technology Licensing, Llc Synchronization of media interactions using context
US9381427B2 (en) 2012-06-01 2016-07-05 Microsoft Technology Licensing, Llc Generic companion-messaging between media platforms
KR101392936B1 (en) * 2012-06-29 2014-05-09 한국과학기술연구원 Customizable interface system and its implementation
CN102779009B (en) * 2012-06-29 2015-04-08 华为终端有限公司 Method and terminal for displaying application program interface
US20140006994A1 (en) * 2012-06-29 2014-01-02 Apple Inc. Device, Method, and Graphical User Interface for Displaying a Virtual Keyboard
US9354721B2 (en) * 2012-09-28 2016-05-31 Apple Inc. Movement based image transformation
US9423886B1 (en) 2012-10-02 2016-08-23 Amazon Technologies, Inc. Sensor connectivity approaches
KR20140065955A (en) * 2012-11-22 2014-05-30 엘지전자 주식회사 Mobile terminal and control method thereof
TWI540492B (en) * 2012-12-20 2016-07-01 Mstar Semiconductor Inc Electronic device and electronic device controlling method
CN103116440A (en) * 2013-01-23 2013-05-22 深圳市金立通信设备有限公司 Method and terminal for icon to move on terminal
US9035874B1 (en) 2013-03-08 2015-05-19 Amazon Technologies, Inc. Providing user input to a computing device with an eye closure
KR20150011885A (en) * 2013-07-23 2015-02-03 삼성전자주식회사 User Interface Providing Method for Device and Device Thereof
KR20150016783A (en) * 2013-08-05 2015-02-13 삼성전자주식회사 Method for user input by using mobile device and mobile device
US9269012B2 (en) 2013-08-22 2016-02-23 Amazon Technologies, Inc. Multi-tracker object tracking
US20150062178A1 (en) * 2013-09-05 2015-03-05 Facebook, Inc. Tilting to scroll
US10055013B2 (en) 2013-09-17 2018-08-21 Amazon Technologies, Inc. Dynamic object tracking for user interfaces
US9367203B1 (en) 2013-10-04 2016-06-14 Amazon Technologies, Inc. User interface techniques for simulating three-dimensional depth
US9367086B2 (en) * 2013-12-10 2016-06-14 Atmel Corporation Smart watch with adaptive touch screen
US9813517B2 (en) 2014-05-29 2017-11-07 Vmware, Inc. Feedback layer for native content display in virtual desktop infrastructure
WO2015189710A2 (en) * 2014-05-30 2015-12-17 Infinite Potential Technologies, Lp Apparatus and method for disambiguating information input to a portable electronic device
EP3002661A1 (en) * 2014-09-30 2016-04-06 Advanced Digital Broadcast S.A. System and method for controlling a virtual input interface
EP3002669A1 (en) * 2014-09-30 2016-04-06 Advanced Digital Broadcast S.A. System and method for controlling a virtual input interface
CN104657073A (en) * 2015-01-22 2015-05-27 上海华豚科技有限公司 Half-screen operating method of mobile phone interface
US10289820B2 (en) * 2015-02-24 2019-05-14 Motorola Mobility Llc Multiuse 3D IR for electronic device
CN104699439B (en) * 2015-03-30 2019-03-12 三星电子(中国)研发中心 Picture display process, device and mobile terminal for mobile terminal
US10310726B2 (en) * 2015-05-14 2019-06-04 Oath Inc. Content navigation based upon motion
CN105807952A (en) * 2016-03-07 2016-07-27 联想(北京)有限公司 Information processing method and electronic equipment

Citations (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6421064B1 (en) * 1997-04-30 2002-07-16 Jerome H. Lemelson System and methods for controlling automatic scrolling of information on a display screen
US6466198B1 (en) 1999-11-05 2002-10-15 Innoventions, Inc. View navigation and magnification of a hand-held device with a display
US6570567B1 (en) * 2000-05-31 2003-05-27 Alan Eaton System and method for using a graphical interface for the presentation of genealogical information
US6573883B1 (en) 1998-06-24 2003-06-03 Hewlett Packard Development Company, L.P. Method and apparatus for controlling a computing device with gestures
US20030201972A1 (en) * 2002-04-25 2003-10-30 Sony Corporation Terminal apparatus, and character input method for such terminal apparatus
US20040201576A1 (en) * 2003-04-09 2004-10-14 Microsoft Corporation Software multi-tap input system and method
US20050012723A1 (en) * 2003-07-14 2005-01-20 Move Mobile Systems, Inc. System and method for a portable multimedia client
WO2005093550A2 (en) 2004-03-01 2005-10-06 Apple Computer, Inc. Methods and apparatuses for operating a portable device based on an accelerometer
US20060033724A1 (en) 2004-07-30 2006-02-16 Apple Computer, Inc. Virtual input device placement on a touch screen user interface
US20060075357A1 (en) * 2004-10-05 2006-04-06 International Business Machines Corporation Method and system for maintaining scroll position in a web user interface
US7036086B2 (en) 2001-01-04 2006-04-25 Intel Corporation Displaying software keyboard images
US20060164382A1 (en) 2005-01-25 2006-07-27 Technology Licensing Company, Inc. Image manipulation in response to a movement of a display
KR20070004466A (en) 2006-11-03 2007-01-09 희 한 The vilture screen and mouse curcer moving method by movement of the display of hand helded information equipment
US20070038952A1 (en) * 2005-06-29 2007-02-15 Erika Reponen Mobile communication terminal
KR20070065590A (en) 2005-12-20 2007-06-25 삼성코닝 주식회사 Fabricating method of setter and setter for planar device
US20080005703A1 (en) * 2006-06-28 2008-01-03 Nokia Corporation Apparatus, Methods and computer program products providing finger-based and hand-based gesture commands for portable electronic device applications
US20080082934A1 (en) * 2006-09-06 2008-04-03 Kenneth Kocienda Soft Keyboard Display for a Portable Multifunction Device
US20080094369A1 (en) * 2006-09-06 2008-04-24 Ganatra Nitin K Email Client for a Portable Multifunction Device
US20080115046A1 (en) * 2006-11-15 2008-05-15 Fujitsu Limited Program, copy and paste processing method, apparatus, and storage medium
US20080174547A1 (en) * 2004-08-09 2008-07-24 Dimitri Kanevsky Controlling devices' behaviors via changes in their relative locations and positions
US20080174564A1 (en) * 2007-01-20 2008-07-24 Lg Electronics Inc. Mobile communication device equipped with touch screen and method of controlling operation thereof
US20080228429A1 (en) * 2006-12-08 2008-09-18 Andrew Shane Huang Systems and methods for location, motion, and contact detection and tracking in a networked audiovisual device
US20080244460A1 (en) * 2007-03-29 2008-10-02 Apple Inc. Cursor for Presenting Information Regarding Target
US7467356B2 (en) * 2003-07-25 2008-12-16 Three-B International Limited Graphical user interface for 3d virtual display browser using virtual display windows

Patent Citations (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6421064B1 (en) * 1997-04-30 2002-07-16 Jerome H. Lemelson System and methods for controlling automatic scrolling of information on a display screen
US6573883B1 (en) 1998-06-24 2003-06-03 Hewlett Packard Development Company, L.P. Method and apparatus for controlling a computing device with gestures
US6466198B1 (en) 1999-11-05 2002-10-15 Innoventions, Inc. View navigation and magnification of a hand-held device with a display
US20020190947A1 (en) * 2000-04-05 2002-12-19 Feinstein David Y. View navigation and magnification of a hand-held device with a display
US6570567B1 (en) * 2000-05-31 2003-05-27 Alan Eaton System and method for using a graphical interface for the presentation of genealogical information
US7036086B2 (en) 2001-01-04 2006-04-25 Intel Corporation Displaying software keyboard images
US20030201972A1 (en) * 2002-04-25 2003-10-30 Sony Corporation Terminal apparatus, and character input method for such terminal apparatus
US20040201576A1 (en) * 2003-04-09 2004-10-14 Microsoft Corporation Software multi-tap input system and method
US20050012723A1 (en) * 2003-07-14 2005-01-20 Move Mobile Systems, Inc. System and method for a portable multimedia client
US7467356B2 (en) * 2003-07-25 2008-12-16 Three-B International Limited Graphical user interface for 3d virtual display browser using virtual display windows
WO2005093550A2 (en) 2004-03-01 2005-10-06 Apple Computer, Inc. Methods and apparatuses for operating a portable device based on an accelerometer
US20060033724A1 (en) 2004-07-30 2006-02-16 Apple Computer, Inc. Virtual input device placement on a touch screen user interface
US20080174547A1 (en) * 2004-08-09 2008-07-24 Dimitri Kanevsky Controlling devices' behaviors via changes in their relative locations and positions
US20060075357A1 (en) * 2004-10-05 2006-04-06 International Business Machines Corporation Method and system for maintaining scroll position in a web user interface
US20060164382A1 (en) 2005-01-25 2006-07-27 Technology Licensing Company, Inc. Image manipulation in response to a movement of a display
US20070038952A1 (en) * 2005-06-29 2007-02-15 Erika Reponen Mobile communication terminal
KR20070065590A (en) 2005-12-20 2007-06-25 삼성코닝 주식회사 Fabricating method of setter and setter for planar device
US20080005703A1 (en) * 2006-06-28 2008-01-03 Nokia Corporation Apparatus, Methods and computer program products providing finger-based and hand-based gesture commands for portable electronic device applications
US20080094369A1 (en) * 2006-09-06 2008-04-24 Ganatra Nitin K Email Client for a Portable Multifunction Device
US20080082934A1 (en) * 2006-09-06 2008-04-03 Kenneth Kocienda Soft Keyboard Display for a Portable Multifunction Device
KR20070004466A (en) 2006-11-03 2007-01-09 희 한 The vilture screen and mouse curcer moving method by movement of the display of hand helded information equipment
US20080115046A1 (en) * 2006-11-15 2008-05-15 Fujitsu Limited Program, copy and paste processing method, apparatus, and storage medium
US20080228429A1 (en) * 2006-12-08 2008-09-18 Andrew Shane Huang Systems and methods for location, motion, and contact detection and tracking in a networked audiovisual device
US20080174564A1 (en) * 2007-01-20 2008-07-24 Lg Electronics Inc. Mobile communication device equipped with touch screen and method of controlling operation thereof
US20080244460A1 (en) * 2007-03-29 2008-10-02 Apple Inc. Cursor for Presenting Information Regarding Target

Also Published As

Publication number Publication date
US20090265627A1 (en) 2009-10-22
WO2009128593A1 (en) 2009-10-22

Similar Documents

Publication Publication Date Title
US9176620B2 (en) Mobile terminal and method for displaying information list thereof
CN101594429B (en) Mobile terminal and transparent display control method thereof
US8284170B2 (en) Touch screen device, method, and graphical user interface for moving on-screen objects without using a cursor
US9652116B2 (en) Mobile terminal and method of controlling the same
US8464182B2 (en) Device, method, and graphical user interface for providing maps, directions, and location-based information
US10140301B2 (en) Device, method, and graphical user interface for selecting and using sets of media player controls
US8624933B2 (en) Device, method, and graphical user interface for scrolling a multi-section document
CN103995724B (en) A portable multifunction device with the interface reconfiguration mode
US9001047B2 (en) Modal change based on orientation of a portable multifunction device
US8972879B2 (en) Device, method, and graphical user interface for reordering the front-to-back positions of objects
US7978182B2 (en) Screen rotation gestures on a portable multifunction device
US8274536B2 (en) Smart keyboard management for a multifunction device with a touch screen display
US8438504B2 (en) Device, method, and graphical user interface for navigating through multiple viewing areas
US9195389B2 (en) Menu executing method and apparatus in portable terminal
US10254927B2 (en) Device, method, and graphical user interface for manipulating workspace views
CN103176701B (en) Apparatus and method for navigation of web pages
US8477139B2 (en) Touch screen device, method, and graphical user interface for manipulating three-dimensional virtual objects
US8723810B2 (en) Terminal for outputting a vibration and method of controlling the same
US8698845B2 (en) Device, method, and graphical user interface with interactive popup views
US7966578B2 (en) Portable multifunction device, method, and graphical user interface for translating displayed content
KR101526995B1 (en) Mobile terminal and method for controlling display thereof
US8548528B2 (en) Mobile terminal and control method thereof
US9112988B2 (en) Terminal and method of controlling the same
US8612894B2 (en) Method for providing a user interface using three-dimensional gestures and an apparatus using the same
CN104219376B (en) A mobile terminal and controlling method

Legal Events

Date Code Title Description
AS Assignment

Owner name: LG ELECTRONICS INC., KOREA, REPUBLIC OF

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KIM, JOO MIN;KWON, YOUK;REEL/FRAME:021812/0260

Effective date: 20080821

STCF Information on status: patent grant

Free format text: PATENTED CASE