WO2010001326A1 - User interface display device - Google Patents

User interface display device Download PDF

Info

Publication number
WO2010001326A1
WO2010001326A1 PCT/IB2009/052802 IB2009052802W WO2010001326A1 WO 2010001326 A1 WO2010001326 A1 WO 2010001326A1 IB 2009052802 W IB2009052802 W IB 2009052802W WO 2010001326 A1 WO2010001326 A1 WO 2010001326A1
Authority
WO
WIPO (PCT)
Prior art keywords
actuation member
interactive component
component
proximate
interactive
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/IB2009/052802
Other languages
French (fr)
Inventor
Patrick Mccarthy
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nokia Inc
Original Assignee
Nokia Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nokia Inc filed Critical Nokia Inc
Publication of WO2010001326A1 publication Critical patent/WO2010001326A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04886Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Definitions

  • the present invention relates to a computing device and a method of making available a user interface element comprising a displayable component and an interactive component.
  • Touch screens are becoming a more common user interface for mobile computing devices.
  • they suffer from the disadvantage that it is necessary to dedicate at least a portion of the screen to user interface components such as buttons, sliders, icons, and so on.
  • These components must be large enough on the screen to permit effective user interaction - for example a button must be large enough not only to be visible and recognisable by the user, but also to be physically accessible to him in order that he can actuate it.
  • a disadvantage with the use of on-screen interface components is that screen real estate must be set aside for such components and will not therefore be available for displaying other information.
  • the disadvantage described above is at its greatest when the total screen area is small (for example in a mobile device), or the interface components must be large (perhaps because the interface is adapted for people who have below-average vision or motor difficulties that would impair their physical interaction with small components).
  • the efficient use of screen area is therefore of particular importance in mobile devices such as mobile phones, personal digital assistants, and smart phones - which are normally small in size and consequently have small screens, and are also mass-market devices that need to be usable by all people regardless of visual acuity or manual dexterity.
  • the desire to minimise the impact of interface components on the area available for displaying other data has led to some touch screens being adapted for stylus use.
  • Some devices have two screens where one of the screens is a touch screen and is permanently dedicated to user input.
  • the use of a second screen increases the total available screen area, but its use will consume device resources such as power, memory and processor time, and its presence will increase the overall size of the device.
  • buttons have a mixture of hardware buttons and a touch screen to minimise the amount of screen area dedicated to user interaction.
  • This use of hardware buttons increases the overall size and weight of the device, and does not provide the inherent flexibility in terms of interface configurability that is possible using an on-screen user interface.
  • the Microsoft Windows Media Player for the Windows XP operating system displays a bar of user interface components (e.g. a play/pause button, and a volume slider) whilst video is being played, but hides these components (and the mouse pointer) when no user input has been received for a certain period of time, and enlarging the video display to use the area of the screen that was previously occupied by the bar.
  • a bar of user interface components e.g. a play/pause button, and a volume slider
  • the video display is shrunk and the component bar and mouse pointer are redisplayed.
  • apparatus comprising: at least one processor; and at least one memory including computer program code, the at least one memory and the computer program code configured to, working with the at least one processor, cause at least the following to be performed: detecting the presence of an actuation member proximate an interactive component of a user interface element that can be contactually actuated by an actuation member acting on said interactive component, wherein the actuation member has not necessarily actuated the interactive component; and displaying a displayable component of the user interface element in response to the detection of an actuation member proximate the interactive component.
  • a method comprising: detecting the presence of an actuation member proximate an interactive component of a user interface element that can be contactually actuated by an actuation member acting on said interactive component, wherein the actuation member has not necessarily actuated the interactive component; and displaying a displayable component of the user interface element in response to the detection of an actuation member proximate the interactive component.
  • a computer readable medium having instructions stored thereon for: detecting the presence of an actuation member proximate an interactive component of a user interface element that can be contactually actuated by an actuation member acting on said interactive component, wherein the actuation member as not necessarily actuated the interactive component; and displaying a displayable component of the user interface element in response to the detection of an actuation member proximate the interactive component.
  • apparatus comprising means for: detecting the presence of an actuation member proximate an interactive component of a user interface element that can be contactually actuated by an actuation member acting on said interactive component, wherein the actuation member has not necessarily actuated the interactive component; and displaying a displayable component of the user interface element in response to the detection of an actuation member proximate the interactive component.
  • apparatus comprising: a display; a user interface input element comprising a displayable component and an interactive component, wherein the input element can be contactually actuated by an actuation member; a presence detector configured to detect the presence of an actuation member that is proximate the interactive component but has not necessarily actuated the interactive component; and a processor configured to cause the display to display the displayable component in response to the detection of an actuation member proximate the interactive component.
  • Suitable user interface input elements include buttons, slide bars, tabs, and radio groups implemented on a touch screen, and soft buttons.
  • a soft button is a user interface element comprising a hardware button whose function is determined by software. Conventionally, a soft button is often implemented as a key placed close to the edge of a display, with a label displayed on the screen in close proximity to the key. The label indicates what function will result when the key is pressed, and both the function and label will change as the user navigates through menu systems and applications. Such soft buttons are common in mobile phones whose small size restricts the number of dedicated hardware buttons that can be provided.
  • the term 'soft button' normally corresponds to a button - i.e. a key in the user interface - the hardware component may instead be implemented as a scroll wheel, a joystick, a track ball, or any other suitable hardware input component.
  • the interactive component comprises the hardware input component through which a user makes the input (e.g. a key) and the displayable component comprises a displayable indication as to the result to be expected by actuating the interactive component.
  • the user interface element may be entirely implemented using a touch screen.
  • Touch screens operate by displaying a graphical representation of an input element (e.g. a button, a slide bar, etc.) and associating an area of the screen with that representation. Normally the associated area of the screen is the area in which the graphical representation is displayed.
  • the user would see the graphical representation of the button displayed on the screen, and would actuate the button by touching the area of the screen associated with it.
  • the displayable component of the user interface element is the graphical representation and the interactive component is the area of touchable screen associated with it.
  • the interactive component is actuated by an actuating member.
  • the type of member that actuates the component will vary according to the component's nature, but common actuating members include fingers and dedicated styli.
  • the interactive component is contactually actuated, meaning that its actuation requires it to be contacted by the actuation member. In many cases mere contact alone may be sufficient to actuate the interactive component (for example, touching a finger against a touch screen), but in other cases a further motion or force may be required (for example pushing against a button, rotating a wheel, moving along a slider bar's path).
  • Contactual actuation encompasses both actuation where contact alone is sufficient and actuation where some other effort is additionally required.
  • An actuating member is proximal the interactive component when it is close to the interactive component.
  • the threshold proximity required for a member to be detected as 'proximal' the component will vary according to the technology used and its specific application. Since actuation of the interactive component requires contact, the actuating member will always be proximal during actuation. However, the actuating member will also be detected as proximal the interactive component when it is touching or close to the component but not actuating it.
  • the display should remain uncluttered by the displayable components of the user interface in order to provide the maximum display area for other purposes, it is important that the displayable components are displayed when the user is using the user interface elements, or is likely to use them. It is therefore not enough just to display the displayable components in response to the actuation of an interactive component.
  • the first input simply triggers the display of the displayable components and is otherwise ignored, then the user will need to repeat his input operation before it is accepted by the operating system.
  • Displaying the displayable component of the user interface element in response to the detection of a proximate actuation member avoids the undesirable operation described above. Since the displayable component is visible to the novice user before he actuates the interactive component, he is less likely to erroneously actuate the wrong component. The expert user who actuates the interactive component without looking at the displayable component is similarly unburdened using the new solution - since the displayable component is displayed in response to the detection of proximity, rather than actuation, there is no longer any reason for the operating system to ignore the first-received input.
  • the processor is preferably configured to cause the display not to display the displayable in response to the detection of no actuation member proximate the interactive component.
  • the processor may be further configured to cause the display to cease to display the displayable a predetermined period of time after an actuation member ceases to be detected proximate the interactive component. This takes advantage of the fact that user inputs often occur in grouped sequences.
  • Contactually actuating the input element in some embodiments comprises touching the actuation member against the interactive component. This touching is sufficient in itself in some embodiments such as those using touch switch buttons or the majority of touch screens.
  • Contactually actuating the input element may comprise pressing the actuation member against the interactive component with greater than a threshold force. This would be the case for a hardware button that need to be pressed, or a pressure-sensitive touch screen.
  • the threshold force is the minimum force required to actuate the interactive element (e.g. to depress the button, or to register a 'touch' on the pressure-sensitive touch screen).
  • the presence detector may be configured to detect the presence of an actuation member proximate the interactive component when the actuation member is actually touching the interactive component. This would be the case in embodiments when the actuation of the interactive component requires something further than a touch, for example the application of greater force, the rotation of a wheel, or the movement of the member along a specific path (e.g. along a slider).
  • the presence detector may be configured to detect the presence of an actuation member proximate the interactive component when the actuation member is detected as located within a threshold range of the interactive component.
  • the threshold range relates to a threshold distance from the interactive component
  • other thresholds may be used when they are at least partially representative of the distance from the component. For example, a detected signal greater than a threshold magnitude will indicate a finger proximate to a capacitive touch screen, even though the magnitude will also be affected by factors other than distance.
  • the presence detector may be configured to detect the presence of an actuation member proximate the interactive component only when the actuation member is detected as actually touching the interactive component. This is possible where something further than a touch is required to actuate the component - for example the application of a force or a particular motion.
  • the display may be a touch screen, and the interactive component may then be a location on the touch screen.
  • the location may be a single point on the screen, or an area - including the whole screen.
  • the location on the touch screen may comprise the location of the displayable component.
  • the input element may be a soft control, where the interactive component is a hardware control and the displayable component describes the function presently associated with the actuation of the hardware control.
  • 'Soft controls' are frequently referred to as 'soft buttons'. In practice a soft control/button need not be implemented as a button - it could be a scroll wheel, joystick, or any other suitable input.
  • the presence detector may include a capacitance detector.
  • the presence detector may include a beam-break detector.
  • Fig. 1 is a schematic representation of a mobile phone
  • Fig. 2a is a schematic representation of a phone of the type shown in Fig
  • Fig. 2b is a schematic representation of a first embodiment of the phone of Fig. 2a;
  • Fig. 2c is a schematic representation of a second embodiment of the phone of Fig. 2a;
  • Fig. 3a is a schematic representation of a phone of the type shown in Fig
  • Fig. 3b is a schematic representation of a first embodiment of the phone of Fig. 3a;
  • Fig. 3c is a schematic representation of a second embodiment of the phone of Fig. 3a;
  • Fig. 3d is a schematic representation of a first embodiment of the phone of Fig. 2a; and Fig. 4 is a flowchart of a method.
  • Fig. 1 illustrates a mobile phone 100 comprising a Central Processing Unit (CPU) 1 10 which is connected to connected to a speaker 120, microphone 130, a keypad 140, a camera 150, storage memory 160, Random Access Memory (RAM) 170, and a mobile phone transceiver 180.
  • the CPU 1 10 is also connected to a display 190.
  • the display 190 is a touch screen operated using an actuating member 195; however it is also provided with soft buttons 140'.
  • the phone 100 runs under the control of the operating system.
  • the operating system includes a user interface that manages interactions between the phone and a user.
  • the user interface receives user input from the keypad 140, the touch screen display 190 and the soft buttons 140'.
  • the user interface also controls the graphical output to the display 190. Audio input via the microphone 130, visual input via the camera 150, and audio output via the speaker 120 may also be controlled by the user interface.
  • the interface with the user may comprise a non-touch screen and buttons arranged to form an alphanumeric keypad, soft buttons, or in a directional pad.
  • buttons arranged to form an alphanumeric keypad, soft buttons, or in a directional pad.
  • a joystick, touch pad, mouse or roller-ball may be replace the buttons or be present in addition to them.
  • a touch screen operates by displaying user interface components on the screen and detecting touch operations relating to the displayable components. For example, a graphical button or slider bar may be displayed at a location on the screen and a press or slide of a finger or stylus in the location on the screen where the button or slider is displayed is interpreted by the operating system as an activation of that component.
  • a finger or dedicated stylus is normally used to perform the touch operation, the screen may be operable using other types of actuating member.
  • the actuation member 195 is a dedicated stylus is provided for contactually actuating the touch screen display 190.
  • Fig. 2a shows an embodiment of the phone 100 where the display 190 is a touch screen.
  • a media player application is running on the phone.
  • the media player application is playing a video, with the playback area 200 occupying the entirety of the display 190.
  • Fig. 2b shows the same phone as Fig. 2a, running the same media player application and playing the same video. However, Fig. 2b additionally shows a number of on-screen buttons 210 that can be operated by the user as part of the user interface. Each of the buttons 210 is displayed in a different location within the display 190 and associated with touchable area of the display 190 at that location. To operate one if the buttons 210, the user contactually actuates its associated area of the display 190 by touching it with his finger.
  • buttons 210 and their associated touchable locations in this case occupy the same locations on the touch screen 190, it is possible that a user interface component may be associated with a separate, remote location on the touch screen 190.
  • buttons 210 are superimposed on the video playback 200 and thus partly obscure it.
  • the graphical representations of the buttons 210 are removed from the user interface under certain pre-defined conditions. Such conditions may include the start of video playback, and when a predefined period has passed since the last user input (e.g. touch upon the touch screen 190 or actuation of the buttons 210).
  • the representations of the buttons 210 are concealed when the user is not expected to use them (e.g. after he has started playback, or after he has not made an input for a while) in order that the visible portion of the video playback 200 be maximised during these times.
  • the touch screen 190 is able to detect the user's finger 195 when it is proximate the display 190 and not just when the finger 195 is in direct contact with it (i.e. to contactually actuate one of the buttons 210).
  • the buttons are actuated contactually (by touch), but other functionality may be initiated based on whether a finger 195 is detected proximate the display 190.
  • proximate means within close to the display, or touching it.
  • buttons 210 Linking the display of the buttons 210 to this proximity detection permits the buttons 210 to be displayed when a finger 195 (or other actuation member, for example a stylus) is close to the screen 190 and a user input is likely to be forthcoming, but otherwise to be concealed on the basis that no input is possible.
  • a finger 195 or other actuation member, for example a stylus
  • the closeness of the finger to the display is preferably defined and detected in terms of a distance (for example 1 cm), in practice this may not be possible and the proximity will relate to the magnitude of a detection signal or value.
  • a resistive touch screen is used.
  • a resistive touch screen can detect actual touches but cannot detect a finger close to but not touching the screen.
  • infrared signals are projected in front of the screen between one or more infrared emitters and detectors. When the grid is broken, the signal from the detector indicates that an actuation member is proximate the screen.
  • the grid can be positioned at a specific distance from the screen, the specific distance defining the maximum distance from the screen that is considered 'proximate' (i.e. a member will be detected as proximate once it has come within this distance of the screen).
  • a capacitive touch screen is used. Capacitive touch screens can detect fingers both actually touching the screen and merely close to it, the two being distinguished by the strength of the detection signal. A finger close to the screen results in a stronger detection signal that one further away, and a finger actually touching the screen results in a signal that is stronger still.
  • the strength of the detected signal will also depend on the nature of the finger, for example its size and whether or not it is wet. The signal strength therefore cannot be relied upon to define an accurate distance bound within which a finger is considered proximate. In this case, it is the strength of the detected signal that defines a finger's proximity, signals stronger than a certain threshold being deemed proximate, regardless of the real distance of the finger from the screen.
  • touch screens require only a touch to perform a contactual actuation of a user interface component. However, some touch screens require a minimum amount of pressure to be applied. In the later case, a member that is touching the screen will not be making a 'contactual actuation', although it will be proximate.
  • buttons 210 are displayed on the screen 190 when an actuating member 195 is detected proximate to it. Once the member 195 is removed from the vicinity of the screen and is no longer proximate, the buttons cease to be displayed. Preferably, the buttons continue to be displayed for a predefined period of time after the member 195 is removed. For example, the buttons 210 may continue to be displayed for a period of about 5 seconds after the member 195 has been removed. The reason for this is that one user input is often closely followed by another.
  • buttons may not necessarily cease to be displayed.
  • a video is being played. If the user input serves to pause the video there is likely to be little detriment to the user if the buttons 210 continue to be displayed until playback is resumed and the buttons 210 may therefore remain visible until playback has resumed.
  • Fig. 2c shows an alternative display of the buttons 210 shown in Fig 2b. So that the buttons 210 do not obscure the video playback, the playback area 200 is decreased in size to create a clear area of the display 190 in which the buttons 210 are be displayed. When the buttons 210 are not displayed, the playback area 200 increases in size to fill the entire display 190.
  • an actuating member that is merely proximate the interface
  • a member that is (also) making a contactual actuation of the interface For example, it is above descried that an infrared grid could be projected over the display 190; however a similar grid can be projected over the keypad 140, allowing members proximate the keys to be detected.
  • the presence of a finger proximate a key may be detected using capacitive sensing.
  • a finger merely touching a pressure sensitive key may be detected using a resistive touch switch integrated into the top surface of the key.
  • Other techniques for detecting proximate actuation members for keys may be used and the most suitable choice will vary according to its application.
  • Fig. 3a shows a phone 100 that is, similar to the phone of Fig. 2a, playing video using the full extent of its display 190.
  • To the side of the display are located the hardware keys 140' of a number of soft buttons.
  • the phone 100 can determine when an actuation member 195 is proximate the keys 140'.
  • their labels are hidden until a member 195 is detected proximate the keys 140'.
  • Fig. 3b shows a first embodiment of the phone 100 of Fig. 3a, in which the labels 310 that correspond to the keys 140' are displayed in response to the detection of a member 195 proximate the keys 140'.
  • the labels 310 are superimposed on the video playback and therefore partially obscure it. For this reason, the labels 310 are removed from the display 190 once it is detected that a member 195 is no longer proximate the keys 140'.
  • the labels may be retained on the screen for a predetermined period of time after the member has been removed.
  • Fig. 3b shows a second embodiment of the phone 100 of Fig. 3a.
  • This embodiment differs from the first in that to avoid obscuring the video playback, the playback area 200 is reduced in size to create space for the labels 310 on the display.
  • the shrinking of the playback area 200 to provide space for the labels 310 and its enlargement once the labels 310 have been removed from the display 190 are tied to the proximity detection in the same manner as for the touch screen embodiment of Fig. 2c.
  • the graphical components of the user interface have been displayed and concealed en masse dependent upon the proximity of an activation member 195 to the user interface.
  • an actuation member 195 is detected as proximate to only a subset of the hardware components of the user interface (for example just one key or group of keys in a larger keypad, or just a first area of a larger touch screen)
  • Fig. 3d shows a further embodiment of the phone 100 of Fig. 3a, in which the user's finger 195 is detected to be proximate key "F2", but not proximate the other keys in the pad 140'.
  • the label 310' associated with key F2 is displayed superimposed over the playback area 200, but the labels associated with the other keys 140' are not displayed. The user is therefore informed of the outcome of pressing key "F2" with minimal obstruction of the playback area 200.
  • Fig. 4 illustrates a method beginning with the detection 410 of the presence of an actuation member proximate an interactive component of a user interface element that can be contactually actuated by an actuation member acting on said interactive component, wherein the actuation member has not necessarily actuated the interactive component, and following with the displaying 420 of a displayable component of the user interface element in response to the detection of an actuation member proximate the interactive component.
  • buttons in an interface are not limited to buttons in an interface.
  • the invention is equally applicable to other interface components, such as slider bars, radio groups and tabs.
  • the invention has been described with specific reference to a video playback application, it is usable with any visual display.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Position Input By Displaying (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

A user interface element is divided into a displayable component and an interactive component. The displayable component is displayed on a screen whilst the user's finger or a stylus is close or touching the interactive component, and hidden otherwise. The user interface element may be implemented on a touch screen or as a soft button.

Description

User interface display device
FIELD OF THE INVENTION
The present invention relates to a computing device and a method of making available a user interface element comprising a displayable component and an interactive component.
BACKGROUND OF THE INVENTION
Touch screens are becoming a more common user interface for mobile computing devices. However, they suffer from the disadvantage that it is necessary to dedicate at least a portion of the screen to user interface components such as buttons, sliders, icons, and so on. These components must be large enough on the screen to permit effective user interaction - for example a button must be large enough not only to be visible and recognisable by the user, but also to be physically accessible to him in order that he can actuate it. A disadvantage with the use of on-screen interface components is that screen real estate must be set aside for such components and will not therefore be available for displaying other information.
The disadvantage described above is at its greatest when the total screen area is small (for example in a mobile device), or the interface components must be large (perhaps because the interface is adapted for people who have below-average vision or motor difficulties that would impair their physical interaction with small components). The efficient use of screen area is therefore of particular importance in mobile devices such as mobile phones, personal digital assistants, and smart phones - which are normally small in size and consequently have small screens, and are also mass-market devices that need to be usable by all people regardless of visual acuity or manual dexterity. The desire to minimise the impact of interface components on the area available for displaying other data has led to some touch screens being adapted for stylus use. The use of a stylus rather than a fingertip increases the accuracy of user interactions and therefore allows smaller interface components to be used. However, some screen space is still taken up by the interface components and the forced use of a stylus is used is inconvenient to the user, who must locate the stylus prior to using the interface and store it safely away at the end of each interaction.
Some devices have two screens where one of the screens is a touch screen and is permanently dedicated to user input. The use of a second screen increases the total available screen area, but its use will consume device resources such as power, memory and processor time, and its presence will increase the overall size of the device. Furthermore, it is normally desirable to present interface components within the same screen as the main display to provide a simple user experience.
Other devices have a mixture of hardware buttons and a touch screen to minimise the amount of screen area dedicated to user interaction. This use of hardware buttons increases the overall size and weight of the device, and does not provide the inherent flexibility in terms of interface configurability that is possible using an on-screen user interface.
It is also known for a device that incorporates user interface components into its display to hide such components whilst particular information is being displayed. For example, the Microsoft Windows Media Player for the Windows XP operating system displays a bar of user interface components (e.g. a play/pause button, and a volume slider) whilst video is being played, but hides these components (and the mouse pointer) when no user input has been received for a certain period of time, and enlarging the video display to use the area of the screen that was previously occupied by the bar. Once the user clicks the mouse or presses a key the video display is shrunk and the component bar and mouse pointer are redisplayed. SUMMARY OF THE INVENTION
According to a first example of the invention, there is provided apparatus comprising: at least one processor; and at least one memory including computer program code, the at least one memory and the computer program code configured to, working with the at least one processor, cause at least the following to be performed: detecting the presence of an actuation member proximate an interactive component of a user interface element that can be contactually actuated by an actuation member acting on said interactive component, wherein the actuation member has not necessarily actuated the interactive component; and displaying a displayable component of the user interface element in response to the detection of an actuation member proximate the interactive component.
According to a second example of the present invention, there is provided a method comprising: detecting the presence of an actuation member proximate an interactive component of a user interface element that can be contactually actuated by an actuation member acting on said interactive component, wherein the actuation member has not necessarily actuated the interactive component; and displaying a displayable component of the user interface element in response to the detection of an actuation member proximate the interactive component.
According to a third example of the present invention, there is provided a computer readable medium having instructions stored thereon for: detecting the presence of an actuation member proximate an interactive component of a user interface element that can be contactually actuated by an actuation member acting on said interactive component, wherein the actuation member as not necessarily actuated the interactive component; and displaying a displayable component of the user interface element in response to the detection of an actuation member proximate the interactive component. According to a fourth example of the present invention, there is provided apparatus comprising means for: detecting the presence of an actuation member proximate an interactive component of a user interface element that can be contactually actuated by an actuation member acting on said interactive component, wherein the actuation member has not necessarily actuated the interactive component; and displaying a displayable component of the user interface element in response to the detection of an actuation member proximate the interactive component.
Also disclosed is apparatus comprising: a display; a user interface input element comprising a displayable component and an interactive component, wherein the input element can be contactually actuated by an actuation member; a presence detector configured to detect the presence of an actuation member that is proximate the interactive component but has not necessarily actuated the interactive component; and a processor configured to cause the display to display the displayable component in response to the detection of an actuation member proximate the interactive component.
Also disclosed is a method of making available a user interface element comprising a displayable component and an interactive component, wherein the input element can be contactually actuated by an actuation member, the method comprising: detecting the presence of an actuation member that is proximate the interactive component but has not necessarily actuated the interactive component; and in response to the detection of an actuation member proximate the interactive component, cause a display to display the displayable component.
Also disclosed is a computer-readable medium having stored thereon computer-readable instructions for performing the immediately-above method.
Suitable user interface input elements include buttons, slide bars, tabs, and radio groups implemented on a touch screen, and soft buttons. A soft button is a user interface element comprising a hardware button whose function is determined by software. Conventionally, a soft button is often implemented as a key placed close to the edge of a display, with a label displayed on the screen in close proximity to the key. The label indicates what function will result when the key is pressed, and both the function and label will change as the user navigates through menu systems and applications. Such soft buttons are common in mobile phones whose small size restricts the number of dedicated hardware buttons that can be provided. Although the term 'soft button' normally corresponds to a button - i.e. a key in the user interface - the hardware component may instead be implemented as a scroll wheel, a joystick, a track ball, or any other suitable hardware input component.
In the case of a soft button, the interactive component comprises the hardware input component through which a user makes the input (e.g. a key) and the displayable component comprises a displayable indication as to the result to be expected by actuating the interactive component.
Alternatively, the user interface element may be entirely implemented using a touch screen. Touch screens operate by displaying a graphical representation of an input element (e.g. a button, a slide bar, etc.) and associating an area of the screen with that representation. Normally the associated area of the screen is the area in which the graphical representation is displayed. In the case of a button, the user would see the graphical representation of the button displayed on the screen, and would actuate the button by touching the area of the screen associated with it. In the case of a touch screen, the displayable component of the user interface element is the graphical representation and the interactive component is the area of touchable screen associated with it.
The skilled person will be aware of other suitable user interface elements that have interactive and displayable components and which would benefit from the present invention.
The interactive component is actuated by an actuating member. The type of member that actuates the component will vary according to the component's nature, but common actuating members include fingers and dedicated styli. The interactive component is contactually actuated, meaning that its actuation requires it to be contacted by the actuation member. In many cases mere contact alone may be sufficient to actuate the interactive component (for example, touching a finger against a touch screen), but in other cases a further motion or force may be required (for example pushing against a button, rotating a wheel, moving along a slider bar's path). Contactual actuation encompasses both actuation where contact alone is sufficient and actuation where some other effort is additionally required.
An actuating member is proximal the interactive component when it is close to the interactive component. The threshold proximity required for a member to be detected as 'proximal' the component will vary according to the technology used and its specific application. Since actuation of the interactive component requires contact, the actuating member will always be proximal during actuation. However, the actuating member will also be detected as proximal the interactive component when it is touching or close to the component but not actuating it.
Although it is normally desirable that the display should remain uncluttered by the displayable components of the user interface in order to provide the maximum display area for other purposes, it is important that the displayable components are displayed when the user is using the user interface elements, or is likely to use them. It is therefore not enough just to display the displayable components in response to the actuation of an interactive component.
If the displayable components are displayed in response to an actuation then the actuation will therefore have been made in the absence of the displayable element and be very prone to user error. If pre-display input is accepted as valid by the operating system, it is highly likely that it will not correspond to the user's intended input.
If, on the other hand, the first input simply triggers the display of the displayable components and is otherwise ignored, then the user will need to repeat his input operation before it is accepted by the operating system. This introduces great unpredictability into the user interface - sometimes the user's inputs will be accepted immediately, and sometimes they will require repeating. Not only is this confusing to novice users, but it also presents a significant unreliability for expert users who have learnt common sequences of inputs (e.g. key presses) which they enter without reference to the display - such expert users may not expect the first input to require repeating.
Displaying the displayable component of the user interface element in response to the detection of a proximate actuation member avoids the undesirable operation described above. Since the displayable component is visible to the novice user before he actuates the interactive component, he is less likely to erroneously actuate the wrong component. The expert user who actuates the interactive component without looking at the displayable component is similarly unburdened using the new solution - since the displayable component is displayed in response to the detection of proximity, rather than actuation, there is no longer any reason for the operating system to ignore the first-received input.
The processor is preferably configured to cause the display not to display the displayable in response to the detection of no actuation member proximate the interactive component.
Still more preferably, the processor may be further configured to cause the display to cease to display the displayable a predetermined period of time after an actuation member ceases to be detected proximate the interactive component. This takes advantage of the fact that user inputs often occur in grouped sequences.
Contactually actuating the input element in some embodiments comprises touching the actuation member against the interactive component. This touching is sufficient in itself in some embodiments such as those using touch switch buttons or the majority of touch screens. Contactually actuating the input element may comprise pressing the actuation member against the interactive component with greater than a threshold force. This would be the case for a hardware button that need to be pressed, or a pressure-sensitive touch screen. The threshold force is the minimum force required to actuate the interactive element (e.g. to depress the button, or to register a 'touch' on the pressure-sensitive touch screen).
The presence detector may be configured to detect the presence of an actuation member proximate the interactive component when the actuation member is actually touching the interactive component. This would be the case in embodiments when the actuation of the interactive component requires something further than a touch, for example the application of greater force, the rotation of a wheel, or the movement of the member along a specific path (e.g. along a slider).
The presence detector may be configured to detect the presence of an actuation member proximate the interactive component when the actuation member is detected as located within a threshold range of the interactive component. Although in principle the threshold range relates to a threshold distance from the interactive component, other thresholds may be used when they are at least partially representative of the distance from the component. For example, a detected signal greater than a threshold magnitude will indicate a finger proximate to a capacitive touch screen, even though the magnitude will also be affected by factors other than distance.
The presence detector may be configured to detect the presence of an actuation member proximate the interactive component only when the actuation member is detected as actually touching the interactive component. This is possible where something further than a touch is required to actuate the component - for example the application of a force or a particular motion.
The display may be a touch screen, and the interactive component may then be a location on the touch screen. The location may be a single point on the screen, or an area - including the whole screen. The location on the touch screen may comprise the location of the displayable component.
The input element may be a soft control, where the interactive component is a hardware control and the displayable component describes the function presently associated with the actuation of the hardware control. 'Soft controls' are frequently referred to as 'soft buttons'. In practice a soft control/button need not be implemented as a button - it could be a scroll wheel, joystick, or any other suitable input.
The presence detector may include a capacitance detector.
The presence detector may include a beam-break detector.
DESCRIPTION OF THE DRAWINGS
The present invention will now be described by way of example with reference to the accompanying drawings. In the drawings:
Fig. 1 is a schematic representation of a mobile phone;
Fig. 2a is a schematic representation of a phone of the type shown in Fig
1 ;
Fig. 2b is a schematic representation of a first embodiment of the phone of Fig. 2a;
Fig. 2c is a schematic representation of a second embodiment of the phone of Fig. 2a;
Fig. 3a is a schematic representation of a phone of the type shown in Fig
1 ;
Fig. 3b is a schematic representation of a first embodiment of the phone of Fig. 3a;
Fig. 3c is a schematic representation of a second embodiment of the phone of Fig. 3a;
Fig. 3d is a schematic representation of a first embodiment of the phone of Fig. 2a; and Fig. 4 is a flowchart of a method.
DETAILED DESCRIPTION
Fig. 1 illustrates a mobile phone 100 comprising a Central Processing Unit (CPU) 1 10 which is connected to connected to a speaker 120, microphone 130, a keypad 140, a camera 150, storage memory 160, Random Access Memory (RAM) 170, and a mobile phone transceiver 180. The CPU 1 10 is also connected to a display 190. In the embodiment shown in Fig. 1 , the display 190 is a touch screen operated using an actuating member 195; however it is also provided with soft buttons 140'.
In operation, the phone 100 runs under the control of the operating system. The operating system includes a user interface that manages interactions between the phone and a user. The user interface receives user input from the keypad 140, the touch screen display 190 and the soft buttons 140'. The user interface also controls the graphical output to the display 190. Audio input via the microphone 130, visual input via the camera 150, and audio output via the speaker 120 may also be controlled by the user interface.
In practice many different combinations and variations of user interface hardware are possible. In simple embodiments, the interface with the user may comprise a non-touch screen and buttons arranged to form an alphanumeric keypad, soft buttons, or in a directional pad. A joystick, touch pad, mouse or roller-ball may be replace the buttons or be present in addition to them.
There has been a recent trend away from the use of hardware buttons and instead towards the use of touch screens. Replacing hardware buttons with a touch screen allows the screen area of a portable device to be maximised, providing a larger display area. A touch screen operates by displaying user interface components on the screen and detecting touch operations relating to the displayable components. For example, a graphical button or slider bar may be displayed at a location on the screen and a press or slide of a finger or stylus in the location on the screen where the button or slider is displayed is interpreted by the operating system as an activation of that component. Although a finger or dedicated stylus is normally used to perform the touch operation, the screen may be operable using other types of actuating member. In the embodiment shown in Fig. 1 , the actuation member 195 is a dedicated stylus is provided for contactually actuating the touch screen display 190.
Fig. 2a shows an embodiment of the phone 100 where the display 190 is a touch screen. In the illustration, a media player application is running on the phone. The media player application is playing a video, with the playback area 200 occupying the entirety of the display 190.
Fig. 2b shows the same phone as Fig. 2a, running the same media player application and playing the same video. However, Fig. 2b additionally shows a number of on-screen buttons 210 that can be operated by the user as part of the user interface. Each of the buttons 210 is displayed in a different location within the display 190 and associated with touchable area of the display 190 at that location. To operate one if the buttons 210, the user contactually actuates its associated area of the display 190 by touching it with his finger.
Although the graphical representation of the buttons 210 and their associated touchable locations in this case occupy the same locations on the touch screen 190, it is possible that a user interface component may be associated with a separate, remote location on the touch screen 190.
As can be seen from Fig 2b, the displayable components of the buttons 210 are superimposed on the video playback 200 and thus partly obscure it. In order to render the playback 200 free from obstruction, the graphical representations of the buttons 210 are removed from the user interface under certain pre-defined conditions. Such conditions may include the start of video playback, and when a predefined period has passed since the last user input (e.g. touch upon the touch screen 190 or actuation of the buttons 210). In this way, the representations of the buttons 210 are concealed when the user is not expected to use them (e.g. after he has started playback, or after he has not made an input for a while) in order that the visible portion of the video playback 200 be maximised during these times.
The touch screen 190 is able to detect the user's finger 195 when it is proximate the display 190 and not just when the finger 195 is in direct contact with it (i.e. to contactually actuate one of the buttons 210). The buttons are actuated contactually (by touch), but other functionality may be initiated based on whether a finger 195 is detected proximate the display 190. In this context, proximate means within close to the display, or touching it. Linking the display of the buttons 210 to this proximity detection permits the buttons 210 to be displayed when a finger 195 (or other actuation member, for example a stylus) is close to the screen 190 and a user input is likely to be forthcoming, but otherwise to be concealed on the basis that no input is possible.
Although it is preferable that the closeness of the finger to the display is preferably defined and detected in terms of a distance (for example 1 cm), in practice this may not be possible and the proximity will relate to the magnitude of a detection signal or value.
In one embodiment a resistive touch screen is used. A resistive touch screen can detect actual touches but cannot detect a finger close to but not touching the screen. In order to provide proximity detection, infrared signals are projected in front of the screen between one or more infrared emitters and detectors. When the grid is broken, the signal from the detector indicates that an actuation member is proximate the screen. The grid can be positioned at a specific distance from the screen, the specific distance defining the maximum distance from the screen that is considered 'proximate' (i.e. a member will be detected as proximate once it has come within this distance of the screen).
In another embodiment, a capacitive touch screen is used. Capacitive touch screens can detect fingers both actually touching the screen and merely close to it, the two being distinguished by the strength of the detection signal. A finger close to the screen results in a stronger detection signal that one further away, and a finger actually touching the screen results in a signal that is stronger still. The strength of the detected signal will also depend on the nature of the finger, for example its size and whether or not it is wet. The signal strength therefore cannot be relied upon to define an accurate distance bound within which a finger is considered proximate. In this case, it is the strength of the detected signal that defines a finger's proximity, signals stronger than a certain threshold being deemed proximate, regardless of the real distance of the finger from the screen.
Most touch screens require only a touch to perform a contactual actuation of a user interface component. However, some touch screens require a minimum amount of pressure to be applied. In the later case, a member that is touching the screen will not be making a 'contactual actuation', although it will be proximate.
In operation, the buttons 210 are displayed on the screen 190 when an actuating member 195 is detected proximate to it. Once the member 195 is removed from the vicinity of the screen and is no longer proximate, the buttons cease to be displayed. Preferably, the buttons continue to be displayed for a predefined period of time after the member 195 is removed. For example, the buttons 210 may continue to be displayed for a period of about 5 seconds after the member 195 has been removed. The reason for this is that one user input is often closely followed by another.
In certain cases, the buttons may not necessarily cease to be displayed. For example, in the embodiment shown in Fig. 2a and 2b a video is being played. If the user input serves to pause the video there is likely to be little detriment to the user if the buttons 210 continue to be displayed until playback is resumed and the buttons 210 may therefore remain visible until playback has resumed.
Fig. 2c shows an alternative display of the buttons 210 shown in Fig 2b. So that the buttons 210 do not obscure the video playback, the playback area 200 is decreased in size to create a clear area of the display 190 in which the buttons 210 are be displayed. When the buttons 210 are not displayed, the playback area 200 increases in size to fill the entire display 190. In this embodiment, the buttons 210 and concealed based on the detection of a proximate member in just the same manner as in the embodiment of Fig. 2b. However, the resizing of the playback area 200 is similarly tied to the proximity detection, so that its size is maximised when the buttons 210 are concealed.
Although the invention has been described in relation to touch screen user interfaces, it finds application in other types of user interface where a distinction can be made between an actuating member that is merely proximate the interface, and a member that is (also) making a contactual actuation of the interface. For example, it is above descried that an infrared grid could be projected over the display 190; however a similar grid can be projected over the keypad 140, allowing members proximate the keys to be detected. In another embodiment, the presence of a finger proximate a key may be detected using capacitive sensing. In still another embodiment, a finger merely touching a pressure sensitive key may be detected using a resistive touch switch integrated into the top surface of the key. Other techniques for detecting proximate actuation members for keys may be used and the most suitable choice will vary according to its application.
Fig. 3a shows a phone 100 that is, similar to the phone of Fig. 2a, playing video using the full extent of its display 190. To the side of the display are located the hardware keys 140' of a number of soft buttons. Using a suitable proximity detection technique, the phone 100 can determine when an actuation member 195 is proximate the keys 140'. In order that the video playback 200 should not be obscured when the user is unlikely to make an input using the keys 140', their labels are hidden until a member 195 is detected proximate the keys 140'.
Fig. 3b shows a first embodiment of the phone 100 of Fig. 3a, in which the labels 310 that correspond to the keys 140' are displayed in response to the detection of a member 195 proximate the keys 140'. The labels 310 are superimposed on the video playback and therefore partially obscure it. For this reason, the labels 310 are removed from the display 190 once it is detected that a member 195 is no longer proximate the keys 140'. As before, the labels may be retained on the screen for a predetermined period of time after the member has been removed.
Fig. 3b shows a second embodiment of the phone 100 of Fig. 3a. This embodiment differs from the first in that to avoid obscuring the video playback, the playback area 200 is reduced in size to create space for the labels 310 on the display. The shrinking of the playback area 200 to provide space for the labels 310 and its enlargement once the labels 310 have been removed from the display 190 are tied to the proximity detection in the same manner as for the touch screen embodiment of Fig. 2c.
In the foregoing embodiments, the graphical components of the user interface have been displayed and concealed en masse dependent upon the proximity of an activation member 195 to the user interface. However, it may be desirable that only a subset of the graphical elements be displayed. For example, when an actuation member 195 is detected as proximate to only a subset of the hardware components of the user interface (for example just one key or group of keys in a larger keypad, or just a first area of a larger touch screen), it may be desirable to display only the graphical components that are associated with that subset.
Fig. 3d shows a further embodiment of the phone 100 of Fig. 3a, in which the user's finger 195 is detected to be proximate key "F2", but not proximate the other keys in the pad 140'. The label 310' associated with key F2 is displayed superimposed over the playback area 200, but the labels associated with the other keys 140' are not displayed. The user is therefore informed of the outcome of pressing key "F2" with minimal obstruction of the playback area 200.
Fig. 4 illustrates a method beginning with the detection 410 of the presence of an actuation member proximate an interactive component of a user interface element that can be contactually actuated by an actuation member acting on said interactive component, wherein the actuation member has not necessarily actuated the interactive component, and following with the displaying 420 of a displayable component of the user interface element in response to the detection of an actuation member proximate the interactive component.
The applicant hereby discloses in isolation each individual feature described herein and any combination of two or more such features, to the extent that such features or combinations are capable of being carried out based on the present specification as a whole in light of the common general knowledge of a person skilled in the art, irrespective of whether such features or combinations of features solve any problems disclosed herein, and without limitation to the scope of the claims. The applicant indicates that aspects of the present invention may consist of any such feature or combination of features. In view of the foregoing description it will be evident to a person skilled in the art that various modifications may be made within the scope of the invention.
In particular, the skilled person will appreciate that the present invention is not limited to a telephone, and is not limited to the use of buttons in an interface. The invention is equally applicable to other interface components, such as slider bars, radio groups and tabs. Similarly, although the invention has been described with specific reference to a video playback application, it is usable with any visual display.

Claims

1. Apparatus comprising: at least one processor; and at least one memory including computer program code, the at least one memory and the computer program code configured to, working with the at least one processor, cause at least the following to be performed: detecting the presence of an actuation member proximate an interactive component of a user interface element that can be contactually actuated by an actuation member acting on said interactive component, wherein the actuation member has not necessarily actuated the interactive component; and displaying a displayable component of the user interface element in response to the detection of an actuation member proximate the interactive component.
2. Apparatus according to claim 1 , wherein the at least one memory and the computer program code are further configured to, working with the at least one processor, cause at least the following to be performed: causing displayable component not to be displayed in response to the detection of the absence of an actuation member proximate the interactive component.
3. Apparatus according to claim 2, wherein the at least one memory and the computer program code are further configured to, working with the at least one processor, cause at least the following to be performed: ceasing to display the displayable component after a predetermined period of time has passed since an actuation member was last detected proximate the interactive component.
4. Apparatus according to any preceding claim, wherein contactually actuating the input element comprises touching the actuation member against the interactive component.
5. Apparatus according to any preceding claim, wherein contactually actuating the input element comprises pressing the actuation member against the interactive component with greater than a threshold force.
6. Apparatus according to claim 5, wherein the at least one memory and the computer program code are further configured to, working with the at least one processor, cause at least the following to be performed: detecting the presence of an actuation member proximate the interactive component when the actuation member is actually touching the interactive component.
7. Apparatus according to any preceding claim, wherein the at least one memory and the computer program code are further configured to, working with the at least one processor, cause at least the following to be performed: detecting the presence of an actuation member proximate the interactive component when the actuation member is detected as located within a threshold range of the interactive component.
8. Apparatus according to claim 6, wherein the at least one memory and the computer program code are further configured to, working with the at least one processor, cause at least the following to be performed: detecting the presence of an actuation member proximate the interactive component only when the actuation member is detected as actually touching the interactive component.
9. Apparatus according to any preceding claim, the interactive component is a location on a touch screen.
10. Apparatus according to claim 9, wherein the location on the touch screen comprises the location of the displayable component.
1 1. Apparatus according to any of claims 1 -9, wherein the input element is a soft control and wherein: the interactive component is a hardware control; and the displayable component describes the function presently associated with the actuation of the hardware control.
12. Apparatus according to any preceding claim, wherein the detection is performed using a capacitance detector.
13. Apparatus according to any preceding claim, wherein the detection is performed using a beam-break detector.
14. A method comprising: detecting the presence of an actuation member proximate an interactive component of a user interface element that can be contactually actuated by an actuation member acting on said interactive component, wherein the actuation member has not necessarily actuated the interactive component; and displaying a displayable component of the user interface element in response to the detection of an actuation member proximate the interactive component.
15. The method of claim 14, further comprising causing preventing the display of the displayable component in response to the detection of the absence of an actuation member proximate the interactive component.
16. The method of claim 15, further comprising ceasing displaying the displayable component after a predetermined period of time has passed since an actuation member was last detected proximate the interactive component.
17. The method of any of claims 14 to 16, wherein contactually actuating the input element comprises touching the actuation member against the interactive component.
18. The method of any of claims 14 to 17, wherein contactually actuating the input element comprises pressing the actuation member against the interactive component with greater than a threshold force.
19. The method of claim 18, wherein detecting the presence of an actuation member proximate the interactive component comprises detecting when the actuation member is actually touching the interactive component.
20. The method of any of claims 14 to 18, wherein detecting an actuation member proximate the interactive component comprises detecting when the actuation member is within a threshold range of the interactive component.
21. The method of any of claims 14 to 18, wherein detecting the presence of an actuation member proximate the interactive component comprises detecting when the actuation member is within a threshold range of the interactive component but not touching the interactive component.
22. The method of any preceding claim, wherein the interactive component is a location on a touch screen.
23. The method claim 22, wherein the location on the touch screen comprises the location of the displayable component.
24. The method of any of claims 14 to 21 , wherein the input element is a soft control and wherein: the interactive component is a hardware control; and the displayable component describes the function presently associated with the actuation of the hardware control.
25. The method of any of claims 14 to 24, wherein the detection is performed using a capacitance detector.
26. The method of any of claims 14 to 24, wherein the detection is performed using a beam-break detector.
27. A computer-readable medium having stored thereon computer- readable instructions for performing the method of any of claims 14 to 26.
28. The computer readable medium of claim 27, wherein the computer- readable instructions comprise an operating system.
29. A computer readable medium having instructions stored thereon for: detecting the presence of an actuation member proximate an interactive component of a user interface element that can be contactually actuated by an actuation member acting on said interactive component, wherein the actuation member has not necessarily actuated the interactive component; and displaying a displayable component of the user interface element in response to the detection of an actuation member proximate the interactive component.
30. Apparatus comprising means for: detecting the presence of an actuation member proximate an interactive component of a user interface element that can be contactually actuated by an actuation member acting on said interactive component, wherein the actuation member has not necessarily actuated the interactive component; and displaying a displayable component of the user interface element in response to the detection of an actuation member proximate the interactive component.
PCT/IB2009/052802 2008-06-30 2009-06-29 User interface display device Ceased WO2010001326A1 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
GB0811946A GB0811946D0 (en) 2008-06-30 2008-06-30 Computing device
GB0811946.3 2008-06-30
GB0823705A GB0823705D0 (en) 2008-06-30 2008-12-31 Computing device
GB0823705.9 2008-12-31

Publications (1)

Publication Number Publication Date
WO2010001326A1 true WO2010001326A1 (en) 2010-01-07

Family

ID=39683393

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2009/052802 Ceased WO2010001326A1 (en) 2008-06-30 2009-06-29 User interface display device

Country Status (2)

Country Link
GB (2) GB0811946D0 (en)
WO (1) WO2010001326A1 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102010032221A1 (en) * 2010-07-26 2012-01-26 Continental Automotive Gmbh Manually controlled electronic display device for motor car, has operating elements arranged on picture screen, where activation of operation field is switchable on operated operation fields by guiding user hand to new operating field
EP2963545A1 (en) * 2014-06-30 2016-01-06 Orange Method of displaying a new rectangular window on a screen
WO2021252759A1 (en) * 2020-06-10 2021-12-16 Snap Inc. Contextual application menu
US11481111B2 (en) 2020-04-24 2022-10-25 Microsoft Technology Licensing, Llc Utilization of predictive gesture analysis for preloading and executing application components
US20250012053A1 (en) * 2023-07-03 2025-01-09 Yanmar Holdings Co., Ltd. Control method of work machine, control program for work machine, display system for work machine, and work machine

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030025676A1 (en) * 2001-08-02 2003-02-06 Koninklijke Philips Electronics N.V. Sensor-based menu for a touch screen panel
WO2006039939A1 (en) * 2004-10-13 2006-04-20 Wacom Corporation Limited A hand-held electronic appliance and method of entering a selection of a menu item
EP1752864A2 (en) * 2005-07-29 2007-02-14 Avago Technologies Fiber IP (Singapore) Pte. Ltd. Methods and systems for detecting selections on a touch screen display
JP2007305110A (en) * 2006-05-09 2007-11-22 Apple Computer Inc Force and location sensitive display
AU2007101053A4 (en) * 2007-01-05 2007-12-13 Apple Inc. Multimedia communication device with touch screen responsive to gestures for controlling, manipulating, and editing of media files
AU2008100004A4 (en) * 2007-01-07 2008-02-14 Apple Inc. Portrait-landscape rotation heuristics for a portable multifunction device

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030025676A1 (en) * 2001-08-02 2003-02-06 Koninklijke Philips Electronics N.V. Sensor-based menu for a touch screen panel
WO2006039939A1 (en) * 2004-10-13 2006-04-20 Wacom Corporation Limited A hand-held electronic appliance and method of entering a selection of a menu item
EP1752864A2 (en) * 2005-07-29 2007-02-14 Avago Technologies Fiber IP (Singapore) Pte. Ltd. Methods and systems for detecting selections on a touch screen display
JP2007305110A (en) * 2006-05-09 2007-11-22 Apple Computer Inc Force and location sensitive display
AU2007101053A4 (en) * 2007-01-05 2007-12-13 Apple Inc. Multimedia communication device with touch screen responsive to gestures for controlling, manipulating, and editing of media files
AU2008100004A4 (en) * 2007-01-07 2008-02-14 Apple Inc. Portrait-landscape rotation heuristics for a portable multifunction device

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102010032221A1 (en) * 2010-07-26 2012-01-26 Continental Automotive Gmbh Manually controlled electronic display device for motor car, has operating elements arranged on picture screen, where activation of operation field is switchable on operated operation fields by guiding user hand to new operating field
EP2963545A1 (en) * 2014-06-30 2016-01-06 Orange Method of displaying a new rectangular window on a screen
US11481111B2 (en) 2020-04-24 2022-10-25 Microsoft Technology Licensing, Llc Utilization of predictive gesture analysis for preloading and executing application components
WO2021252759A1 (en) * 2020-06-10 2021-12-16 Snap Inc. Contextual application menu
US20250012053A1 (en) * 2023-07-03 2025-01-09 Yanmar Holdings Co., Ltd. Control method of work machine, control program for work machine, display system for work machine, and work machine

Also Published As

Publication number Publication date
GB0811946D0 (en) 2008-07-30
GB0823705D0 (en) 2009-02-04

Similar Documents

Publication Publication Date Title
US7777732B2 (en) Multi-event input system
JP6429981B2 (en) Classification of user input intent
CA2779706C (en) Three-state touch input system
US10331219B2 (en) Identification and use of gestures in proximity to a sensor
CN105359065B (en) Multi-function keys that provide additional functions and preview of each function
US20100060588A1 (en) Temporally separate touch input
US20070236474A1 (en) Touch Panel with a Haptically Generated Reference Key
US20130207905A1 (en) Input Lock For Touch-Screen Device
US20120105367A1 (en) Methods of using tactile force sensing for intuitive user interface
US20090066659A1 (en) Computer system with touch screen and separate display screen
US20130300704A1 (en) Information input device and information input method
US20150100911A1 (en) Gesture responsive keyboard and interface
EP3044660A1 (en) Multi-touch virtual mouse
US20120075202A1 (en) Extending the touchable area of a touch screen beyond the borders of the screen
US8970498B2 (en) Touch-enabled input device
WO2010001326A1 (en) User interface display device
US20140298275A1 (en) Method for recognizing input gestures
US20150106764A1 (en) Enhanced Input Selection
US20130154967A1 (en) Electronic device and touch control method thereof
KR20160027063A (en) Method of selection of a portion of a graphical user interface

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09772986

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 09772986

Country of ref document: EP

Kind code of ref document: A1