WO2016042303A1

WO2016042303A1 - Navigation system for a menu-driven graphical user interface

Info

Publication number: WO2016042303A1
Application number: PCT/GB2015/052654
Authority: WO
Inventors: Adam Bond
Original assignee: Adam Bond
Priority date: 2014-09-17
Filing date: 2015-09-14
Publication date: 2016-03-24
Also published as: GB201416410D0

Abstract

Embodiments of the invention provide an improved keyboard-based navigation system for menu-based operating systems and desktop applications. An embodiment provides a drop-down (or pull-down) menu interface operated by providing the keyboard function keys, i.e. keys not assigned to a specific ASCII printable code, with context-driven functionality, i.e. where the operation caused by the press of a given function key depends on the state of the menu interface (e.g. the active portion thereof) at the time the function key is pressed. An embodiment provides a GUI having an application switch interface comprising a tiled display, wherein each tile has a function key associated with it, and wherein each open application is assigned to a tile to enable a user to switch to an open application by selecting the corresponding function key.

Description

NAVIGATION SYSTEM FOR A MENU-DRIVEN GRAPHICAL USER INTERFACE

FIELD OF THE INVENTION

The invention relates to a system for navigating within a menu-driven graphical user interface. In particular, the invention relates to a keyboard-based navigation system in which navigation and execution operations are mapped to programmable function keys.

BACKGROUND TO THE INVENTION

Menu-driven graphical user interfaces are well known. For example, the Microsoft® Office suite of desktop

applications all incorporate a top level menu from which various functions and operations are accessible. Similarly, applications running in the desktop operating systems of Apple Inc. also use a menu-driven system for navigation and control.

Conventionally, menu-driven graphical user interfaces are designed primarily for operation with a computer mouse or trackpad, where a user moves a cursor to a desired screen location by recording a corresponding physical movement with the mouse or trackpad. However, for keyboard intensive activities, the repeated use of a mouse to hit relatively small targets (pull down menu titles, up/down arrows, etc.) is a subconscious stressor.

Accordingly, it is also normal for operations to be accessible using the keyboard. For example, common operations are often accessible directly using a keyboard shortcut, which requires simultaneous pressing of two or more keys (e.g.

Ctrl+C, Ctrl+Alt+Del, etc.) . In another example, the menu system may be "activated" for use with the keyboard by

pressing an appropriate trigger key (e.g. Alt) . When the keyboard is activated in this way, the menu system may be navigated using multiple presses of the cursor keys.

Alternatively, once activated, the menu system may display available shortcut keys next to menu items to allow those operations to be quickly accessed. SUMMARY OF THE INVENTION

At its most general, the present invention seeks to provide an improved keyboard-based navigation system for menu- based operating systems and desktop applications. The invention proposes a drop-down (or pull-down) menu interface operated by providing the keyboard function keys with context- driven functionality, i.e. where the operation caused by the press of a given function key depends on the state of the menu interface (e.g. the active portion thereof) at the time the function key is pressed.

With the invention, a single press of a keyboard function key can be used to open each of the pull down menus. Using the function keys in this manner facilitates display of the available functions in any given state of the menu interface, e.g. in the form ' Fl - File', ' F2 - Edit', ' F3 - View', etc. When a pull down menu is opened, the menu interface may change state, so that all commands within the selected menu item (i.e. within an active area of the display) are shown with corresponding function keys, thereby allowing the user to access each command by pressing a function key.

The pull-down menu interface of the invention may be used in addition to a native navigation system for any given interface, and may itself also be operated using a cursor, e.g. using a mouse or touch screen. However, the keyboard- based navigation system of the invention may provide a complete single-keystroke navigation solution, which is advantageous for computer users who are unable to use a mouse or press key combinations.

The pull-down menu interface may operate throughout the entire computer interface, i.e. on the desktop (operating system), within programs (e.g. file manager and applications) and even within intra-application operations (e.g. dialogue boxes ) .

Herein, reference to a "function key" is intended to indicate a key on a computer keyboard that is not assigned to a specific ASCII printable code. In a preferred embodiment the term "function key" refers to the F-keys Fl to F12 that are found on a conventional computer keyboard. In addition to the F-keys, the invention may also use the Esc, Enter and Tab keys. These keys can be programmed to cause an operating system or application to perform certain actions. In the present invention, this functionality is extended such that the action performed by each function key depends on the state of the pull-down menu interface or dialogue box of the invention.

Thus, according to the invention there is provided a control system for a menu-driven graphical user interface, the control system comprising: a computer keyboard having a plurality of function keys that are not assigned to a specific ASCII printable code; a processor connectable to receive an input signal from the keyboard, the input signals

corresponding to operation of one or more of the function keys; a memory stored code that is executable by the

processor; and a display for showing a menu-driven graphical user interface, wherein the code is operable to: detect a state of the menu-driven graphical user interface; determine an operation to be performed in response to a received input signal from the keyboard based on the detected state; execute the determined operation; and display on the display an updated menu-driven graphical user interface following execution of the determined operation. The processor, memory and display may be those of a conventional computing device. The keyboard may also be a conventional computer keyboard, whereby the function keys are Fl to F12.

The code may be arranged to detect the state of the menu- driven graphical user interface by detecting an active portion of the menu-driven graphical user interface, wherein the active portion includes a plurality of available operations.

Herein, references to a menu-driven graphical user interface may mean any of a menu bar with pull-down sub-menu listings, as well as dialogue boxes which may be launched from such a menu.

The memory may store a one-to-one mapping between the plurality of function keys and the plurality of available operations. For static displays, such as dialogue boxes and menu structure, the one-to-one mapping may be predetermined and stored in the memory. These static active portions may include unique identifiers which enable the relevant one-to- one mapping to be accessed (i.e. looked up) easily. The code may then be arranged to determine the operation to be performed by comparing the received input signal with the current one-to-one mapping for the detected active portion.

For dynamic displays, the code may be arranged to set the one-to-one mapping on the fly.

The code may be arranged to update the state of the menu- driven graphical user interface after the determined operation is executed. This may comprise determining an updated active portion having a plurality of available operations, assigning (either dynamically or by looking up a stored mapping) an updated one-to-one mapping between the plurality of available operations and the plurality of function keys, and storing the updated one-to-one mapping in the memory as the current one- to-one mapping.

The code is arranged to display on the display the current one-to-one mapping for the updated active portion.

Preferably this is done by providing a label for each function key next to its corresponding operation. The labels may be provided only on the active portion, to make it easier to see which area of the display is active, but in practice in may be desirable to maintain the labels on display even in inactive regions of the display.

The menu-driven graphical user interface may comprise a horizontal menu bar having a plurality of selectable menu items, whereby selection of a menu item causes a pull-down menu to be displayed beneath the menu bar. The plurality of selectable menu items of the horizontal menu bar may be context-sensitive, whereby the code may detect an active underlying process being executed by the processor, and determine the plurality of selectable menu items of the horizontal menu bar based on the active underlying process.

This enables the menu-driven graphical user interface to be used seamlessly across all operations on a computer. For example, the underlying process may be any one of an operating system desktop, a word processing application, a file

management application, a data entry application, a program dialogue box, and a shutdown dialogue box. To provide the context sensitivity, the memory may include a look-up table storing a plurality of selectable menu items for each of a plurality of available processes executable by the processor. As an alternative to the pull-down menu, the plurality of selectable menu items may be presented in a column, with the elements of the pull-down bar being displayed in a row with their corresponding selectable menu item. There may be more than one column of selectable menu items, and there may be sub-rows within each element of the pull-down bar.

In use, each of the plurality of selectable menu items may be assigned to a respective function key. One or more of the plurality of selectable menu items may be adjustable by holding down its respective function key for longer than a predetermined length of time (e.g. more than 0.2 seconds but less than 1 second) . This enables the user to customise the menu bars and pull down menu items. It may also be possible for a function key to be associated with two actions,

depending on the duration of the key press.

The menu-driven graphical user interface may comprise an application switch control screen having a tiled display, wherein each tile has a function key associated with it, and wherein each open application is assigned to a tile to enable a user to switch to the open application by selecting the corresponding function key. The application switch control screen may be accessed by a single key stroke, e.g. of a dedicated key such as the Esc key. The application switch control screen may be a resizable screen. The tiles of the application switch control screen may display indicators corresponding to the status of the applications they

represent .

The menu-driven graphical user interface may comprise one or more horizontal quick command bars (QCBs) selectively displayable beneath the menu bar, each QCB comprising a plurality of selectable command items that are mapped to the plurality of function keys when that QCB forms an active portion of the menu-driven graphical user interface.

The menu-driven graphical user interface may comprise a window resizing control screen having a tiled display, wherein each tile of the tiled display has a window resize option and a function key associated with it to enable a user to switch to a desired window resize option by selecting the

corresponding function key.

In one embodiment, the code may be arranged to perform a different operation depending on whether a single or double tap of a function key is received. The code may thus be arranged, upon receiving an input signal indicating that a double-tappable function key has been pressed once, to generate a double tap animation for display on the display, wherein the double tap animation indicates the time remaining within which a further tap of the function key will register as a double tap. This animation therefore makes it easier for the user to avoid accidentally double tapping.

In one embodiment, after the code has determined the operation to be performed, it may be arranged to trigger playback of an audio signal (e.g. synthesised voice) that is indicative of the operation. The audio signal may have a different pitch (e.g. adult, child or male, female) depending on the state of the menu-driven graphical user interface.

The code may be arranged to defer execution of the operation until an additional confirmatory input signal (e.g. another press of the relevant function key) is received.

In another aspect, the invention may provide a computer- implemented method of operating a menu-driven graphical user interface, the method comprising: detecting a state of the menu-driven graphical user interface; determining, based on the detected state, an operation to be performed in response to an input signal received from a keyboard, wherein the input signal corresponding to operation of one or more of a

plurality of function keys that are not assigned to a specific ASCII printable code; executing the determined operation; and displaying on a display an updated menu-driven graphical user interface following execution of the determined operation.

Detecting the state of the menu-driven graphical user interface may comprise detecting an active portion of the menu-driven graphical user interface (e.g. a specific pull down menu or a dialogue box) , wherein the active portion includes a plurality of available operations.

The method may include assigning or looking up from a computer memory a current one-to-one mapping between the plurality of function keys and the plurality of available operations, wherein the step of determining the operation to be performed comprises comparing the input signal with the current one-to-one mapping for the detected active portion.

The method may include updating the state of the menu- driven graphical user interface after the determined operation is executed, e.g. by determining an updated active portion having a plurality of available operations; and assigning or looking up from a computer memory an updated one-to-one mapping between the plurality of available operations and the plurality of function keys.

In a different aspect, the invention may provide a computer system with a graphical user interface comprising: a computer mouse; a processor connectable to receive an input signal from the computer mouse; a memory stored code that is executable by the processor; and a display for showing the graphical user interface, wherein the code is operable to: display the graphical user interface, the interface comprising a dialogue box having a user interaction region, display the position of a cursor whose position in the dialogue box corresponds to the inputs from the computer mouse, provide the user interaction region with a cursor movement limitation zone that constrains cursor movement within the user interaction region .

The cursor movement limitation zone may comprise a deceleration region, and decelerate the cursor when the cursor intersects the deceleration region.

The deceleration region may allow the cursor to pass through the deceleration region, if the user accelerates the cursor after the deceleration region has decelerated it.

Further, the deceleration region may act to decelerate the cursor only if the cursor is moving in a specific direction through the deceleration region. Said another way, the deceleration region may not act to decelerate the cursor if the cursor is moving through the deceleration region in a particular direction.

In embodiments of this aspect, the deceleration region may be a border of the user interaction region which partially bounds the user interaction region. The deceleration region may be positioned on an opposite side of the user interaction region to the cursor. In this case, the deceleration region may automatically track the cursor, so that it remains on an opposite side of the user interaction region to the cursor at all times. Therefore when the cursor impinges the deceleration region it is within the user interaction region. The cursor may be initially located at a point equidistant from all buttons in the dialogue box. In an embodiment, the

deceleration region may be placed between a title bar and primary function bar of the dialogue box, wherein there is a gap in the deceleration region to allow the user access to the title bar without traversing the deceleration region.

In other embodiments, the deceleration region may entirely overlap the user interaction region. Therefore when the cursor impinges a periphery of the user interaction region, it is decelerated or halted. In addition to

decelerating the cursor, the deceleration region may cause the cursor to jump to a central point of the user interaction region and set the cursor velocity to zero. Alternatively, the deceleration region may cause the cursor to decelerate at a rate such that it comes to rest within the user interaction region. In both cases, the user interaction region may be a button provided in the dialogue box.

In an embodiment, the cursor movement limitation zone may comprise or consist of a boundary that the cursor cannot cross in one or both directions . The boundary may be around a part of the user interaction region.

In yet a further aspect, the invention may provide a computer system with a graphical user interface comprising: an input device; a processor connectable to receive an input signal from the input device; a memory stored code that is executable by the processor; and a display for showing the graphical user interface, wherein the code is operable to: display the graphical user interface, the interface comprising a moveable interface box containing only: an escape button corresponding to an escape command for the interface, a move button for moving the moveable interface box, and a hide button for hiding the moveable interface box. Pressing either of the hide or move buttons via input signals may change the state of the moveable interface box for a predetermined time.

In embodiments of this aspect, the predetermined time may be independent of subsequent input signals. The move button may be used by providing a constant input signal via the input device. Depressing the hide button may cause the moveable input box to transform from an unhidden state into a hidden state, the hidden state having smaller dimensions than the unhidden state. In the hidden state, the moveable input box may appear as a timer bar, the timer bar indicating the predetermined time during which the moveable input box will remain in the hidden state. The move button may transform the moveable input box into a move configuration for the predetermined time wherein the move button occupies

substantially all of the moveable input box. Elapsed time of the predetermined time may be indicated by a timer bar. The move button and hide button may be provided in an unexpanded state, wherein a region of the moveable interface box may be used to transform the moveable interface box into an expanded state, wherein the move button and hide button are visible. The input device may be a touch screen, such that input signals are provided by the user physically touching a section of the screen.

In another aspect, the invention may provide a computer system with a graphical user interface comprising: an input device; a processor connectable to receive an input signal from the input device; a memory stored code that is executable by the processor; and a display for showing the graphical user interface, the interface comprising a cursor, and a case indicator, wherein the case indicator indicators whether the input device is in an upper case mode or a lower case mode. The case indicator may be a letter, e.g. ^Λ0' , denoting that the input device is in a particular case mode e.g. upper case.

Alternatively the case indicator may be some other graphical element that indicates to the user that the input device is in a particular case mode such as a bar or solid block of colour. The input device may be a keyboard. The keyboard may be in an upper case mode when, for example, the user has toggled the

Caps Lock key to ^xon' or is holding down the Shift key.

The invention may also provide a computer program product comprising a computer-readable storage medium having computer- executable instructions (i.e. software) stored thereon, wherein the computer-executable instructions are executable by a processor to perform a method as set out herein.

BRIEF DESCRIPTION OF THE DRAWINGS An example of the present invention is discussed below in detail with reference to the accompanying drawings, in which:

Fig. 1 is a screen shot of a pull-down menu interface for a computer application program that is a first embodiment of the present invention;

Figs 2A, 2B and 2C are screen shots illustrating a first navigation operation in the menu interface of Fig. 1; Figs 3A, 3B and 3C are screen shots illustrating a second navigation operation in the menu interface of Fig. 1;

Fig. 4 is a screen shot showing a first extendable menu within the menu interface of Fig. 1;

Fig. 5 is a screen shot showing a second extendable menu within the menu interface of Fig. 1;

Fig. 6 is a screen shot of a pull-down menu interface for desktop operations that is a second embodiment of the present invention;

Fig. 7 is a series of images illustrating an animation associated with an edit function of the menu interface of Fig. 6;

Fig. 8 shows screen shots illustrating a shutdown operation in the menu interface of Fig. 6;

Fig. 9 is a screen shot of a print dialogue box that is launched from the menu interface of Fig. 1;

Figs. 10A and 10B shows screen shots of a print dialogue box and a select printer sub-dialogue box that is launched therefrom;

Fig. 11 is a screen shot showing a tiled application switch function of a pull-down menu interface that is an embodiment of the invention;

Figs. 12A, 12B and 12C are a series of images

illustrating additional sub-menu command bars that are accessible from the menu interface shown in Fig. 1;

Fig. 13 is a screen shot of a sub-menu command bar dialogue box that is accessible from the menu interface shown in Fig. 1;

Fig. 14 is a series of images illustrating a first process of editing the content of a sub-menu command bar;

Figs. 15A and 15B present a series of images illustrating a second process of editing the content of a sub-menu command bar;

Fig. 15C is a dialogue box for a process of selecting a display format for a plurality of sub-menu command bars;

Fig. 15D is a series of partial screen shots showing a scrolling transition between two sub-menu command bars in a viewport ;

Fig. 16 is a screen shot illustrating a data entry screen with fields that are directly accessible using function keys; Fig. 17 is a series of images illustrating a process of resizing windows that is accessible from the menu interface of Fig. 1;

Fig. 18 is a series of images illustrating a double click time limit animation that can be used with the menu interface of the present invention;

Figs. 19A to 19F present a series of images illustrating a function key-driven file management navigation scheme that is another embodiment of the present invention;

Figs. 20A to 20C are screen shots illustrating a

navigation operation in the menu interface of Fig. 19; and

Fig. 21 is a screen shot illustrating a file selection operation in the menu interface of Fig. 19;

Figs. 22 to 24 present a series of images illustrating the use of a two column quick command bar;

Figs. 25 and 26 are screen shots illustrating the use of smart button borders in a dialogue box which is another embodiment of the invention;

Fig. 27 is a screen shot illustrating the use of a smart menu border for the title bar of a dialogue box which is another embodiment of the invention.

Fig. 28 is a series of screen shots illustrating a floating escape menu for touch screens which is another embodiment of the present invention;

Fig. 29 is a series of images illustrating a progress bar for the hidden floating escape menu of Fig. 28;

Fig. 30 is a series of images illustrating further functionality of the floating escape menu of Fig. 28;

Fig. 31 is a series of images illustrating further functionality of the floating escape menu of Fig. 28;

Fig. 32 is a series of images illustrating the process in which the floating escape menu of Fig. 28 is expanded to show more options

Fig. 33 is a series of screen shots illustrating

embodiments of the case indicator.

DETAILED DESCRIPTION; FURTHER OPTIONS AND PREFERENCES

The accompanying drawings illustrate various aspects of a function key-driven menu interface, any one or more or all of which may be incorporated into an embodiment of the invention. Fig. 1 shows a function key-driven pull-down menu interface 100 that is an embodiment of the invention. This embodiment may be used (e.g. displayed) during running of an application (computer program) within an operating system on a user's computer. The menu interface 100 comprises a

horizontal menu bar 102 which lists a set of available pulldown menus. Each pull-down menu has a keyword title (e.g. "File", "Edit", etc.) as is conventional. In addition to the keyword title, each pull-down menu has a function key label (e.g. Fl, F2, etc.) which indicates the function key that must be pressed in order for that pull-down menu to be activated.

There are twelve function keys on a conventional keyboard. The embodiments of the invention discussed herein are based on a twelve key system, although the invention is equally applicable if more or fewer function keys are

provided.

In this embodiment, the menu bar 102 can be displayed at the top of a user's screen when running the relevant

application. The menu bar 102 may be movable. The pull down menus 106 of the menu bar 102 can be customised to contain any commands used by the application, as explained below with reference to Figs. 6 and 7. Similarly the titles of each button on the menu bar 102 may be changed.

The pull-down menu items are grouped into three sets of four on the menu bar 102. Each set is separated from the next by a divider 104. At the left hand end of the menu bar 102, there is an Esc button 105. If the Esc key on the keyboard or the Esc button 105 is pressed when the menu bar is active, the application switch control screen is launched, as discussed below with reference to Fig. 11. The Esc button 105 on the menu bar may remain clickably active (e.g. for a mouse or on a touchscreen) even if the rest of the menu bar is not active, e.g. if a QCB is active.

The menu bar items and each of the pull-down menu items is also a selectable button, which means it can be activated by clicking a cursor thereon, e.g. using a mouse or

touchscreen .

When the menu bar 102 is active and a function key is depressed, the pull-down menu 106 associated with that function key is displayed and made active. The pull-down menu is arranged to display a list of available functions. Each available function has a function key associated with it, and displayed next to it in the menu. As with the menu bar, each command may also be selected by a click action, e.g. via a mouse or touch screen. Moreover, the pull-down menu 106 is separated into three groups of four keys by divider 111. This three-way division may be replicated on all menu bars, pulldown menus and lists displayed in the interface.

One or more of the commands in the pull-down menu may be associated with a listing or fly-out menu, e.g. of sub- commands, "favourites" or "most recent documents". In Fig. 1, command "Fl Open previous" is associated with such a list, as shown by the small arrow at the right hand end of the command button. When the pull-down menu 106 is active, pressing Fl will cause the listing 108 associated with that command to be displayed. The listing 108 is arranged to display a list of available items. Each available item has a function key associated with it, and displayed next to the relevant item. As with the menu bar and pull-down menu, each item may also be selected by a click from a suitable pointing device.

The invention may use highlighting to indicate which of the menu bar 102, pull-down menu 106 or listing 108 is active. In a preferred embodiment, the function key labelling is only provided on the active part of the interface.

By way of illustration, the following actions can be achieved from the menu bar 102 simply by using a respective sequence of functions keys:

- to open previous document 'Letterl . doc' : Fl Fl Fl

- to launch an open document dialogue box: Fl F2

- to start a new document: Fl F3

- to close the current document: Fl F4

- to launch the exit program dialogue: Fl F12

None of these sequences needs to be memorised by the user, because the user remembers their approximate spatial location (i.e. screen position), and will automatically learn the ones they use the most, such as Fl F2 to open a document, etc .

Figs. 2A, 2B and 2C show a sequence of images to

illustrate the changes in appearance that are associated with operation of the functions keys in the invention. Fig. 2A shows a portion of the menu bar 102 of Fig. 1 after function key Fl has been pressed. Pull-down menu 106 for the item "Fl File" is therefore displayed and may be indicated as active by being highlighted in a different colour from the menu bar 102 or by some other method.

As shown in Fig. 2B, upon pressing Fl the "Fl Open previous" command 107 on the pull-down menu 106 is highlighted in a different colour from the rest of the menu and the listing 108 associated with that command is displayed. The listing 108 may be indicated as active by being displayed in a different colour from the pull-down menu 106.

As shown in Fig. 2C, upon pressing Fl again, the item 109 on the listing 108 which has the Fl designation is highlighted to show it is selected. This item is then launched.

Figs. 3A, 3B and 3C show a sequence of images

illustrating the changes in appearance that are associated with operation of the Esc key in the invention. Each list of function keys also includes an Esc option for returning to the previous level. Fig. 3A shows a portion of the menu bar 102 of Fig. 1 after function key Fl has been pressed. Pull-down menu 106 for the item "Fl File" is therefore displayed. At the top of the pull-down menu there is an Esc option 110 which has an upward arrow which indicates that pressing the Esc key will return to the menu bar 102. As shown in Fig. 3B, if the Esc key is pressed when the pull-down menu 106 is active, the Esc option 110 is highlighted. The pull-down menu thus disappears and the menu bar 102 becomes active again, as shown in Fig. 3C.

If there are more than twelve items or commands for a pull-down menu or listing, the interface may automatically allocate one of the function keys, e.g. F12, to launch a supplementary list. Fig. 4 shows this arrangement for an example of the listing 108 shown in Fig. 1. The listing 108 using function keys Fl to Fll to designate the first eleven items. Function key F12 is used to display a first

supplementary listing 112. Depending on the number of items, the first supplementary listing 112 may also use function key F12 to display a second supplementary listing and so on.

Alternatively, function key F12 may open a plurality of supplementary listing, e.g. to show as many as possible previous items. In this arrangement, the F12 key may be used simply to move between already open supplementary listings. Thus, in the example shown in Fig. 4, function key F12 may be used to navigate forward along (i.e. render active) each supplementary listing of 11 recent files. The item desired can be selected by pressing the appropriate function key when the supplementary listing containing that item is active. The Esc key would be used to navigate back through the

supplementary listings. If you pressed Esc while on the first of the fly-out menus, then they would all close at once. They are either all displayed, or none displayed, or a number of the user's choosing are displayed or not displayed, i.e. one, or four, etc. As mentioned above (but not shown) the active supplementary listing may be highlighted in a different colour .

Fig. 5 shows a similar arrangement of supplementary listings to Fig. 4 but for the activity of selecting a font (e.g. F2 F8 from the menu bar) . In this case the active supplementary listing is highlighted in a first colour, and the selected font is highlighted in a second colour.

Figs. 22 to 24 show examples of a two column QCB which is a development of the QCB arrangements discussed above. In Fig. 22, a QCB 700 is split into a first column 701 and a second column 702. Each column 701 and 702 in this example has six rows and 12 sub-columns, and therefore 144 commands are displayed on the screen. As is shown in Fig. 22, selection by the user of a row 703 can be performed by depressing the relevant function key (in this case 'Fl' ) . In this example, the selected row 703 is highlighted in a different colour to the unselected rows. Fig. 23 shows a different example where, along with highlighting, the selected row 703 is shown with a shadow 704 below the selected row, indicating it is at a different height to the unselected rows. If the user then selects a column within the selected row 704, in the case of Fig. 24 this is F5' , a sub-row 705 appears. This sub-row 705 can be a fly out menu, and also may have a shadow indicating height with relation to the QCB. The use of sub-rows in this example can increase the number of available commands to

1,728. The user may navigate through use of the function keys and the Esc key as discussed previously. Notably, the Esc key can be used to navigate back through selected rows and columns. The two column QCB can be used as an alternative to the pull down menus discussed with respect to Figs. 4 and 5. For example, the user may indicate through depressing *F12' twice in quick succession that they wish to navigate through the two column QCB rather than the sequential pull down menus.

In a development of the menu interface discussed above, navigation may be facilitated by additional audio prompts. The audio prompts are preferably spoken phrases, e.g.

corresponding to the contents of the menu bar, pull-down menu, etc. Preferably, a different pitch or tone is used to indicate the level of the interface which is active. For example, an older voice may be used when the menu bar is active, whereas a younger voice may be used when the pull down menu is active. Alternatively the different levels may be indicated by male and female voices, and/or by using different musical tones, e.g. in the background to the spoken

information .

The audio prompts may enable the invention to be used by blind and partially sighted users. If audio prompts are enabled, pressing a function key a first time will cause playback of an audio prompt for the item on the active list that is associated with that function key. For example, if the user presses Fl, the interface may play back a prompt that says "Fl File". If they wish to select that menu, they press Fl again, and hear "Fl File selected". The pull down menu is then displayed, and the user can then press function keys to hear the commands available in the pull down menu, in a younger voice, so they know they are in the pull down menu, not the menu bar. For example, the user may press Fl and hear "Fl Open Previous", or press F2 and hear "F2 Open", and so on. When they hear the command they wish to use, they just press the same function key again, and will hear "F2 Open selected", etc.

An advantage of this feature is that it may enable blind users to operate all programs covered by the function-key driven interface of the invention without any need to modify those programs specifically for blind users.

Fig. 6 shows a function key-driven pull-down menu interface 200 that is another embodiment of the invention. This embodiment is similar to that discussed above with reference to Fig. 1. However, whereas the interface shown in Fig. 1 is intended for use within a given application, the embodiment shown in Fig. 6 is intended for use on a user's desktop. Similarly to Fig. 1, the menu interface 200 comprises a horizontal menu bar 202, which in this case lists appropriate functions for desktop activities. For example, the menu bar 102 may be provided with conventional Control Panel option under the Fll key, and system commands, such as Task Manager, Run, Command Prompt, etc. under F12.

Using the menu interface 200 may mean that no icons are needed on the desktop, although they can also be displayed, if wanted .

With the menu interface 200, it is possible to open programs, documents, music files, pictures, etc. by operating the function keys instead of having to search through lots of icons, or navigate through mouse- or cursor-controlled menus.

As mentioned above, the menu interface 200 of the invention may be used in addition to native navigational tools, so the conventional mouse-driven and touch navigation options may still be available on the user's device.

In principle, the menu interface gives direct access to 144 separate items using only two keystrokes. It may thus represent a 12x12 matrix of function keystroke pairs. In practice, to make the interface extendable, the function keystroke pair F12 F12 may be reserved to open a supplementary command bar (not shown), e.g. to be displayed directly beneath the menu bar 102. Thus, if there are more than 143 programs or files to be included on the desktop menu, the additional items can be included on a supplementary command bar. This arrangement can be repeated as often as necessary, i.e. the user can create as many supplementary command bars as they want, and these can be displayed in any order the user likes under the main desktop menu bar.

The concept of supplementary command bars is discussed below in more detail with reference to Figs. 12 to 15.

To change the programs or files that are displayed in each pull-down menu, or to change the menu titles, the user may hold down the function key, rather than tap it. Holding down the function key triggers the Edit animation shown in

Fig. 7. As the function key is held down, an "Edit..." button 206 rises to cover the button 204 that indicates the item associated with the depressed function button. If the function key is released before the "Edit..." button 206 completely covers the button 204, the operation is terminated and the "Edit..." button 206 slides back down. The user therefore has a graphical indication of whether or not the function key has been held down long enough to trigger the Edit functionality. Any suitable graphical indication may be used for this purpose, e.g. the new button can come down from the top, left, right, fade in, etc. The hold duration may be less than 1 second, preferably less than 0.6 seconds, e.g. 0.5 seconds. The hold duration may be adjustable by the user.

Once the "Edit..." button 206 had covered the button 204, a dialogue box (not shown) or file manager (not shown) opens. The dialogue box may list the available items in groups of eleven associated with function keys Fl to Fll, with function key F12 reserved to launch a supplementary list if there are more than eleven available items. The file manager may operate as shown in Fig. 19 below. The user may thus use the function keys to select or navigate within the dialogue box or file manager to select a program or file.

Pressing the Esc key when the menu bar 202 of the desktop menu interface 200 is active may launch the shutdown dialogue box 208 shown in Fig. 8. From the shutdown dialogue box, the user may press function keys in order to select Fl Program Picker (application switch control screen) , F2 Switch User, F3 Log Off, F4 Lock, F5 Sleep, F6 Restart or F7 Shut Down. If F7 Shut Down is selected, for example, a final confirmation dialogue box 210 is displayed, from which the user may press Enter to shut down or Esc to cancel. All buttons from F2 to F7 may have similar confirmation dialogue boxes. The use of such boxes (with on screen buttons for all keyboard commands) throughout the interface of the invention means that the interface is still usable by mouse only users and touchscreen users, as well as keyboard users (and blind users) .

Figs. 25 and 26 illustrate examples of interfaces using smart button borders. In Fig. 25 solid black lines 801 represent the smart button borders, and are displayed

partially bounding each button of a dialogue box 800. These smart button borders represent 'walls' to a cursor 803 in that the cursor 803 is unable to pass through a border i.e. the borders acts as a barrier to prevent movement of the cursor 803 past the borders. The borders may prevent movement of the cursor in one direction only, i.e. away from the button as viewed from an initial position of the cursor. For example, the borders may be located on the opposite side of the button to a starting position of the cursor 803 and the direction mentioned previously may be one passing from the starting position of the cursor 803 through the border. This is illustrated in Fig. 25 where the border around button 804 is in an elongate 'U' shape, whereas the border around button 805 is in an L' shape. This means that the user does not need to decelerate the cursor 804 themselves in order that it remains within a button. It would be advantageous for the cursor 803 to be placed centrally in the dialogue box 800 automatically when the dialogue box 800 is created, so that all buttons are accessible without having to initially pass through a smart button border.

Fig. 26 illustrates a different dialogue box 900 using the smart button border principle. In this case, the buttons are lined around the edge of the dialogue box. Again the user need only direct the cursor towards the button they wish to press, and does not need to decelerate the cursor for it to remain within the active zone of a button.

Another example of the use of a smart border is shown in Fig. 27. Here a smart menu border 850 is positioned between the primary menu bar 851 and title bar 852 of an application. When the user moves the cursor towards the primary menu bar 851 it engages the smart menu border 850 and is halted. In some examples the user may be able to traverse the smart menu border 850 by moving the cursor upwards again once it has been halted by the smart menu border 850. For example, the user may need to push the cursor against the smart menu border 850 for a time e.g. 0.3 seconds. In doing so the smart menu border 850 may disappear or fade from view after the time has elapsed. Once the user moves the cursor back below the smart menu border 850 it may reappear. In the example shown in Fig. 27, a gap 853 is provided in the smart menu border 850 to allow the user to manipulate the title bar 852 e.g. minimize, close, or move the dialogue box. This gap 853 could be provided instead of or in addition to allowing the cursor to traverse the smart menu border 850.

In these examples of the smart border, it may be that the border provides a barrier to the cursor only in one direction. For example, in Fig. 27 it may be that if the cursor is above the smart menu border 850 (i.e. within the title bar 852) the cursor can traverse the smart menu border 850 without being impeded.

Fig. 9 illustrates an example of a dialogue box 300 that is fully operable using function keys. Although the example shows a print dialogue box, it can be understood that the principles of the invention are equally applicable in all other types of dialogue box. The dialogue box utilises function keys for every available operation, in order to negate the need for the mouse. Function key-driven dialogue boxes may make repetitive tasks easier by removing the reliance on cursor-based selection. When the audio

functionality discussed above is implemented, the interface also allows blind users to use the dialogue boxes without the need for a second interface.

Where there are more than 12 available operations, the dialogue box may be sub-divided into a plurality of

activatable portions, and one of the function keys (e.g. F12) may be reserved to move to subsequent portions (either right or down) , the Esc key being used to move to previous portions (either left or up) .

In the dialogue box 300 there is a first portion 302 in the upper part of the dialogue box 300 and a second portion 304 in the lower part thereof. The first portion has eleven operations labelled with function keys Fl to Fll and a "More" button labelled with F12 that enables a user to move to the second portion 304. In Fig. 9 the second portion 304 is active, indicated by the fact it is highlighted in a different colour. The second portion 304 includes an Esc key button for returning to the first portion 302.

The dialogue box may be closed by pressing the Esc key. The position of the function keys in the dialogue box may be selected to mirror as closely as possible their physical position on the keyboard. Thus, the Esc option is positioned at the top left corner of the dialogue box, mirroring its position on the computer keyboard. The Enter option, which may be used to execute operation following setting of the relevant parameters using the function keys, may be located towards the bottom right.

We read from left to right, top to bottom, so the top left of the dialogue box is the 'beginning' , thus

Escape/Cancel is top and left, and Enter is bottom and right. This also mirrors the physical position of the keys on a keyboard .

It is desirable for the function key elements to go left to right across the dialogue box, from Fl to F12 (or Fl to Fll if more than 12 are needed) .

In Fig. 9, the options labelled Fl and F2 are 'Multi- Toggle' buttons, that enable the user to cycle through the options very quickly, where only one option can be selected at a given time. Highlighting is used to indicate the option currently selected.

In the top right hand section of the dialogue box is a graphical representation 305 of the number of pages in the document to be printed. Where a subset of these pages is selected to be printed (pages 1 and 2 in Fig. 9) , those pages are highlighted (e.g. in a different colour) in the graphical representation 305.

The options labelled F3 to F8 are parameter adjusters arranged in pairs around corresponding parameter boxes. The parameter adjusters are used to step the parameter in the parameter box up or down respectively. When pressed, the number in the parameter box is immediately selected, which enables the user to enter their own value using the keyboard if they prefer. Thus F3 to F8 can also be used as selectors, to quickly select the value the user wants to change by typing a value .

The option labelled F9 is a simple radio switch that is toggled on and off by pressing F9. Options F10 and Fll in the first portion 302 and options Fl to F4 in the second portion 304 launch further dialogue boxes, which may operate in a similar manner.

Fig. 10A shows another example of a print dialogue box 301 that is fully operable using function keys. The print dialogue box 301 is the same as the one shown in Fig. 9 except that the Fl select printer option in the first portion 302 is arranged to launch a further dialogue box, e.g. because the number of available printers is too large to list within the main dialogue box. A further dialogue box may be used for any of the options in the main dialogue box in order to save space or avoid clutter. For example, a further dialogue box may be desirable whenever there are more than four items to select, because a Multi-Toggle option may take up too much space. Fig. 10B shows a further dialogue box 306 that is launched when the user selected function key Fl from the dialogue box 301 shown in Fig. 10A. In this example, the further dialogue box 306 presented a list of more than twelve selectable options. In order to operate under the schema of the invention, the list is therefore subdivided into two activatable portions comprising an upper block 308 and a lower block 310. The upper block includes one function key option (F12 in the example) reserved for moving from the upper block 308 to the lower block 310.

When F12 is pressed while the upper block 308 is active, the dialogue box changes configuration as shown in the right hand diagram in Fig. 10B. The function key legends may disappear from the upper block 308, and appear instead on the lower block 310. This may indicate that the lower block 310 is active. Moreover, the F12 button disappears from the upper block 308, and an Esc button appears on the lower block 310. The Esc button enables the user to move back to the upper block 308, whereupon it disappears and the upper block 308 becomes selected again.

In this example, the user presses F3 when the lower block 310 is active in order to select 'Laserjet VI'. The new selection 311 is shown near the top of the dialogue box under 'Current: ' . The dialogue box is then closed by pressing the Enter key 313.

In order to facilitate switching between applications, the menu interface of the invention may configure the Esc key such that if it is depressed when the menu bar of an

application is active, a tiled application switch interface 400 as shown in Fig. 11 is launched. This application switch interface is referred to herein as the 'Program Picker' . As an alternative to the Esc key being used, a different key such as the CTRL, tilde, PrtScn, Scroll Lock or Pause keys may be held down or pressed to activate the 'Program Picker' . The tiled application switch interface 400 presents a 4 x 3 matrix in which each cell of the matrix is associated with an open application and labelled with a function key. A miniature screen shot of the application in question may be included in each cell.

In use, functions keys Fl to F10 can be pressed to instantly go to an open application shown in the relevant cell. Function key F12 is reserved for going back to the desktop, or to open a supplementary display, containing up to another 12 open applications, if there are more than 11 programs running. F12 can be reserved in this way on as many supplementary displays as necessary to show all open

applications .

In this arrangement, the two keystroke sequence Esc Fl would take you to the previous program you were looking at.

Pressing the Esc key from the tiled application switch interface 400 closes it, and takes the user back to the program that was active before the tiled application switch interface 400 was launched.

If the interface is running on a Windows® operating system, function key Fll may be reserved to provide a link to the Windows Power User Menu (i.e. Win-X commands) . Pressing Fll would open a dialogue box, in which you could select the Power User Menu items, using function keys.

As an expansion of the Program Picker, non-function keys could be utilized and the cells representing applications disposed so as to mirror the layout of a standard QWERTY keyboard. In a similar development, Power User Menu commands (such as Device Manager) could be mapped directly to Program Picker cells.

In the expanded or any other example of the Program Picker, a cell representing an application may have different graphical effects applied to indicate the status of the application. For example, applications not presently running may be represented by a greyscale screenshot of the

application or its icon. Whereas applications presently running, and open, may be presented in full colour. As an alternative to modifying the hue and saturation, icons may be placed in the corner of each cell to indicate the current status of the program.

As discussed above, it may be desirable to provide additional menu bars if there is not enough capacity in a single menu bar to cover all desired options. However, in a development of this idea, the menu interface of the invention may include additional menu bars that provide short cut (i.e. single keystroke) access to favourite commands. This type of additional menu is referred to herein as a Quick Command Bar (or QCB) . Each QCB differs from a primary menu bar because it does not have pull down menus. Instead a QCB simply displays selected commands, which can therefore be accessed directly with one function key press when the corresponding QCB is active.

Each QCB includes an Esc button to move up to the QCB immediately above it (when more than one QCB is open) , or to the primary menu bar if only one QCB is open. The currently active bar is highlighted.

Figs. 12A to 12C show how a QCB is launched from a primary menu bar. Fig. 12A shows the result of pressing F12 F10 when the primary menu bar 102 is active. This two- keystroke sequence launches a dialogue box to select one or more QCBs to be displayed. This dialogue box is discussed below with reference to Fig. 13.

Once a QCB is selected, it is displayed as a horizontal menu bar 120 under the primary menu bar 102. The QBC has an ID label 122 at its right hand end.

It is possible to have more than one QCB open at the same time. In Fig. 12C, a second QCB 124 has been opened

underneath the first QCB 120. The active QCB is highlighted. The Esc key is configured on each QCB to navigate up to the next menu bar. In order to get to the primary menu bar 102 from the second QCB 124, it is therefore necessary to press Esc Esc. The highlighting that indicates the active menu moves each time Esc is pressed.

Similarly, the F12 function key is reserved on each QCB to navigate down to a QCB located below. Where no QCB below is present, there may be no F12 option (see Fig. 12B) . The text in the F12 command position of the F12 pull-down menu of the primary menu bar 102 will change to say "F12 QCB 1 T" or "F12 QCB 2 T", etc. depending on which QCB is directly below the primary menu bar 102.

If more than one QCB is running, the position of the F12 command in the F12 pull-down menu of the primary menu bar 102 will state "Show QCB" or "Show QCBs" if none are currently displayed under the primary menu bar 102. Thus, pressing F12 F12 will thus display any running QCBs, and also immediately make the topmost QCB active.

Function key Fll in the F12 pull-down menu of the primary menu bar 102 will say 'Hide QCBs' , so that the user can immediately hide all displayed QCBs. Function key F10 in the F12 pull-down menu of the primary menu bar 102 will launch a 'Quick Command Bars' dialogue box, discussed below with reference to Fig. 13.

Thus, when the primary menu 102 is active and more than one QCB is running, the first QCB may be selected (made active) by the two-keystroke sequence F12 F12. The second QCB can then be accessed by pressing F12 again.

If the second QCB 124 is active, the first QCB can be selected (i.e. made active) by pressing the Esc key

There is no limit to the number of QCBs that can be used. Each QCB may be fully customisable using one of the editing techniques discussed below.

In one embodiment, a plurality of QCBs may be running, but only a subset (e.g. one or more) of them may be displayed at any given time through a viewport, in order to save

vertical screen space. The user may move between the running QCBs using the Esc key and F12 as discussed above. The selected QCB may then scroll up, while the previously selected QCB scrolls up out of sight. This enables multiple QCBs to be open (i.e. running) at the same time without taking up too much screen space. The nature of this multiple QCB display may be configured by the user in the QCB viewport, which is discussed below with reference to Fig. 15C.

Fig. 13 shows a QCB selection dialogue box 550. In an upper portion thereof a user can select a number of already configured QCBs using radio buttons that toggle on and off through pressing a corresponding function key. Once the desired QCBs are selected, the user presses Enter and the selected QCBs will automatically be displayed under the primary menu bar. The first of the selected QCBs will be active, not the primary menu bar.

In a lower portion thereof, a user may create additional QCBs by pressing function key F6. A previously created custom QCB is selectable using function key F5. In this example, function key F5 is a dual state button, in that it has two operations associated with it. The operation that is

performed depends on the duration of the key press. Thus, if function key is tapped (short duration) , the custom QCB is selected/deselected. If function key F5 is held down, the

'Edit Quick Command Bar' dialogue box for that particular custom QCB is launched. Note that, as before, the F5 box, radio button and 'Colour editing' label box in it are all clickable .

The title (label) of each QCB may be changed by providing a suitable option within the create and edit options. For example, function key F12, which is reserved when a QCB is active for downward navigation between QCBs may be used in the edit or create state to change the QCB label.

The custom QCBs may be edited by selecting a dedicated edit option present in the lower portion of the QCB selection dialogue box 550, e.g. by pressing function key F7, or by pressing and holding the function key associated with the QCB, which will then invoke a Dual State Button, as shown in Fig. 7. Either of these actions may launch a separate QCB edit dialogue box. In the embodiment shown in Fig. 13, the first four QCBs may be preset and uneditable.

A further option may be presented in the lower portion of the QCB selection dialogue box 550. This option may allow the user to map the first eleven commands of any pull down menu of the currently open program, to a custom QCB. The user will then be able to view each of the pull down menus of the program, within another dialogue box, and press 'Enter' when the one they want to map is open. This will then appear in the QCB selection dialogue box 550 as a new custom QCB. All custom QCBs can be renamed by the user, and the ID label 122 will appear at the right hand end.

Fig. 14 shows a first editing technique for a QCB. This editing technique may be available at any time when the QCB is active, and/or after the user has confirmed that an edit action is required. To begin editing, the QCB 120 is first displayed to the user. The user may then change the contents of a given function key by holding down the relevant function key for a predetermined length of time (e.g. at least 0.2 seconds) . During this time the button will fill with another colour and the word "Edit...", as shown by the animation sequence 130 in Fig. 14.

If the user lets go before the new "Edit..." button 132 has risen completely up, it scrolls back down out of sight. In this way, the user can learn by accident that there is a second option for these buttons, if they accidentally hold them down a split second longer than usual, but are not then committed to the "Edit..." command, unless they continue to hold down the key.

After the "Edit..." button 132 has risen up to completely cover the button underneath it, an Edit dialogue box 134 is displayed. The Edit dialogue box 134 may include instructions on how to select a new command to display on the QCB.

Effectively the edit function activates a dummy version of the primary menu bar which the user can navigate using the function keys to arrive at a command that is to be selected for the QCB. For example, if a user presses Fl, the main menu bar first pull-down menu is displayed, then the user presses F4 for 'Save', and ' Fl Save' then appears on the QCB, in the place of ' Fl Bold' . The new selection may flash three times in the QCB so that the user can see it has gone in.

Figs. 15A and 15B illustrate an alternative QCB edit technique. In this technique, instead of or in addition to the ability to hold down a function key to edit it, the QCB may include a dedicated command 138 (e.g. associated with function key Fll) for editing the QCB. Upon selecting this dedicated command, an Edit QCB button box 140 is displayed.

This box may lead the user through the steps required for editing the QCB. The Edit QCB button box 140 contains a copy of the currently selected QCB.

In a first step the user must select one of the QCB commands to be changed. In Fig. 15A, the command "Fl Bold" is selected. After this selection is made, the edit function displays a dummy version of the primary menu bar which the user can navigate using the function keys to arrive at a command that is to replace the selected command. This is done by fading in the primary menu bar above the dummy copy of the

QCB being edited as shown in Fig. 15B. In the illustrated example, the user then navigates to the Fl File F5 Save command. Once this command is selected, the Edit QCB dialogue box closes and the real QCB Fl position updates to display 'Fl Save' and flashes three times to confirm the change.

Throughout each step, nomenclature is updated to tell the user what to do next, as shown in Fig. 15A and Fig. 15B.

Fig. 15C shows a QCB viewport dialogue box 144. This dialogue box allows the user to configure the appearance of the QCBs that are running so that only some of the currently active QCBs are actually visible, through a QCB viewport underneath the main menu bar. At the left hand end of the box is a summary of the currently running QCBs . In the example, the function keys are associated with different display options. Thus, function key Fl is associated with the

"display 1 of 3" option, function key F2 is associated with the "display 2 of 3" option, and function key F3 is associated with the "display 3 of 3" option. Further options can be displayed if more than 3 QCBs are running.

Fig. 15D shows a number of partial screen shots to illustrate a scrolling transition between two QCBs in a viewport 146 located under the menu bar 102. The viewport in this example is wide enough to show a single QCB. In the top pane of Fig. 15D, a first QCB 148 (labelled as ==QCB 1==) is displayed. Function key F12 on the first QCB 148 is assigned to cause a second QCB to be displayed in the viewport. If function key F12 is pressed while the first QCB is active, the F12 button is highlighted and the first QCB scrolls up as shown in the middle pane of Fig. 15D. When the scrolling transition terminates, the second QCB 149 is displayed in the viewport, as shown in the bottom pane of Fig. 15D.

Fig. 16 shows another example of how function keys may be used to improve a data entry interface. Here the set of data fields is broken down into groups (or blocks) that can be navigated using the Esc and F12 function key. The active block may be highlighted in some way. The function keys can be used as a means for jumping the cursor directly to the relevant field for rapid data entry.

Fig. 17 shows an example of how function keys can be used to control window sizing on a desktop viewing area. This functionality may be accessed from a pull-down menu associated with a running application. Upon selecting the appropriate command from the program menu bar or QCB, the interface may operate to show four or more screen resize options as shown in the top part of Fig. 17. Each option may indicate how the currently active application window will be resized and/or repositioned and may be associated with a respective function key. Pressing the relevant function key will cause the display to change as indicated.

The process may continue by displaying an option to select which other window you wanted to resize and/or

reposition to the remaining part of the display. To do this, the process displays a dialogue box as shown in the lower part of Fig. 17, with all the running programs shown with a function key next to each one. Selection of the appropriate function key will cause the respective application window to appear in the relevant position. The displayed

representations of the new program size and position would change depending on which resizing/repositioning option was chosen in the previous step, in the upper part of Fig. 17. Other options in the same dialogue box could allow the user to move the current application window to another monitor on a dual or triple monitor setup.

Fig. 18 illustrates an example of a graphical technique of showing if you are about to "double tap" an item - i.e. that the second tap you are about to make is going to run the item. After the first tap, a small animated bar 150 is displayed at the right hand end of the item. The animated bar 150 may rapidly fall downwards during the time left to hit the key again, to double tap. Other animations may be used, e.g. a bar shrinking to the left or right, or fading out, etc. The duration for recording a double tap may be configurable by the user .

In the present invention, the ability to double tap an item is not required in any of the preceding discussion.

However, the double tap functionality is useful for running an item or opening a folder when the menu interface of the invention is used in a file manager, as discussed below.

Figs. 19A to 19F illustrate navigation and file

manipulation within a file management structure. As shown in Figs. 19A and 19B, the currently accessible (i.e. active) folders /files are always contained between two horizontal bars

(which may be coloured or otherwise distinct) , the top being 'Esc' to go up (if available) and the bottom containing 'Enter - Expand current folder' or "Enter - Collapse current folder' , ^λ TAB - Focus right pane' or "TAB - Focus left pane, 'Fll Commands... ' and ' F12 MoreT' (if available) . In addition, the active list of files or folders are each labelled with a respective function key to permit quick selection (single tap) or execution or expanding/collapsing a folder (double tap) . This configuration makes it very easy for the user to see which set of folders/files are currently accessible by the function keys. Fig. 19A shows a window 600 of a file manager application in which the leftmost pane 602 of the window shows a folder tree under the "Desktop" primary folder. The right hand pane 604 shows details of the contents of the selected folder in the leftmost pane 602, which in this example is the primary

"Desktop" folder itself.

To navigate to a sub-folder of the "Desktop" primary folder, an appropriate function key is pressed. Fig. 19B shows the result of selecting function key F3. In the leftmost pane 602, the item labelled F3 is highlighted, and its contents are shown in the right hand pane 604. Because it is now possible to navigate to a higher level in the folder tree structure, an upper Esc band 606 appears above the Fl labelled folder. A lower band 608 is located at the bottom of the active list. Pressing the Esc key in the arrangement shown in Fig. 19B thus returns to the "Desktop" primary folder .

To open the selected sub-folder, a double tap operation is performed, as discussed with reference to Fig. 18 above, or the Enter key can be pressed, as indicated in the lower horizontal bar. A result of double tapping the F3 labelled subfolder "This PC" in Fig. 19B is shown in Fig. 19C. The leftmost pane 602 now displays a list of items available within the "This PC" folder. This is now the active list, and is labelled with the function keys. The active list is bounded by the upper and lower bands 606, 608. Again there are less than ten items, so it is not necessary to utilise the F12 option to navigate down the list of items.

Although the double tap operation opens the folder structure, it does not change the selection of the folder, so the right hand pane is unchanged between Fig. 19B and 19C.

However, if a function key, e.g. Fl, is then selected from the active list, that item is highlighted and its contents are displayed in the right hand pane as shown in Fig. 19D.

In order to access the right hand pane, the user can press the Tab key. A result of doing so is shown in Fig. 19E. The active list is now in the right hand pane. As there are more than ten items in the list, the lower band 608 now includes the "F12 More" command, which allows the user to select the next ten items in the list, by moving the upper and lower bands 606 and 608 or by scrolling the contents of the lower ten items up. Once the user has begun moving down the list, an "A Esc" band 606 will appear above the first item (which begins with "Fl") to allow the user to navigate back up through the list, in the same manner as shown in Fig 19B. For very long lists, it may be desirable to provide a means of more quickly jumping to a location in the list. To do this, the user may open the "Fll Commands" QCB as shown in Fig 21 by pressing Fll (as shown in the lower band 608), and then select the "Fll Jump to..." command in the QCB. A dedicated "Jump To" QCB will then open beneath the "Commands" QCB, and may assign a fixed percentage or fraction to each function key, whereby a selection of that key may move directly to the equivalent position in the list, for example "Fl Start" will move the user to the beginning of the file list, "F2 10%" will move the user 10% down the list, etc. One of the function keys may be reserved to open a dialogue box to permit a user to enter the first characters of the item name they wish to jump to - the list will update in real time to coincide with what the user has typed in. The user then presses Enter to close the dialogue box and go to that part of the list.

As shown in Fig. 19F, the user may select (single tap), for example, multiple items in the active list. When all relevant items are highlighted, manipulation or management commands can be accessed by pressing Fll to launch the

"Commands..." QCB (not shown) . To deselect an item, the user presses its function key again, after the Double Click

Indicator Bar has vanished (to prevent double clicking) .

Figs. 20A to 20C show the operation of the Esc key in the file management scenario of Fig. 19. In Fig. 20A the leftmost pane shows that a subfolder of the "This PC" folder is highlighted in the active list. If the Esc key is pressed at this point, the active list moves up a level to the subfolder of the "Desktop" primary folder, as shown in Fig. 20B. In order to collapse the subfolder tree under the "This PC" folder, the Enter key is pressed, and the subfolders of the

"Desktop" folder (F4 to F7) move up underneath the "This PC" folder. The result of this action is shown in Fig. 20C.

Fig. 21 shows a screen shot of a file manager application in which the present invention is used to select a large range of files /folders . In this technique a first file of a range is selected by pressing its function key (e.g. F5 in Fig. 21) . Then, the user may press "Fll Commands..." then "F5 Select from... to...". This may cause the currently selected file/folder to slowly flash (e.g. fade from white on black to black on white) , and a notification box will open which says "180.zip selected - navigate to the end file in your selection and press its function key". The user then navigates to the file at the end of the range they want to select, either using F12, Esc, or "Fll Commands..." then "Fll Jump to...", or any conventional key-based navigation tool, e.g. the arrow keys, and the Home, End, PgUp and PgDn keys. Once the active range includes the last file of the range, the user selects it by pressing its function key, and then the range from the first file selected to the last file selected will be highlighted in some way. The notification window will update to say "180.zip to 1942.zip selected: 16 files" so that the user can see how many files are selected.

Then the user can press Fll Commands... Fl Cut, etc. to perform operations on the selected range of files.

Alternatively or additionally, the user may hold down the function key for the file they wish to begin the selection range with, and a dual state Button will rise up over the file name, saying "Select from...". The currently selected file/folder will slowly flash (fade from white on black to black on white) , and a notification box will display which says "Navigate to the end file in your selection and tap its function key". The file at the end of the range can be selected in the same way as described above.

Fig. 28 shows a process in which a floating escape menu 900 is hidden from the user. The floating escape menu 900 comprises three buttons: an escape button 901, move button 902, and hide button 903. This floating escape menu 900 is provided, for example, on a touch screen device where the user is expected to physically touch the location on the display device corresponding to any particular button. The escape button 901 may be the equivalent to the Esc key on the keyboard. The move button 902 allows the user to move the location of the floating escape menu 900, and the hide button 903 allows the user to hide the floating escape menu 900. As is shown in the subsequent images in Fig. 28, pressing the hide button 903 initiates a process in which the floating escape button 900 becomes increasingly thinner. The result of this process is shown in Fig. 29, where a progress bar 910 is shown. The progress bar 910 fills (e.g. gradually with a different colour from the left) with time, as is shown in the subsequent images. When the progress bar 910 is filled, the floating escape button 900 may become visible once more. The time during which the floating escape menu 900 is hidden may be configured by the user.

In Fig. 30, a process is shown after the user has pressed the move button 902. The move button 902 increases in size until it fills the floating escape menu 900. This enables the user to drag the floating escape menu 900 around their screen, without risk of accidentally pressing the escape 901 or hide 903 button. As is illustrated in Fig. 31, whilst the move button 902 occupies substantially all of the floating escape menu 900, a timer bar 920 may decrease in length. When the timer bar 920 is empty (i.e. has a minimal length) the floating escape menu 900 may revert to the original

configuration in which all three buttons are visible.

Alternatively, a section of the move button 902 may be usable as a separate button which initiates the transition to the original configuration.

Fig. 32 illustrates an alternative configuration of a floating escape menu 950, in which only the escape button 951 is initial visible. Ellipsis 952 are provided in a corner of the floating escape menu 950 which, when pressed by the user, causes the floating escape menu 950 to expand thereby

revealing the move button 953 and hide button 954. The floating escape menu 950 in this example can therefore be considered a dual-state menu, as it can transition between a first condensed state and a second expanded state. In this example the floating escape menu 950 can remain in the expanded state until either the user presses a button in the menu, or until a pre-set time has elapsed.

As an alternative to the ellipsis 952, which may be too small a target, the user may press and hold the Esc button for a pre-determined time (for example from 0.3 to 0.5 seconds) . The pre-determined time may be specified by the user. In this example, the Esc button is functioning as a dual state button as discussed above in that it operates as an Esc key or as a means of expanding the floating escape menu 950. Fig. 33 illustrates embodiments of the case indicator. Three screen shots of a text editor 960 are shown, with a cursor 961. A case indicator 962, 963, 964 is provided on or near the cursor 961 to indicate that the user will be typing in upper case. This can be triggered by, for example, toggling to Caps Lock key or holding down the Shift key. In the embodiments shown, the case indicator may be an alphanumerical symbol such as 'C 962, an extension 963 of the cursor 961 that will be clearly visible to the user, or a block of solid colour 964. The invention is not necessarily limited to these particular examples, any indicator may be used so long as it clearly identifies to the user the state of the case.

In summary, the disclosure herein describes an addition to the current menu bar interface, which uses a function key to open each of the pull down menus, displayed in the form ' Fl

- File', 'F2 - Edit', ' F3 - View', etc. When the pull down menu has been opened, all commands within the pull down menu may use function keys in the same manner, allowing the user to access each command by pressing a function key. Function keys are also used in dialogue boxes to select options, such as page number up and page number down in the Print dialogue box, and to select files in all file dialogue boxes. Every command has a clearly displayed function key next to it at all times. The Esc key is displayed and used to go up, left, or Out' of an open dialogue window, menu or program (and is also used on the desktop to open a menu of shut down options, all of which use function keys) , the Enter key is used to go right, down or to select 'Okay' in an open dialogue window or menu. All onscreen displays of functions keys, Esc or Enter are also clickable/touchable, allowing disabled users who can only use pointing devices to use all functions, as well as making all programs touchscreen compatible.

In 'Blind mode', the user is required to press a function key (and Esc or Enter, where appropriate) once to initiate play back of the name of the function key and the command it represents on a voice synthesizer, and again to select the menu title or command. The menu bar titles can be read out in an adult voice, the pull down menu commands can be read out in a child voice, so that a blind user knows which bar they are on. The desktop will also have a pull down menu bar, which the user can customise in any way they like - each title can be customised, each 'command space' in the pull down menus can contain any program or file the user wants - for example, a picture or song file can be entered into one of the pull down menu spaces so that the user can access their favourite song, or a document they are working on, by pressing ' Fl Fl ' , for example, as soon as they boot to the desktop. The single keystroke paradigm may thus be used throughout the entire computer interface, from the desktop, to programs, to dialogue boxes, and file management.

Claims

1. A control system for a menu-driven graphical user interface, the control system comprising:

a computer keyboard having a plurality of function keys that are not assigned to a specific ASCII printable code;

a processor connectable to receive an input signal from the keyboard, the input signals corresponding to operation of one or more of the function keys;

a memory stored code that is executable by the processor; and

a display for showing a menu-driven graphical user interface,

wherein the code is operable to:

detect a state of the menu-driven graphical user interface ;

determine an operation to be performed in response to a received input signal from the keyboard based on the detected state;

execute the determined operation; and

display on the display an updated menu-driven graphical user interface following execution of the determined operation .

2. A control system according to claim 1, wherein the code is arranged to detect the state of the menu-driven graphical user interface by detecting an active portion of the menu-driven graphical user interface, wherein the active portion includes a plurality of available operations.

3. A control system according to claim 2, wherein the memory stores a current one-to-one mapping between the plurality of function keys and the plurality of available operations, and wherein the code is arranged to determine the operation to be performed by comparing the received input signal with the current one-to-one mapping for the detected active portion.

4. A control system according to any preceding claim, wherein the code is arranged to update the state of the menu- driven graphical user interface after the determined operation is executed.

5. A control system according to claim 3, wherein the code is arranged to update the state of the menu-drive

graphical user interface after the determined operation is executed by:

determining an updated active portion having a plurality of available operations,

assigning an updated one-to-one mapping between the plurality of available operations and the plurality of

function keys, and

storing the updated one-to-one mapping in the memory as the current one-to-one mapping.

6. A control system according to claim 5, wherein the code is arranged to display on the display the current one-to- one mapping for the updated active portion.

7. A control system according to claim 6, wherein the current one-to-one mapping is displayed by providing a label for each function key next to its corresponding operation.

8. A control system according to any preceding claim, wherein the menu-driven graphical user interface comprises a horizontal menu bar having a plurality of selectable menu items, and wherein selection of a menu item causes a pull-down menu to be displayed beneath the menu bar.

9. A control system according to claim 8, wherein the plurality of selectable menu items of the horizontal menu bar is context-sensitive, whereby the code is arranged to detect an active underlying process being executed by the processor, and determine the plurality of selectable menu items of the horizontal menu bar based on the active underlying process.

10. A control system according to claim 9, wherein the underlying process is any one of an operating system desktop, a word processing application, a file management application, a data entry application, a program dialogue box, and a shutdown dialogue box.

11. A control system according to claim 9 or 10, wherein the memory includes a look-up table storing a plurality of selectable menu items for each of a plurality of available processes executable by the processor.

12. A control system according to claim 3 or 5, wherein each mapping between one of the plurality of function keys and its respective available operation is adjustable by holding down the respective function key for longer than a

predetermined length of time.

13. A control system according to any preceding claim, wherein the menu-driven graphical user interface comprises_, an application switch interface having a tiled display, wherein each tile has a function key associated with it, and wherein each open application is assigned to a tile to enable a user to switch to the open application by selecting the

corresponding function key.

14. A control system according to any one of claims 8 to 12, wherein the menu-driven graphical user interface comprises one or more horizontal quick command bars (QCBs) selectively displayable beneath the menu bar, each QCB comprising a plurality of selectable operations that are mapped to the plurality of function keys when that QCB forms an active portion of the menu-driven graphical user interface.

15. A control system according to any preceding claim, wherein the menu-driven graphical user interface comprises a window resizing dialogue box, wherein there are a number of window resize options, each with an associated function to enable a user to switch to a desired window resize option by selecting the corresponding function key.

16. A control system according to any preceding claim, wherein the code may be arranged to perform a different operation depending on whether a single or double tap of a function key is received, and wherein the code is arranged, upon receiving an input signal indicating that a double- tappable function key has been pressed once, to generate a double tap animation for display on the display, wherein the double tap animation indicates the time remaining within which a further tap of the function key will register as a double tap .

17. A control system according to any preceding claim, wherein upon determining the operation to be performed, the code is arranged to trigger playback of an audio signal that is indicative of the operation, and wherein the operation is executed upon receiving an additional confirmatory input signal .

18. A control system according to claim 17, wherein the audio signal includes a voice speaking the operation.

19. A control system according to claim 17 or 18, wherein the audio signal has a different pitch depending on the state of the menu-driven graphical user interface.

20. A computer-implemented method of operating a menu- driven graphical user interface, the method comprising:

detecting a state of the menu-driven graphical user interface;

determining, based on the detected state, an operation to be performed in response to an input signal received from a keyboard, wherein the input signal corresponding to operation of one or more of a plurality of function keys that are not assigned to a specific ASCII printable code;

executing the determined operation; and

displaying on a display an updated menu-driven graphical user interface following execution of the determined

operation .

21. A method according to claim 20, wherein detecting the state of the menu-driven graphical user interface

comprises detecting an active portion of the menu-driven graphical user interface, wherein the active portion includes a plurality of available operations.

22. A method according to claim 21 including assigning or looking up from a computer memory a current one-to-one mapping between the plurality of function keys and the plurality of available operations, wherein the step of determining the operation to be performed comprises comparing the input signal with the current one-to-one mapping for the detected active portion.

23. A method according to any one of claims 20 to 22 including updating the state of the menu-driven graphical user interface after the determined operation is executed.

24. A method according to claim 23, wherein the step of updating the state of the menu-drive graphical user interface comprises :

determining a updated active portion having a plurality of available operations;

assigning or looking up from a computer memory an updated one-to-one mapping between the plurality of available

operations and the plurality of function keys.

25. A computer program product comprising a computer- readable storage medium having computer-executable

instructions stored thereon, wherein the computer-executable instructions are executable by a processor to perform a method according to any one of claims 20 to 24.