FR2928752A1 - Movable man-machine interface, has keys for receiving specific command and alternate different colors, where command initializes selection of unordinary object on screen and keystroke of one of keys initiates selection of selectable object - Google Patents

Abstract

The interface has keyboard with keys for receiving a specific command (CS), digit, letters or symbols relative to an input. The keys alternatively receive different colors (C1, C2), and the command initializes selection of arbitrary object on a screen. A keystroke of one of the keys initiates the selection of selectable object (OS) formed from a color image converts the image into black and white image (INB) and places a partial transparent selection indicator (IS) on the image. The object receives a direction indicator indicating a direction in which the key is found relative to the object.

Description

The invention relates to a mobile human-machine interface based on the keyboard-screen couple. It consists of an improvement of certain aspects of a previous invention which was the subject of patent INPI 02 06510 filed in June 2002, patent INPI 0406579000 filed in June 2004 and patent application 06 10749.

Each keyboard key according to the previous invention receives a command, a numeric character and a variable number of alphabetic characters or other symbols relating to the input. An object on the screen is said active when it has a selection indicator equal to one of the numeric characters of the keyboard and it will be activated by a keystroke on the key whose numeric character is equal to this indicator. The ambiguity created by the double command and alphanumeric meaning of each key is lifted thanks to a BA toolbar which receives at all times all the active commands on the current screen. In addition to the active objects, the screen may contain non-active but non-selectable OS objects. The selection of such an object can be performed by moving a cursor with the direction controls, but it can be done by a specific method of the interface. This method is always initialized by a keystroke on a certain TO key and consists either of placing selection indicators on each selectable object or of displaying a selection checklist over the entire selection space concerned by the selection process. A specific command of the interface O has for role to pass the keyboard from the input mode to the command mode and vice versa. In an alternative embodiment, the keys of the keyboard alternately have the color C 1 and the color C2 so as to provide a checkerboard structure.

The improvements provided by the present invention relate to: initialization of the selection method according to the preceding invention, the method of placing the selection indicators on the selectable objects of the screen, the introduction of direction indicators, Selection shortcut indicators and identity indicators that facilitate the selection process, Predictive mode input, A method of triggering a zoom on an image, A method of triggering the scrolling of a list, A shortcut keyboard for moving the cursor, A method of accessing keyboard shortcuts functional.

Other features and advantages of the invention will become apparent on reading the description given by way of illustrative and nonlimiting example, with reference to the appended figures in which: FIGS. 1 and 2 illustrate two methods of placing selection indicators on color images displayed on the current screen, - Figure 3 shows how direction indicators ID make it easier to identify and hit the key that will select any object on the screen with a selection indicator, - Figures 4 and 5 illustrate the role of IRS selection shortcut indicators in the quick selection of any object on the screen, - Figure 6 illustrates the role of the object ID indicators of objects selectable from the screen - Figures 7 to 11 illustrate the method of entering characters in predictive mode, - Figure 12 illustrates one of the access modes to functional keyboard shortcuts. of the interface.

The object selection according to the preceding invention requires, when the selection space is the entire screen, either to give it a selection indicator which reduces its readability and causes a loss of space, or to initialize the process by a menu command which increases the overall time of the selection. A specific command CS is instead placed on the keyboard according to the present invention, whose role is to initialize the selection of any object belonging to the current screen. According to the embodiment of Figure 1, this command is placed on the # key and is represented by the symbol in the form of mouse pointer e •.

A specific image-inlay method according to the present invention allows a selection indicator IS to be very legibly assigned to a selectable object OS when it is a color image. It consists in converting this image into a black and white INB image, converting the IS indicator into a semi-transparent object and placing it on the INB black and white image (FIG. 1). Each detail of the image remains in these conditions perfectly identifiable and the indicators themselves are legible because their color (eg red) is very different from (gray) image 1NB. In an alternative embodiment, the initial image is again converted to black and white, but a semitransparent mask is placed on the entire image and the selection indicator placed itself above the mask. When the object to be selected is an icon, only the black and white drawing that underlies this icon is retained before placing the selection indicator. The user who has previously identified in color the icon he wants to activate continues to identify it easily thanks to the outline that is presented to him.

The selection of any point of a color image according to the present invention is initialized by transforming the image into a two-color image in the following manner (FIG. 2): the image is previously converted into a black image and white, 35/11 - a homologous selection checkerboard of the keyboard or part of the keyboard is virtually placed on this image, defining a homology of position between each box of this checkerboard and a key T of the keyboard, - if we call Ti and T2 the shades relating to the colors C1 and C2 which are alternately received by the keys of the keyboard, each pixel of the image then receives the same shade, T1 or T2, as that of the color of the key T homologous to the box CA to which it belongs, while maintaining the same brightness and the same saturation, - each box receives a selection indicator identical to the numerical character of its homologous key. This indicator can be full or partial transparency.

Each detail of the image thus transformed remains perfectly identifiable and the color identity is added to the homology of position to make it very easy identification of the key T homologue any CA box of the checkerboard.

The selection according to the previous invention of an object of the screen by the placement of a selection indicator IS on this object is improved according to the present invention by the display of direction indicators (Figure 3). The direction indicator ID of an OS selectable object is a line segment that starts from the OS object and takes the direction of the T key that will select the OS object. It indicates accordingly the direction in which is the key T with respect to the object that it designates and thus facilitates the identification of this key already known by the numerical character that is inscribed there. In FIG. 3, the direction indicators thus eliminate, without affecting the readability of the assembly, the discomfort created by the difference that exists between the vertical position of an icon and that of its homologous key.

In an alternative embodiment of the present invention, the selection of a selectable object OS can also be obtained, as a keyboard shortcut, by a long press on the key T which will select the object OS after the keystroke of the key TO which initiates the selection process. Long press on the T key will have the same effect as the sequence of keystrokes <TO, T>. The existence of this keyboard shortcut is then indicated to the user by an IRS selection shortcut indicator that can be:

- A symbol placed preferably at the top of the screen (figure 4). In the embodiment illustrated by this figure, this symbol represents in dotted line the central part of the keyboard. The text on the screen contains hypertext links with an A-note call. These links are supposed to be the only selectable objects on the screen. The TO key that will initialize one of these objects is the # key that carries the CS () selection command as shown by its presence on the toolbar. The user therefore knows in advance the sequence of keystrokes that will select one of the hypertext links of the text. The Navarre link, for example, with the call of / 11 note 5, will be activated by the keystroke sequence <4, 5>; the presence at the top of the screen of the IRS selection shortcut indicator further indicates that it may also perform a long press on the key 5. -A set of IRSS numerical characters representing the keys that are affected by the selection keyboard shortcut (Figure 5). This set can be either grouped in a corner of the screen (for example at the top right of the screen as before), or placed under the part of the screen where are the selectable objects and in a layout that facilitates the perception of the homology between each object and the key whose long press will select it. Thus, in FIG. 5, the arrangement of the IRSS indicators directs the user's gaze towards the key that will select the target object because the slight horizontal shift between selectable objects of the same column and the keys that select them is compensated by the oblique slope that draws on the screen the IRSS indicators of the said column of objects. It is therefore very clear for example that the OS object of Figure 5 will be selected by the key 4.

The interface according to the present invention is able to provide an indicator of identity IID objects that will be selected after the keystroke of the TO key that initiates the selection (Figure 6). The identity indicator IID of an object OS thus designates the key that will select this object after the TO key, but it does not allow the long press on this key to directly select the object O. It is distinguished from an IRS selection shortcut indicator with a different style or color. The identity indicators IID of FIG. 6, which designate the items of a list, are thus identified by the outline style. Thanks to them the user knows in advance the sequence of keystrokes <T0, IID> which will select a selectable object from the screen. Long press on the IID key remains available for another action.

Predictive writing is performed on the interface according to the present invention by permanently displaying on the screen a selection checker DA rigorously homologous to the two-color keyboard. This checker receives the + selection indicator which is placed on the homologue box of the + key. At each hit of a predictive write key, the underlying software places a number of words on the checkerboard DA as follows (Figure 7-10):

- If there are dictionary words compatible with the first keystrokes of the word, the software puts one of them, preferably the one whose occurrence is most common in a standard text of the language, on the box CO of the damier DA homologue of the key 9 of the keyboard which carries the command space. This box CO preferably receives a cursor (highlighted in FIG. 7). The other compatible words are placed on the adjacent cells of the CO box. Thus in Figure 7, where the user is supposed to want to grasp the word basically, the compatible words / 11 with the first five keystrokes of this word: Found, Found, Found, Honda are placed on the checkerboard. The word founds, the most frequent, occupies the box CO.

- the software places on the remaining boxes of the checkerboard strings compatible with the first keystrokes of the word entry, ie strings which are the beginning of a dictionary word compatible with the said first keystrokes . They are followed by zero, one or two points. When they are not followed by any point, they are necessarily dictionary words like the word Fondettes in Figure 7 (box 7 of the checkerboard). When followed by a single point, these are dictionary words that are not invariable in terms of gender and / or number, as for example the basic word of Figure 7 (box 4). Finally, character strings followed by two points are strings compatible with the first keystrokes of the seizure, such as the fondam chain of Figure 7 (box 2).

- The verbs compatible with the first n keys are displayed on the checkerboard in the infinitive mode, the last letters of this mode being in parentheses (at least in some languages such as French and German): the verb to found appears on Figure 7 in the bottom form (er).

At each step of the prediction mode, the user has the choice between continuing this entry in predictive mode or selecting one of the chains of the checkerboard DA. The selection of the CHO chain located in the CO box will thus be obtained by pressing the key 9 which carries the space character. The selection of another CH chain will be by pressing the + key to activate the entire array followed by a hit of the counterpart T key of the said channel, but the user can also directly select the CH channel. by pressing and holding the T key.

The word grounds placed on the box CO will be definitively selected by a hit of the key 9 which carries the command space: a space will consequently be placed behind this word in the text being entered and the strike that will follow will necessarily the first keystroke of entering the next word. the word "fonda", located just above the word "fonda", will be definitively selected by pressing the + key and then pressing the 6 key, or by pressing and holding the 6 key.

The selection of the foundation chain. , carried out by the sequence of keystrokes <+, 4> or by the long press on the key 4, will involve the display on the checkerboard of a new table consisting solely of the word foundation and its plural foundations. This table is shown in Figure 8: the word foundation is preselected and will be permanently selected by a new key on the key 4. The word foundations will be selected by a key strike 1. In total these two words foundation and foundations could have been entered in 7 keystrokes (including a long press) instead of respectively 10 and 11 with a keyboard with a letter by key, because it is necessary to add / 11 in this case the hit of the key space to the strikes of the entering the word stricto sensu. Note that the plural of a word will always be placed on the checkerboard DA just above this word, except when this word is itself placed on the first row of squares of the checkerboard in which case it will be. mirrored on the box just below. After a brief period of use, the user will know in advance the two keystrokes that will select the word in the singular and the plural word :: he will therefore no longer need to consult the table in Figure 8 to select definitely the target word. In the same way the feminine of a word if it exists will always be placed on the right of this word, except if it is impossible in which case it will be placed on its left always by mirror effect.

Selecting a character string followed by two points will have the same effect as the sequence of keystrokes that should have been done in the predictive mode to arrive at that string. Thus in FIG. 9, the selection of the functionalised chain will place this string on the homologous box of the key 9 and will save the typing of the substring ionnalis when entering the words functionalism and functionalisms.

The CH strings of the DA checkerboard can always be divided into two substrings CH1 and CH2, the first being the longest substring common to the CH chain and the preselected CHO chain, the second being the rest of the chain. CH. Thus the fundamental chain of the array of Fig. 7 has the substring merges in common with the preselected base chain and can therefore be divided into two sub-chains, background and am. In the variant embodiment shown in FIG. 7, the substring CH2 receives a style or a color different from that of the substring CHI. The chain fondam thus appears in the form bottom, in If the style which differentiates CH2 from CH 1 is the style Contour. In the same way the word foundation appears in the form fond5i and the word Honda in the form T L-. This process allows the user to identify more quickly the word he is looking for or the string that will lead him to the word he is looking for. It is obvious that a difference of color is more effective than a difference of style in attaining this.

When all the chains of the checkerboard DA are only followed by zero or a point as in FIG. 10, an audible or visual signal is preferably sent to the user to indicate to him that the word he wants to enter, if he in the dictionary, can now be reached by a selection of this word in the checkerboard. It will then be enough for the user to grasp the word fundamentally to raise their eyes on the table, located just above the keyboard, where he will find very easily this word. However, this will not prevent it from continuing if it prefers predictive mode.

When a verb is selected on the checkerboard DA, the predictive writing software according to the invention displays on this checkerboard the different possible endings of this verb relating to different modes, different times and different people of this verb. Figure 1.1 thus shows the result of the selection of the verb of the first group background (er) on the checkerboard of Figure 7. The different endings of / 11 verbs of the first group are distributed on the checkerboard in an order consistent with the order alphabetical. As of course this distribution will always be the same for the verbs of the same group, the user will quickly memorize the box where the most frequent endings are: he will then tap on their homologous key without having to consult the screen of Figure 11.

In an embodiment variant not illustrated, a zoom can be performed on an image that has just been selected on the screen. A zoom in on the image will thus be achieved by placing a finger D on one of the keys T of the keyboard, provided that this key is neither one of the arrow keys nor the key which carries the command alt, and while maintaining it on this key, then by pressing a second finger D 'on another key T' situated on a line of keys situated above the line of keys to which belongs the key T. As long as the finger D 'is maintained on the key T ', the image will grow gradually. If the user lifts the finger D 'of the T' key, this progressive zoom in is stopped but not canceled. If the user lifts the finger D of the key T while holding the finger D 'on the key T', then rest the finger D on the key T, a progressive zoom out is triggered. The process will be completed and validated when the user has raised both fingers D and D '. A zooming out of the image will also be done by placing a finger D on one of the keys T of the keyboard and by pressing and holding this key, then pressing a second finger D 'on another key T' located on a line key itself below the key line to which the T key belongs. The process will continue in the same way as before.

In an embodiment variant not illustrated, the scrolling of a list is obtained in the following way: Scrolling the list downwards will be done by placing a finger D on the key which carries the direction control upwards and now press this key, then pressing a second finger D 'on one of the two keys located just below on the same column of keys. If the user takes the finger D ', this scrolling will be stopped but not canceled. If the user lifts the finger D and then relies on the T key, scroll down the list will be engaged. The process will be completed when the user has raised both fingers D and D '. Scroll up will be initialized, completed and terminated in a similar fashion.

During a character input session, the keyboard interface according to the previous invention as in the present invention switches from the input mode to the command mode, and vice versa, by pressing the OO command located on the + key in the embodiment variant shown in FIG. 1. When, in input mode, a command C of the keyboard is grayed out on the toolbar, a long press on this command not only executes this command, but also causes the switching of the keyboard in command mode. This long support has the same result as the sequence <OO, C>. The opposite is not possible, however, because the long press on a direction control or on the delete command has the role of triggering the continuous movement of the cursor on the screen. In a variant / 11 not illustrated embodiment of the present invention this disadvantage is overcome by the following convention, which can be programmed and deprogrammed by the user:

- In command mode, a brief press followed immediately by a long press on a direction control or the erase command triggers a first elementary movement of the cursor, then a continuous movement of the cursor in the corresponding direction. The time interval between the short press and the long press must be less than a certain delay, for example the double-click delay. - the long press in command mode of any key T has the same result as the sequence of keystrokes <, T>: switch to entry mode and entry of an alphanumeric character, unless the key T carries a direction command or the erase command and that long press immediately follows a brief press on the same key.

This convention makes it possible to fully use the long key as an input shortcut when the keyboard is operating in command mode. On the other hand, it entails only a very slight slowdown when the user wants to make a continuous displacement of the cursor in the text being entered.

In an alternative embodiment of the present invention, the keyboard shortcut access alt command, located at the bottom right of the keyboard in FIG. 12, is used as follows:

- Long press on the key which carries this command alt results in the display of a homologous checker of the keyboard where each active box receives an indicator of selection IS, - This IS indicator receives; a certain C color or a certain S style when a keyboard shortcut has been programmed on its peer T key; the meaning of the keyboard shortcut may or may not be added explicitly in the box. In all cases, pressing the T key will activate the corresponding keyboard shortcut, - The IS indicator receives another color C 'or another style S' when a keyboard shortcut has not been programmed on its peer T key . Typing it will then initiate a process that will assign a new keyboard shortcut,

Simultaneous pressing of the alt command and any other T key will immediately trigger, if it exists, the shortcut relative to the T key, without simultaneous display of the DA checkerboard on the screen.

Claims (11)

1. Human-machine interface characterized in that the keys of the keyboard receive both a command, a numeric character and letters or other symbols relating to the input, characterized in that the said keys on the keyboard alternatively receive two colors different C1 and C2 and characterized in that one of the CS commands has the role of initializing the selection of any object on the screen. 2. Human-machine interface according to claim 1 characterized in that the typing of the TO key that initiates the selection of a selectable object OS consisting of a color image has the consequence of converting this image into a black and white image INB and placing on the image an IS selection indicator with partial transparency. 3. Human-machine interface according to claim 1, characterized in that the keystroke which initializes the selection of a color image IC transforms this image into a two-color image homologous to the keyboard or part of the keyboard, obtained by converting this color image. IC in a black and white INB image, possibly reinforcing the contrast of this INB image, by partitioning this INB image so as to establish a homology of position between each zone created by this partition and a key T of the keyboard, in giving each pixel of said zone the same shade T1 or T2 as that of the key T, finally placing a selection indicator IS identical to the numerical character of the key T. 4. Human-machine interface according to claim 1 characterized in that after the key to key TO which initiates the selection of object on the screen or a part of the screen, each OS selectable object receives not only a selection indicator IS, but also a direction indicator ID which indicates the direction in which the key T is located with respect to OS whose numeric character is identical to IS. 5. Human-machine interface according to claim 1 characterized in that a global IRS selection shortcut indicator is placed on the screen to mark that each object of this screen which will be selected by a certain key T after the striking of the TO key that initiates the selection process can also be directly by a long press on said key T. 6. Human-machine interface according to claim 1 characterized in that a set of IRSS selection shortcut indicators is placed on the screen to mark that the selectable objects of this screen can be selected by a keyboard shortcut long press , characterized in that each IRSS indicator has a value identical to the numeric character of the key that will select one of the objects of the screen, and characterized in that the disposition of / 11 the set of IRSS indicators is such that it facilitates the identification of the key that will select by long press any selectable object of the screen. 7. Human-machine interface according to claim 1 characterized in that in predictive entry mode checker DA counterpart of the keyboard is permanently placed on the screen, characterized in that each new typing of the word being entered causes the placement on the checkerboard of a set of CH character strings one of which CHO is preselected, each string being followed by zero, one or two points, or a parenthesis, depending on whether it is a dictionary word, a dictionary word whose number and genre will be designated by a subsequent typing, a string that is the beginning of one or more words compatible with the first keystrokes, or a verb whose mode, time and person will be defined by a subsequent tap, and characterized in that the preselected channel CHO will be selected by a tap on the space key, and characterized in that the other CH channels will be selected or by a toggle of the TO key which designates the checkerboard as a whole followed by a striking of their homologous key T, or by a long press on the key T. 8. Human-machine interface according to claims 1 and 7, characterized in that the character strings CH other than the preselected CHO appear divided into two substrings CHI and CI-12 of different colors or styles, the first being identical all or part of the CHO chain, and characterized in that a sound or visual signal is sent by the underlying software when the checkerboard DA has only character strings followed by a maximum point. 9. Human-machine interface according to claim 1 characterized in that the zooming on an image or the scrolling of a list can be obtained by placing a finger on a key T and keeping this finger pressed on the said key, then in placing another finger on another key T 'and keeping this finger pressed on said key. 10. Human-machine interface according to claim 1 characterized in that during a word processing session a brief keystroke, performed in command mode on a direction command or erase command and followed, before does not flow a certain time interval I, a prolonged press on the same command, has the effect of making a continuous movement of the cursor until the user raises his finger, and characterized in that a support long operation in the command mode on a key T has the same effect as the short hit of the selection command followed by the short hit of the key T, unless it carries a direction command or the erase command, which it was preceded by a strike of the same key T and these two keystrokes were separated by a time interval less than the time interval I. 11. Human-machine interface according to claim 1 characterized in that a key T of the keyboard receives an alt command access to functional keyboard shortcuts of the current application, characterized in that the long press of the T key displays a checker counterpart of the keyboard and indicating for each key T 'whether its keystroke will or will not trigger a keyboard shortcut, and characterized in that the meaning of the latter is likely to appear explicitly in the homologous box of the key T'.