WO2023137099A2

WO2023137099A2 - Contextual keyboard input method, system, and techniques

Info

Publication number: WO2023137099A2
Application number: PCT/US2023/010666
Authority: WO
Inventors: Daniel POHORYLES
Original assignee: Bierman, Ellen, M.
Priority date: 2022-01-14
Filing date: 2023-01-12
Publication date: 2023-07-20
Also published as: WO2023137099A3

Abstract

Methods, systems, and techniques for contextual keyboards for improving user keyboard input are provided. Example embodiments provide a Contextual Keyboard System ("CKS"), which enables users to type more accurately with fewer user actions (e.g., pressing of keys and/or switching between modes of a keyboard), and provides enhanced feedback mechanisms to assist in reducing user input errors. For example, the CKS provides emphasis of keys that typically follow the current input based upon a dictionary associated with the current designated language, provides deemphasis of keys that do not typically follow the current input based upon a dictionary associated with the current designated language, provides various forms of autocompletion based upon context and dictionaries, generates suggestions of alternative words that may have been intended by the user based upon key typing position, and the like.

Description

CONTEXTUAL KEYBOARD INPUT METHOD, SYSTEM, AND TECHNIQUES

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Patent Application No. 17/994,972, titled “CONTEXTUAL KEYBOARD INPUT METHOD, SYSTEM, and TECHNIQUES,” filed November 28, 2022; which claims the benefit of U.S. Provisional Patent Application No. 63/266,789, titled “METHODS FOR IMPROVING KEYBOARD USAGE,” filed January 1 , 2022; which applications are incorporated herein by reference in their entireties.

TECHNICAL FIELD

[0002] The present disclosure relates to methods, techniques, and systems for improved keyboarding and, in particular, to methods, techniques, and systems for an improved virtual and/or physical keyboard that supports contextual input and other automated functions.

BACKGROUND

[0003] Mobile devices, especially those of smaller footprints such as mobile phones, prove difficult for users to type on, particularly when the keys, virtual or physical, are small and/or when the user is visually or dexterity challenged. For example, typographical errors are prolific on mobile phones - and typing speed is typically diminished even for users that would otherwise be touch key typists on a traditional QWERTY keyboard. Such struggles may dissuade users who are prone to typing errors from using mobile devices for entering text.

[0004] Some keyboard applications have been created with limited ability to provide alternative virtual keyboards (software-based keyboards) on mobile devices. For example, on APPLE’S IOS operating system, keyboards with alternative themes, some autocompletion capabilities, ability to add emojis, etc. have been made available. The operating system of such mobile devices allows users to configure use of the phone with such alternative keyboards. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 is a block diagram of an example contextual keyboard deployment.

[0006] Figures 2A-2B is an example block diagram of overview logic flow of an example contextual keyboard processing system.

[0007] Figures 3A and 3B is an example flow diagram of processing initial settings on an example contextual keyboard.

[0008] Figure 4 is an example screen display of keyboard behavior settings on an example contextual keyboard.

[0009] Figure 5 is an example screen display of result bar settings on an example contextual keyboard.

[0010] Figure 6 is an example screen display of sound settings on an example contextual keyboard.

[0011] Figure 7 is an example screen display of color settings on an example contextual keyboard.

[0012] Figure 8 is an example screen display of borders settings on an example contextual keyboard.

[0013] Figure 9 is an example screen display of languages settings on an example contextual keyboard.

[0014] Figures 10-16 are example screen displays illustrating dynamic highlighting in response to typing an input sequence of an example word using an example contextual keyboard.

[0015] Figure 17 is an example screen display illustrating feedback settings of an example software contextual keyboard fora non-recommended key input on an example contextual keyboard.

[0016] Figures 18A-18C is an example flow diagram of dynamic highlighting logic of an example contextual keyboard.

[0017] Figure 19 is an example screen display of a context key on an example contextual keyboard.

[0018] Figures 20A-20C is an example flow diagram of logic for implementing a context key of an example contextual keyboard. [0019] Figure 21 is an example flow diagram of logic for selection of aggregate keys for an example contextual keyboard.

[0020] Figures 22A and 22B is an example table excerpt of terms of a dictionary for selection of aggregate keys.

[0021] Figure 23 is an example of aggregate keys on a contextual keyboard.

[0022] Figures 24A-24C is an example flow diagram of logic for implementing aggregate keys of an example contextual keyboard.

[0023] Figure 25 is an example screen display of an adjacent key on an example contextual keyboard.

[0024] Figure 26 is an example flow diagram of logic for implementing adjacent keys of an example contextual keyboard.

[0025] Figures 27-28 are example screen displays illustrating a replacement word area for user inputted errors on an example contextual keyboard.

[0026] Figures 29A-29C is an example flow diagram of logic for implementing replacement words of an example contextual keyboard.

[0027] Figure 30 is an example screen display of expansion key area of an example contextual keyboard.

[0028] Figure 31 is an example flow diagram of logic for implementing expansion keys of an example contextual keyboard.

[0029] Figure 32 is an example block diagram of a computing system for practicing embodiments of an example contextual keyboard.

DETAILED DESCRIPTION

[0030] Embodiments described herein provide enhanced computer- and network-based methods, techniques, and systems for improving user keyboard input, especially on smaller footprint mobile devices. Example embodiments provide a Contextual Keyboard (“CK”) and Contextual Keyboard System (“CKS”), which enables users to type more accurately with fewer use actions (e.g., pressing of keys and/or switching between modes of a keyboard), and provides enhanced feedback mechanisms to assist in reducing user input errors. For example, the CKS provides emphasis of keys that typically follow the current input based upon a dictionary associated with the current designated language, provides deemphasis of keys that do not typically follow the current input based upon a dictionary associated with the current designated language, provides various forms of autocompletion based upon context and dictionaries, generates suggestions of alternative words that may have been intended by the user based upon key typing position, and the like.

[0031] The example contextual keyboards and context keyboard systems described herein provide a keyboard input method for mobile and non-mobile devices. It has initially been deployed on Android and IOS operating systems for mobile devices and can be adapted to other operating systems by “hooking” into the input flow of these systems. The CKS input method adds new elements and capabilities to the current keyboards used in conjunction with mobile and non-mobile devices. Although described herein mostly in conjunction with mobile devices, the contextual keyboard may also be used with hardware keyboards with suitable adaptations. It may be used with keyboards that are physically part of a device such as those connected to laptop computers but it may also be used by stand-alone hardware keyboards that are linked to a device with Bluetooth, wired, or other wireless communications.

[0032] The contextual keyboard and contextual keyboard system was initially developed to achieve the following goals:

• Diminish the number of typing errors in mobile keyboards

• Help all users who are challenged in hand-eye coordination achieve a better typing experience in mobile keyboards

• Help fix errors when errors occur

[0033] Among other aspects, the contextual keyboard and contextual keyboard system provides an improvement over traditional keyboards and existing software keyboard applications by further diminishing user typing errors (and helping challenged users) by, for example:

• Dynamically highlighting the recommended keys when typing a sequence of characters;

• Increasing the size of the keys; and

• Reducing the number of keys to type.

[0034] In addition, the contextual keyboard and contextual keyboard system provides improved techniques fixing errors by: • Providing dynamic feedback when an error is suspected

• Reducing the number of keys needed to type for correcting errors

• Allowing replacement of a word with one touch

• Allowing multiple corrections in one go for a full message

• Generating suggestions of alternative words to correctly spelled words that may not have been the intended word based on the key typing position

[0035] The contextual keyboard and contextual keyboard system are able to achieve these improved results by taking advantage of several factors, including its knowledge of the language used to type the input, the proficiency of the user in the input language, its knowledge of the specific details of the user typing the input, the layout of the keyboard, and the input that was typed. To achieve these improved results, the CK and CKS provides a dynamic keyboard that let the user make decisions and does not attempt to do any automatic text replacement or auto-correction except when specifically instructed by the user.

[0036] The contextual keyboard and contextual keyboard system provides the following additional capabilities beyond what is provided by existing keyboards:

- Adjacent keys - correction suggestions based upon near neighbor keys

- Recommended and non-recom mended key emphasis

- Context key - input insertion based upon context especially when desired key not currently displayed

- Aggregate keys - assigning multiple corresponding input values to single key allows reducing keys on keyboard and providing of larger keys

- Auto-completion by the keyboard itself when only one word available

- Replacement word - correction suggestions based upon dictionary

- Expansion keys - suggested key sequences to follow based upon dictionary

- Ability to change the height of the result bar of the keyboard (e.g., making the result line a different size than the standard (e.g., QWERTY) keys line)

Each of these are described below with respect to the various figures.

[0037] Preliminary tests of the contextual keyboard system described herein have demonstrated that using a combination of these improved capabilities have reduced errors by 40% and has resulted in increasing typing speed by 20%. [0038] Many current keyboards used with mobile devices (for example, the virtual/software keyboard provided by default with APPLE or ANDROID devices, typically divide the keyboard in two parts: the keys which are in four rows and a prediction line which appears above the keys. The prediction line is dynamic and presents in general three words which are the predictions of the words that the user will type next.

[0039] Figure 1 is a block diagram of an example contextual keyboard deployment. The contextual keyboard 100 of an example contextual keyboard system comprises a selectable key area 110 (e.g., a QWERTY keyboard with four rows of user selectable keys) and a result bar 101 , which appears above the keys. In one deployment, the contextual keyboard does not include a prediction line, but such line may be added as a sixth line. The result bar includes a “globus” key 103 for language selection, a settings key (e.g., “gear” key) 102 for selection of various contextual keyboard settings, adjacent key area 104, current user typed key area 105, and expansion key area/replacement key sequence area 106. The operating system maintains a current input display area for keys typed in the keyboard (not shown in Figure 1 , see e.g., Figure 10, input area 1000).

[0040] Figures 2A-2B is an example block diagram of overview logic flow of an example contextual keyboard processing system. In overview, in block 201 , the CKS reads the designated dictionaries, the settings, and draws an initial keyboard. It also 'hooks’ into the keyboard and mouse (I/O drivers) events of the operating system to be able to receive notice of keyboard and mouse events. For example, using API in IOS and in Android operating systems, an application can declare itself as a keyboard. The user then can select the contextual keyboard instead of or in addition to the default system keyboard from the system settings interface.

[0041] The logic then waits to receive (be notified of) a tap, and upon notification, proceeds to block 204. In block 204, the logic determines what key was pressed and keeps track of the adjacent keys. For example, in one embodiment each key is divided into 4 selection zones: lower left quadrant, a lower right quadrant, an upper left quadrant, and an upper right quadrant. These selection zones are used to determine which other keys are the adjacent keys. For example, pressing on the lower left quadrant of the “d” key, may result in adjacent keys “s,” “z,” and “x.” Adjacent keys are described below with respect to Figures 25 and 26. [0042] In block 205, the logic determines whether a regular “letter” (or equivalent input) has been selected (e.g., pressed, input, formed, etc.). For the purposes of this description, a key on the contextual keyboard generally corresponds to a letter or a character, including alphanumeric input, numbers, symbols, emojis, images, or the like. The examples herein describe letters commonly used with a QWERTY keyboard on a mobile device, and may refer to these as “regular letters” (e.g., the symbols that correspond to the keys on the primary keyboard display) although it is to be understood that any key supported by the operating system or provided by the contextual keyboard (such as the special context key) is operable with the methods, systems, and techniques described herein albeit they may be detected using different logic. When a regular letter (alphabetically or equivalent for the designated language) is determined, the logic continues to block 206, otherwise continues to determine the input key in block 207. In block 206, the logic resolves any pending “placeholders” (e.g., used in autocompletion, replacement words, etc.), adds the letter to the current word, and causes the keyboard appearance to be updated to concatenate the determined letter to the currently displayed input to form a revised currently displayed input (and, for example, to deemphasize all the non-recom mended keys). The logic then returns to wait for the next key input (before block 202).

[0043] In block 207, the logic determines whether a control character has been selected and if so continues in block 208 to draw the requested pane such as numeric input display, an emoji input display, etc., and then returns to wait for the next key input (before block 202). If not, the logic continues to determine the input key in block 209.

[0044] In block 209, the logic determines whether a selection has been made in the result bar, and if so, handles the “tap” in block 210, and then returns to wait for the next key input (before block 202). If not, the logic continues to determine the input key in block 211 .

[0045] In block 211 , the logic determines whether a special character has been selected and if so continues in block 212 to handle the special character (e.g., context key, aggregate key, etc.), and then returns to wait for the next key input (before block 202). If not, the logic continues to determine the input key in block 213.

[0046] In block 213, the logic handles any deferred placeholders, handles the end of word processing after detecting an appropriate character, and then returns to wait for the next key input (before block 202). An End of Word may be detected by examining the input of certain keys such as a Return key, or a punctuation key, or a numeric key except when these keys are part of a legitimate word in the designated dictionary (e.g., o’clock, longterm, am/pm, and the like). End of word processing may include performing checks on the word, including examining the word to see it is a legitimate word in the designated dictionary, showing a replacement word if necessary (depending upon the CKS settings) and if the word is not found the logic will either add the word to the user dictionary or increase the user’s language level (if the word occurs in the designated dictionary at a more expert language level). Then the logic will clean up appropriate internal structures that are used to track the current word, in order to reset these structures for the next word.

[0047] The CKS implements several operating system methods (not shown) in order to handle certain types of keyboard events that may be first detected by the operating system. Keys such as a Return key or a Send (arrow) key may trigger end of input/message processing. For example, when an end of input/message is determined by the Android operating system, it calls the method OnFinishlnput() in order to handle end of input/message processing. The CKS implementation of this method (e.g., OnFinishlnput()) examines the last word of the input from the characters that have been accumulated thus far. If the characters do not represent a word from the currently designated dictionary, it adds the unknown word to user dictionary (or increases the user’s language level as described below if the word occurs in the designated dictionary at a more expert language level) and handles the end of a input/message by cleaning up all internal structures in order to be able to start typing a new message. Note that a replacement word is not suggested because the input has already been sent (the input is finished as determined by the operating system).

[0048] The CKS system also implements a method to determine when/whether to remove the keyboard. For example, in the Android operating system, the CKS implements the method OnDestroy() to close the dictionary and settings files and then exits.

[0049] The techniques of a contextual keyboard and contextual keyboard system are generally applicable to any type of keyboard and language. Also, although certain terms are used primarily herein, other terms could be used interchangeably to yield equivalent embodiments and examples. In addition, terms may have alternate spellings which may or may not be explicitly mentioned, and all such variations of terms are intended to be included.

[0050] Example embodiments described herein provide applications, tools, data structures and other support to implement a contextual keyboard that improves user accuracy and precision based upon word frequency based dictionaries and contextual factors such as the language, proficiency level of the user and the like. Other embodiments of the described techniques may be used for other purposes. In the following description, numerous specific details are set forth, such as data formats and code sequences, etc., in order to provide a thorough understanding of the described techniques. The embodiments described also can be practiced without some of the specific details described herein, or with other specific details, such as changes with respect to the ordering of the logic, different logic, etc. Thus, the scope of the techniques and/or functions described are not limited by the particular order, selection, or decomposition of aspects described with reference to any particular routine, module, component, and the like.

[0051] Example contextual keyboard systems include four components: a settings engine, a dictionary engine, a keyboard engine (for processing the keyboard input), and a user interface for drawing the keyboard allocated to the CKS, for detecting input and coordinating input and resulting output between the user, the system user interface, and keyboard engine. Each of these components is described.

Settings Engine

[0052] When the contextual keyboard is initialized it will establish the initial settings. The initial settings screens (or user input controls) allow the program to determine the keyboard layout, the languages file to use and to build the initial user dictionary.

[0053] Figures 3A and 3B is an example flow diagram of processing initial settings on an example contextual keyboard. In block 301 , the logic determines the languages (from a system list) used in the system. Blocks 302-308 implement a loop performed by each such language. In block 302, the logic selects the next language to process starting with the first. In block 303, if more than one layout for the language exists, then the logic determines a preferred layout in block 304, and continues in block 305. If not, the logic continues in block 305. In block 305, the logic determines the language “level” (proficiency) of the user using several parameters. This is described further below with respect to the dictionary. In one deployment, the user is also given an opportunity to confirm his/her level by querying familiarity with word meaning near or slightly above the system determined level. For example, if the system determines that the user’s level is a “10” then the user may be asked if he/she is familiar with a word at a level “15.” In block 306, if the system determined language level is not confirmed by the user, then the logic adjusts the language level in block 307 and continues in block 308. In block 308, the logic determines whether there are additional languages to process and if so returns to block 302 to process the next language; otherwise, continues in block 309. In block 309, the logic queries and save the user’s preferred language. In block 310, the logic determines whether the user has granted access to the user’s contact list and if so in block 311 adds the user’s contacts to the user dictionary, and continues in block 312. If not, the logic continues in block 312. In block 312, the logic saves the determined settings and exits.

[0054] Figures 4-9 illustrate example screen displays of additional settings available to a user that turn on the various improved capabilities described above for a contextual keyboard. These additional settings are selected by tapping on the settings icon 102 located in the result bar 101 as shown in Figure 1 . When settings icon 102 is selected, a settings window is opened. The settings are used to allow the CKS to control the display of the rows of keys and of the result bar. The settings allow a user to turn on or off any of the improvements overviewed above and control how they will be presented. For example, these settings allow control of colors, border sizes, and sound feedback. They also allow the possibility to add languages.

[0055] When settings are changed by the user the current settings window passes the changes to the user interface engine which passes these changes to the keyboard engine which is responsible to save them in a file and to generate a redraw action when necessary. Other additional settings can be incorporated similarly.

[0056] Figure 4 is an example screen display of keyboard behavior settings on an example contextual keyboard. This display can be used to turn on and off deemphasize (e.g., dimming) of non-recommended keys (auto-guide), auto-completion, context key and aggregate key capabilities. [0057] Figure 5 is an example screen display of result bar settings on an example contextual keyboard. This display can be used to determine the size of the result bar and whether adjacent keys, typed keys, replacement words, and expansion key sequences are shown.

[0058] Figure 6 is an example screen display of sound settings on an example contextual keyboard. This display can be used to control sound associated with non-recommended keys.

[0059] Figures 7-9 are example screen displays for color settings, borders settings, and languages for an example contextual keyboard.

The Dictionary Engine and the CKS Dictionaries

[0060] For each language supported by the contextual keyboard, there is a data file that contains the language dictionary for that language. A language dictionary file used for CKS contains all the words of the dictionary for this language (it is a data structure that lists words). The words are arranged (preferably) in a compressed fast access trie structure that also contains additional information needed to support the recommended/non- recommended keys, the context key, the auto-completion, the replacement word, the expansion key sequences.

[0061] In an example embodiment, a CKS dictionary (list of valid words) is ordered by the frequency of the usage of the word in the language, and include additional information for foreign language speakers like “false friends” (words that are written or pronounced the same in two languages but have a different meaning in each language). Dictionaries and information regarding word frequency for a particular language may be purchased, for example, from different online vendors such as “www.wordfrequency.info/intro.asp” (a resource for obtaining data from the Corpus of Contemporary American English (COCA)). Each word also includes the information on possible words that are close by if type and could have been intended to be typed instead of the word itself. To include this information in a CKS dictionary, example contextual keyboard systems include additional code logic that determines the possible words that could have been typed if an adjacent key had been typed. Each word in the CKS dictionary includes this additional information. [0062] To optimize performance, the whole list can be compressed in a modified trie structure. (Other data structures may be similarly incorporated.) In addition, some CKS embodiments use a paging technique for reducing the memory footprint of the various dictionaries.

[0063] Frequency of occurrence has an impact on several features of a contextual keyboard system, including replacement words, key highlighting (recommended and nonrecommended), and in the aggregate key mechanism. When the user starts with the contextual keyboard, at the time of the initial settings, users are asked about their languages and about their level of proficiency in these languages (see Figures 3A and 3B). For example, the user’s proficiency level may be initially determined based upon parameters such as age, origin (native language), and education level. During the usage of the keyboard, the language level may be adjusted automatically, and it may also be adjusted manually in the settings (not shown). For example, if the CKS determines that the user knows more words than the initially set level then the user’s language level may be raised. For example, before the CKS adds an “unknown” word to the user’s dictionary, the CKS may check to make sure the word doesn’t already exist at a more advanced language level (in the full dictionary). If it does, then, instead of adding the “unknown” word into the user’s dictionary, the CKS increases the language level of the user.

[0064] Language level is comparable to the proficiency profile of the user. For example, in some embodiments, there is a level for “expert” native speaker and a level for an “average” non-native speaker. Additional levels may be added for example that may roughly correspond to years of education in a particular language.

[0065] The list of words is dynamically adjusted according to the language level so that the CKS does not indicate replacement words that are beyond the proficiency level of the user. Also, based upon the selected/determined proficiency level, the CKS will not resolve aggregate keys with words that are beyond the user’s current proficiency level settings (as determined by the system) and will not highlight keys that correspond to words that are beyond the user’s current proficiency level settings (as determined by the system). Note that in embodiments that allow the user to select the user’s level, the system provides replacement words, aggregate keys, and highlighting (emphasis) according to the user’s selection of proficiency level. [0066] For example, in one embodiment, the word “sez” is in the CKS list of words. According to the New Oxford American Dictionary “sez” is the nonstandard spelling of “says” used in representing uneducated speech. In the contextual keyboard when the CKS needs to resolve the aggregate key “z/x” and the user has typed the keys “s” “e” ”z/x” it is likely that the CKS will resolve “z/x” with an “x” and not a “z”. If the user is associated with the regular level (not the expert level), the CKS will not show any replacement words as most likely “sez” is not the intended word and is beyond the level of a regular user. If instead, the user is associated with an expert level, then the CKS will still resolve the “z/x” with “x” because “sex” has a higher frequency than “sez” but it will show “sez” as a possible replacement word in the replacement word area 106 of Figure 1 . The frequency is useful for limiting potential replacement words for users below expert level especially children or non-native speaker of a language.

[0067] In addition to the language files that exist for each language, there is a single user dictionary file that is built using the initial settings and is updated when the user types a word that is not included in one of the language dictionaries and is not considered to be a typing error.

[0068] The dictionary engine is responsible for communicating to the keyboard engine the following elements: the recommended and non-recom mended keys, the context key, the replacement word, the aggregate key.

[0069] When the dictionary engine receives a key from the keyboard engine its logic processes special keys (control keys, context key, aggregate key, replacement word, adjacent keys, punctuation, backspace), then updates the recommended and nonrecommended key structures, and passes the results back to the keyboard engine for further processing.

The Keyboard Engine

[0070] The keyboard engine is responsible for computing the boundaries of all the elements that need to be drawn by the keyboard user interface based on the language and the layout from the settings. It is also responsible for determining which element of the keyboard user interface has been selected after the contextual keyboard user interface is notified of a key press from the underlying system (see block 202 in Figure 2A). [0071] The keyboard engine receives the coordinates of the user touch from the keyboard user interface module (which in some example deployments come from the underlying system), as well as the touch action (regular press or long press). Then, the keyboard engine uses the current keyboard layout data stored to establish the typed key and associated functions (e.g. , finding adjacent keys) and sends the result back to the keyboard user interface module. The keyboard user interface module then provides the initial feedback of the typed key to the user.

[0072] In some embodiments the keyboard user interface module is provided as part of the CKS and obtains notification of user actions with the touch screen keyboard both as to location and duration from the underlying operating system. The CKS user interface then communicates the location and duration (e.g., detection of a long vs short press) to the keyboard engine. In other embodiments, the keyboard user interface module is provided by the underlying operating system and the CKS user interface functions are integrated into the operating system. Whichever, technique is used, once the system/CKS user interface has displayed the current input, it passes it to whichever application is associated with receiving keyboard input.

[0073] When a user taps on a key, the tap is transferred from the system/CKS user interface to the keyboard engine to determine the actual typed key (e.g., a “u”). The keyboard engine then passes this recognition (that the typed key is a “u”) back to the user interface. If the user selects this typed key (e.g., by raising the user’s finger), then the confirmation is sent back to the keyboard engine. If not, for example, if the user drags the user’s finger to another area of the display (e.g., signaling cancelation of the action), then the key is ignored.

[0074] Upon determining a typed key, the keyboard engine updates the adjacent keys list and passes the typed key to the dictionary engine which processes the typed key, gives back a result and update the structures containing the information about the recommended and non-recommended keys.

[0075] The keyboard engine has also access to the bitmap of the context keyboard user interface (which is responsible, for example, for drawing the selectable key area 110 and the result bar 101 in Figure 1 ). The keyboard engine updates the recommended and nonrecommended keys display by drawing directly on the bitmap of the keyboard (or by sending the updates to the context keyboard user interface) and sends the desired result bar actions to the contextual keyboard user interface engine such that when the contextual keyboard user interface redraws the keyboard, the display is appropriately updated.

The User Interface

[0076] As mentioned above, the keyboard user interface module may be provided by the underlying operating system or as part of the CKS or both. When starting the program, the user interface engine is responsible to pass the rectangle size of the keyboard area to the keyboard engine. Once this rectangle is passed, the keyboard engine can compute the boundaries of the various elements and pass them back to the user interface.

[0077] The CKS user interface will then draw the keyboard and the result bar according to the layout and the settings received from the keyboard engine. After drawing the keyboard, the user interface will pass the bitmap representing the keyboard back to the keyboard engine so that the keyboard engine will be able to optimize the redrawing of the keyboard keys.

[0078] After initialization, the CKS user interface is responsible for monitoring the user taps (e.g., presses, selections, etc.) and to pass the tap coordinates to the keyboard engine together with the tap action (short or long press). After passing the coordinate of the tap and the tap action to the keyboard engine, the CKS user interface will receive from the engine a result allowing it to update the current input display of the contextual keyboard and to pass the characters generated to the application requesting the services of the contextual keyboard (for example, via the system user interface). In some deployments, such as that described below with respect to Figures 10-16, the operating system user interface/application is responsible for updating the currently displayed input, as shown, for example, in currently displayed input area 1000 in Figure 10.

Improved Keyboard Functions

[0079] The contextual keyboard user interface of the contextual keyboard implements the improvements overviewed above including: adjacent keys, recommended and nonrecommended key emphasis, context key, aggregate keys, auto-completion, replacement word, and expansion keys. a. Dynamic Highlighting of Recommended and Non-Recommended Keys: [0080] Figures 10-16 are example screen displays illustrating dynamic highlighting in response to typing an input sequence of an example word using an example contextual keyboard. The screen displays illustrate the sequence of typing the word “context” and showing which keys are “recommended,” based upon the designated dictionary and level of the user, by dynamically emphasizing them. In some example deployments, the emphasis can be through color, associated with a sound, haptics, emboldened, or the like.

[0081] For example, in one embodiment (shown in Figures 10-16), the dynamic highlighting of the recommended keys is achieved by changing the color of the non-recommended keys and giving them a color close to the background color of the key. By giving to the nonrecommended keys such a color the effect is to dim these keys. The dimming effect of the non-recommended keys diminishes the probability for a user to tap on such keys and automatically increases the probability fora user to tap on a recommended key. In addition, the non-recommended keys may have a distinct audio feedback and/or a distinctive visual feedback in the result bar. The effect of dimming the non-recommended keys and providing them special feedback diminishes the probability of typing errors.

[0082] An example of the dynamic highlighting when typing the word “context” is shown in the sequence of Figures 10-16. For example, in Figure 10, the current input display is shown in current input display area 1000. This area is controlled typically by the underlying operating system user interface but in some embodiments can be controlled by the CKS user interface. The selectable key area 1005 is shown with the recommended keys in regular font and emphasis and the non-recommended keys dimmed. The CKS keyboard bitmap (as displayed in area 1006) is controlled by the CKS user interface and the keyboard engine. When the user types a “c,” the CKS determines using the dictionary that the keys eligible to follow include any one from the set of {w, e, r, t, y, u, i, o, p, a, s, d, h, I, z, n}. When the user types a “cont” sequence (see Figure 13), the recommended keys change to any key from the set of {e, r, u, i, o, a, I}. Figures 11 ,12, and 14-16 operate similarly.

[0083] Figure 17 is an example screen display illustrating feedback settings of an example software contextual keyboard fora non-recommended key input on an example contextual keyboard. Example feedback for a non-recommended key is shown in Figure 17. The display shows that the user typed a “v” after typing the input “p” (as shown in current input display area 1700). The non-recommended key “v” is emphasized in typed key area 1710 (for example with an orange color - not shown) and emboldening to show that there is no word starting with “pv” in the designated dictionary.

[0084] Figures 18A-18C is an example flow diagram of dynamic highlighting logic of an example contextual keyboard.

[0085] In addition to the general highlighting explained in the text above, the flowchart provides that if a user taps two consecutive dimmed keys (the two is configurable), then the contextual keyboard assumes that the inputted keys are part of a potentially desired word that do not appear in the dictionary. In this case, the contextual keyboard will restart highlighting all the keys. If this potentially desired word is not corrected in the replacement word area, then the potentially desired word will automatically be stored in the user dictionary or the language level will be adjusted if this word was present in the dictionary but beyond the user current language level.

[0086] In block 1801 , the logic initializes itself and starts by highlighting all keys. The logic then waits to receive (be notified of) a key tap, and upon notification in block 1802, proceeds to block 1803. In block 1803, the logic determines what key was pressed.

[0087] In block 1804, the logic determines whether an end of word is detected, and if not continues to block 1805, otherwise continues to block 1816 (see block 213 in Figure 2B). In block 1805, the logic determines whether the pressed key was highlighted, and if not continues in block 1811 , otherwise (if so) continues to block 1806 to determ ine whether the user intended a potentially user-defined word (by determining whether the count of the number of dimmed keys pressed is greater than the number used to determine whether the user may have intended to input a word outside the dictionary). For example if 2 dimmed keys are pressed in succession, then dimcounter = 2 and if restartHighlight has been defined as 1 the keyboard will always be highlighted from this point on until the end of the word. If so, the logic increments the dimmed key counter and returns to wait for the next key input (before block 1802). Otherwise, if the count of the number of dimmed keys pressed is less than the number used to determine a potential user-defined word (e.g., dimmed keys <=1 ), then the logic proceeds to block 1808 to highlight all of the keys (e.g., to reset the keyboard before deciding which key to dim), and then in block 1809 dims all non-recom mended keys (keys that cannot follow the current key according to the designated dictionary).

[0088] In block 1810, the logic stores the key’s current state and otherwise returns to wait for the next key input (before block 1802).

[0089] If in block 1805 the tapped key was not a highlighted key, then in block 1811 the logic determines whether a backspace key was tapped and, if so, continues to block 1815, otherwise continues in block 1812 (no backspace was pressed so the key is a nonrecommended key) to dim all of the keys and increment the count of the number of dimmed keys. Then in block 1813, the logic determines whether to see if a potential user-defined word is detected (dimcount > restartHighlight), and if so, continues in block 1814 to highlight all keys (assuming the user-defined word) and returns to wait for the next key input (before block 1802). Otherwise, in block 1813 if a potential user-defined word is not detected, then the logic returns to wait for the next key input (before block 1802).

[0090] In block 1815, if a backspace character is detected, then the logic resets the highlight of the keyboard selection area to the previous key state (see block 1810), decrements the count of dimmed keys, and return to wait for the next key input (before block 1802).

[0091] In block 1816, if an end of word is detected in block 1804, then the logic checks to see if a potential user-defined word is detected (dimcount > restartHighlight) and if so, shows a different word in the replacement word area and proceeds to block 1818. If not, the logic ends. In block 1818, if the logic detects that the user has selected a checkmark (or other indicator of confirmation) to indicate that a mistake was made and the replacement word should be selected, then the logic ends. The relevant portion of the displayed input is then updated to be replaced by the different word (not shown). Otherwise, the logic adds the user-defined word to the user dictionary or adjusts the language level of the user as appropriate and ends. b. Context Key:

[0092] The context key allows to generate a character that does not appear on the keyboard presented on the screen if the character it would generate unequivocally matches a character not presented. For example, if the user wants to type the contraction “isn’t” the user can type it with 5 keys, an “i”, then an “s”, then an “n”, then the context key and then a “t.” This replaces the sequence of 7 keys “i”, ”s”, “n”, “123” (to invoke the numeric keyboard), “ ’ ,” ”abc” (to invoke the alphabet keyboard), “t” used in a non CKS keyboard. Similarly, when some words exist in the user dictionary with emoji spelling like “im # ment”, then the context key will generate the proper emoji without the user needing to switch keyboards.

[0093] Figure 19 is an example screen display of a context key on an example contextual keyboard. The context key 1905 is represented on keyboard 1900. Other representations and symbols may be used for this function and in other placements on keyboard 1900.

[0094] When the keyboard engine detects that a context key has been typed, it will pass notification of the context key to the dictionary engine and the dictionary engine will assign the appropriate character for this context. The dictionary engine will then pass back the appropriate character to the keyboard engine which will pass it to the user interface. In most cases the context key diminishes the number of characters typed by the user from 3 to 1 and in some rarer cases the number of characters remains unchanged at 3. Generally, the context key increases the typing speed and diminishes the probability of errors. The context key does not exist on today’s keyboard.

[0095] Figures 20A- 20C is an example flow diagram of logic for implementing a context key of an example contextual keyboard. The logic 2000 begins in block 2001 with some initializations. The logic then waits to receive (be notified of) a key tap, and upon notification in block 2002, proceeds to block 2003. In block 2003, the logic determines what key was pressed. In block 2004, the logic determines whether a context key was pressed and if so continues in block 2005, otherwise continues in block 2015. In block 2005, the logic keeps track of the number of possible keys that can follow the current word input so far which are not currently displayed in the selectable key area. In block 2006, the logic stores the possible following keys as “contextstring” (which are the set of possible keys that can follow the current word input so far which are not currently displayed) and proceeds to block 2007.

[0096] In block 2007 if the number of such possible follow-on keys identically 1 , then the logic knows the intended context and in block 2008 generates the first character of the contextstring as input. In block 2009 the logic causes the display of the keyboard to be updated, resets the context key related variables, and returns to wait for the next key input (before block 2002).

[0097] In block 2007 if on the other hand there is no context key possible or more than one context key possible, the logic continues in block 2010 to determine if there are no keys that meet the possible following key definition (e.g., not presented on the current keyboard) and if so, continues in block 2011 . Otherwise the logic continues in block 2014 because more than one follow-on key is possible and it stores a placeholder (and highlights all keys that could follow any of the follow-on keys) and returns to wait for the next key input (before block 2002).

[0098] In block 2011 , if the logic determines that the end of a word has been reached, then in block 2012 it generates a comma for the context key and then returns to wait for the next key input (before block 2002). If an end of word has not been reached, the logic continues with block 2013. In block 2013, the logic causes a switch to the numeric keyboard because the logic cannot predict what the user intended and returns to wait for the next key input (before block 2002).

[0099] In block 2015, when no context key has been selected, the logic determines whether there are any pending follow-on keys (from a placeholder) and if not the logic ends. Otherwise, in block 2016 the logic determines whether it can now resolve the prior placeholder and, if not returns to wait for the next key input (before block 2002). If so, then in block 2017 the logic replaces the placeholder (see block 2014) and then ends. c. Aggregate Keys:

[00100] The aggregate keys allow to combine two characters on one key. The contextual keyboard is in charge of generating the proper character according to the context. Similarly to the context key, when the keyboard engine detects that an aggregate key has been typed, it will pass the aggregate key to the dictionary engine and the dictionary engine will assign the appropriate character for this context. The dictionary engine will then pass this character to the keyboard engine which will pass it to the CKS user interface.

[00101] The aggregate keys are dependent on the designated language and on the keyboard layout. It is recommended to generate aggregate keys that are composed of low usage characters in the language or of characters that are mutually exclusive. [00102] Figure 21 is an example flow diagram of logic for selection of aggregate keys for an example contextual keyboard. For the QWERTY keyboard and the English language, words have been determined that could be generated by pair of adjacent keys by replacing one key by the other and selecting the pair of keys that result in the fewest words by frequency in the language. Thus, for QWERTY and English, the CKS uses three aggregate keys, the first is the q/w which can generate 6 words, the second is the j/k which can generate 40 words and the third is the z/x which can generate 16 words. For most of the pairs of words, one word has a much higher frequency than the other and because the individual letters are low frequency letters in English, most of the words themselves are low frequency usage words. As such the algorithm to replace the aggregate key with the appropriate single key in a word is extremely accurate. Even in the extremely low probability that the aggregate key was not properly resolved, the correct word will still appear as a replacement word in the result bar.

[00103] Specifically, in block 2101 , the logic selects keys that are mutually exclusive. In block 2102, the logic joins mutually exclusive keys as aggregate keys. In block 2103, if there are more than one key reductions per row in the selectable key area, then the logic ends. Otherwise, in block 2014, the logic selects pairs of adjacent keys. In block 2105, the logic determines the number of words that are present in the dictionary when switching the adjacent keys and their frequency in the language. In block 2106, the logic then selects the pairs of adjacent keys resulting in the lowest total frequency of switching works and ends.

[00104] Figures 22A-22B are an example table excerpt of terms of a dictionary for selection of aggregate keys. A list of the pairs of words for the chosen aggregate keys and their frequency in a very large word corpus of American English is provided in Figures 22A and 22B. The aggregate keys allow the CKS to reduce the number of keys on the keyboard and therefore to increase the screen size allocated to each key. The increase in the screen size for each key and the reduction in the number of keys reduces the probability of errors. Aggregate keys do not exist on today’s keyboard.

[00105] Figure 23 is an example of aggregate keys on a contextual keyboard. The aggregate key 2301 represents a “q” or a “w.” The aggregate key 2302 represents a “z” or an “x.” The aggregate key 2303 represents a “j” or a “k.” [00106] Figures 24A-24C is an example flow diagram of logic for implementing aggregate keys of an example contextual keyboard. The logic 2400 begins in block 2401 with some initializations. The logic then waits to receive (be notified of) a key tap, and upon notification in block 2402, proceeds to block 2403. In block 2403, the logic determines what key was pressed. In block 2404, the logic determines whether an aggregate key was pressed (as determined in Figure 21 ) and if so continues in block 2405, otherwise continues in block

2414.

[00107] In block 2405, the logic sets the count of the numAggregates variable to the number of follow-on characters that can follow the word thus far inputted given the aggregate. In block 2407, the logic determines whether the number of follow-on characters is 1 , and if so, generates the character as the next available character in the input in block 2408, otherwise continues in block 2410. In block 2409, the logic causes the keyboard to be updated and highlights all of the keys (as recommended) that can be follow-on keys to the next available character and returns to wait for the next key input (before block 2402).

[00108] In block 2410, the logic determines whether there are no follow-on keys given the aggregate. If true, the logic continues in block 2412, otherwise there are multiple follow-on keys and entry of a placeholder is needed until the next key can be determined. In block 2411 , the logic thus generates a placeholder because either the word is not in the dictionary or there is an error, and tracks the number of placeholders. Optionally, the logic highlights the keyboard according to the characters that can follow the potentially follow-on characters. The logic then returns to wait for the next key input (before block 2402).

[00109] In block 2412, when there are no keys that can follow the aggregate, the logic generates a placeholder in case the user has introduced a word not in the dictionary or an error. In block 2413, the logic updates the keyboard to highlight all of the keys that can follow the aggregate, and returns to wait for the next key input (before block 2402).

[00110] In block 2414, when no aggregate key is pressed (for example, after a placeholder), if the number of placeholders is 0, then the logic finishes, otherwise, continues in block

2415. In block 2415, the logic determ ines whether it can resolve the placeholder and, if so, in block 2419 replaces the placeholder and finishes. Otherwise, in block 2416, the logic checks to see if it is at the end of the word (see block 213 in Figure 2B) and, if so, continues in block 2417, otherwise returns to wait for the next key input (before block 2402).

[oom] In block 2417, the logic replaces the placeholder with the character that generates the word with the highest frequency of occurrence, and in block 2418, generates (and causes to be displayed) a replacement word for the word with the lowest frequency of occurrence, and finishes. This allows the user to easily indicate an alternative. Replacement words are described elsewhere and the user selects one proactively, e.g., by tapping a “checkmark” or equivalent to indicate approval.

[00112] Although not described in this flow diagram, it is still possible to reach any single letter of the aggregate key by doing a long press. In this case a pop-up appears that allow the selection of any of the keys that are combined in the aggregate key.

[00113] Other languages may have different forms for selection and representation of aggregate keys which can be similarly processed. d. Auto-completion:

[00114] The auto-complete allows the contextual keyboard to automatically complete a word when this word is the only one available in the current language designated dictionary or in the user dictionary. For example, if a user is typing in English and has typed the sequence “air”, then if the user types the letter “e” the contextual keyboard will generate the letters “eady” automatically as well as a space.

[00115] When an auto-complete key is typed, the CKS may also generate a different sound or other feedback, for example, when distinctive sounds are associated with the various key types in a sound scheme. e. Adjacent Keys:

[00116] The adjacent keys are the keys that are the closest keys to the key that was typed. When they represent recommended keys, the adjacent keys may appear dynamically in the result bar. For example, if the sequence “cont” has been typed in English on a QWERTY keyboard, then the possible adjacent keys are “y”, “f”, “r,” and “g”. The adjacent keys are used to correct typing errors and are shown in adjacent key area (see area 104 in Figure 1 ). When there is a tap on one of the adjacent keys in the adjacent key area, this key will replace the last key that was typed. In the example above, if the intention was to type the word “confirm” then typing on the adjacent key “f” will replace the “t” by an “f”. The adjacent key diminishes the number of characters to type for correcting an error from 2 to 1. By decreasing the number of characters to type, the adjacent key improvement increases the typing speed and diminishes the probability of errors.

[00117] Figure 25 is an example screen display of an adjacent key on an example contextual keyboard. Here, when the user types a “g,” the CKS places an “h” as an adjacent key 2501 in the adjacent key area because the “h” key is adjacent the “g” and a possible next letter in the dictionary associated with the current language. Of note, “t” “f” “c” “v” and “b” are also adjacent, but they cannot follow the “g” according to the dictionary and therefore are not selected as adjacent keys and “y” is adjacent and can follow “g” according to the dictionary but the tap on the “g” was on the lower right quadrant so “y” was not selected.

[00118] Figure 26 is an example flow diagram of logic for implementing adjacent keys of an example contextual keyboard. Whether appearing in the result bar or not (e.g., in the adjacent key area), the adjacent keys are computed for each tap and saved internally as they work also as a support to the replacement word and are used in the logic of the replacement word.

[00119] The logic 2600 begins in block 2601 with some initializations. Specifically it determines the number of adjacency zones possible for each key. In block 2602, the logic determines and associates adjacent keys for each zone for each key.

[00120] As described above, an example CKS deployment divides the keys into four zones: a lower left quadrant, a lower right quadrant, an upper left quadrant, and an upper right quadrant. These selection zones are used to determine which other keys are the adjacent keys. Other deployments can use a different number of zones and the logic is changed accordingly.

[00121] The logic then waits to receive (be notified of) a key tap, and upon notification in block 2603, proceeds to block 2604. In block 2604, the logic determines what key was pressed. In block 2605, the logic determines the zone associated with the location where the key was pressed. The CKS user interfaces typically forwards this tap location from the underlying operating system. In 2606, the logic displays the keys associated with the determined zone for the determined input key if they are highlighted (recommended) keys. Other deployments may display such keys whether or not they are recommended keys. In block 2607, the logic stores all of the keys associated with the zone for replacement word processing, and then ends. f. Replacement Word:

[00122] The replacement word appears dynamically in the result bar when a word has been completed and a potential error has been detected. When a replacement word appears, a checkmark (or other symbol or user interface control for acceptance) and a cross mark (or other symbol or user interface control for rejection) will also appear. Tapping on the checkmark results in replacing the word that was typed by the replacement word that is currently being displayed in the replacement key sequence area (see area 106 in Figure 1 ). Tapping on the cross mark will clear the replacement word. By taping on the checkmark in the result bar, it is possible in most of the cases to fix the error with one user action (tap). Note that if not cleared by tapping on the cross mark and not used to replace a word with the checkmark, the replacement word data is kept in a list for possible use later. It is possible to type a full message with several words containing errors and to correct all these errors one after each other when the message is completed. It is also possible to correct all of them in one tap by doing a long press on the checkmark in the result bar at the end of the message.

[00123] Consider that an average word in English is 5 letters and that an error will occur in the middle of the word on average, then correcting an error typically will need 5 taps on average. In English, the replacement word diminishes the number of characters to type for correcting an error from 5 to 1. By decreasing the number of characters to type, the replacement word increases the typing speed and diminishes the probability of errors. By providing indication of potential errors, the replacement word improves the clarity of the messages.

[00124] Figures 27 and 28 are example screen displays illustrating a replacement word area for user inputted errors on an example contextual keyboard. Two types of replacement words are shown. Figure 27 illustrates a spelling error and Figure 28 illustrates a potential typing error. More specifically, the replacement word in Figure 27 is for a direct spelling mistake of “rabit” entered (see 2700), but the replacement word in Figure 28 is for a potential typing error due to the closeness of the keys “r” and “f.” For example, if the user’s tap on “r” key was close to the “f” key then the “r” and “f” keys have been saved as adjacent keys according to the adjacent keys logic (see Figure 26). It is possible that the intention was to type “pretty face” and not “pretty race”. When a word could have been typed by using an adjacent key and this word has a higher language frequency that the word that was effectively typed by the user, then a replacement word appears in the replacement key sequence area. If the intention was really to type “pretty race” then taping on the cross mark in the result bar will eliminate the replacement suggestion, and if the intention was to type “pretty face” then taping on the check mark will generate the replacement of “race” by “face” and the deleting of the suggestion. In both cases, Figures 27 and 28, tapping on the check mark or tapping on the cross mark will remove the replacement word from the list of replacement words.

[00125] Figures 29A-29C is an example flow diagram of logic for implementing replacement words of an example contextual keyboard.

[00126] The logic 2900 begins in block 2901 with some initializations. The logic then waits to receive (be notified of) a key tap, and upon notification in block 2902, proceeds to block 2903. In block 2903, the logic determines what key was pressed and determines the adjacent keys for the pressed key. In block 2904, the logic determines whether the pressed key is a non-recommended (dimmed) key, and, if so, in block 2905 keeps track of where in the inputted word this non-recommended key appears. If not, the logic continues in block 2906.

[00127] In block 2906, the logic determines whether the word is complete (at the end of the word) and if so continues in block 2908. The word may exist in the dictionary such as the case of “race” and “face.” In this situation, the logic determines whether there is another word in the alternate list that the user may have intended to type and if the alternate list indicates a word with a higher frequency then the logic will show this word. If not, then in block 2907 the logic tracks where the unexpected character is located in the input word and returns to wait for the next key input (before block 2902). If the word is complete, then in block 2908, the logic finds all possible words with one adjacent key replacement (using the dictionary) and adds them to the alternative word list.

[00128] In block 2909, the logic determines if the word (completed) is in the dictionary and if so continues in block 2910, otherwise continues in block 2912. In block 2910, the logic determines if there are alternative words (in the alternate word list) and if so, completes processing, otherwise continues in block 2911 to display the checkmark, crossmark, (or equivalents), and the alternative word list as replacement words in the result bar and then ends.

[00129] In block 2912, the logic determines whether the user “mistyped” any keys (tapped non-recom mended keys which may be error or intentional), and if so continues in block 2917, otherwise continues in block 2913. In blocks 2913-2916, the logic finds and stores all alternative words in the alternative word list by:

• switching the key at the input index that was non-recommended with the next key and storing those resultant words;

• switching the key at the input index that was non-recommended with the previous key and storing those resultant words;

• switching the key at the input index that was non-recommended according to “sound rules” (rules that can be used to find similar sounding words) and storing those resultant words; and

• switching the key at the input index that was non-recommended according to “spelling rules” (rules that can be used to find similarly spelled words) and storing those resultant words;

[00130] In block 2917, the logic finds close words according to the spelling rules (not by replacing a key) and stores those resultant words in the alternative word list. In block 2918, the logic displays the checkmark, crossmark, (or equivalents), and the alternative word list as replacement words in the result bar and then ends. g. Expansion Keys:

[00131] The expansion keys represent a unique sequence of characters that could appear following the sequence of letters that has been typed. For example, if the input is in English and if the sequence “insem” has been typed, then if one wants to continue to type the unique sequence that will be typed is “inat” because the only words in the dictionary that start with “insem” are “inseminate”, “insemination”, “inseminating” and “inseminator”. In this case the sequence “inat” will appear in the expansion key area of the result bar (area 106 of Figure 1 ) and it may be entered in one touch. When an expansion keys sequence exists, it diminishes the number of characters to type. As such, the expansion keys increase the typing speed and diminishes the probability of errors. [00132] Figure 30 is an example screen display of expansion key area of an example contextual keyboard. In Figure 30, the currently displayed input is shown in input area 3000. The expansion key sequence “inat” is shown in expansion key sequence area 3005. Note that this is only a portion of the word, as the word may be completed by, for example, “ed,” “ing,” “or,” and the like.

[00133] Figure 31 is an example flow diagram of logic for implementing expansion keys of an example contextual keyboard. In block 3101 , the logic 3100 determines whether there is only one “recommended” character (that is one character can follow the input at this point, given the designated dictionaries). If so, the logic continues in block 3102, otherwise ends, because a unique expansion key sequence is not available.

[00134] In block 3102, the logic determines whether there is a unique sequence of at least 3 characters and if so, in block 3103 shows all of the 3 characters as a sequence in the expansion key sequence area of the result bar and continues in block 3104. If not, the logic ends.

[00135] In block 3104, the logic determines whether it has received a user tap of the expansion key sequence and if so, in block 3105 generates the expansion key sequence by causing it to be concatenated it to the currently displayed input, and then ends. If a tap of the expansion keys is not received, then the logic ends.

Example Computing Systems

[00136] Figure 32 is an example block diagram of a computing system for practicing embodiments of an example contextual keyboard described herein. Note that one or more general purpose virtual or physical computing systems suitably instructed or a special purpose computing system may be used to implement an contextual keyboard system (“CKS”). Further, the CKS may be implemented in software, hardware, firmware, or in some combination to achieve the capabilities described herein.

[00137] Note that one or more general purpose or special purpose computing systems/devices may be used to implement the described techniques. However, just because it is possible to implement the contextual keyboard on a general purpose computing system does not mean that the techniques themselves or the operations required to implement the techniques are conventional or well known. [00138] The computing system 3200 may comprise one or more server and/or client computing systems and may span distributed locations. In addition, each block shown may represent one or more such blocks as appropriate to a specific embodiment or may be combined with other blocks. Moreover, the various blocks of the contextual keyboard system 3210 may physically reside on one or more machines, which use standard (e.g., TCP/IP) or proprietary interprocess communication mechanisms to communicate with each other.

[00139] In the embodiment shown, computer system 3200 comprises a computer memory (“memory”) 3201 , a display 3202, one or more Central Processing Units (“CPU”) 3203, Input/Output devices 3204 (e.g., keyboard, mouse, CRT or LCD display, etc.), other computer-readable media 3205, and one or more network connections 3206. The CKS 3210 is shown residing in memory 3201. In other embodiments, some portion of the contents, some of, or all of the components of the CKS 3210 may be stored on and/or transmitted over the other computer-readable media 3205. The components of the contextual keyboard system 3210 preferably execute on one or more CPUs 3203 and manage the implementation of a contextual keyboard, as described herein. Other code or programs 3230 and potentially other data repositories, such as data repository 3206, also reside in the memory 3201 , and preferably execute on one or more CPUs 3203. Of note, one or more of the components in Figure 32 may not be present in any specific implementation. For example, some embodiments embedded in other software may not provide means for user input or display.

[00140] In a typical embodiment, the CKS 3210 includes one or more settings engines 3211 , one or more dictionary engines 3212, one or more keyboard engines and one or more CKS user interfaces 3214. In at least some embodiments, the dictionary engine is provided external to the CKS and is available, potentially, over one or more networks 3250. Other and /or different modules may be implemented. In addition, the CKS may interact via a network 3250 with application or client code 3255 that uses results computed by the CKS 3210, one or more client computing systems 3260, and/or one or more third-party information provider systems 3265, such as purveyors of dictionary information used in dictionary data repository 3215. Also, of note, the CKS keyboard and state data repository 3216 may be provided external to the CKS as well, for example accessible over one or more networks 3250.

[00141] In an example embodiment, components/modules of the CKS 3210 are implemented using standard programming techniques. For example, the CKS 3210 may be implemented as a “native” executable running on the CPU 103, along with one or more static or dynamic libraries. In other embodiments, the CKS 3210 may be implemented as instructions processed by a virtual machine. In general, a range of programming languages known in the art may be employed for implementing such example embodiments, including representative implementations of various programming language paradigms, including but not limited to, object-oriented, functional, procedural, scripting, and declarative.

[00142] The embodiments described above may also use well-known or proprietary, synchronous or asynchronous client-server computing techniques. Also, the various components may be implemented using more monolithic programming techniques, for example, as an executable running on a single CPU computer system, or alternatively decomposed using a variety of structuring techniques known in the art, including but not limited to, multiprogramming, multithreading, client-server, or peer-to-peer, running on one or more computer systems each having one or more CPUs. Some embodiments may execute concurrently and asynchronously and communicate using message passing techniques. Equivalent synchronous embodiments are also supported.

[00143] In addition, programming interfaces 3217 to the data stored as part of the CKS 3210 (e.g., in the data repositories 3215 and 3216) can be available by standard mechanisms such as through C, C++, C#, and Java APIs; libraries for accessing files, databases, or other data repositories; through scripting languages such as XML; or through Web servers, FTP servers, or other types of servers providing access to stored data. The data repositories 3215 and 3216 may be implemented as one or more database systems, file systems, or any other technique for storing such information, or any combination of the above, including implementations using distributed computing techniques.

[00144] Also the example CKS 3210 may be implemented in a distributed environment comprising multiple, even heterogeneous, computer systems and networks in some embodiments. Different configurations and locations of programs and data are contemplated for use with techniques of described herein. In addition, the server and/or client may be physical or virtual computing systems and may reside on the same physical system. Also, one or more of the modules may themselves be distributed, pooled or otherwise grouped, such as for load balancing, reliability or security reasons. A variety of distributed computing techniques are appropriate for implementing the components of the illustrated embodiments in a distributed manner including but not limited to TCP/IP sockets, RPC, RMI, HTTP, Web Services (XML-RPC, JAX-RPC, SOAP, etc.) and the like. Other variations are possible. Also, other functionality could be provided by each component/module, or existing functionality could be distributed amongst the components/modules in different ways, yet still achieve the functions of an CKS.

[00145] Furthermore, in some embodiments, some or all of the components of the CKS 3210 may be implemented or provided in other manners, such as at least partially in firmware and/or hardware, including, but not limited to one or more application-specific integrated circuits (ASICs), standard integrated circuits, controllers executing appropriate instructions, and including microcontrollers and/or embedded controllers, field-programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), and the like. Some or all of the system components and/or data structures may also be stored as contents (e.g., as executable or other machine-readable software instructions or structured data) on a computer-readable medium (e.g., a hard disk; memory; network; other computer-readable medium; or other portable media article to be read by an appropriate drive or via an appropriate connection, such as a DVD or flash memory device) to enable the computer- readable medium to execute or otherwise use or provide the contents to perform at least some of the described techniques. Some or all of the components and/or data structures may be stored on tangible, non-transitory storage mediums. Some or all of the system components and data structures may also be stored as data signals (e.g., by being encoded as part of a carrier wave or included as part of an analog or digital propagated signal) on a variety of computer-readable transmission mediums, which are then transmitted, including across wireless-based and wired/cable-based mediums, and may take a variety of forms (e.g., as part of a single or multiplexed analog signal, or as multiple discrete digital packets or frames). Such computer program products may also take other forms in other embodiments. Accordingly, embodiments of this disclosure may be practiced with other computer system configurations.

[00146] All of the above U.S. patents, U.S. patent application publications, U.S. patent applications, foreign patents, foreign patent applications and non-patent publications referred to in this specification and/or listed in the Application Data Sheet, including but not limited to U.S. Patent Application No. 17/994,972, titled “CONTEXTUAL KEYBOARD INPUT METHOD, SYSTEM, and TECHNIQUES,” filed November 28, 2022 and U.S. Provisional Patent Application No., 63/266,789, titled “METHODS FOR IMPROVING KEYBOARD USAGE,” filed January 14, 2022, are incorporated herein by reference, in their entireties.

[00147] From the foregoing it will be appreciated that, although specific embodiments have been described herein for purposes of illustration, various modifications may be made without deviating from the spirit and scope of the invention. For example, the methods and systems for implementing a contextual keyboard discussed herein are applicable to architectures other than an IOS or ANDROID architecture. Also, the methods and systems discussed herein are applicable to differing application and operating system specific protocols, communication media (optical, wireless, cable, etc.) and devices (such as wireless handsets, electronic organizers, personal digital assistants, portable email machines, game machines, pagers, navigation devices such as GPS receivers, etc.).

Claims

1 . A method in a computing system for implementing a software keyboard that facilitates a user minimizing typographical errors and minimizing number of keystrokes entered to complete words, comprising: presenting in a selectable key area a plurality of selectable keys, each key corresponding to a letter or character key present in a designated language; receiving indication of a user selected key from the plurality of selectable keys displayed in the selectable key area, concatenating the indicated key to keystrokes previously received from the user to form a current input; and displaying the current input in a current input display area; determining one or more adjacent keys associated with the indicated user selected key and the current input, each adjacent key indicative of possible input errors based in part upon prediction of valid terms in a dictionary associated with the designated language; for each adjacent key, determining whether addition of the adjacent key to the current input would result in a valid term in the dictionary and, when it is determined that the addition of the adjacent key to the current input would result in the valid term, displaying the adjacent key in an adjacent key area distinct from the selectable key area; and upon receiving indication of user selection of at least one of the displayed adjacent keys, automatically replacing the indicated user selected key in the displayed current input with the indicated displayed adjacent key, thereby correcting an input error in a single user action.

2. The method of claim 1 wherein each of the plurality of selectable keys is divided into a plurality of selection zones and where the determination of the one or more adjacent keys is based upon determining a selection zone associated with the user indication of the user selected key.

33

3. The method of claims 1 or 2 wherein each of the selectable keys has at least four selection zones that include a lower left quadrant, a lower right quadrant, an upper left quadrant, and an upper right quadrant.

4. The method of at least one of the above claims wherein when it is determined that the addition of the adjacent key to the current input would not result in the valid term, not displaying the adjacent key in the adjacent key area.

5. The method of at least one of the above claims wherein the presenting in the selectable key area the plurality of selectable keys further comprises: determining a next collection of possible valid selectable keys based upon a prediction of valid terms in the dictionary associated with the designated language and a determination of which of the plurality of selectable keys when concatenated to the displayed current input would result in valid terms; and displaying the next collection of possible valid selectable keys with emphasis to enable easier user selection of the valid selectable keys.

6. The method of claim 5 wherein the emphasis comprises highlighting each valid selectable key using at least one of color, font change, or audio or haptic feedback.

7. The method of at least one of the above claims, further comprising: determining a next collection of possible invalid selectable keys based upon a prediction of valid terms in the dictionary associated with the designated language and a determination of which of the selectable keys when concatenated to the displayed current input would not result in valid terms; and presenting in a dimmed manner the next collection of possible invalid selectable keys to deemphasize user selection of the invalid selectable keys.

34

8. The method of at least one of the above claims, further comprising: determining a next collection of possible invalid selectable keys based upon a prediction of valid terms in the dictionary associated with the designated language and a determination of which of the selectable keys when concatenated to the displayed current input would not result in valid terms; and receiving indication of a second user selected key from the plurality of selectable keys displayed in the selectable key area; determining whether the second user selected key is one of the next collection of possible invalid selectable keys; and when it is determined that the second user selected key is one of the next collection of possible invalid selectable keys, presenting the key with emphasis in a separate typing result area.

9. The method of at least one of the above claims, further comprising: determining one or more valid keys that could directly following the current input, based upon a prediction of all valid terms in a dictionary associated with the designated language creatable from the current input; and when it is determined that only one unique sequence of valid keys can be concatenated to the displayed current input and would result in a single unique valid term, automatically concatenating the unique sequence of valid keys to form a revised current input, and automatically displaying the revised current input as the current input in the current input display area without user intervention.

10. The method of at least one of the above claims, further comprising: determining one or more valid keys that could directly following the current input, based upon predicting all valid terms in a dictionary associated with the designated language creatable from the current input; and when it is determined that a plurality of sequences of valid keys can follow the current input, displaying the plurality of sequences of valid keys in an expansion key area; upon receiving indication of user selection of one of the plurality of sequences of valid keys, concatenating the indicated one of the of the plurality of sequences of valid keys to form a revised current input, and automatically displaying the revised current input as the current input in the current input display area.

11 . The method of claim 10 wherein each plurality of sequences of valid keys comprises at least three keys.

12. The method of at least one of the above claims, further comprising: based upon the determining of the one or more adjacent keys associated with the indicated user selected key and the current input and the possible input errors based in part upon the prediction of valid terms in the dictionary associated with the designated language, determining one or more replacement key sequences that correspond to valid terms for at least a portion of the displayed current input; and when it is determined that one or more replacement key sequences for at least a portion of the displayed current input exist, displaying the one or more replacement key sequences in a replacement key sequence area.

13. The method of claim 12, further comprising: receiving indication of a selection of one replacement key sequence in the replacement key sequence area; and automatically replacing the portion of the displayed current input with the indicated one replacement key sequence.

14. The method of at least one of claims 12 or 13 wherein the displaying of the one or more replacement key sequences in the replacement key sequence area is accessed through an additional user interface control when the number of characters comprising the one or more replacement key sequences exceed space available in the replacement key sequence area.

15. The method of at least one of the above claims, further comprising: presenting a selectable context key in the selectable key area; receiving indication of user selection of the presented context key; determining a collection of possible next valid selectable keys based upon a ranked ordering of valid terms in the dictionary associated with the designated language and a determination of which one next valid key of the collection of possible next valid selectable keys when concatenated to the displayed current input would result in a most frequent occurring valid term; and automatically concatenating the determined next valid key to form a revised current input, and automatically displaying the revised current input as the current input in the current input display area without user intervention.

16. The method of claim 15 wherein the next valid key is not currently displayed in the plurality of selectable keys presented in the selectable key area, thereby enabling the user to insert a letter or character not presently displayed on the software keyboard without changing the displayed plurality of selectable keys presented in the selectable key area.

17. The method of at least one of the above claims wherein one or more of the plurality of selectable keys presented in the selectable key area are displayed as aggregate keys, wherein each aggregate key is associated with at least two letters or characters present in the designated language, and further comprising: receiving indication of a user selected aggregate key; determining, based upon prediction of frequency of valid terms in the dictionary associated with the designated language, which one of the at least two letters or characters associated with the indicated aggregate key when concatenated to the displayed current input would result in a most frequent of valid terms; and automatically concatenating the determined one of the at least two letters or characters associated with the indicated aggregate key to form a revised current input, and

37 automatically displaying the revised current input as the current input in the current input display area without user intervention.

18. A computing system configured to implement a software keyboard structured to enable a user to minimize the number of keystrokes performed and to minimize the number of errors on devices with virtual keyboards, comprising: a processor; a memory; a display; and a logic component including user interface logic, display logic, dictionary logic, and rule logic stored in the memory, that is configured when executed on the processor to: receiving indication of inputted letters or characters previously received from a user and displayed as a current input sequence in a first area of the display in a current input display region; present, in a second area of the display, a selectable key region, wherein the logic is structured to: display a plurality of user selectable keys each corresponding to at least one letter or character key present in a designated language, each user selectable key indicating a corresponding input of at least one letter or character key when the key is selected; and update a tracked current input sequence by concatenating a key corresponding to an indicated user selectable key to form a revised current input sequence; cause code external to code included in the logic component to update the current input sequence displayed in the current input display region to the revised current input sequence and display the revised current input sequence in the current input display region; and present, in a third area of the display, an adjacent key area that is distinct from the presented selectable key region, wherein the logic is structured to: display one or more adjacent keys, each adjacent key determined in response to an indicated user selectable key, wherein each adjacent key corresponds to a

38 selectable key proximate to the indicated user selectable key when the letter or character corresponding to the proximate user selectable key concatenated to the displayed current input is predicted by the dictionary logic to form a valid term in a dictionary associated with the designated language; and upon receiving indication of a selection of one of the displayed adjacent keys, automatically cause the external code to replace the letter or character indicated by the indicated user selectable key with the indicated letter or character corresponding to the indicated one of the displayed adjacent keys, thereby correcting an input error in a single user action.

19. The computing system of claim 18 wherein each of the plurality of user selectable keys is divisible into a plurality of at least four selection zones and wherein each user selectable key proximate to the indicated user selectable key is based upon each of the at least four selection zones.

20. The computing system of at least one of claims 18 or 19 wherein the logic is further structured to not display an adjacent key in the adjacent key area when the adjacent key is indicative of a user selectable key proximate to the indicated user selectable key and the logic determines that the letter or character corresponding to the proximate user selectable key concatenated to the displayed current input is predicted by the dictionary logic to not form a valid term in a dictionary associated with the designated language.

21. The computing system of at least one of claims 18 through 20 wherein the logic is further structured to: determine a collection of possible valid selectable keys based upon a prediction of valid terms in the dictionary associated with the designated language and a determination of which of the plurality of selectable keys when concatenated to the displayed current input would result in valid terms; and

39 display in the selectable key region each of the collection of possible valid selectable keys with emphasis to enable easier user selection of the valid selectable keys.

22. The computing system of at least one of claims 18 through 21 wherein the logic is further structured to: determine a collection of possible invalid selectable keys based upon a prediction of valid terms in the dictionary associated with the designated language and a determination of which of the plurality of selectable keys when concatenated to the displayed current input would not result in valid terms; and display in the selectable key region each of the collection of possible invalid selectable keys in a dimmed manner to deemphasize user selection of the invalid selectable keys.

23. The computing system of at least one of claims 18 through 22 wherein the logic is further structured to determine one or more valid letters or characters that could directly follow the current input, based upon a prediction of all valid terms in a dictionary associated with the designated language creatable from the current input; and when it is determined that only one unique sequence of valid letters or characters can be concatenated to the displayed current input that would result in a single unique valid term, automatically concatenating the unique sequence of valid letters or characters to form a revised current input, and automatically display the revised current input as the current input in the current input display region without user intervention.

24. The computing system of at least one of claims 18 through 23 wherein the logic component is further configured to: present, in a fourth area of the display, an expansion key area containing one or more valid key sequences that are able to follow the displayed current input and result in a valid term in the dictionary, and wherein the logic is structured to determine the one or more valid key sequences based upon predicting a set of unique key sequences that can follow the letter or character corresponding to the indicated user selectable key.

40

25. The computing system of at least one of claims 18 through 24 wherein the logic component is further configured to: present, in a fifth area of the display, a replacement sequence area containing one or more valid key sequences that each begin with the letter or character that corresponds to one of the displayed one or more adjacent keys and that are able to be concatenated with the displayed current input to result in a valid term in the dictionary, and wherein the logic is structured to determine the one or more valid key sequences based upon predicting a set of key sequences that can follow the letter or character corresponding to each adjacent key.

26. The computing system of at least one of claims 18 through 25 wherein the logic component is further configured to: present, in the selectable key region of the display, a context key, wherein the logic is structured to determine a collection of possible next valid selectable keys based upon a ranked ordering of valid terms in the dictionary associated with the designated language and a determination of which one next valid key of the collection of possible next valid selectable keys when concatenated to the displayed current input would result in a most frequent occurring valid term; and automatically concatenate the determined next valid key to form a revised current input, and automatically display the revised current input as the updated current input in the current input display region without user intervention; wherein the next valid key is not current displayed in the plurality of selectable keys presented in the selectable key region, thereby enabling the user to insert a letter or character not presently displayed on the software keyboard without changing the displayed plurality of selectable keys presented in the selectable key region.

27. The computing system of at least one of claims 18 through 26 wherein logic component is further configured to: present, in the selectable key region of the display, one or more user selectable aggregate keys, wherein each aggregate key is associated with at least two

41 letters or characters present in the designated language, and wherein the logic is structured to: determine, based upon prediction of frequency of valid terms in the dictionary associated with the designated language and an indicated user selectable aggregate key, which one of the at least two letters or characters associated with the indicated aggregate key when concatenated to the displayed current input would result in a most frequent of valid terms; and automatically concatenate the determined one of the at least two letters or characters associated with the indicated aggregate key to form a revised current input, and automatically display the revised current input as the current input in the current input display region without user intervention.

28. A keyboard for use with a computing system comprising: a plurality of user selectable keys each having a label corresponding to at least one letter or character key present in a designated language, each user selectable key label indicating a corresponding input of at least one letter or character key when the key is selected; and a separate context key that, when user selected, with no further user action, causes the computing system to generate a letter or character not indicated by the plurality of user selectable key labels and generates as input a predicted letter or character based upon context of an already inputted letter or character sequence, wherein the predicted letter or character corresponds to a letter or character predicted to most likely result in a most frequently occurring valid term in the designated language.

29. The keyboard of claim 28, wherein the keyboard is a physical keyboard.

30. The keyboard of claim 28, wherein the keyboard is a virtual keyboard.

42