WO2014146131A1 - Space optimizing micro keyboard method and apparatus - Google Patents

Space optimizing micro keyboard method and apparatus Download PDF

Info

Publication number
WO2014146131A1
WO2014146131A1 PCT/US2014/031121 US2014031121W WO2014146131A1 WO 2014146131 A1 WO2014146131 A1 WO 2014146131A1 US 2014031121 W US2014031121 W US 2014031121W WO 2014146131 A1 WO2014146131 A1 WO 2014146131A1
Authority
WO
WIPO (PCT)
Prior art keywords
word
keys
user
fragment
virtual keyboard
Prior art date
Application number
PCT/US2014/031121
Other languages
French (fr)
Inventor
Forbes Holten 3rd NORRIS
Original Assignee
Norris Forbes Holten Rd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Norris Forbes Holten Rd filed Critical Norris Forbes Holten Rd
Priority to CA2941832A priority Critical patent/CA2941832A1/en
Priority to BR112015023799A priority patent/BR112015023799A2/en
Priority to CN201480027568.0A priority patent/CN105247452A/en
Priority to JP2016502610A priority patent/JP2016513849A/en
Priority to US14/776,742 priority patent/US10235042B2/en
Priority to MX2015013077A priority patent/MX2015013077A/en
Priority to EP14765337.2A priority patent/EP2972689A4/en
Publication of WO2014146131A1 publication Critical patent/WO2014146131A1/en
Priority to US16/270,545 priority patent/US11061561B2/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04886Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/02Input arrangements using manually operated switches, e.g. using keyboards or dials
    • G06F3/023Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
    • G06F3/0233Character input methods
    • G06F3/0236Character input methods using selection techniques to select from displayed items
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/02Input arrangements using manually operated switches, e.g. using keyboards or dials
    • G06F3/023Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
    • G06F3/0233Character input methods
    • G06F3/0237Character input methods using prediction or retrieval techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04842Selection of displayed objects or displayed text elements
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0489Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using dedicated keyboard keys or combinations thereof
    • G06F3/04897Special input arrangements or commands for improving display capability
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/20Natural language analysis
    • G06F40/274Converting codes to words; Guess-ahead of partial word inputs

Definitions

  • This invention pertains to the entry of text on computer screens having a virtual keyboard and limited screen space, for example as would be found on a handheld computer, smartphone, or automobile dashboard.
  • Virtual keyboards are a text input method employed on computer screens used as computer user interfaces, in which the computer renders an image of a keyboard, and the user selects "keys" directly from image of the key.
  • Virtual keyboards may rely on touch screens, in which the user touches the screen with a finger or a stylus, but other virtual keyboards are possible, relying on mouse key selections, eye movements, or other actions taken by a user to select a key on a virtual keyboard.
  • virtual keyboards do not have actual physical keys with a physical switch, as in a conventional computer keyboard, but rely entirely on the touching of the screen or other selection method to select the key desired by the user.
  • Each "key" on a virtual keyboard is nothing more than a region on a screen that is selectable.
  • Virtual keyboards are commonplace in a plethora of devices, including smartphones, tablet computers, automobile dashboards, industrial assets, etc.
  • Virtual keyboards are often presented in the traditional "QWERTY' format, but alternative keyboard layouts have been developed, and can be readily implemented using a virtual keyboard.
  • a typical feature of virtual keyboards regardless of the device, is that virtual keyboards are usually employed in situations involving devices with rigid space limitations.
  • Prime examples are smartphones or other handheld computers, in which small physical size is a critical attribute. Such small size creates challenges in entering and displaying text and graphics.
  • the '233 patent discloses a typical prior art embodiment for predictive word completion in which a third region is rendered between the virtual keyboard and edit content regions of the screen that displays (typically in a single line) several word completion options. See, for example Fig. 10 in the '233 patent.
  • the '233 patent itself discloses word completion candidates or fragments thereof that appear in the area that would otherwise be used by the virtual keyboard space bar (Fig. 11 and col. 9, 1 40).
  • This invention discloses methods and apparatus that transform normal keyboards into highly dynamic menu systems where keyboard layouts and key labels are high flexible and can change and adapt to provide users the largest possible keys on a keystroke by keystroke basis.
  • a limited set of keys is generated dynamically to generate keys optimizing the space, size, and visibility of each key.
  • this invention provides a method and apparatus for word completion during text input by a user on a computer apparatus having a virtual keyboard user interface, in which the user interface has limited screen space to display the virtual keyboard and any other information.
  • the method and apparatus employ a sequence in which the user enters text by selecting keys on the virtual keyboard for one or more letters comprising a partial word, wherein each partial word has a "last letter key,” which is the last key (in time) selected by the user. This last key need not be the final key in a word, or word fragment.
  • the last letter key includes letter that are embedded within a word or word fragment.
  • the next step in the sequence of this embodiment is that the user is presented with a computer generated set of keys on the virtual keyboard in response to the last letter key, called "responsive keys.”
  • the responsive keys are prioritized by some method, which may include statistical occurrence of such letters, or may be based on some other rational context, such as locations of nearby features on a map.
  • a final step of the method or apparatus in this embodiment is that the user selects one of the responsive keys to partially complete or complete a word.
  • a computer implemented menu generation system is provided, based on word trees. BRIEF DESCRIPTION OF THE DRAWINGS
  • Figure 1 shows the inventive word tree systems.
  • Figure 2 shows variable size keys resized in their approximately normal QWERTY location.
  • Figure 3 shows a sequence of typing examples illustrating the variable reduced key set menus.
  • Figure 4 shows circular menu formats.
  • Figure 5 shows single row mode of operation.
  • Figure 6 shows key menus overlapping with edit content.
  • This invention describes methods and apparatus that are particularly adapted for use on space constrained virtual keyboards, such as are used on smart phones, compact handheld computers, automobile dashboards, and other applications where cost or physical limitations (for example, the need to hold a device in a human hand) limit the screen space available.
  • space constrained devices present challenges in the efficient entry of text and data.
  • inventive methods and apparatus are not limited to space constrained keyboards, and can also be valuable in larger devices to expedite text entry in certain situations.
  • the instant invention provides modified keyboards, particularly suited to virtual keyboard computer interfaces, in which word fragment keys can be rendered showing one or more letters, in non-standard shapes and modified locations on the region of the screen used as the virtual keyboard. Additionally, this invention may only display keys relevant to specific word completion scenarios. In another embodiment, key size and shape can be dynamically determined. In another embodiment, various modes of operation can changed automatically based on the appropriate context. By the phrase “modes of operation,” it is meant that different modes can be displayed as the context requires, for example, switching from a standard QWERTY display to a spring loaded mode displaying a limited number of keys, as is more fully described herein.
  • All features of this invention can be dynamically responsive to very specific word completion scenarios on a keystroke by keystroke basis.
  • words can be constructed from a sequence of word fragments, where a word fragment is defined in this context as one or more letters, which function as building blocks of longer words, where fragments themselves can even be complete words.
  • a word fragment is defined in this context as one or more letters, which function as building blocks of longer words, where fragments themselves can even be complete words.
  • Traditional keyboards such as having a "QWERTY' layout, use a static layout of single letter (fragment) key input to construct words, thus requiring an entire alphabet of keys.
  • word fragments longer than one letter reduce the number of key presses required to enter desired text, and require fewer keys and less space.
  • a database is constructed comprising trees of words emanating from a root word fragment.
  • This database is constructed in advance.
  • Such databases actually exist, and can be accessed for example, at htl7p://www.scrabblefindei com/starts-with/xxx/ (where "xxx" represents a root word fragment).
  • a root word fragment can be a complete word, like "main.” But even a complete word like "main” can also be a root fragment for longer words, for example "maintain.”
  • Each branch in the word tree database terminates in a node that is a suffix fragment that can be combined with the root word fragment.
  • the combined root and suffix make a complete word.
  • the combined root and suffix can be used as a child root fragment in the word tree database that leads to a subset of branches, narrowing the choices of complete words that can be generated as the root grows in size.
  • the typing (data entry) of words can be visualized as navigating or walking down the branches of a word completion tree, via a menu of fragment choices.
  • a much more compact form of this type of tree has branch leaf nodes representing word fragments, where "word fragment” or just “fragment” is defined in this context as one or more letters, which function as pieces or building blocks of longer words, where fragments themselves can even be complete words.
  • word “MAINTAINING” can be formed from the fragment sequence “MAIN” "T", “AIN”, "ING".
  • the starting fragment of each word forms the root parent node of a word completion tree, with branch leaves represented by child fragment nodes.
  • Each node can be the parent of child nodes extending downwards another branch level. This branching pattern is repeated downwards until all possible words deriving from the starting root node fragment are represented, where terminal nodes (those lacking child nodes) represent the final fragments of complete words.
  • each letter or fragment in a word represents a node in a word completion tree, where each node represents a fragment building block suffix that is incrementally added to a prefix of prior fragments in the node branch navigation path.
  • Each partial node path forms a new incremental word fragment representing the start or root building block of still longer words.
  • each additional letter or fragment entered greatly reduces the number of relevant next letters or fragments choices, particularly after just the first 3 starting letters of most words.
  • Each node's child node group effectively represents a set of unique menu keys, each providing a potential building block suffix to the current word being edited.
  • WORD TREE DATABASE [0042] This invention requires a computer method and database containing the entire vocabulary of words derivable from a common word fragment prefix. Any possible word fragment, or database node, may be connected by branches to other word fragments, or child nodes. The only instances in which there is no further child node connection is at the termini of branches, where no child nodes are possible. In such cases, the terminal nodes are complete words that are not fragments of other longer words.
  • the terminal nodes are complete words that are not fragments of other longer words.
  • combination of a parent node and a child node is either a complete word (at the terminus of a branch), or a longer valid word fragment that gives rise to a subset of further children branches and nodes.
  • each fragment's child node fragments where possible, are chosen for maximum flexibility, meaning factors like brevity and minimum character length, resulting in fewer non-terminal nodes, and bigger trees with richer branch structure.
  • each node fragment is a suffix added to the sum of all ancestor node fragments traversed in a descending path starting from the tree root node fragment.
  • Each node fragment can be the end of a branch from which no additional child nodes with that parent node fragment embedded are possible.
  • a node has a set of one or more branches emanating from it leading to unique child node fragments from which longer words can be built.
  • Node fragments preferably have maximum flexibility, based on factors such as brevity and minimum character length. This results in fewer non-terminal nodes, providing fragments which tend to provide word completion building blocks, rather than full word completion endings. [0047] Thus trees are built from a set of the most common intermediate step fragments, resulting in a rich branch structure, which compared to
  • MAIN is both a word, but also the beginning root word fragment of approximately 35 other words, all deriving from any of the letters F, L, M, S, T appended to the end of "MAIN.”
  • a new parent word fragment root is "MAINT,” from which about nine possible child words may be derived from any of just four letter/word fragments menu choices: "AIN,” “ENANCE,” “OP,” and “OPS.”
  • Another important benefit to this approach is a greatly reduced key set required, particularly within just 2 to 3 starting letters of most words, which allows fewer characters to appear on screen at any one time, allowing for fewer, but larger keys, which are easier to see and use.
  • Another advantage of this method over prior art approaches is that the individual tree branches can be built out in a more dynamic and incremental fashion as the user starts typing (thus walking deeper down a particular branch completion path).
  • Another advantage over prior art is that the individual tree branches can be built out in a more dynamic and incremental fashion as the user starts typing, thus walking deeper down a particular branch completion path.
  • menus are provided of generally full length terminal word completion endings, similar to
  • the inventive tree building approach is subtly but importantly different in that it constructs words from shorter length intermediate word fragments, which can potentially take a more keystrokes, but allows for fewer larger keys on average.
  • terminal fragment completion options like "TAIN,” or "TENANCE,” rarely if ever single letter fragments except perhaps at end of words.
  • the invention allows keys to display word fragments in place of normal single letters, but retaining familiar, relative keyboard layout position based on the initial fragment letter.
  • An example is seen in Fig. 2C, element 112 showing key "ENANCE" in the normal QUERTY layout 'E' key position.
  • the invention runs a priority determination algorithm to ascertain which potential subsequent letters and fragments (keys) are likely to be most and least relevant and assigns a relative priority value to each key with the goal of providing more space for the most important keys, making them easier to use on small screens.
  • Key size is determined by multiple factors such as the layout format mode, but is generally a function of relative priority, the higher priority the larger the relative size, the lower the priority, the smaller the relative size.
  • keyboard/keyboard menu display can be navigated into view via
  • Letter/key priority can be based on multiple factors, such as potential word completion trees, and the most common word, letter frequencies and usage patterns, but can also factor in contextual data from myriad sources, such as GPS location, and building, vehicular or biomechanical device sensors, etc.
  • Word completion trees can include custom, personalized vocabulary, including shortcuts, slang and abbreviations, etc.
  • Work completion options can include both single letters and word fragments of variable length, where a word fragment is defined as 1 or more letters.
  • keys with varying priorities can share uniform size and also shape; an example is seen the alternating menus of Fig. 3C.
  • keys with significantly different priorities higher priority keys can be given relatively larger size and lower priority keys relatively smaller size, particularly when there's limited screen real estate. This is illustrated in Fig. 3F.
  • a static keyboard layout can constrain the size and shape of larger keys because of the need to retain the original layout relative key positioning. This can cause some keys to expand in size to a more rectangular shape seen in Fig 3E, the R key has expanded more horizontally resulting in a
  • the D key has also expanded unevenly, but in downward direction, creating a vertically oriented rectangular shape.
  • Non-uniform key shapes are permitted to achieve sizing objectives, e.g. more rectangular.
  • the general rule is that keys can expand to fill adjacent vacant spaces; such expansion usually happens in the normal column/row orientation of most keyboards.
  • Alternating menus is a mode that shows two rows of menu keys that appear above and below each other, thus conserving space. This mode is more appropriate when key menus have a relatively similar number of keys which fit comfortably into a single onscreen row, a circumstance more common when editing closer to the end of a word.
  • Menu and submenu display can occur in alternating directions, with a repeating vertical pattern of up/down style, or a side to side left/right style (exact pattern determined by the first direction of menu display). This allows the user to navigate into indefinitely deep submenu levels while using a maximum screen area equivalent to just two submenus, since the menu screen space is constantly reused.
  • SPRING LOADED ALTERNATING MENUS SPRING LOADED ALTERNATING MENUS
  • Another big advantage is that the 'bread crumbs' of the user's pointer navigation path through the key menus can be used to automatically enter text associated with each key thus not requiring explicit key selection. This can be extremely helpful in many tricky situations, for example single hand device usage, etc.
  • This invention permits the use of multiple keyboard menu styles that can change mid- word literally with every keystroke.
  • a sizing algorithm determines that there is adequate space and common enough priority for all vowel keys to be resized in larger uniform shape, with keys positioned in row fashion above and adjacent to the just triggered 'M' key, shown in Fig. 3B.
  • the layout algorithm produces a more circular layout pattern of "A,E,I,0,U,Y" in Fig. 3C, designed to bring the next key menu options closer to the previously triggered 'M' key, which can reduce pointer travel distance to the next key selection.
  • Fig. 3F optionally shows a more limited key set for "MAI”, hiding very low priority keys, available however via an "ALT" control key, resulting in much larger keys, with the "S, N” being good examples.
  • Figs. 3G, 3H, and 31 illustrate a mechanism where menu selections are shown bolded.
  • Fig 3H shows selection of "T”, building “MAINT”, then displaying the next menu key set "AIN, Enance, OP, OPS” in a downward alternating direction.
  • "Enance” is shown with a larger “E” and smaller following letters. Note how word fragment keys are display in alphabetical order.
  • Figure 31 shows a similar pattern with the selection of "AIN”, hiding the “F,L,M,S,T” menu, then displaying the child menu key options of the "AIN" fragment, "S, ER, ED, ING".
  • the invention circumstantially makes the most relevant letter/key options easier to use by temporarily increasing relative size and/or
  • highlighting while optionally de-emphasizing less relevant keys via any combination of decreased relative size, disabling, un-highlighting, graying out, or hiding, etc.
  • Fig. 2A illustrates how the typing of "main" dynamically results in larger, highlighted "F, L, M, S, T" keys, which retain their relative QWERTY keyboard locations, particularly relative to each other; and how they overlap and in some cases hide irrelevant keys.
  • Fig 2B illustrates how overlapped keys underneath can also be displayed via translucence to further help retain layout familiarity and thus guide the user.
  • Space permitting, keys can display any combination of relevant single letters and/or word fragments, including just partial fragments, or even entire complete words.
  • Word fragments are always positioned in the same approximate normal keyboard locations as their optionally highlighted starting letter; for example in Figure 2C, “ain” is displayed approximately where the "A” key is normally positioned, with the letter A capitalized for highlight purposes.
  • a letter has multiple associated word fragment completions, they can be displayed using multiple keys or via popup menu.
  • Starting letter highlighting can take any form, for example font type, style, size, color, graphic embellishments, etc.
  • Overlapped keys can, if desired, remain selectable by gestures such as longer and/or harder (variable pressure and/or duration) key press, or sliding motion into the overlapped key region.
  • relative sizing/highlighting can also be applied, reflecting the most and least likely choices, thus helping guide the user.
  • the most likely choice "enance” has a much larger relative key size and font, compared to the other completion key options.
  • the system can optionally display each key menu in circular layout formats, containing a center area displaying the currently selected key, surrounded outside by the center selection's next key menu options, arranged concentrically radiating out from the center.
  • Layouts can be in concentric pie slice format (Figs. 4A and 4C), or just traditional keys (Fig. 4B)
  • user can select the next menu item by moving the pointer from the center into a key selection area. Selection can occurs only after the pointer has moved outwards far enough to remove any possible ambiguity of choice when a heavily populated menu selection is presented, or selection can involve a gesture like longer key press or a small pointer movement pattern, etc.
  • the center area displays the menu item currently selected - whenever the pointer is over the center, the key regions display the selected items child branch menu items.
  • each key area region represents a single menu item
  • each menu item can be represented as buttons arranged in either concentric circular rows surrounding the center area, or in spoke fashion.
  • Buttons can be irregular in shape and size to better accommodate and optimize varying button label requirements, and also take advantage of greater space availability, for example farther away from the center.
  • Figs 4A to 4G illustrate circular menu operations
  • Fig. 4D shows keyboard selection of "N”, triggering a display of keys arranged in a circular fashion, showing the new word fragment "MAIN”, also showing newly selected "N” in center, surrounded by the next menu of possible word completion options;
  • Fig. 4E also shows the next selected menu item "T” in bold, triggering a new menu of word fragments in a circular arrangement around the central letter T.
  • "T” is selected by sliding the pointer from center ("N") into “T” and back into the center again to complete.
  • "T” can be simply tapped or selected by pointer location pause, or mouse click.
  • Figs 4F and 4G continue the same pattern resulting in the terminal word completion of "MAINTAINING".
  • Tiny screens often lack space for display of more than a single row of keys large enough to be truly usable. This invention solves this problem by allowing keyboards and keyboard menus to temporarily overlap and float on top of text display content editing areas.
  • either one or both rows of alternating menus can overlap edit content areas.
  • An example is illustrated in Fig. 6.
  • Sequential word completion menus can also be shown in single row mode if extremely limited space.
  • the current menu row disappears and the next row menu sequence appears in its place - this can be accomplished with slide/scroll/dissolve effects if desired.
  • key T is selected, creating "MAINT”, next causing key menu "F, L, M, S, T” to be replaced by key menu "AIN, ENANCE, OP, OPS", seen in Fig. 5B.
  • Next "AIN” is selected, creating “MAINTAIN”, triggering

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Input From Keyboards Or The Like (AREA)
  • User Interface Of Digital Computer (AREA)
  • Machine Translation (AREA)

Abstract

A method and apparatus is disclosed for entering text on a computer screen with a virtual keyboard and limited space, such as would be found on a mobile device such as smartphone, tablet, handheld computer or an automobile dashboard, or an appliance with a small screen. The method requires a sequence of events in which text is selected on the virtual keyboard, and a last key selected generates a new limited set of responsive keys, potentially based on word completion word fragments, for display on the screen, allowing responsive keys to be dynamically larger, variable in shape, or positioned on the virtual keyboard based on priority and availability of adjacent space to maximize the ease of use of the highest priority responsive keys. The user selects one of the responsive keys to partially complete or complete text entry.

Description

SPACE OPTIMIZING MICRO KEYBOARD METHOD AND
APPARATUS
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This application claims priority to provisional applications No.
61/799,004, filed March 15, 2013, and 61/853,575 filed April 6, 2013, the entire contents of which are incorporated herein by reference.
TECHNICAL FIELD
[0002] This invention pertains to the entry of text on computer screens having a virtual keyboard and limited screen space, for example as would be found on a handheld computer, smartphone, or automobile dashboard.
BACKGROUND OF THE INVENTION
[0003] Virtual keyboards are a text input method employed on computer screens used as computer user interfaces, in which the computer renders an image of a keyboard, and the user selects "keys" directly from image of the key. Virtual keyboards may rely on touch screens, in which the user touches the screen with a finger or a stylus, but other virtual keyboards are possible, relying on mouse key selections, eye movements, or other actions taken by a user to select a key on a virtual keyboard. Thus, virtual keyboards do not have actual physical keys with a physical switch, as in a conventional computer keyboard, but rely entirely on the touching of the screen or other selection method to select the key desired by the user. Each "key" on a virtual keyboard is nothing more than a region on a screen that is selectable. [0004] Virtual keyboards are commonplace in a plethora of devices, including smartphones, tablet computers, automobile dashboards, industrial
equipment, and home appliances.
[0005] Virtual keyboards are often presented in the traditional "QWERTY' format, but alternative keyboard layouts have been developed, and can be readily implemented using a virtual keyboard.
[0006] A typical feature of virtual keyboards, regardless of the device, is that virtual keyboards are usually employed in situations involving devices with rigid space limitations. Prime examples are smartphones or other handheld computers, in which small physical size is a critical attribute. Such small size creates challenges in entering and displaying text and graphics.
[0007] A common issue with virtual keyboards on small screens is that the small keys are hard to see and accurately select, make typing slow, tedious, difficult, and highly error prone.
[0008] However, even if a virtual keyboard is not presented on a highly space constrained device, typing and data entry on a virtual keyboard is usually more difficult than a conventional full size keyboard. There is a lack of tactile feedback and other cues present on conventional keyboards, which makes typing even on a larger virtual keyboard slow, difficult, and error prone.
[0009] Computer makers on devices such as smartphones have addressed this issue with various methods for predictive word completion systems. One such method is disclosed in U.S. Patent No. 7,886,233, (the '233 patent) which discloses various menus that appear when text is typed that guesses what the user might want to say. [0010] Typically, such systems start with two computer screen regions. A lower region displays the virtual keyboard, and an upper region displays edit content, such as a text message, email message, or other document.
[0011] The '233 patent discloses a typical prior art embodiment for predictive word completion in which a third region is rendered between the virtual keyboard and edit content regions of the screen that displays (typically in a single line) several word completion options. See, for example Fig. 10 in the '233 patent. The '233 patent itself discloses word completion candidates or fragments thereof that appear in the area that would otherwise be used by the virtual keyboard space bar (Fig. 11 and col. 9, 1 40).
[0012] Another prior art approach is shown in U.S. Patent No. 6,002,390, in which substantial menus showing word completion candidates are displayed.
SUMMARY OF THE INVENTION
[0013] This invention discloses methods and apparatus that transform normal keyboards into highly dynamic menu systems where keyboard layouts and key labels are high flexible and can change and adapt to provide users the largest possible keys on a keystroke by keystroke basis. A limited set of keys is generated dynamically to generate keys optimizing the space, size, and visibility of each key.
[0014] Traditional keyboards display full and extensive sets of letters and symbols, creating significant usage problems on small screens; by contrast, this invention filters and strips these down to bare essential keys, thus creating a dynamically changing keyboard menu system, designed to selectively optimize size and ease of use of each menu of keys. [0015] Thus, in an embodiment, this invention provides a method and apparatus for word completion during text input by a user on a computer apparatus having a virtual keyboard user interface, in which the user interface has limited screen space to display the virtual keyboard and any other information. The method and apparatus employ a sequence in which the user enters text by selecting keys on the virtual keyboard for one or more letters comprising a partial word, wherein each partial word has a "last letter key," which is the last key (in time) selected by the user. This last key need not be the final key in a word, or word fragment. Thus, the last letter key includes letter that are embedded within a word or word fragment.
[0016] The next step in the sequence of this embodiment is that the user is presented with a computer generated set of keys on the virtual keyboard in response to the last letter key, called "responsive keys." The responsive keys are prioritized by some method, which may include statistical occurrence of such letters, or may be based on some other rational context, such as locations of nearby features on a map.
[0017] There may be a computer generated means to adjust the appearance of the responsive keys into uniform or non-uniform shapes dynamically variable in size, shape, or position on the virtual keyboard based on priority and availability of adjacent space to maximize the ease of use of the highest priority responsive keys.
[0018] A final step of the method or apparatus in this embodiment is that the user selects one of the responsive keys to partially complete or complete a word.
[0019] In another embodiment, a computer implemented menu generation system is provided, based on word trees. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 shows the inventive word tree systems.
[0021] Figure 2 shows variable size keys resized in their approximately normal QWERTY location.
[0022] Figure 3 shows a sequence of typing examples illustrating the variable reduced key set menus.
[0023] Figure 4 shows circular menu formats.
[0024] Figure 5 shows single row mode of operation.
[0025] Figure 6 shows key menus overlapping with edit content.
DETAILED DESCRIPTION OF THE INVENTION
[0026] This invention describes methods and apparatus that are particularly adapted for use on space constrained virtual keyboards, such as are used on smart phones, compact handheld computers, automobile dashboards, and other applications where cost or physical limitations (for example, the need to hold a device in a human hand) limit the screen space available. Such space constrained devices present challenges in the efficient entry of text and data. However, the inventive methods and apparatus are not limited to space constrained keyboards, and can also be valuable in larger devices to expedite text entry in certain situations.
[0027] Traditional prior art keyboards typically have these features:
• Static layout ( e.g. QWERTY ) • Relatively single mode of operation
• display of the entire alphabetical letter set at all times
• single letter keys
• uniform key size and symmetric shape.
• Few if any additional modes of operation, which if present, generally require manual switching
[0028] In an embodiment, the instant invention provides modified keyboards, particularly suited to virtual keyboard computer interfaces, in which word fragment keys can be rendered showing one or more letters, in non-standard shapes and modified locations on the region of the screen used as the virtual keyboard. Additionally, this invention may only display keys relevant to specific word completion scenarios. In another embodiment, key size and shape can be dynamically determined. In another embodiment, various modes of operation can changed automatically based on the appropriate context. By the phrase "modes of operation," it is meant that different modes can be displayed as the context requires, for example, switching from a standard QWERTY display to a spring loaded mode displaying a limited number of keys, as is more fully described herein.
[0029] All features of this invention can be dynamically responsive to very specific word completion scenarios on a keystroke by keystroke basis.
[0030] During text entry, words can be constructed from a sequence of word fragments, where a word fragment is defined in this context as one or more letters, which function as building blocks of longer words, where fragments themselves can even be complete words. [0031] Traditional keyboards, such as having a "QWERTY' layout, use a static layout of single letter (fragment) key input to construct words, thus requiring an entire alphabet of keys. However, word fragments longer than one letter reduce the number of key presses required to enter desired text, and require fewer keys and less space.
[0032] In the inventive method, a database is constructed comprising trees of words emanating from a root word fragment. This database is constructed in advance. Such databases actually exist, and can be accessed for example, at htl7p://www.scrabblefindei com/starts-with/xxx/ (where "xxx" represents a root word fragment). A root word fragment can be a complete word, like "main." But even a complete word like "main" can also be a root fragment for longer words, for example "maintain."
[0033] Each branch in the word tree database terminates in a node that is a suffix fragment that can be combined with the root word fragment. In some cases, the combined root and suffix make a complete word. Regardless of whether the combined root and suffix make a complete word, the combined root and suffix can be used as a child root fragment in the word tree database that leads to a subset of branches, narrowing the choices of complete words that can be generated as the root grows in size.
[0034] In the inventive method, the typing (data entry) of words can be visualized as navigating or walking down the branches of a word completion tree, via a menu of fragment choices.
[0035] A much more compact form of this type of tree has branch leaf nodes representing word fragments, where "word fragment" or just "fragment" is defined in this context as one or more letters, which function as pieces or building blocks of longer words, where fragments themselves can even be complete words. For example the word "MAINTAINING" can be formed from the fragment sequence "MAIN" "T", "AIN", "ING".
[0036] An example is seen in Fig.XX, where the fragment "MAINT" has branch node fragment completion options "AIN," "ENANCE," "OP," and "OPS" (element 100 ).
[0037] The starting fragment of each word forms the root parent node of a word completion tree, with branch leaves represented by child fragment nodes.
[0038] Each node can be the parent of child nodes extending downwards another branch level. This branching pattern is repeated downwards until all possible words deriving from the starting root node fragment are represented, where terminal nodes (those lacking child nodes) represent the final fragments of complete words.
[0039] Thus each letter or fragment in a word represents a node in a word completion tree, where each node represents a fragment building block suffix that is incrementally added to a prefix of prior fragments in the node branch navigation path. Each partial node path forms a new incremental word fragment representing the start or root building block of still longer words.
[0040] As words are typed, each additional letter or fragment entered greatly reduces the number of relevant next letters or fragments choices, particularly after just the first 3 starting letters of most words.
[0041] Each node's child node group effectively represents a set of unique menu keys, each providing a potential building block suffix to the current word being edited.
WORD TREE DATABASE [0042] This invention requires a computer method and database containing the entire vocabulary of words derivable from a common word fragment prefix. Any possible word fragment, or database node, may be connected by branches to other word fragments, or child nodes. The only instances in which there is no further child node connection is at the termini of branches, where no child nodes are possible. In such cases, the terminal nodes are complete words that are not fragments of other longer words. The
combination of a parent node and a child node is either a complete word (at the terminus of a branch), or a longer valid word fragment that gives rise to a subset of further children branches and nodes.
[0043] In a preferred word completion tree embodiment, each fragment's child node fragments, where possible, are chosen for maximum flexibility, meaning factors like brevity and minimum character length, resulting in fewer non-terminal nodes, and bigger trees with richer branch structure.
[0044] In a preferred embodiment of each word completion tree, each node fragment is a suffix added to the sum of all ancestor node fragments traversed in a descending path starting from the tree root node fragment.
[0045] Each node fragment can be the end of a branch from which no additional child nodes with that parent node fragment embedded are possible. Alternatively, a node has a set of one or more branches emanating from it leading to unique child node fragments from which longer words can be built.
[0046] Node fragments preferably have maximum flexibility, based on factors such as brevity and minimum character length. This results in fewer non-terminal nodes, providing fragments which tend to provide word completion building blocks, rather than full word completion endings. [0047] Thus trees are built from a set of the most common intermediate step fragments, resulting in a rich branch structure, which compared to
conventional methods, provides faster, more convenient branch navigation to every possible word derivable from any root or node fragment anywhere in the tree.
[0048] For example "MAIN" is both a word, but also the beginning root word fragment of approximately 35 other words, all deriving from any of the letters F, L, M, S, T appended to the end of "MAIN."
[0049] For example, if the letter T is selected as the next letter in the sequence, a new parent word fragment root is "MAINT," from which about nine possible child words may be derived from any of just four letter/word fragments menu choices: "AIN," "ENANCE," "OP," and "OPS."
[0050] An important benefit of this approach is that it allows maximum flexibility in constructing long and complex words from small fragment building block suffixes.
[0051] Another important benefit to this approach is a greatly reduced key set required, particularly within just 2 to 3 starting letters of most words, which allows fewer characters to appear on screen at any one time, allowing for fewer, but larger keys, which are easier to see and use.
[0052] Another advantage of this method over prior art approaches is that the individual tree branches can be built out in a more dynamic and incremental fashion as the user starts typing (thus walking deeper down a particular branch completion path).
[0053] Another benefit is this approach allows fewer characters to appear on screen at any one time, allowing for larger key and key label sizes, which are easier to see and use. This is a critical advantage on small virtual keyboard screens.
[0054] Another advantage over prior art is that the individual tree branches can be built out in a more dynamic and incremental fashion as the user starts typing, thus walking deeper down a particular branch completion path.
[0055] By contrast, as shown in the '233 patent, menus are provided of generally full length terminal word completion endings, similar to
conventional word completion methods. So instead of providing menus of suggested completion word endings, only entire words are shown in prior art approaches. This greatly limits the flexibility of the word completion method, and requires too many options than are possible to be displayed efficiently, especially in a space constrained keyboard.
[0056] The inventive tree building approach is subtly but importantly different in that it constructs words from shorter length intermediate word fragments, which can potentially take a more keystrokes, but allows for fewer larger keys on average.
[0057] For example when the inventive system receives the word "MAIN" (Fig. 1, no. 100), the next key menu may be "F, L, M, S, T" (Fig. 1, no. 102). By contrast, the method disclosed in the '233 patent would show full
(terminal fragment) completion options like "TAIN," or "TENANCE," rarely if ever single letter fragments except perhaps at end of words.
[0058] The invention allows keys to display word fragments in place of normal single letters, but retaining familiar, relative keyboard layout position based on the initial fragment letter. An example is seen in Fig. 2C, element 112 showing key "ENANCE" in the normal QUERTY layout 'E' key position. [0059] Following the entry of every new letter or fragment, the invention runs a priority determination algorithm to ascertain which potential subsequent letters and fragments (keys) are likely to be most and least relevant and assigns a relative priority value to each key with the goal of providing more space for the most important keys, making them easier to use on small screens.
[0060] Keys representing totally irrelevant fragments are hidden or shown de-activated (grayed out in the background), and remaining keys are allowed to expand in size to fill resulting vacant spaces, thus becoming easier to use.
[0061] For example in Fig. 1, following entry of letter N, creating MAIN, (100) the letters "F, L, M, S, T" are displayed in menu row 102, become the only relevant next additional letter/key entry options , thus the rest of the keys can be hidden, allowing much more space for relevant keys.
[0062] Key size is determined by multiple factors such as the layout format mode, but is generally a function of relative priority, the higher priority the larger the relative size, the lower the priority, the smaller the relative size.
[0063] Larger keys also permit the clearer display of significantly more data in each key, such as word fragments, helpful functionality notably lacking in prior art word completion mechanisms.
[0064] For example in Fig. 1, as letter T is entered creating "MAINT" (110), word fragment "AIN" ( element 114 ) then becomes a possible word fragment ending option; larger keys allow the full display of "AIN" text in normal keys, which can also be positioned in familiar keyboard locations; for example the "AIN" key can be displayed in the normal keyboard layout location of its starting letter Έ', shown in element XX of Fig. 3H. [0065] As the user starts typing a new word, there may little or no context information to help guide the key filtering process, thus a potentially large number of keys may need to be efficiently displayed. Off screen
keyboard/keyboard menu display can be navigated into view via
scrolling/swiping type gestures, or the use of control keys, etc. But even so, word and letter frequency can be used to control the relative size, position and accessibility of keys.
DETERMINING LETTER/FRAGMENT KEY PRIORITY
[0066] Letter/key priority can be based on multiple factors, such as potential word completion trees, and the most common word, letter frequencies and usage patterns, but can also factor in contextual data from myriad sources, such as GPS location, and building, vehicular or biomechanical device sensors, etc.
[0067] Word completion trees can include custom, personalized vocabulary, including shortcuts, slang and abbreviations, etc.
[0068] Work completion options can include both single letters and word fragments of variable length, where a word fragment is defined as 1 or more letters.
DETERMINATION OF RESIZED KEY SHAPE
[0069] Given adequate space, for example due to extremely few keys being shown, all displayed keys will normally expand in size to occupy available space vacated by hidden or de-activated keys.
[0070] In some cases keys with varying priorities can share uniform size and also shape; an example is seen the alternating menus of Fig. 3C. [0071] However, if keys have significantly different priorities, higher priority keys can be given relatively larger size and lower priority keys relatively smaller size, particularly when there's limited screen real estate. This is illustrated in Fig. 3F.
[0072] A static keyboard layout can constrain the size and shape of larger keys because of the need to retain the original layout relative key positioning. This can cause some keys to expand in size to a more rectangular shape seen in Fig 3E, the R key has expanded more horizontally resulting in a
rectangular shape; the D key has also expanded unevenly, but in downward direction, creating a vertically oriented rectangular shape.
[0073] Non-uniform key shapes are permitted to achieve sizing objectives, e.g. more rectangular. The general rule is that keys can expand to fill adjacent vacant spaces; such expansion usually happens in the normal column/row orientation of most keyboards.
ALTERNATING MENUS MODE
[0074] Alternating menus is a mode that shows two rows of menu keys that appear above and below each other, thus conserving space. This mode is more appropriate when key menus have a relatively similar number of keys which fit comfortably into a single onscreen row, a circumstance more common when editing closer to the end of a word.
[0075] Menu and submenu display can occur in alternating directions, with a repeating vertical pattern of up/down style, or a side to side left/right style (exact pattern determined by the first direction of menu display). This allows the user to navigate into indefinitely deep submenu levels while using a maximum screen area equivalent to just two submenus, since the menu screen space is constantly reused. SPRING LOADED ALTERNATING MENUS
[0076] Spring loaded functionality works very well as a driver of alternating menus as it allows next menus to open when a menu key region is not selected, but merely touch activated, thus providing extremely rapid navigation through a complex branching tree of options, making it well suited to word completion trees.
[0077] It also provides rapid "look ahead" previewing of possible next menu key options without requiring specific key selection, which is a huge help with word completion. Another advantage is that the user doesn't need to move a finger or pointer device off the screen at any stage.
[0078] Another big advantage is that the 'bread crumbs' of the user's pointer navigation path through the key menus can be used to automatically enter text associated with each key thus not requiring explicit key selection. This can be extremely helpful in many tricky situations, for example single hand device usage, etc.
COMBINING MULTIPLE KEYBOARD MENU STYLES AND MODES
[0079] This invention permits the use of multiple keyboard menu styles that can change mid- word literally with every keystroke.
[0080] For example in Fig. 3B, the user begins typing the word
"MAINTAINING" starting with the letter 'M', triggering display of vowel menu key set "Α,Ε,Ι,Ο,υ,Υ', the only word completion fragments that generally apply in this situation.
[0081] Next a sizing algorithm determines that there is adequate space and common enough priority for all vowel keys to be resized in larger uniform shape, with keys positioned in row fashion above and adjacent to the just triggered 'M' key, shown in Fig. 3B.
[0082] In an alternate menu embodiment, the layout algorithm produces a more circular layout pattern of "A,E,I,0,U,Y" in Fig. 3C, designed to bring the next key menu options closer to the previously triggered 'M' key, which can reduce pointer travel distance to the next key selection.
[0083] Note how the vowel key menus are displayed above a de-activated full QWERTY keyboard.
[0084] Next the user selects the letter Ά', which triggers a new cycle of prioritization, sizing and layout algorithms, resulting the in the QWERTY compatible keyboard menu seen in Fig. 3D, which combines with key hiding with key re-sizing, with many keys morphing from uniform square to nonuniform rectangular shapes.
[0085] Next the user selects the "I" key, creating "MAI", resulting in the new key menu set seen in Fig. 3E.
[0086] Fig. 3F optionally shows a more limited key set for "MAI", hiding very low priority keys, available however via an "ALT" control key, resulting in much larger keys, with the "S, N" being good examples.
[0087] Next the user selects the "N" key, creating "MAIN" in Fig. 3G, then displaying the "F,L,M,S,T" key menu. Because subsequent key menus are so small, the system can switch to an alternating menu mode, including spring loaded action, illustrated in Fig 3H.
ALTERNATING MENUS EXAMPLE [0088] Figs. 3G, 3H, and 31 illustrate a mechanism where menu selections are shown bolded.
[0089] In Fig 3G, the "N" is selected appending "N" to end of "MAI" forming word fragment "MAIN" in the editing display area, which triggers display of the next word completion menu keys, "F,L,M,S,T", displayed in alphabetical order from left to right, making it much easier to find the next key.
[0090] Fig 3H shows selection of "T", building "MAINT", then displaying the next menu key set "AIN, Enance, OP, OPS" in a downward alternating direction. To conserve space, "Enance" is shown with a larger "E" and smaller following letters. Note how word fragment keys are display in alphabetical order.
[0091] Figure 31 shows a similar pattern with the selection of "AIN", hiding the "F,L,M,S,T" menu, then displaying the child menu key options of the "AIN" fragment, "S, ER, ED, ING".
DYNAMIC KEY RELATIVE SIZING
[0092] The invention circumstantially makes the most relevant letter/key options easier to use by temporarily increasing relative size and/or
highlighting, while optionally de-emphasizing less relevant keys via any combination of decreased relative size, disabling, un-highlighting, graying out, or hiding, etc.
[0093] If the user prefers a static display mode, e.g. QUERTY, the invention requires all resized keys to remain positioned as close as possible to the user's normal keyboard layout location, thus retaining continuity with a familiar layout, eliminating or substantially reducing learning curves. [0094] Fig. 2A illustrates how the typing of "main" dynamically results in larger, highlighted "F, L, M, S, T" keys, which retain their relative QWERTY keyboard locations, particularly relative to each other; and how they overlap and in some cases hide irrelevant keys.
[0095] Fig 2B illustrates how overlapped keys underneath can also be displayed via translucence to further help retain layout familiarity and thus guide the user.
[0096] Space permitting, keys can display any combination of relevant single letters and/or word fragments, including just partial fragments, or even entire complete words.
[0097] For example, in Fig 2C, after typing "maint" the possible word fragment completions, "ain", "enance", "op", are displayed as key labels.
[0098] Word fragments are always positioned in the same approximate normal keyboard locations as their optionally highlighted starting letter; for example in Figure 2C, "ain" is displayed approximately where the "A" key is normally positioned, with the letter A capitalized for highlight purposes.
[0099] If a letter has multiple associated word fragment completions, they can be displayed using multiple keys or via popup menu.
[0100] If a letter has multiple associated word fragment completions, they can be displayed, space permitting, using multiple contiguous keys, illustrated in Figure 2C
[0101] Starting letter highlighting can take any form, for example font type, style, size, color, graphic embellishments, etc. [0102] Overlapped keys can, if desired, remain selectable by gestures such as longer and/or harder (variable pressure and/or duration) key press, or sliding motion into the overlapped key region.
[0103] Within the limited subset of word completion choices, relative sizing/highlighting can also be applied, reflecting the most and least likely choices, thus helping guide the user. For example in Figure 28, the most likely choice "enance" has a much larger relative key size and font, compared to the other completion key options.
CIRCULAR DISPLAY FORMATS
[0104] The system can optionally display each key menu in circular layout formats, containing a center area displaying the currently selected key, surrounded outside by the center selection's next key menu options, arranged concentrically radiating out from the center.
[0105] Layouts can be in concentric pie slice format (Figs. 4A and 4C), or just traditional keys (Fig. 4B)
[0106] Traditional key press type selection can be used, or a semi spring loaded alternating menu style is possible.
[0107] In non-traditional mode, user can select the next menu item by moving the pointer from the center into a key selection area. Selection can occurs only after the pointer has moved outwards far enough to remove any possible ambiguity of choice when a heavily populated menu selection is presented, or selection can involve a gesture like longer key press or a small pointer movement pattern, etc. [0108] The center area displays the menu item currently selected - whenever the pointer is over the center, the key regions display the selected items child branch menu items.
[0109] For example if the starting word fragment is "main" and the user selects "T" as the next letter, the center will display "T" and the outer pie slices will display the menu choices of "AIN, ENANCE, OP, OPS."
[0110] Normally each key area region represents a single menu item;
however if there are more menu items than can easily, conveniently fit, then each menu item can be represented as buttons arranged in either concentric circular rows surrounding the center area, or in spoke fashion.
[0111] The more likely high priority key fragments will normally be placed in the innermost rows or spoke locations.
[0112] Buttons can be irregular in shape and size to better accommodate and optimize varying button label requirements, and also take advantage of greater space availability, for example farther away from the center.
[0113] Figs 4A to 4G illustrate circular menu operations
[0114] Fig. 4D shows keyboard selection of "N", triggering a display of keys arranged in a circular fashion, showing the new word fragment "MAIN", also showing newly selected "N" in center, surrounded by the next menu of possible word completion options;
[0115] Fig. 4E also shows the next selected menu item "T" in bold, triggering a new menu of word fragments in a circular arrangement around the central letter T. In a spring loaded menu embodiment, "T" is selected by sliding the pointer from center ("N") into "T" and back into the center again to complete. Alternatively, "T" can be simply tapped or selected by pointer location pause, or mouse click.
[0116] Figs 4F and 4G continue the same pattern resulting in the terminal word completion of "MAINTAINING".
KEYBOARD OVERLAP WITH EDIT CONTENT
[0117] Tiny screens often lack space for display of more than a single row of keys large enough to be truly usable. This invention solves this problem by allowing keyboards and keyboard menus to temporarily overlap and float on top of text display content editing areas.
[0118] For example either one or both rows of alternating menus can overlap edit content areas. An example is illustrated in Fig. 6.
SINGLE ROW KEYBOARD MENUS
[0119] Sequential word completion menus can also be shown in single row mode if extremely limited space. When a key is selected, the current menu row disappears and the next row menu sequence appears in its place - this can be accomplished with slide/scroll/dissolve effects if desired. For example in Fig. 5A, key T is selected, creating "MAINT", next causing key menu "F, L, M, S, T" to be replaced by key menu "AIN, ENANCE, OP, OPS", seen in Fig. 5B. Next "AIN" is selected, creating "MAINTAIN", triggering
replacement of current key menu with "S, ER, ED, ING" in Fig. 5C.

Claims

A method for word completion during text input by a user on a computer apparatus having a user interface comprising a virtual keyboard, wherein said user interface has limited screen space to display the virtual keyboard and any other information, comprising in sequence: a. the user entering text by selecting keys on the virtual keyboard for one or more letters comprising a partial word, wherein each partial word has a last letter key comprising the last key selected by the user; b. a computer generated means for presenting the user with a set of keys on the virtual keyboard in response to said last letter key, wherein said set of responsive keys is prioritized; c. a computer generated means to adjust the appearance of said set of responsive keys into uniform or non-uniform shapes dynamically variable in size, shape, or position on the virtual keyboard based on priority and availability of adjacent space to maximize the ease of use of the highest priority responsive keys; and d. wherein the user selects one of the responsive keys to partially complete or complete a word.
The method of claim 1, wherein the responsive keys comprise single letters or word fragments.
The method of claim 1, repeated iteratively, wherein selection of a responsive key generates a second set of prioritized responsive keys.
4. The method of claim 1, wherein higher priority responsive keys are displayed with a larger area than lower priority responsive keys.
5. The method of claim 1, wherein the virtual keyboard is dynamically modified to display a set of responsive keys within the virtual keyboard and in close proximity to the last letter key.
6. The method of claim 5, wherein the responsive keys are displayed in alphabetical order.
7. The method of claim 5, wherein the responsive keys vary in size based on priority, and are displayed in relative positions that such keys would appear on a standard QWERTY keyboard.
8. The method of claim 5, wherein the responsive keys vary in size based on priority and are arranged on the virtual keyboard above, right, left, and below the last letter key.
9. The method of claim 5, wherein the responsive keys are displayed in a single row.
10. The method of claim 5, wherein the responsive keys are displayed in two rows.
11. The method of claim 1, further comprising a spring loaded menu
comprising a means for displaying menu items in response to a pointer mechanism entering a display trigger zone for said menu, and hiding said menu items when the pointer exits said display trigger zone, wherein no selection of a menu is made by such pointer mechanism entering said display trigger zone; wherein the responsive keys are displayed in a spring loaded menu.
12. The method of claim 1, wherein each first letter of a suffix selected by the user creates a new final letter key for the generation of additional suffixes.
13. The method of claim 1, wherein the last letter key is embedded within a word or partial word.
14. The method of claim 1, wherein the virtual keyboard is a user interface for a compact handheld computer.
15. The method of claim 1, wherein the responsive dynamically shaped keys expand to fill available space using adjacent space while remaining anchored in a relatively familiar location based on the original layout.
16. The method of claim 1, wherein the responsive dynamically shaped keys vary in shapes selected from generally square, rectangular, triangular, and circular shapes.
17. The method of claim 1, wherein the prioritization of responsive keys is determined by a computer algorithm based on factors selected from statistical incidence of letters and words in a language being employed and predetermined contextual significance.
18. The method of claim 1, wherein the higher priority responsive keys are visually highlighted using a method selected from: using a larger font than surrounding keys; a bold font; a bold border compared to surrounding keys; blinking of background, font, or border of key; and contrasting key background color, font color or border color.
19. The method of claim 1, wherein the higher priority keys have increased trigger sensitivity compared to lower priority keys.
20. A method for word completion during text input by a user on a computer apparatus having a user interface comprising a virtual keyboard, wherein said user interface has limited screen space to display the virtual keyboard and any other information, and said user interface is divided into at least two regions, comprising a first region comprising a virtual keyboard and at least a second region comprising edit content, the method comprising, in sequence: a. the user entering text by selecting keys on the virtual keyboard for one or more letters comprising a partial word, wherein each partial word has a last letter key comprising the last key selected by the user; b. a computer generated means for presenting the user with a set of keys, generated in response to said last letter key, wherein said set of responsive keys floats over the edit content region of the user interface; c. a computer generated means to adjust the appearance of said set of responsive keys into uniform or non-uniform shapes dynamically variable in size, shape, or position on the virtual keyboard based priority and availability of adjacent space to maximize the ease of use of the highest priority responsive keys; and d. wherein the user selects one of the responsive keys to partially complete or complete a word.
21. The method of claim 20, wherein the virtual keyboard is hidden from view after the responsive keys are presented to the user.
22. A method for word completion during text input by a user on a computer apparatus having a user interface comprising a QWERTY virtual keyboard, wherein said user interface has limited screen space to display the virtual keyboard and any other information, comprising in sequence: a. the user entering text by selecting keys on the virtual keyboard for one or more letters comprising a partial word, wherein each partial word has a last letter key comprising the last key selected by the user; b. a computer generated means for presenting the user with a set of keys on the virtual keyboard in response to said last letter key, wherein said set of responsive keys is prioritized; c. a computer generated means to adjust the appearance of said set of responsive keys into uniform or non-uniform shapes dynamically variable in size, shape, or position on the virtual keyboard based priority and availability of adjacent space to maximize the ease of use of the highest priority responsive keys, while retaining the relative position of each responsive key in its normal arrangement in the QWERTY keyboard layout; d. wherein the user selects one of the responsive keys to partially complete or complete a word.
23. A user interface for word completion during text input by a user on a computer apparatus comprising a virtual keyboard, wherein said user interface has limited screen space to display the virtual keyboard and any other information, said interface comprising in sequence: a. the user entering text by selecting keys on the virtual keyboard for one or more letters comprising a partial word, wherein each partial word has a last letter key comprising the last key selected by the user; b. a computer generated means for presenting the user with a set of keys on the virtual keyboard in response to said last letter key, wherein said set of responsive keys is prioritized; c. a computer generated means to adjust the appearance of said set of responsive keys into uniform or non-uniform shapes dynamically variable in size, shape, or position on the virtual keyboard based on priority and availability of adjacent space to maximize the ease of use of the highest priority responsive keys; and d. wherein the user selects one of the responsive keys to partially complete or complete a word.
A computer-implemented menu generation method for a text input method for optimized data entry on a space constrained keyboard, wherein a user at a computer interface enters a root word fragment comprising one or more letters, the method comprising— a. a word tree database, comprising nodes comprising all partially spelled and complete words that branch from any possible root word fragment, wherein each node on a branch in the word tree database is a word completion candidate, and wherein each node is also a starting root word fragment for child branches of a subset of word completion candidates, if any, that can be derived from said node; b. wherein in response to the entry of a root word fragment in the computer interface, the menu generation method retrieves a list of all word completion candidates from the word tree database for the specific root word fragment; c. wherein for the list of all word completion candidates, the menu generation method creates a non-redundant list of suffix word fragments of one or more letters for each word completion candidate; d. wherein a second computer implemented method presents each non-redundant suffix word fragment to the user in a menu, and the computer interface permits the user to select one word fragment on the menu, wherein the combination of the specific root word fragment and the selected word fragment from the menu creates a child word fragment, wherein the child word fragment may comprise a complete word or a child specific word fragment, wherein any child specific word fragment may be used to retrieve word completion candidates from the child branch in the word tree database based on the child specific word fragment.
25. The method of claim 24, wherein the non-redundant list of suffix word fragments is prioritized.
26. A computer-implemented menu generation method for a text input method for optimized data entry on a space constrained keyboard, wherein a user at a computer interface enters a root word fragment comprising one or more letters, the method comprising— a word tree database, comprising nodes comprising all partially spelled and complete words that branch from any possible root word fragment, wherein each node on a branch in the word tree database is a word completion candidate, and wherein each node is also a starting root word fragment for child branches of a subset of word completion candidates, if any, that can be derived from said node; wherein in response to the entry of a root word fragment in the computer interface, the menu generation method retrieves a list of all word completion candidates from the word tree database for the specific root word fragment; wherein for the list of all word completion candidates, the menu generation method creates a non-redundant list of suffix word fragments of one or more letters for each word completion candidate, wherein each letter in the non-redundant list is assigned a priority; wherein a second computer implemented method presents each non-redundant suffix word fragment to the user in a menu, wherein the display of each letter in the menu is dependent on the priority that letter, and the computer interface permits the user to select one word fragment on the menu, wherein the combination of the specific root word fragment and the selected word fragment from the menu creates a child word fragment, wherein the child word fragment may comprise a complete word or a child specific word fragment, wherein any child specific word fragment may be used to retrieve word completion candidates from the child branch in the word tree database based on the child specific word fragment.
27. The prioritized list of claims 24 or 26, wherein the computer interface comprises a virtual keyboard, and each letter on the menu is displayed on the virtual keyboard as a special key, and the size, shape, visibility, position, or selection sensitivity of each special key is dependent on the priority.
28. The method of claims 24 or 26, wherein the priority is based on
statistical occurrences of word completion candidates.
29. The method of claims 24 or 26, wherein the priority is based on the geographical context of the user.
30. The method of claims 24 or 26, wherein the priority is based on the linguistic context of the subject matter being typed.
31. The method of claims 24 or 26, wherein each node in the word tree is prioritized.
32. The method of claims 24 or 26, wherein the word tree database
branches are tuned according to priority or according to predetermined factors selected from minimum fragment length, maximum fragment length, and average fragment length, wherein the tuning also depends on screen size and screen space available at any given moment.
33. The method of any of claims 24 to 32 in which the tuning and
prioritization steps are selected so as to minimize the number of word construction steps for a user to arrive at a desired complete word.
PCT/US2014/031121 2013-03-15 2014-03-18 Space optimizing micro keyboard method and apparatus WO2014146131A1 (en)

Priority Applications (8)

Application Number Priority Date Filing Date Title
CA2941832A CA2941832A1 (en) 2013-03-15 2014-03-18 Space optimizing micro keyboard method and apparatus
BR112015023799A BR112015023799A2 (en) 2013-03-15 2014-03-18 SPACE OPTIMIZING MICRO METHOD OF KEYBOARD AND DEVICE
CN201480027568.0A CN105247452A (en) 2013-03-15 2014-03-18 Space optimizing micro keyboard method and apparatus
JP2016502610A JP2016513849A (en) 2013-03-15 2014-03-18 Method and apparatus for space optimized micro keyboard
US14/776,742 US10235042B2 (en) 2013-03-15 2014-03-18 Space optimizing micro keyboard method and apparatus
MX2015013077A MX2015013077A (en) 2013-03-15 2014-03-18 Space optimizing micro keyboard method and apparatus.
EP14765337.2A EP2972689A4 (en) 2013-03-15 2014-03-18 Space optimizing micro keyboard method and apparatus
US16/270,545 US11061561B2 (en) 2013-03-15 2019-02-07 Space optimizing micro keyboard method and apparatus

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US201361799004P 2013-03-15 2013-03-15
US61/799,004 2013-03-15
US201361853575P 2013-04-06 2013-04-06
US61/853,575 2013-04-06

Related Child Applications (2)

Application Number Title Priority Date Filing Date
US14/776,742 A-371-Of-International US10235042B2 (en) 2013-03-15 2014-03-18 Space optimizing micro keyboard method and apparatus
US16/270,545 Continuation US11061561B2 (en) 2013-03-15 2019-02-07 Space optimizing micro keyboard method and apparatus

Publications (1)

Publication Number Publication Date
WO2014146131A1 true WO2014146131A1 (en) 2014-09-18

Family

ID=51538194

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2014/031121 WO2014146131A1 (en) 2013-03-15 2014-03-18 Space optimizing micro keyboard method and apparatus

Country Status (7)

Country Link
US (2) US10235042B2 (en)
EP (1) EP2972689A4 (en)
JP (1) JP2016513849A (en)
CN (1) CN105247452A (en)
BR (1) BR112015023799A2 (en)
MX (1) MX2015013077A (en)
WO (1) WO2014146131A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017023185A1 (en) * 2015-08-06 2017-02-09 Общество С Ограниченной Ответственностью "1С Виарабл" Method, device and system for inputting and displaying data on a touchscreen
WO2017176335A1 (en) * 2016-04-04 2017-10-12 Google Inc. Dynamic key mapping of a graphical keyboard

Families Citing this family (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101742359B1 (en) * 2013-12-27 2017-05-31 주식회사 두산 Organic electro luminescence device
KR101561783B1 (en) * 2014-10-14 2015-10-19 천태철 Method for inputing characters on touch screen of terminal
US10725659B2 (en) 2014-10-14 2020-07-28 Tae Cheol CHEON Letter input method using touchscreen
US20170031461A1 (en) * 2015-06-03 2017-02-02 Infosys Limited Dynamic input device for providing an input and method thereof
US10853378B1 (en) * 2015-08-25 2020-12-01 Palantir Technologies Inc. Electronic note management via a connected entity graph
US20180204227A1 (en) * 2015-09-21 2018-07-19 Asheesh Mohindru Golf Pace of Play
US20170083231A1 (en) * 2015-09-23 2017-03-23 Honeywell International Inc. Database driven input system
CN106610730A (en) * 2015-10-23 2017-05-03 深圳富泰宏精密工业有限公司 Electronic equipment and character correction method
JP6702177B2 (en) 2016-12-26 2020-05-27 トヨタ自動車株式会社 Vehicle display device and display method in vehicle display device
US10234985B2 (en) * 2017-02-10 2019-03-19 Google Llc Dynamic space bar
US10671181B2 (en) * 2017-04-03 2020-06-02 Microsoft Technology Licensing, Llc Text entry interface
AT520031A1 (en) * 2017-06-07 2018-12-15 Caretec Int Gmbh Apparatus and methods of machine writing and virtual reading of volatile tactile characters and acoustic sounds
CN107168552A (en) * 2017-06-20 2017-09-15 北京京东尚科信息技术有限公司 Input method, input unit and electronic equipment
KR102054517B1 (en) * 2017-11-15 2019-12-11 주식회사 비트바이트 Method for providing interactive keyboard and system thereof
KR102608953B1 (en) * 2018-09-06 2023-12-04 삼성전자주식회사 Electronic apparatus and controlling method thereof
WO2020240578A1 (en) * 2019-05-24 2020-12-03 Venkatesa Krishnamoorthy Method and device for inputting text on a keyboard
US11983382B2 (en) * 2022-06-13 2024-05-14 Illuscio, Inc. Systems and methods for generating three-dimensional menus and toolbars to control computer operation

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6611253B1 (en) * 2000-09-19 2003-08-26 Harel Cohen Virtual input environment
US20040104896A1 (en) * 2002-11-29 2004-06-03 Daniel Suraqui Reduced keyboards system using unistroke input and having automatic disambiguating and a recognition method using said system
US20090193334A1 (en) * 2005-05-18 2009-07-30 Exb Asset Management Gmbh Predictive text input system and method involving two concurrent ranking means
KR20110088310A (en) * 2010-01-28 2011-08-03 박태운 Hanalgeul input method for korean text
US8300016B2 (en) * 2008-05-02 2012-10-30 Hon Hai Precision Industry Co., Ltd. Electronic device system utilizing a character input method

Family Cites Families (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2741575B2 (en) * 1994-09-22 1998-04-22 日本アイ・ビー・エム株式会社 Character recognition character completion method and computer system
US7177797B1 (en) * 2000-08-31 2007-02-13 Semantic Compaction Systems Linguistic retrieval system and method
US20090040184A9 (en) * 2001-10-04 2009-02-12 Infogation Corporation Information entry mechanism
TWI235942B (en) 2001-10-29 2005-07-11 Oqo Inc Keyboard with variable-sized key
US7886233B2 (en) 2005-05-23 2011-02-08 Nokia Corporation Electronic text input involving word completion functionality for predicting word candidates for partial word inputs
US8185841B2 (en) 2005-05-23 2012-05-22 Nokia Corporation Electronic text input involving a virtual keyboard and word completion functionality on a touch-sensitive display screen
US7443316B2 (en) * 2005-09-01 2008-10-28 Motorola, Inc. Entering a character into an electronic device
US8605039B2 (en) * 2009-03-06 2013-12-10 Zimpl Ab Text input
KR20110109551A (en) * 2010-03-31 2011-10-06 삼성전자주식회사 Touch screen device and method for processing input of the same
US8918734B2 (en) * 2010-07-28 2014-12-23 Nuance Communications, Inc. Reduced keyboard with prediction solutions when input is a partial sliding trajectory
US20120047454A1 (en) * 2010-08-18 2012-02-23 Erik Anthony Harte Dynamic Soft Input
WO2012037200A2 (en) * 2010-09-15 2012-03-22 Spetalnick Jeffrey R Methods of and systems for reducing keyboard data entry errors
US8719724B2 (en) 2011-03-16 2014-05-06 Honeywell International Inc. Method for enlarging characters displayed on an adaptive touch screen key pad
US20130002553A1 (en) * 2011-06-29 2013-01-03 Nokia Corporation Character entry apparatus and associated methods
US9122672B2 (en) 2011-11-10 2015-09-01 Blackberry Limited In-letter word prediction for virtual keyboard
CN104160361A (en) * 2012-02-06 2014-11-19 迈克尔·K·科尔比 Character-string completion
US8850350B2 (en) * 2012-10-16 2014-09-30 Google Inc. Partial gesture text entry
US8843845B2 (en) * 2012-10-16 2014-09-23 Google Inc. Multi-gesture text input prediction
US20150089435A1 (en) * 2013-09-25 2015-03-26 Microth, Inc. System and method for prediction and recognition of input sequences
US9678664B2 (en) * 2015-04-10 2017-06-13 Google Inc. Neural network for keyboard input decoding
US9952764B2 (en) * 2015-08-20 2018-04-24 Google Llc Apparatus and method for touchscreen keyboard suggestion word generation and display

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6611253B1 (en) * 2000-09-19 2003-08-26 Harel Cohen Virtual input environment
US20040104896A1 (en) * 2002-11-29 2004-06-03 Daniel Suraqui Reduced keyboards system using unistroke input and having automatic disambiguating and a recognition method using said system
US20090193334A1 (en) * 2005-05-18 2009-07-30 Exb Asset Management Gmbh Predictive text input system and method involving two concurrent ranking means
US8300016B2 (en) * 2008-05-02 2012-10-30 Hon Hai Precision Industry Co., Ltd. Electronic device system utilizing a character input method
KR20110088310A (en) * 2010-01-28 2011-08-03 박태운 Hanalgeul input method for korean text

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP2972689A4 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017023185A1 (en) * 2015-08-06 2017-02-09 Общество С Ограниченной Ответственностью "1С Виарабл" Method, device and system for inputting and displaying data on a touchscreen
WO2017176335A1 (en) * 2016-04-04 2017-10-12 Google Inc. Dynamic key mapping of a graphical keyboard
CN108701123A (en) * 2016-04-04 2018-10-23 谷歌有限责任公司 The dynamic key of graphic keyboard maps
US10146764B2 (en) 2016-04-04 2018-12-04 Google Llc Dynamic key mapping of a graphical keyboard

Also Published As

Publication number Publication date
EP2972689A4 (en) 2016-11-23
US11061561B2 (en) 2021-07-13
US20190171357A1 (en) 2019-06-06
MX2015013077A (en) 2016-09-29
CN105247452A (en) 2016-01-13
EP2972689A1 (en) 2016-01-20
BR112015023799A2 (en) 2017-08-22
US10235042B2 (en) 2019-03-19
JP2016513849A (en) 2016-05-16
US20160034181A1 (en) 2016-02-04

Similar Documents

Publication Publication Date Title
US11061561B2 (en) Space optimizing micro keyboard method and apparatus
US9176668B2 (en) User interface for text input and virtual keyboard manipulation
US9063580B2 (en) Keyboardless text entry
KR101366723B1 (en) Method and system for inputting multi-touch characters
US20120062465A1 (en) Methods of and systems for reducing keyboard data entry errors
KR20080097114A (en) Apparatus and method for inputting character
US8922492B2 (en) Device and method of inputting characters
KR101039284B1 (en) Touch type character input apparatus and method
JP5977764B2 (en) Information input system and information input method using extended key
JP2014530423A (en) Character input device and character input method in portable terminal
CA2941832A1 (en) Space optimizing micro keyboard method and apparatus
KR20140018436A (en) System for inputting information utilizing word prediction and method thereof
KR20150132896A (en) A remote controller consisting of a single touchpad and its usage
WO2013157157A1 (en) Input character string conversion device, electronic device, input character string conversion method and character string conversion program
KR101624255B1 (en) Character input apparatus and method
US9563355B2 (en) Method and system of data entry on a virtual interface
KR101157084B1 (en) Method for inputting characters by drag
KR20120097921A (en) System for inputting information utilizing extension key and method thereof
JP6739083B2 (en) Data input device, data input method, and program for switching and displaying character input buttons according to two-direction input
WO2010149225A1 (en) User interface for a device
JP4610200B2 (en) Character input device and character input method
JP2020161179A (en) Data input device, method for input data, and program
US20120120012A1 (en) Method for inputting digital characters
KR101328555B1 (en) Character input apparatus and character input method
KR20230020711A (en) Apparatus for inputing english text for improving speech recognition performance and method using the same

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14765337

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2016502610

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: MX/A/2015/013077

Country of ref document: MX

WWE Wipo information: entry into national phase

Ref document number: 241649

Country of ref document: IL

REEP Request for entry into the european phase

Ref document number: 2014765337

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2014765337

Country of ref document: EP

REG Reference to national code

Ref country code: BR

Ref legal event code: B01A

Ref document number: 112015023799

Country of ref document: BR

ENP Entry into the national phase

Ref document number: 2941832

Country of ref document: CA

ENP Entry into the national phase

Ref document number: 112015023799

Country of ref document: BR

Kind code of ref document: A2

Effective date: 20150915