US6948136B2 - System and method for automatic control device personalization - Google Patents

System and method for automatic control device personalization Download PDF

Info

Publication number
US6948136B2
US6948136B2 US10/342,002 US34200203A US6948136B2 US 6948136 B2 US6948136 B2 US 6948136B2 US 34200203 A US34200203 A US 34200203A US 6948136 B2 US6948136 B2 US 6948136B2
Authority
US
United States
Prior art keywords
user
configuration
control device
control signals
control
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime, expires
Application number
US10/342,002
Other languages
English (en)
Other versions
US20040064597A1 (en
Inventor
Sharon M. Trewin
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Lenovo PC International Ltd
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
US case filed in Court of Appeals for the Federal Circuit litigation Critical https://portal.unifiedpatents.com/litigation/Court%20of%20Appeals%20for%20the%20Federal%20Circuit/case/22-1284 Source: Court of Appeals for the Federal Circuit Jurisdiction: Court of Appeals for the Federal Circuit "Unified Patents Litigation Data" by Unified Patents is licensed under a Creative Commons Attribution 4.0 International License.
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Priority to US10/342,002 priority Critical patent/US6948136B2/en
Assigned to INTERNATIONAL BUSINESS MACHINES CORPORATION reassignment INTERNATIONAL BUSINESS MACHINES CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: TREWIN, SHARON M.
Priority to TW092121006A priority patent/TWI229787B/zh
Priority to CNB03154990XA priority patent/CN100375052C/zh
Publication of US20040064597A1 publication Critical patent/US20040064597A1/en
Priority to HK05101587A priority patent/HK1068023A1/xx
Assigned to LENOVO (SINGAPORE) PTE LTD. reassignment LENOVO (SINGAPORE) PTE LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: INTERNATIONAL BUSINESS MACHINES CORPORATION
Publication of US6948136B2 publication Critical patent/US6948136B2/en
Application granted granted Critical
Assigned to LENOVO PC INTERNATIONAL reassignment LENOVO PC INTERNATIONAL NUNC PRO TUNC ASSIGNMENT (SEE DOCUMENT FOR DETAILS). Assignors: LENOVO (SINGAPORE) PTE LTD.
Adjusted expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/02—Input arrangements using manually operated switches, e.g. using keyboards or dials
    • G06F3/023—Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/60—Software deployment
    • G06F8/65—Updates
    • G06F8/656—Updates while running
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/20—Pc systems
    • G05B2219/25—Pc structure of the system
    • G05B2219/25084—Select configuration as function of operator

Definitions

  • the present invention relates generally to input devices used to control computer systems or other technology. More specifically, the present invention relates to the automatic, dynamic configuration of such devices in order to accommodate the control requirements of users who may have disabilities, or be operating the devices in contexts which affect their control of the device.
  • Computer control devices such as keyboards, mice, switches and touch pads are often accompanied by software which allows users to configure the device response to suit their physical abilities, situation and task.
  • appropriate configuration can be vital for computer users with physical disabilities, because the default mode of operation of the device may be unusable for them. For example, on a standard computer keyboard, a key that is held down will start to generate repeated characters after a certain time delay.
  • a person with a disability which inhibits movement may find it difficult or impossible to raise their finger off the key before this delay is up. As a result, they will generate many unwanted repeated characters, making accurate typing impossible. Appropriate configuration of this delay value will eliminate unwanted key repeats, enabling accurate typing for this user.
  • Configuration tools such as Wizards (e.g., Microsoft Corporation's Accessibility Wizard) provide a mechanism for finding the available settings but still require users to choose their own settings, and to use the interface in order to configure it. These approaches lead to a catch-22 situation in which users must configure their systems in order to be able to use them, but need to use them in order to configure them. This effectively makes independent access impossible for users with more severe disabilities.
  • a further aspect of input device configuration not addressed by these approaches is the dynamic nature of physical needs. While some user needs are permanent (e.g. amputee will always type with one hand and require an alternative way to generate multiple key presses such as Control-Alt-Delete), others may change over the short term (e.g. due to fatigue, or background noise) or long term (e.g. due to accident, healing, progression of disease, or the normal aging process). Existing configuration mechanisms do not accommodate the dynamic nature of users' requirements.
  • the keyboard user modeling techniques described in these two references have been incorporated into a configuration tool called the “Keyboard Optimizer.”
  • This tool allows users to demonstrate their typing and provides them with suggestions for configuration modifications. Users then choose whether to try those settings, and can accept or reject them.
  • the Keyboard Optimizer is easier to use and more accessible than other configuration mechanisms it does not completely solve the problems outlined above because users must still know that the program exists and be capable of launching it before they can benefit from its suggestions.
  • speech recognition programs such as IBM's Via Voice build user models from samples of user input. However, they require a user to go through a sequence of configuration tasks in order to create this user model, in this case a voice profile. This necessitates the use of some alternate control technique while the voice profile is being built, and so does not provide independent access for voice-only users.
  • a system and method for adjusting configurable parameters of a control device to improve the performance and control of the device according to needs of a user comprising: an input monitoring mechanism, effective regardless of the user's current activity; a mechanism for analyzing the user's actions in real time and inferring appropriate configuration options for the user; and, a configurer, which sets the configuration options chosen by the analysis mechanism.
  • the system may optionally implement a mechanism for identifying when a different user having different configuration needs starts to use the device.
  • the analyzing module produces an assessment of the ideal configuration options for the device, and this is updated with every new user action on the device.
  • identifying module if implemented, detects a change in user, it causes analyzing module to reinitialize.
  • the recommendations made by the analyzing module are input to the configurer module, which implements them as closely as possible in the current operating environment.
  • the system and method is of particular benefit to people with disabilities that negatively impact their ability to control the device in its unconfigured state, novice users, and people whose configuration requirements change frequently.
  • FIG. 1 illustrates an overall block diagram of a system providing for the automatic configuration of devices according to the present invention
  • FIG. 2 illustrates the software architecture for an embodiment of the present invention and information flow between the components
  • FIG. 3 depicts a flowchart detailing the method for capturing user actions with a device as a stream of control signals
  • FIG. 4 illustrates a stream of control signals generated by a keyboard
  • FIG. 5 is a flow chart illustrating a method for analyzing a user's actions and inferring appropriate configuration values
  • FIG. 6 is a flow chart illustrating a method for recognizing user changes
  • FIG. 7 is a flow chart illustrating a method for updating the active configuration
  • FIG. 8 depicts the software architecture according to an embodiment of the invention depicted in FIG. 2 , for configuring the key repeat delay of a keyboard device;
  • FIG. 9 is a flow chart depicting an embodiment of the invention illustrated in FIG. 3 for capturing keyboard control signals
  • FIG. 10 is a flow chart illustrating an embodiment of the invention depicted in FIG. 5 for inferring an appropriate key repeat delay for a keyboard user
  • FIG. 11 is a flow chart illustrating an embodiment of the invention depicted in FIG. 6 for recognizing changes of user of a keyboard
  • FIG. 12 is a flow chart illustrating an embodiment of the invention depicted in FIG. 7 for setting the key repeat delay of a keyboard.
  • FIG. 13 illustrates a set of configuration recommendations for a standard keyboard.
  • FIG. 1 depicts an overview of the present invention and particularly, illustrates the main actors involved in the configuration process for enabling a user to operate a control device in order to control a target, which may be an electronic device or service.
  • a user 10
  • a disability which affects motor control, speech or other function, or may be in a situation which affects their abilities in these areas (e.g., driving a car affects one's ability to operate a push button device).
  • a control device 11
  • This device ( 11 ) may comprise a physical device such as a keyboard, mouse or binary switch, or it may be less tangible, such as speech input, which would be represented by a microphone and speech recognition software.
  • the user wishes to use the control device in order to control a target device ( 12 ) which may comprise a personal computer, a household appliance (in which the input device may be physically built in to the target) or a service provided over the Internet via the user's device, for example accessing a currency conversion service through a cell phone.
  • the input/control device ( 11 ) may be separate from or integrated with the target, and may include an integrated output mechanism such as a display.
  • the input device ( 11 ) has a number of configuration options ( 13 ) associated with it that may be built into the device, the target, or both.
  • Configuration options ( 13 ) for input/control devices include, but are not limited to: the delay before a key on a keyboard starts to repeat, the rate at which keys repeat, the volume of a microphone, the voice profile being used by a speech recognition package, the distance moved by a cursor on a computer screen for a given physical distance moved by a mouse, etc.
  • These configuration options ( 13 ) are implemented in software and can be programmatically controlled. This control ability may be physically located at the device ( 11 ) or the target ( 12 ).
  • the final actor is the automatic configuration agent ( 14 ) according to the preferred embodiment of the invention.
  • This configuration agent ( 14 ) is connected to the output of the input/control device ( 11 ) for receiving control signals ( 15 ) therefrom, and, provides the control point ( 16 ) of the configuration options ( 13 ). Particularly, the configuration agent ( 14 ) reads the output signals ( 15 ) of the control device ( 11 ) but does not modify them. It passes the output as signals ( 17 ) on to the target ( 12 ) which processes the output signals ( 17 ) on the target in the way it would normally be handled.
  • the target receives the output signals ( 17 ) as control signals.
  • the configuration agent ( 14 ) analyses the user's control signals ( 15 ) and makes the appropriate (application program interface) API calls to adjust the configuration so as to optimize the configuration for the current user.
  • the key repeat delay can be set to approximately 1 second by the Windows system call: SystemParametersInfo(SPI_SETKEYBOARDDELAY, 3, NULL, SPI_SENDCHANGE). Pointing devices and other keyboard configuration options may be controlled through the same function.
  • FIG. 2 illustrates a more detailed view of the automatic configuration agent ( 14 ) component of FIG. 1 .
  • the automatic configuration agent ( 14 ) comprises: an input monitoring mechanism ( 20 ); a user change recognizer mechanism ( 21 ) for identifying when a different user starts to use the device; an analyzer mechanism ( 22 ) for analyzing the user's actions in real time and inferring appropriate configuration options for the user; and, a configurer component ( 23 ), which sets the configuration options chosen by the analysis mechanism.
  • the method by which the configuration agent operates is specialized for a specific form of control device, and operates on a single device. Different instantiations of the method could be used to handle multiple control devices. A different instantiation of the method could omit the user change recognizer mechanism ( 21 ).
  • the input monitor component ( 20 ) captures user actions with the device as a stream of control signals ( 24 ).
  • FIG. 4 illustrates example information comprising such a stream of signals.
  • the input monitor component ( 20 ) copies this stream to both the analyzer ( 22 ) and the user change recognizer ( 21 ), and passes it on to the target ( 28 ).
  • the user change recognizer ( 21 ) detects whether a new user with different configuration requirements to the previous user has recently started using the device, and may generate a probability value ( 25 ) ranging between 0.0 and 1.0 to indicate the probability of such a change.
  • a value of 0 indicates that the same user, or a user with similar needs, is using the device.
  • a value of 1 indicates that a very different user has taken over.
  • the user change recognizer may directly send a reset command ( 26 ) to the analyzer ( 22 ) and configurer component ( 23 ) when a different user is detected.
  • the analyzer ( 22 ) inspects the result of the user change recognizer. If the probability of a change in user ( 25 ) is above a given threshold (e.g., 0.75), then the analyzer performs a reset command ( 26 ). Regardless of whether a reset has occurred, the analyzer ( 22 ) produces a set of recommended configuration settings ( 27 ), and passes these to the configurer ( 23 ).
  • a set of configuration recommendations for the keyboard may be provided. Some of these recommendations may be ‘unknown’, indicating that no recommendation is being made. After a reset operation, all recommended configuration values are ‘unknown’.
  • the configurer ( 23 ) communicates with the device or target in order to implement the recommended configuration settings.
  • FIG. 3 illustrates in greater detail the method for operation of the input monitor component ( 20 ).
  • the input monitor first waits for control signals ( 30 ) from the control device ( 11 ). When such a signal arrives, it captures the signal reported by the control device ( 31 ), sends the signal on to the appropriate receiver within the control device or target ( 32 ), provides a copy of the signal to the user change recognizer ( 33 ), if implemented; and provides a copy of the signal to the analyzer ( 34 ). It then processes the next control signal ( 35 ) by returning to capture step ( 31 ).
  • FIG. 4 illustrates an example stream of control signals generated by a control device, e.g., a keyboard device, with each successive signal represented as events ( 40 , 41 , 42 , 43 ).
  • FIG. 4 specifically illustrates four events, with each event comprising attributes such as an event time ( 44 ), event type ( 45 ) and event data ( 46 ).
  • an event time ( 44 ) a keyboard Shift key is depressed.
  • An ‘a’ key is then pressed down ( 41 ).
  • the ‘a’ key is raised at time 0012795 ( 42 ) and the Shift key is raised at later time 0012810 ( 43 ). It is understood that the nature of control signals will be dictated by the form and modality of the control device.
  • FIGS. 5 and 10 respectively illustrate a general method and specific embodiment of the technique for analyzing a user's actions and inferring appropriate configuration values in the analyzer component ( 22 ) of the automatic configuration agent ( 14 ) of FIG. 2 .
  • the user change recognizer sets its control signal history to be empty, and zeros any counts or ongoing analysis. It sets every value in the configuration recommendations ( 52 ) to ‘unknown’.
  • a control signal (event) arrives ( 54 ) from the input monitor, it adds this to its event history ( 55 ).
  • the event history includes an ordered list of all of the events since initialization or the last reset.
  • the user change recognizer then provides this new history to individual modules ( 561 - 563 ) for each aspect of configuration under consideration.
  • Each module ( 561 - 563 ) then examines the new history, analyses the control signals, and updates ( 571 - 573 ) a respective configuration recommendation, or set of recommendations ( 521 - 523 ) with which it deals.
  • These recommendations are then combined to form an overall recommendation ( 581 ) and a decision is made ( 582 ) as to whether to actively pass this recommendation on to the configurer component ( 23 ) (FIG. 2 ).
  • the recommendation may be sent only when it is different to the previous recommendation, or different by a predetermined margin. If the recommendation is to be passed on, it is then sent to the configurer ( 592 ).
  • the algorithm then processes, or waits for, the next event ( 54 ).
  • the configurer component may request information on the current recommendation ( 591 ).
  • the analyzer handles such requests after it has processed an event ( 592 ) and responds by sending the full set of current recommendations.
  • the analyzer handles reset commands ( 51 ) immediately upon receiving them.
  • FIGS. 6 and 11 respectively illustrate a general method and specific embodiment of the technique for recognizing changes of user as implemented in the user change recognizer ( 21 ) of FIG. 2 .
  • the user change recognizer sets its control signal history to be empty, and zeros any counts or ongoing analysis. It sets an “evidence” value to 0.5.
  • a control signal (event) arrives ( 62 ) from the input monitor, it adds this to its event history ( 63 ).
  • the event history includes an ordered list of all of the events since initialization or the last reset.
  • the user change recognizer then provides this new history to individual modules ( 641 - 643 ) for each feature of the input stream which contributes to the final assessment.
  • Each of these modules ( 641 - 643 ) then calculates an independent probability of a different user being present ( 651 - 653 ), and those probabilities are combined (for example, using the well known Bayes Law formula) to produce a single evidence value ( 66 ).
  • This value may be made directly available to the analyzer ( 22 ).
  • the user change recognizer also assesses the probability value ( 66 ) and decides whether a new user is present ( 67 ). If it is determined that a new user is present, it will send a reset message to the analyzer ( 68 ), and will itself reset ( 61 ), returning to state 60 .
  • FIG. 7 is a flow chart illustrating a method for implementing the configurer component ( 23 ) in the agent 14 if FIG. 2 .
  • the configurer 70 first receives a configuration recommendation from the analyzer ( 22 ), either by requesting one, or by one being sent unsolicited.
  • the configurer stores this as the current configuration recommendation ( 71 ).
  • the configurer queries the system for the currently active configuration settings ( 72 ). With this information, the configurer calculates which aspects of the recommendation may be implemented ( 73 ). This calculation is achieved by prioritizing configuration options and incorporating information about constraints and dependencies between options into the configurer. Configuration options are prioritized such that those most crucial to accessibility of the device have higher priority than others.
  • a debounce time is a period of time which starts after a key is raised.
  • the same key will not register if pressed again. This is useful for people with tremor that causes them to press keys multiple times.
  • These options are therefore prioritized, so that if a recommendation for a debounce time and an acceptance delay is made, a decision as to which to implement can be made.
  • the configurer gives priority to the settings already in force on the system. Once the options to be activated have been chosen, specific values for those options are chosen.
  • the analyzer ( 22 ) may provide a recommendation which does not take into account the available values which may be implemented on the underlying system.
  • a debounce time of 175 msec may be recommended, while the underlying system may recognize only values of 0, 300, 500 and 700 msec.
  • the recommended value is rounded up to the next highest available setting. In this case 300 msec.
  • the configurer adjusts the recommended configuration to one that is implementable on the underlying system and compatible with the current settings.
  • the next step ( 75 ) is to decide whether the configuration should be updated. This decision is made with reference to a history of configuration changes ( 74 ) which records the time, position in the event stream and nature of all changes made to the configuration in the current session.
  • a threshold value constituting a “recent” change may be 5 minutes or 100 key events (i.e. 50 key presses).
  • FIG. 8 is diagram depicting the software architecture for implementing the configuration agent methodology described with respect to FIG. 2 , for an example embodiment of configuring a key repeat delay function of a keyboard.
  • a user 80
  • a keyboard 81
  • a personal computer system 82
  • the delay before keys start to repeat may be configured via software ( 83 ) provided by the operating system of the personal computer.
  • KRD key repeat delay
  • the automatic configuration agent is a software program ( 85 ) residing on the personal computer.
  • Input events ( 84 , 86 ) are viewed by the automatic configuration agent during the course of their processing within the target and are not changed by the automatic configuration agent.
  • the events are processed by an input monitor ( 850 ), the operation of which is described in greater detail herein with respect to FIG. 9 .
  • the monitor ( 850 ) passes these events to both the analyzer ( 851 ) and the user change recognizer ( 852 ).
  • the analyzer uses the events to calculate an appropriate key repeat delay (KRD) for the keyboard, and will be described in greater detail herein with respect to FIG. 10 .
  • KRD key repeat delay
  • the user change recognizer ( 852 ) uses these events to identify places in the input stream where a different user begins to operate the computer, as will be described in greater detail herein with respect to FIG. 11 .
  • the user change recognizer When the user change recognizer identifies a change in user it sends a reset command ( 853 ) to the analyzer.
  • the analyzer responds by restarting its calculations.
  • the analyzer passes details of the recommended KRD ( 855 ) to the configurer ( 854 ) whenever the recommendation changes.
  • the configurer ( 854 ) adjusts the recommendation to match the key repeat delay options available on the personal computer, then compares the recommended KRD with the currently active KRD. If the recommended KRD is different, then the current KRD on the personal computer is updated to match the recommendation ( 86 ). Further details regarding the configurer functionality will be described in greater detail herein with respect to FIG. 12 .
  • FIG. 9 is diagram depicting the software architecture for implementing the input monitor ( 90 ) methodology described with respect to FIG. 3 for capturing keyboard control signals.
  • the target is a personal computer running the Microsoft Windows operating system.
  • the MS Windows operating system enables a program to be activated via a keyboard event by installing a system hook ( 91 ) (calling the application program interface (API) function SetWindowsHookEx) which gives access to all the keyboard events (i.e., keyboard control signals) ( 92 ) as they are reported from the keyboard within the operating system ( 93 ).
  • the system hook captures the event information ( 94 ), reads the events and passes them on to the next hook function ( 95 ).
  • the system hook additionally copies the information to a memory address or buffer ( 96 ) which is read by the analyzer ( 97 ) and user change recognizer ( 98 ).
  • This information includes the time of the event, the nature of the event (e.g. key up, key down) and other information such as which key was pressed, as illustrated in the example embodiment of control signals (keyboard events) as depicted FIG. 4 .
  • FIG. 10 is diagram depicting the software architecture for implementing the analyzer methodology described with respect to FIG. 5 for inferring an appropriate key repeat delay for a keyboard user.
  • initialization 1000
  • an empty history is created and a store of key press lengths is created.
  • a count of overlong keystrokes and a count of normal keystrokes are set to zero.
  • the initial key repeat delay recommendation is ‘unknown’.
  • step ( 1002 ) accesses this event by retrieving it from the keyboard event buffer ( 96 ) of FIG. 9 .
  • step 1003 it is added to the history.
  • the new history is used to assess the key repeat delay, then in step 1005 a new recommendation for the repeat delay is generated.
  • step 1006 a decision is made as to whether to forward this recommendation to the configurer.
  • an analyzer may decide to forward the key repeat delay recommendation only when it is more than 50 milliseconds different from the previous recommendation sent. If the recommendation is to be sent, processing passes to step 1007 , otherwise processing returns to step 1002 to fetch the next event.
  • the recommendation is sent to the configurer which component may also request a recommendation ( 1008 ) at any time. When such a request is received, the analyzer finishes processing the current keystroke then responds to the request by sending the recommendation in step 1007 . After sending a recommendation, the analyzer returns to step 1002 to fetch the next event ( 1009 ).
  • the analyzer may receive a reset event at any time ( 1010 ). When a reset event is received, the event is immediately processed and the analyzer returns to step 1000 for reinitialization.
  • a comparison is made to determine whether the length of the most recent keystroke exceeds a threshold value.
  • step 1005 of FIG. 10 for generating a new key repeat delay recommendation is now further described with respect to FIG. 10 .
  • the overlong keystroke count is compared with the count of normal keystrokes. If there are more overlong keystrokes than normal keystrokes, processing passes to step 10052 , where a recommendation of ‘off’ is made (i.e., no key repeats). Otherwise, if the count of overlong keystrokes does no exceed the normal keystroke count, processing passes to step 10053 where a key repeat delay is calculated from the data in the store.
  • this calculation may return the average length of key presses plus some constant. As a second example, it may also return the length of the longest key press.
  • FIG. 11 is diagram depicting the software architecture for implementing the user change recognizer methodology described with respect to FIG. 6 for recognizing a change of user based on key press lengths for a keyboard user. More particularly, FIG. 11 illustrates an embodiment of the method depicted in blocks ( 641 - 643 ) of FIG. 6 for recognizing instances of a change of keyboard user.
  • the methodology ( 1100 ) described in the example embodiment depicted in FIG. 11 is based on a specific feature of the input stream: key press lengths.
  • the current evidence value ( 1105 ) is initialized. For example, a value of 0.5 may be used, indicating that there is no strong evidence as to whether the user has recently changed.
  • the method ( 1100 ) is passed a history of recent keyboard events ( 1101 ) as described in FIG. 6 .
  • the recognizer module examines the most recent event in the history. If this was a key up event (a key being released), and the key being released is one which is unlikely to be deliberately held down, then processing proceeds to step 1103 . Otherwise, processing stops ( 1104 ) and the probability ( 1105 ) of a different user being present remains unchanged.
  • the likelihood of a key being deliberately held down may be obtained by considering the purpose of the key. Alphanumeric keys are not typically held down, while modifier keys, arrow keys and editing keys such as delete and backspace are often held down. For safety, processing is limited to the number and letter keys.
  • Punctuation may sometimes be repeated (e.g. holding down the dash to create a separating line in a document) and so for the purposes of this algorithm punctuation is treated the same way as navigation and editing keys.
  • the key press length (keyUpTime minus keyDownTime) of the most recent keystroke is contributed to the average key press length for all keystrokes examined so far in this step.
  • the same calculation is performed for a number of the most recent of these keystrokes. In the embodiment depicted in FIG. 11 , this number may be twenty ( 20 ) of the most recent keystrokes.
  • step 1107 the two values obtained in steps 1103 and 1106 are compared, and the difference between these values forms the basis of the calculation for updating the evidence value ( 1105 ).
  • FIG. 12 is diagram depicting the software architecture for implementing the configurer component methodology described with respect to FIG. 7 for updating the active key repeat delay for a keyboard user. More particularly, FIG. 12 illustrates an embodiment of the method depicted in FIG. 7 for setting the key repeat delay of a keyboard.
  • the configurer receives a key repeat delay recommendation from the analyzer component of the configuration agent in step 1203 and stores it as the current KRD recommendation ( 1201 ). This may either be for example, a positive millisecond value or a negative value ‘off’ indicating that key repeats are to be suppressed.
  • the configurer finds the currently active keyboard configuration ( 1202 ).
  • the configurer examines the current keyboard settings ( 1202 ). If any active settings are incompatible with the proposed KRD value, the existing settings take priority and the active configuration is not changed ( 1205 ). More specifically, if specialized keyboard accessibility features, specifically the debounce time and the key acceptance delay, are non-zero, then the key repeat delay cannot be adjusted because it is incompatible with those settings.
  • the configurer would prioritize these in order to handle incompatibilities that may exist. If the KRD can be adjusted, the configurer proceeds to step 1206 , in which it transforms the recommended value to the nearest greater value that is implementable in the current system. The set of legal values is built into the configurer, which is specific to the personal computer operating system on which it is running.
  • the configurer examines the recent history ( 1208 ) of changes to the KRD. This history indicates the time in milliseconds, the input event count, and the value set for the KRD since the last reset, or since the start of the session. The history is used in step 1207 to decide whether the KRD should be adjusted.
  • Adjustments are recorded so as to avoid the situation where the settings are repeatedly toggled between two values. If the recommended adjustment is an increase in KRD, and the previous adjustment was also an increase, or if both were decreases, then the change is made. If the previous change was opposite to the current recommendation then the time and event count of the change are examined. The system will only be updated if the previous change was sufficiently long ago. ‘long ago’ is defined by a threshold value time which may be adjusted to suit the situation. If the system is to be updated, the configurer proceeds to step 1209 . It calls the functions necessary to update the KRD on the target system. On the Windows® operating system, there are two functions which control the key repeat delay. Both are adjusted by calling the SystemParametersInfo system function.
  • One call refers to the SPI_SETKEYBOARDSPEED option, while the other refers to the SPI_SETFILTERKEYS option.
  • the appropriate function to call depends on the current state of the FilterKeys Windows data structure. If the FilterKeys structure indicates that it is active, then the SPI_SETFILTERKEYS option must be used to control the KRD. If FilterKeys is not active, the appropriate function call depends on the value to be implemented, as the two functions have different ranges of values. If the SPI_SETFILTERKEYS option is chosen, the other values in the structure must be set so as not to activate any of the additional FilterKeys features. In step 1209 , if these calls are successful, the value, time and event count are stored in the history ( 1208 ). The configurer then returns ( 1210 ) to step 1203 and waits for a new recommendation from the analyzer.
  • FIG. 13 illustrates an example set of configuration recommendations ( 1302 , 1303 , 1304 , 1305 , 1306 ) for a standard keyboard.
  • Five configuration parameters are shown, each on a separate line with each parameter having attributes such as a name ( 1300 ), and a value ( 1301 ).
  • Configuration values 1302 , 1303 1304 and 1305 thus comprise integers representing, for example, a millisecond value.
  • One parameter, the sicky key parameter ( 1306 ) is also an integer, type, for example, a negative integer ⁇ 5. This is one of a set of special values which includes ‘On’, ‘Off’, and ‘Unknown’ and all have negative values.
  • a positive value for a parameter indicates a specific setting, however, for some parameters, only a negative value is valid. For other parameters, both positive and negative values are permitted.
  • the parameters in the example embodiment depicted in FIG. 13 are:

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Software Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • User Interface Of Digital Computer (AREA)
  • Input From Keyboards Or The Like (AREA)
US10/342,002 2002-09-30 2003-01-14 System and method for automatic control device personalization Expired - Lifetime US6948136B2 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
US10/342,002 US6948136B2 (en) 2002-09-30 2003-01-14 System and method for automatic control device personalization
TW092121006A TWI229787B (en) 2002-09-30 2003-07-31 System and method for automatic control device personalization
CNB03154990XA CN100375052C (zh) 2002-09-30 2003-08-26 自动控制设备个人化的系统和方法
HK05101587A HK1068023A1 (en) 2002-09-30 2005-02-24 System and method for automatic control device personalization

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US41516502P 2002-09-30 2002-09-30
US10/342,002 US6948136B2 (en) 2002-09-30 2003-01-14 System and method for automatic control device personalization

Publications (2)

Publication Number Publication Date
US20040064597A1 US20040064597A1 (en) 2004-04-01
US6948136B2 true US6948136B2 (en) 2005-09-20

Family

ID=32033332

Family Applications (1)

Application Number Title Priority Date Filing Date
US10/342,002 Expired - Lifetime US6948136B2 (en) 2002-09-30 2003-01-14 System and method for automatic control device personalization

Country Status (4)

Country Link
US (1) US6948136B2 (zh)
CN (1) CN100375052C (zh)
HK (1) HK1068023A1 (zh)
TW (1) TWI229787B (zh)

Cited By (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050064916A1 (en) * 2003-09-24 2005-03-24 Interdigital Technology Corporation User cognitive electronic device
US20050099395A1 (en) * 2003-11-06 2005-05-12 Marsden Randal J. Assistive technology interface
US20070147932A1 (en) * 2004-01-06 2007-06-28 Giles Susan L Computer keyboard
US20070168486A1 (en) * 2005-06-09 2007-07-19 Whirlpool Corporation Software architecture system and method for operating an appliance exposing key press functionality to a network
US20080005679A1 (en) * 2006-06-28 2008-01-03 Microsoft Corporation Context specific user interface
US20080209269A1 (en) * 2004-04-16 2008-08-28 International Business Machines Corporation Active Probing for Real-Time Diagnosis
US20080275616A1 (en) * 2007-05-01 2008-11-06 Ronald Van Houten Seat belt/accelerator behavioral system
US20090113306A1 (en) * 2007-10-24 2009-04-30 Brother Kogyo Kabushiki Kaisha Data processing device
US20090138268A1 (en) * 2007-11-28 2009-05-28 Brother Kogyo Kabushiki Kaisha Data processing device and computer-readable storage medium storing set of program instructions excutable on data processing device
US20090150787A1 (en) * 2007-12-11 2009-06-11 Brother Kogyo Kabushiki Kaisha Data processing device
US20100180238A1 (en) * 2005-08-15 2010-07-15 Koninklijke Philips Electronics, N.V. User interface system for a personal healthcare environment
US20100201624A1 (en) * 2007-10-19 2010-08-12 Razer (Asia-Pacific) Pte Ltd Unintended Displacement Identification and Correction Method and System
US20100295788A1 (en) * 2009-05-21 2010-11-25 Microsoft Corporation Method of visualizing an input location
US20110296505A1 (en) * 2010-05-28 2011-12-01 Microsoft Corporation Cloud-based personal trait profile data
US20130191755A1 (en) * 2012-01-23 2013-07-25 Zonoff, Inc. Commissioning devices for automation systems
US9298695B2 (en) 2013-09-05 2016-03-29 At&T Intellectual Property I, Lp Method and apparatus for managing auto-correction in messaging
US9400548B2 (en) 2009-10-19 2016-07-26 Microsoft Technology Licensing, Llc Gesture personalization and profile roaming
US20190018478A1 (en) * 2017-07-11 2019-01-17 Sony Corporation Sensing viewer direction of viewing to invoke accessibility menu in audio video device
US10432705B2 (en) * 2016-03-23 2019-10-01 Lutron Technology Company Llc Configuring control devices operable for a load control environment
US10650702B2 (en) 2017-07-10 2020-05-12 Sony Corporation Modifying display region for people with loss of peripheral vision
US10805676B2 (en) 2017-07-10 2020-10-13 Sony Corporation Modifying display region for people with macular degeneration
US10845954B2 (en) 2017-07-11 2020-11-24 Sony Corporation Presenting audio video display options as list or matrix
US10963087B2 (en) 2012-03-02 2021-03-30 Microsoft Technology Licensing, Llc Pressure sensitive keys
US10999420B2 (en) * 2012-07-19 2021-05-04 Srk Technology Llc Adaptive communication mode for recording a media message
USRE48963E1 (en) 2012-03-02 2022-03-08 Microsoft Technology Licensing, Llc Connection device for computing devices
US11445817B2 (en) 2019-09-13 2022-09-20 Ergotron, Inc. Workstation height-adjustment monitoring

Families Citing this family (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050257175A1 (en) * 2004-05-13 2005-11-17 Brisben Amy J Electronic communication interface
JP4731149B2 (ja) * 2004-10-06 2011-07-20 任天堂株式会社 情報処理装置および情報入力プログラム
US7554522B2 (en) * 2004-12-23 2009-06-30 Microsoft Corporation Personalization of user accessibility options
US20060172267A1 (en) * 2005-01-31 2006-08-03 Microsoft Corporation Input device training and automatic assignment
US7903087B2 (en) * 2006-02-13 2011-03-08 Research In Motion Limited Method for facilitating navigation and selection functionalities of a trackball incorporated upon a wireless handheld communication device
US20080030466A1 (en) * 2006-08-03 2008-02-07 Leigh Simeon Keates System and method for correcting positioning and triggering errors for aim-and-trigger devices
JP2008048112A (ja) * 2006-08-15 2008-02-28 Fujitsu Ltd キー割当て可能な携帯端末装置
US9471333B2 (en) * 2006-11-03 2016-10-18 Conceptual Speech, Llc Contextual speech-recognition user-interface driven system and method
US20080140798A1 (en) * 2006-12-08 2008-06-12 Aten International Co., Ltd. Storage adapter and method thereof
CN101317784B (zh) * 2007-06-08 2011-09-07 深圳迈瑞生物医疗电子股份有限公司 多配置模块设备及其模块自动配置方法
CN105653070B (zh) * 2007-10-19 2018-09-25 雷蛇(亚太)私人有限公司 无意位移之识别及其修正的方法、系统以及装置
CN101923474B (zh) * 2009-06-15 2013-10-30 联想(北京)有限公司 一种程序运行参数配置方法及计算机
CN101853102B (zh) * 2010-06-01 2012-05-09 深圳市汇顶科技有限公司 一种触控设备配置参数的自动生成方法、系统
US10601457B2 (en) * 2010-07-27 2020-03-24 Comcast Cable Communications, Llc Configuring remote control behavior to limit a maximum amount of transmissions for continuous press of a button
CN102023894A (zh) * 2010-11-18 2011-04-20 华为终端有限公司 一种用户操作界面的变换方法及终端
US20130085927A1 (en) * 2011-10-04 2013-04-04 Research In Motion Limited Providing Increased Ability To Perform A Transaction Based On Locale
US9348508B2 (en) 2012-02-15 2016-05-24 International Business Machines Corporation Automatic detection of user preferences for alternate user interface model
US9548012B1 (en) * 2012-08-29 2017-01-17 Amazon Technologies, Inc. Adaptive ergonomic keyboard
US8665238B1 (en) * 2012-09-21 2014-03-04 Google Inc. Determining a dominant hand of a user of a computing device
CN103970260B (zh) * 2013-01-31 2017-06-06 华为技术有限公司 一种非接触式手势控制方法及电子终端设备
CN104251802B (zh) * 2013-06-26 2016-08-17 名硕电脑(苏州)有限公司 按键回弹时间的量测装置与量测方法
CN105068699A (zh) * 2015-08-20 2015-11-18 常州智顺电子科技有限公司 一种新型无驱触摸屏参数设置系统
CN107957786B (zh) * 2016-10-14 2020-12-22 东莞宝德电子有限公司 输入系统、周边装置及设定方法
US10715604B1 (en) * 2017-10-26 2020-07-14 Amazon Technologies, Inc. Remote system processing based on a previously identified user
US10567515B1 (en) 2017-10-26 2020-02-18 Amazon Technologies, Inc. Speech processing performed with respect to first and second user profiles in a dialog session
US10776135B2 (en) * 2017-11-20 2020-09-15 International Business Machines Corporation Automated setting customization using real-time user data
CN111221441B (zh) * 2019-12-29 2023-05-05 歌尔科技有限公司 一种Touch按键的感应方法、系统及可穿戴设备
TWI755809B (zh) * 2020-08-05 2022-02-21 華碩電腦股份有限公司 電子裝置之控制方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5692140A (en) * 1995-06-12 1997-11-25 Intellitools, Inc. Methods and apparatus for synchronizing application and utility programs
US20020118223A1 (en) * 2001-02-28 2002-08-29 Steichen Jennifer L. Personalizing user interfaces across operating systems
US20040010328A1 (en) * 2002-06-10 2004-01-15 Carson Barry R. Method and system for controlling ergonomic settings at a worksite

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5355305A (en) * 1992-10-29 1994-10-11 Johnson Service Company Pattern recognition adaptive controller
US6360339B1 (en) * 1999-06-29 2002-03-19 Intel Corporation Method and system of dynamically selecting a data coalesce technique in a computer system
CN1369793A (zh) * 2001-02-12 2002-09-18 英业达股份有限公司 自动检测和设定键盘语系及其配置的方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5692140A (en) * 1995-06-12 1997-11-25 Intellitools, Inc. Methods and apparatus for synchronizing application and utility programs
US20020118223A1 (en) * 2001-02-28 2002-08-29 Steichen Jennifer L. Personalizing user interfaces across operating systems
US20040010328A1 (en) * 2002-06-10 2004-01-15 Carson Barry R. Method and system for controlling ergonomic settings at a worksite

Non-Patent Citations (8)

* Cited by examiner, † Cited by third party
Title
C. Stephanidis, et al., "Adaptable and Adaptive User Interfaces for Disabled Users in the AVANTI Project", in Intelligence in Services and Networks; Technology for Ubiquitous Telecom Services. 5<SUP>th </SUP>International Conference on Intelligence in Services and Networks, IS & N 98 Proceedings, Springer-Verlag, 153-166 (1998).
Lynds, J.S.; "DARCI TOO-A Computer Input Device for People with Disabilities"; Proceedings of the Johns Hopkins National Search for Computing Applications to Assist Persons with Disabilities; Feb. 1-5, 1992; pp.: 110-112. *
S. Trewin, "Configuration Agents, Control and Privacy", In Proceedings of the 1<SUP>st </SUP>ACM Conference on Universal Usability, Nov. 15-17, Washington DC, USA, Abstract and pp. 2-8, ACM Press (2000).
S. Trewin, "Input Errors by Keyboard and Mouse Users with Motor Disabilities", Encyclopedia of Microcomputers, vol. 27, Supp. 6, pp 163-177, (2001).
S. Trewin, "Towards Intelligent, Adaptive Input Devices for Users with Physical Disabilities", PhD Thesis. University of Edinburgh, Great Britain; pp. ii-viii and 236-241, (1998).
S. Trewin, et al., "A model of keyboard configuration requirements", Behaviour and Information Technology Special issue on Assistive Technologies for People with Disabilities, vol. 18, No. 1, pp 27-35, (1999).
S. Trewin, et al., "Accellerating Assessment with Self-optimizing Devices", Proceedings of the 8<SUP>th </SUP>International conference on Computers Helping People with Special Needs, Linz, Austria, 3 pages, (Jul. 2002).
S. Trewin, et al., "Keyboard and mouse errors due to motor disabilities", International Journal of Human-Computer Studies, vol. 50, pp 109-144, (1999).

Cited By (43)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050064916A1 (en) * 2003-09-24 2005-03-24 Interdigital Technology Corporation User cognitive electronic device
US20050099395A1 (en) * 2003-11-06 2005-05-12 Marsden Randal J. Assistive technology interface
US20070147932A1 (en) * 2004-01-06 2007-06-28 Giles Susan L Computer keyboard
US20080209269A1 (en) * 2004-04-16 2008-08-28 International Business Machines Corporation Active Probing for Real-Time Diagnosis
US9009301B2 (en) 2004-04-16 2015-04-14 International Business Machines Corporation Active probing for real-time diagnosis
US8171130B2 (en) 2004-04-16 2012-05-01 International Business Machines Corporation Active probing for real-time diagnosis
US20070168486A1 (en) * 2005-06-09 2007-07-19 Whirlpool Corporation Software architecture system and method for operating an appliance exposing key press functionality to a network
US9401822B2 (en) * 2005-06-09 2016-07-26 Whirlpool Corporation Software architecture system and method for operating an appliance exposing key press functionality to a network
US20100180238A1 (en) * 2005-08-15 2010-07-15 Koninklijke Philips Electronics, N.V. User interface system for a personal healthcare environment
US20080005679A1 (en) * 2006-06-28 2008-01-03 Microsoft Corporation Context specific user interface
US7937202B2 (en) 2007-05-01 2011-05-03 Ronald Van Houten Seat belt/accelerator behavioral system
US20080275616A1 (en) * 2007-05-01 2008-11-06 Ronald Van Houten Seat belt/accelerator behavioral system
KR101486442B1 (ko) * 2007-10-19 2015-01-26 레이저 (아시아-퍼시픽) 피티이 엘티디 디스플레이 장치상에서 물체의 의도하지 않은 이동을 막는 방법 및 포인터 장치
US20100201624A1 (en) * 2007-10-19 2010-08-12 Razer (Asia-Pacific) Pte Ltd Unintended Displacement Identification and Correction Method and System
US9046938B2 (en) * 2007-10-19 2015-06-02 Razer (Asia-Pacific) Pte Ltd Unintended displacement identification and correction method and system
US20090113306A1 (en) * 2007-10-24 2009-04-30 Brother Kogyo Kabushiki Kaisha Data processing device
US20090138268A1 (en) * 2007-11-28 2009-05-28 Brother Kogyo Kabushiki Kaisha Data processing device and computer-readable storage medium storing set of program instructions excutable on data processing device
US8707183B2 (en) * 2007-12-11 2014-04-22 Brother Kogyo Kabushiki Kaisha Detection of a user's visual impairment based on user inputs or device settings, and presentation of a website-related data for sighted or visually-impaired users based on those inputs or settings
US20090150787A1 (en) * 2007-12-11 2009-06-11 Brother Kogyo Kabushiki Kaisha Data processing device
US8416193B2 (en) * 2009-05-21 2013-04-09 Microsoft Corporation Method of visualizing an input location
US20100295788A1 (en) * 2009-05-21 2010-11-25 Microsoft Corporation Method of visualizing an input location
US9400548B2 (en) 2009-10-19 2016-07-26 Microsoft Technology Licensing, Llc Gesture personalization and profile roaming
US20110296505A1 (en) * 2010-05-28 2011-12-01 Microsoft Corporation Cloud-based personal trait profile data
US9274594B2 (en) * 2010-05-28 2016-03-01 Microsoft Technology Licensing, Llc Cloud-based personal trait profile data
US20130191755A1 (en) * 2012-01-23 2013-07-25 Zonoff, Inc. Commissioning devices for automation systems
US9246757B2 (en) * 2012-01-23 2016-01-26 Zonoff, Inc. Commissioning devices for automation systems
USRE48963E1 (en) 2012-03-02 2022-03-08 Microsoft Technology Licensing, Llc Connection device for computing devices
US10963087B2 (en) 2012-03-02 2021-03-30 Microsoft Technology Licensing, Llc Pressure sensitive keys
US10999420B2 (en) * 2012-07-19 2021-05-04 Srk Technology Llc Adaptive communication mode for recording a media message
US11750730B2 (en) * 2012-07-19 2023-09-05 Srk Technology Llc Adaptive communication mode for recording a media message
US20210203766A1 (en) * 2012-07-19 2021-07-01 Srk Technology Llc Adaptive communication mode for recording a media message
US9298695B2 (en) 2013-09-05 2016-03-29 At&T Intellectual Property I, Lp Method and apparatus for managing auto-correction in messaging
US10127222B2 (en) 2013-09-05 2018-11-13 At&T Mobility Ii Llc Method and apparatus for managing auto-correction in messaging
US10432705B2 (en) * 2016-03-23 2019-10-01 Lutron Technology Company Llc Configuring control devices operable for a load control environment
US11019135B2 (en) 2016-03-23 2021-05-25 Lutron Technology Company Llc Configuring control devices operable for a load control environment
US11856051B2 (en) 2016-03-23 2023-12-26 Lutron Technology Company Llc Configuring control devices operable for a load control environment
US11418579B2 (en) 2016-03-23 2022-08-16 Lutron Technology Company Llc Configuring control devices operable for a load control environment
US10650702B2 (en) 2017-07-10 2020-05-12 Sony Corporation Modifying display region for people with loss of peripheral vision
US10805676B2 (en) 2017-07-10 2020-10-13 Sony Corporation Modifying display region for people with macular degeneration
US20190018478A1 (en) * 2017-07-11 2019-01-17 Sony Corporation Sensing viewer direction of viewing to invoke accessibility menu in audio video device
US10845954B2 (en) 2017-07-11 2020-11-24 Sony Corporation Presenting audio video display options as list or matrix
US11445817B2 (en) 2019-09-13 2022-09-20 Ergotron, Inc. Workstation height-adjustment monitoring
US11839293B2 (en) 2019-09-13 2023-12-12 Ergotron, Inc. Workstation height-adjustment monitoring

Also Published As

Publication number Publication date
CN100375052C (zh) 2008-03-12
TW200411348A (en) 2004-07-01
US20040064597A1 (en) 2004-04-01
HK1068023A1 (en) 2005-04-22
TWI229787B (en) 2005-03-21
CN1527204A (zh) 2004-09-08

Similar Documents

Publication Publication Date Title
US6948136B2 (en) System and method for automatic control device personalization
US8250001B2 (en) Increasing user input accuracy on a multifunctional electronic device
US20180329510A1 (en) Arrangement, method and computer program for controlling a computer apparatus based on eye-tracking
KR102423712B1 (ko) 루틴 실행 중에 클라이언트 디바이스간 자동화 어시스턴트 루틴 전송
Schmandt et al. Augmenting a window system with speech input
US7409690B2 (en) Application module for managing interactions of distributed modality components
US8233613B1 (en) Monitoring key-press delay and duration to determine need for assistance
CN112313924A (zh) 提供用于控制各种连接设备的复合图形助理界面
EP3751823B1 (en) Generating iot-based notification(s) and provisioning of command(s) to cause automatic rendering of the iot-based notification(s) by automated assistant client(s) of client device(s)
US20090153366A1 (en) User interface apparatus and method using head gesture
WO2007063714A1 (ja) 入出力装置、入出力方法及びそのプログラム
JP3822357B2 (ja) マルチモーダル入出力装置のインタフェース装置及びその方法
JP2004355426A (ja) タッチパネル操作性向上ソフトウェア及び端末
WO2019022835A1 (en) INTELLIGENT RESPONSE USING A LOOK
CN111427454A (zh) 一种手势控制系统及方法
US20080068195A1 (en) Method, System And Device For The Haptically Controlled Transfer Of Selectable Data Elements To A Terminal
CN114981772A (zh) 在不需要自动助理的基于语音的调用的情况下基于检测到的环境状况来选择性地调用自动助理
Pokhariya et al. Navigo--Accessibility Solutions for Cerebral Palsy Affected
CN116802597A (zh) 在某些情形下响应于助理调用来选择性地渲染键盘接口
US20240231607A1 (en) Predictive Input Interface Having Improved Robustness for Processing Low Precision Inputs
WO2023224644A1 (en) Predictive input interface having improved robustness for processing low precision inputs
CN117666913A (zh) 一种控制方法、装置及输入设备
Papadopoulou et al. Two approaches to handling proactivity in pervasive systems

Legal Events

Date Code Title Description
AS Assignment

Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:TREWIN, SHARON M.;REEL/FRAME:013666/0065

Effective date: 20030114

FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

AS Assignment

Owner name: LENOVO (SINGAPORE) PTE LTD.,SINGAPORE

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:INTERNATIONAL BUSINESS MACHINES CORPORATION;REEL/FRAME:016891/0507

Effective date: 20050520

Owner name: LENOVO (SINGAPORE) PTE LTD., SINGAPORE

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:INTERNATIONAL BUSINESS MACHINES CORPORATION;REEL/FRAME:016891/0507

Effective date: 20050520

STCF Information on status: patent grant

Free format text: PATENTED CASE

CC Certificate of correction
REMI Maintenance fee reminder mailed
FPAY Fee payment

Year of fee payment: 4

SULP Surcharge for late payment
FPAY Fee payment

Year of fee payment: 8

AS Assignment

Owner name: LENOVO PC INTERNATIONAL, HONG KONG

Free format text: NUNC PRO TUNC ASSIGNMENT;ASSIGNOR:LENOVO (SINGAPORE) PTE LTD.;REEL/FRAME:037160/0001

Effective date: 20130401

FPAY Fee payment

Year of fee payment: 12