WO2015084644A1 - Compensation d'une latence lors de l'affichage d'une partie d'un mouvement initié par la main - Google Patents

Compensation d'une latence lors de l'affichage d'une partie d'un mouvement initié par la main Download PDF

Info

Publication number
WO2015084644A1
WO2015084644A1 PCT/US2014/067366 US2014067366W WO2015084644A1 WO 2015084644 A1 WO2015084644 A1 WO 2015084644A1 US 2014067366 W US2014067366 W US 2014067366W WO 2015084644 A1 WO2015084644 A1 WO 2015084644A1
Authority
WO
WIPO (PCT)
Prior art keywords
segment
path
contact point
motion
parameter
Prior art date
Application number
PCT/US2014/067366
Other languages
English (en)
Inventor
Steven Bathiche
Jesse R. CHEATHAM, III
Paul H. Dietz
Matthew G. Dyor
Philip A. Eckhoff
Anoop Gupta
Kenneth P. Hinckley
Roderick A. Hyde
Muriel Y. Ishikawa
Jordin T. Kare
Craig J. Mundie
Nathan P. Myhrvold
Andreas G. Nowatzyk
Robert C. Petroski
Danny Allen Reed
Clarence T. Tegreene
Charles Whitmer
Lowell L. Wood, Jr.
Victoria Y. H. Wood
Original Assignee
Elwha Llc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US14/095,641 external-priority patent/US20150153855A1/en
Priority claimed from US14/095,612 external-priority patent/US20150153890A1/en
Priority claimed from US14/095,623 external-priority patent/US20150153898A1/en
Application filed by Elwha Llc filed Critical Elwha Llc
Publication of WO2015084644A1 publication Critical patent/WO2015084644A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/0416—Control or interface arrangements specially adapted for digitisers
    • G06F3/0418—Control or interface arrangements specially adapted for digitisers for error correction or compensation, e.g. based on parallax, calibration or alignment
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures

Definitions

  • an embodiment of the subject matter described herein includes an apparatus.
  • the apparatus includes a touch tracking circuit configured to detect a segment of a path defined by a user contact point moving across a touch sensitive display.
  • the apparatus includes a motion analysis circuit configured to determine a parameter descriptive of a motion of the user contact point during its movement across the detected segment of the path (hereafter "motion parameter").
  • the apparatus includes a predictive filter configured to predict in response to the motion parameter a next contiguous segment of the path defined by the user-contact point moving across the touch sensitive display.
  • the apparatus includes a latency compensation circuit configured to initiate a display by the touch sensitive display of the detected segment of the path and the predicted next segment of the path.
  • the apparatus includes an updating circuit configured to initiate an update of the detected segment of the path and the predicted next contiguous segment of the path as the user contact point moves across the touch sensitive display.
  • the apparatus includes the touch sensitive display.
  • the apparatus includes a computing device that includes the touch sensitive display.
  • the apparatus includes a receiver circuit configured to receive a signal generated by a handheld stylus.
  • the apparatus includes a learning circuit configured to adaptively learn a motion parameter associated with a specific user based upon a history of at least two motion parameters determined in response to the path defined by a user contact point moving across the touch sensitive display.
  • the learning circuit is further configured to store in a computer readable storage media the adaptively learned motion parameter in an association with an identifier of the specific user.
  • the apparatus includes a learning circuit configured to adaptively learn a motion parameter associated with a specific software application running on the apparatus and based upon a history of at least two motion parameters determined in response to a path defined by the user contact point moving across the touch sensitive display.
  • the learning circuit is further configured to store in a computer readable storage media the learned motion parameter in an association with an identification of the specific software application running on the apparatus.
  • the apparatus includes a non-transitory computer readable storage media.
  • an embodiment of the subject matter described herein includes a method implemented in a computing environment.
  • the method includes detecting a segment of a path defined by a user contact point moving across a touch sensitive display.
  • the method includes determining a parameter descriptive of a motion of the user contact point during its movement across the detected segment of the path (hereafter "motion parameter").
  • the method includes predicting in response to the motion parameter a next contiguous segment of the path of the user contact point moving across the touch sensitive display.
  • the method includes displaying a human- perceivable rendering of the detected segment of the path and the predicted next segment of the path.
  • the method includes updating the detected segment of the path and the predicted next contiguous segment of the path as the user contact point moves across the touch sensitive display.
  • an embodiment of the subject matter described herein includes a method implemented in a computing environment.
  • the method includes detecting a first segment of a path defined by a user contact point moving across a touch sensitive display of the computing device.
  • the method includes determining a first parameter descriptive of a first motion of the user contact point during its movement across the detected first segment of the path (hereafter "first motion parameter").
  • the method includes predicting in response to the first motion parameter a second contiguous segment of the path of the user contact point moving across the touch sensitive display.
  • the method includes displaying on the touch sensitive display the detected first segment of the path and the predicted second segment of the path.
  • the method includes detecting a second segment of the path defined by the user contact point moving across the touch sensitive display of the computing device.
  • the method includes determining a second parameter descriptive of a second motion of the user contact point during its movement across the detected second segment of the path (hereafter "second motion parameter").
  • the method includes predicting in response to the second motion parameter a third contiguous segment of the path defined by the user contact point moving across the touch sensitive display.
  • the method includes displaying on the touch sensitive display the detected first segment, the detected second segment, and the predicted third segment of the path.
  • the method includes determining a parameter descriptive of a motion of the user contact point during its movement across the detected segment of the path (hereafter "motion parameter").
  • motion parameter a parameter descriptive of a motion of the user contact point during its movement across the detected segment of the path.
  • the method includes selecting responsive to the motion parameter a time-interval forecasted to improve a correspondence between a predicted next contiguous segment of the path defined by the user contact point and a subsequently detected next contiguous segment of the path.
  • the method includes predicting in response to the motion parameter and the selected time-interval a next contiguous segment of the path defined by the user contact point.
  • the method includes initiating a display by the touch sensitive display of the detected segment of the path and the predicted next segment of the path.
  • the method includes initiating an update of the detected segment of the path, and the predicted next contiguous segment of the path as the user contact point moves across the touch sensitive display.
  • FIG. 1 illustrates an example embodiment of an environment 19 that includes a thin computing device 20 in which embodiments may be implemented;
  • FIG. 2 illustrates an example embodiment of an environment 100 that includes a general-purpose computing system 110 in which embodiments may be implemented;
  • FIG. 3 schematically illustrates an example environment 200 in which embodiments may be implemented
  • FIGS. 4A-4C illustrate examples of the detected and predicted segments of a path defined by a user contact point moving across a touch sensitive display of an apparatus 205;
  • FIG. 5 illustrates an example operational flow 300 implemented in a computing device
  • FIG. 6 illustrates an example operational flow 400 implemented in a computing device
  • FIG. 7 schematically illustrates an example environment 500 in which embodiments may be implemented
  • FIG. 8 illustrates an example operational flow 600 implemented in a computing device
  • FIG. 9 illustrates an example apparatus 700
  • FIG. 10 schematically illustrates an example environment 800 in which embodiments may be implemented
  • FIG. 11 illustrates an example operational flow 900 implemented in a computing device
  • FIG. 12 schematically illustrates an example environment 1000 in which embodiments may be implemented.
  • FIG. 13 illustrates an example operational flow 1 100 implemented in a computing device.
  • implementations may include software or other control structures suitable to implement an operation.
  • Electronic circuitry may manifest one or more paths of electrical current constructed and arranged to implement various logic functions as described herein.
  • one or more media are configured to bear a device-detectable implementation if such media hold or transmit a special-purpose device instruction set operable to perform as described herein.
  • this may manifest as an update or other modification of existing software or firmware, or of gate arrays or other programmable hardware, such as by performing a reception of or a transmission of one or more instructions in relation to one or more operations described herein.
  • an implementation may include special- purpose hardware, software, firmware components, and/or general-purpose components executing or otherwise invoking special-purpose components. Specifications or other implementations may be transmitted by one or more instances of tangible transmission media as described herein, optionally by packet transmission or otherwise by passing through distributed media at various times.
  • implementations may include executing a special-purpose instruction sequence or otherwise invoking circuitry for enabling, triggering, coordinating, requesting, or otherwise causing one or more occurrences of any functional operations described below.
  • operational or other logical descriptions herein may be expressed directly as source code and compiled or otherwise invoked as an executable instruction sequence.
  • C++ or other code sequences can be compiled directly or otherwise implemented in high-level descriptor languages (e.g., a logic-synthesizable language, a hardware description language, a hardware design simulation, and/or other such similar mode(s) of expression).
  • some or all of the logical expression may be manifested as a Verilog-type hardware description or other circuitry model before physical
  • electromechanical system includes, but is not limited to, electrical circuitry operably coupled with a transducer (e.g., an actuator, a motor, a piezoelectric crystal, a Micro Electro Mechanical System (MEMS), etc.), electrical circuitry having at least one discrete electrical circuit, electrical circuitry having at least one integrated circuit, electrical circuitry having at least one application specific integrated circuit, electrical circuitry forming a general purpose computing device configured by a computer program (e.g., a general purpose computer configured by a computer program which at least partially carries out processes and/or devices described herein, or a microprocessor configured by a computer program which at least partially carries out processes and/or devices described herein), electrical circuitry forming a memory device (e.g., forms of memory (e.g., random access, flash, read only, etc.)), electrical circuitry forming a communications device (e.g., a modem, module, communications switch, optical-electrical equipment, etc.), and/or any
  • electro-mechanical systems include but are not limited to a variety of consumer electronics systems, medical devices, as well as other systems such as motorized transport systems, factory automation systems, security systems, and/or communication/computing systems.
  • electromechanical as used herein is not necessarily limited to a system that has both electrical and mechanical actuation except as context may dictate otherwise.
  • electrical circuitry includes, but is not limited to, electrical circuitry having at least one discrete electrical circuit, electrical circuitry having at least one integrated circuit, electrical circuitry having at least one application specific integrated circuit, electrical circuitry forming a general purpose computing device configured by a computer program (e.g., a general purpose computer configured by a computer program which at least partially carries out processes and/or devices described herein, or a microprocessor configured by a computer program which at least partially carries out processes and/or devices described herein), electrical circuitry forming a memory device (e.g., forms of memory (e.g., random access, flash, read only, etc.)), and/or electrical circuitry forming a communications device (
  • a typical image processing system may generally include one or more of a system unit housing, a video display device, memory such as volatile or non- volatile memory, processors such as microprocessors or digital signal processors, computational entities such as operating systems, drivers, applications programs, one or more interaction devices (e.g., a touch pad, a touch-sensitive screen or display surface, an antenna, etc.), control systems including feedback loops and control motors (e.g., feedback for sensing lens position and/or velocity; control motors for moving/distorting lenses to give desired focuses).
  • An image processing system may be implemented utilizing suitable
  • a data processing system generally includes one or more of a system unit housing, a video display device, memory such as volatile or non- volatile memory, processors such as microprocessors or digital signal processors, computational entities such as operating systems, drivers, graphical user interfaces, and applications programs, one or more interaction devices (e.g., a touch pad, a touch-sensitive screen or display surface, an antenna, etc.), and/or control systems including feedback loops and control motors (e.g., feedback for sensing position and/or velocity; control motors for moving and/or adjusting components and/or quantities).
  • a data processing system may be implemented utilizing suitable commercially available components, such as those typically found in data computing/communication and/or network computing/communication systems.
  • FIGS. 1 and 2 provide respective general descriptions of several environments in which implementations may be implemented.
  • FIG. 1 is generally directed toward a thin computing environment 19 having a thin computing device 20, and
  • FIG. 2 is generally directed toward a general purpose computing environment 100 having general purpose computing device 110.
  • general purpose computing device 110 As prices of computer components drop and as capacity and speeds increase, there is not always a bright line between a thin computing device and a general purpose computing device. Further, there is a continuous stream of new ideas and applications for environments benefited by use of computing power. As a result, nothing should be construed to limit disclosed subject matter herein to a specific computing environment unless limited by express language.
  • FIG. 1 and the following discussion are intended to provide a brief, general description of a thin computing environment 19 in which embodiments may be implemented.
  • FIG. 1 illustrates an example system that includes a thin computing device 20, which may be included or embedded in an electronic device that also includes a device functional element 50.
  • the electronic device may include any item having electrical or electronic components playing a role in a functionality of the item, such as for example, a refrigerator, a car, a digital image acquisition device, a camera, a cable modem, a printer an ultrasound device, an x-ray machine, a non-invasive imaging device, or an airplane.
  • the electronic device may include any item that interfaces with or controls a functional element of the item.
  • the thin computing device may be included in an implantable medical apparatus or device.
  • the thin computing device may be operable to communicate with an implantable or implanted medical apparatus.
  • a thin computing device may include a computing device having limited resources or limited processing capability, such as a limited resource computing device, a wireless communication device, a mobile wireless communication device, a smart phone, an electronic pen, a handheld electronic writing device, a scanner, a cell phone, a smart phone (such as an Android® or iPhone® based device), a tablet device (such as an iPad®) or a Blackberry® device.
  • a thin computing device may include a thin client device or a mobile thin client device, such as a smart phone, tablet, notebook, or desktop hardware configured to function in a virtualized environment.
  • the thin computing device 20 includes a processing unit 21 , a system memory 22, and a system bus 23 that couples various system components including the system memory 22 to the processing unit 21.
  • the system bus 23 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures.
  • the system memory includes read-only memory (ROM) 24 and random access memory (RAM) 25.
  • ROM read-only memory
  • RAM random access memory
  • BIOS input/output system
  • ROM 24 containing the basic routines that help to transfer information between sub-components within the thin computing device 20, such as during start-up, is stored in the ROM 24.
  • a number of program modules may be stored in the ROM 24 or RAM 25, including an operating system 28, one or more application programs 29, other program modules 30 and program data 31.
  • a user may enter commands and information into the computing device
  • An input interface may include a touch-sensitive screen or display surface, or one or more switches or buttons with suitable input detection circuitry.
  • a touch-sensitive screen or display surface is illustrated as a touch-sensitive display 32 and screen input detector 33.
  • One or more switches or buttons are illustrated as hardware buttons 44 connected to the system via a hardware button interface 45.
  • the output circuitry of the touch-sensitive display 32 is connected to the system bus 23 via a video driver 37.
  • Other input devices may include a microphone 34 connected through a suitable audio interface 35, or a physical hardware keyboard (not shown).
  • Output devices may include the display 32, or a projector display 36.
  • the computing device 20 may include other peripheral output devices, such as at least one speaker 38.
  • Other external input or output devices 39 such as a joystick, game pad, satellite dish, scanner or the like may be connected to the processing unit 21 through a USB port 40 and USB port interface 41, to the system bus 23.
  • the other external input and output devices 39 may be connected by other interfaces, such as a parallel port, game port or other port.
  • the computing device 20 may further include or be capable of connecting to a flash card memory (not shown) through an appropriate connection port (not shown).
  • the computing device 20 may further include or be capable of connecting with a network through a network port 42 and network interface 43, and through wireless port 46 and corresponding wireless interface 47 may be provided to facilitate communication with other peripheral devices, including other computers, printers, and so on (not shown). It will be appreciated that the various components and connections shown are examples and other components and means of establishing communication links may be used.
  • the computing device 20 may be primarily designed to include a user interface.
  • the user interface may include a character, a key-based, or another user data input via the touch sensitive display 32.
  • the user interface may include using a stylus (not shown).
  • the user interface is not limited to an actual touch-sensitive panel arranged for directly receiving input, but may alternatively or in addition respond to another input device such as the microphone 34. For example, spoken words may be received at the microphone 34 and recognized.
  • the computing device 20 may be designed to include a user interface having a physical keyboard (not shown).
  • the device functional elements 50 are typically application specific and related to a function of the electronic device, and are coupled with the system bus 23 through an interface (not shown).
  • the functional elements may typically perform a single well-defined task with little or no user configuration or setup, such as a refrigerator keeping food cold, a cell phone connecting with an appropriate tower and transceiving voice or data information, a camera capturing and saving an image, or communicating with an implantable medical apparatus.
  • the thin computing device 20 may be deemed not necessary and omitted. In other instances, one or more other elements may be deemed necessary and added to the thin computing device.
  • FIG. 2 and the following discussion are intended to provide a brief, general description of an environment in which embodiments may be implemented.
  • FIG. 2 illustrates an example embodiment of a general-purpose computing system in which embodiments may be implemented, shown as a computing system environment 100.
  • Components of the computing system environment 100 may include, but are not limited to, a general purpose computing device 110 having a processor 120, a system memory 130, and a system bus 121 that couples various system components including the system memory to the processor 120.
  • the system bus 121 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures.
  • bus architectures include Industry Standard Architecture (ISA) bus, Micro Channel
  • MCA Multimedia Architecture
  • EISA Enhanced ISA
  • VESA VESA local bus
  • PCI Peripheral Component Interconnect
  • Computer-readable media may include any media that can be accessed by the computing device 110 and include both volatile and nonvolatile media, removable and non-removable media.
  • Computer-readable media may include computer storage media.
  • computer-readable media may include a communication media.
  • Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data.
  • Computer storage media includes, but is not limited to, random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other memory technology, CD-ROM, digital versatile disks (DVD), or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage, or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by the computing device 110.
  • a computer storage media may include a group of computer storage media devices.
  • a computer storage media may include an information store.
  • an information store may include a quantum memory, a photonic quantum memory, or atomic quantum memory.
  • Communication media may typically embody computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and include any information delivery media.
  • modulated data signal means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.
  • communications media may include wired media, such as a wired network and a direct-wired connection, and wireless media such as acoustic, RF, optical, and infrared media.
  • the system memory 130 includes computer storage media in the form of volatile and nonvolatile memory such as ROM 131 and RAM 132.
  • a RAM may include at least one of a DRAM, an EDO DRAM, a SDRAM, a RDRAM, a VRAM, or a DDR
  • a basic input/output system (BIOS) 133 containing the basic routines that help to transfer information between elements within the computing device 110, such as during start-up, is typically stored in ROM 131.
  • RAM 132 typically contains data and program modules that are immediately accessible to or presently being operated on by the processor
  • FIG. 2 illustrates an operating system 134, application programs 135, other program modules 136, and program data 137.
  • the operating system 134 offers services to applications programs 135 by way of one or more application programming interfaces (APIs) (not shown). Because the operating system 134 incorporates these services, developers of applications programs 135 need not redevelop code to use the services. Examples of APIs provided by operating systems such as Microsoft's "WINDOWS" ® are well known in the art.
  • the computing device 110 may also include other removable/nonremovable, volatile/nonvolatile computer storage media products.
  • FIG. 2 illustrates a non-removable non-volatile memory interface (hard disk interface) 140 that reads from and writes for example to non-removable, non- volatile magnetic media.
  • FIG. 2 also illustrates a removable non-volatile memory interface 150 that, for example, is coupled to a magnetic disk drive 151 that reads from and writes to a removable, non- volatile magnetic disk 152, or is coupled to an optical disk drive 155 that reads from and writes to a removable, non-volatile optical disk 156, such as a CD ROM.
  • removable/non-removable, volatile/non-volatile computer storage media that can be used in the example operating environment include, but are not limited to, magnetic tape cassettes, memory cards, flash memory cards, DVDs, digital video tape, solid state RAM, and solid state ROM.
  • the hard disk drive 141 is typically connected to the system bus 121 through a non-removable memory interface, such as the interface 140, and magnetic disk drive 151 and optical disk drive 155 are typically connected to the system bus 121 by a removable non-volatile memory interface, such as interface 150.
  • the drives and their associated computer storage media discussed above and illustrated in FIG. 2 provide storage of computer-readable instructions, data structures, program modules, and other data for the computing device 110.
  • hard disk drive 141 is illustrated as storing an operating system 144, application programs 145, other program modules 146, and program data 147. Note that these components can either be the same as or different from the operating system 134, application programs 135, other program modules 136, and program data 137.
  • the operating system 144, application programs 145, other program modules 146, and program data 147 are given different numbers here to illustrate that, at a minimum, they are different copies.
  • a user may enter commands and information into the computing device
  • input devices such as a microphone 163, keyboard 162, and pointing device
  • a mouse commonly referred to as a mouse, trackball, or touch pad.
  • Other input devices may include at least one of a touch-sensitive screen or display surface, joystick, game pad, satellite dish, and scanner. These and other input devices are often connected to the processor 120 through a user input interface 160 that is coupled to the system bus, but may be connected by other interface and bus structures, such as a parallel port, game port, or a universal serial bus (USB).
  • USB universal serial bus
  • a display 191 such as a monitor or other type of display device or surface may be connected to the system bus 121 via an interface, such as a video interface 190.
  • a projector display engine 192 that includes a projecting element may be coupled to the system bus.
  • the computing device 110 may also include other peripheral output devices such as speakers 197 and printer 196, which may be connected through an output peripheral interface 195.
  • the computing system environment 100 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 180.
  • the remote computer 180 may be a personal computer, a server, a router, a network PC, a peer device, or other common network node, and typically includes many or all of the elements described above relative to the computing device 110, although only a memory storage device 181 has been illustrated in FIG. 2.
  • the network logical connections depicted in FIG. 2 include a local area network (LAN) and a wide area network (WAN), and may also include other networks such as a personal area network (PAN) (not shown).
  • LAN local area network
  • WAN wide area network
  • PAN personal area network
  • Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets, and the Internet.
  • the computing system environment 100 When used in a networking environment, the computing system environment 100 is connected to the network 171 through a network interface, such as the network interface 170, the modem 172, or the wireless interface 193.
  • the network may include a LAN network environment, or a WAN network environment, such as the Internet.
  • program modules depicted relative to the computing device 110, or portions thereof, may be stored in a remote memory storage device.
  • FIG. 2 illustrates remote application programs 185 as residing on memory storage device 181. It will be appreciated that the network connections shown are examples and other means of establishing a
  • one or more elements of the computing device 110 may be deemed not necessary and omitted. In other instances, one or more other elements may be deemed necessary and added to the computing device.
  • FIG. 3 schematically illustrates an example environment 200 in which embodiments may be implemented.
  • the environment includes a device 205, illustrated as a computing device, and a user 290.
  • the device may include the thin computing device 20 illustrated in the computing environment 19 described in conjunction with FIG. 1.
  • the device may include the general purpose computing device 110 described in conjunction with the general purpose computing environment 100.
  • the device includes a touch sensitive display 210.
  • the environment includes an apparatus 220, which includes a touch tracking circuit 222 configured to detect a segment of a path 280 defined by a user contact point 292 moving across the touch sensitive display.
  • the path may be defined by the user contact point moving across a relatively small portion of the touch sensitive display, such when forming a letter or a word, such as when forming an element of a graphic, such as when forming a swipe.
  • FIG. 4A illustrates an embodiment that includes a segment 282 of the path 280 defined by the user contact point moving across the touch sensitive display.
  • the apparatus includes a motion analysis circuit 224 configured to determine a parameter descriptive of a motion of the user contact point during its movement across the detected segment of the path
  • FIG. 4A illustrates the motion of the user contact point by a motion 294.
  • the apparatus includes a predictive filter 226 configured to predict in response to the motion parameter a next contiguous segment of the path defined by the user-contact point moving across the touch sensitive display.
  • FIG. 4B illustrates the predicted next contiguous segment 284P of the path.
  • a touch screen tracking system often lags behind the actual user-contact point because of latency inherent in the tracking system.
  • the predicted next contiguous segment is predicting where the user contact point has actually moved but of which detection has not been achieved because of the latency inherent in the touch screen tracking system.
  • the apparatus includes a latency compensation circuit 228 configured to initiate a display by the touch sensitive display of the detected segment of the path and the predicted next segment of the path.
  • the apparatus includes an updating circuit 232 configured to initiate an update of the detected segment of the path and the predicted next contiguous segment of the path as the user contact point moves across the touch sensitive display.
  • the latency compensation circuit updates the detected and predicted segments displayed by the touch sensitive display.
  • FIG. 4C illustrates an example of the updating.
  • the touch sensitive display presents a detected second segment 284D of the path and a second predicted segment 286P of the path.
  • a second parameter of the motion of the user contact point is determined, which is illustrated by a motion 296.
  • the updating circuit is configured to initiate a dynamic updating of the detected segment of the path and the predicted next contiguous segment of the path as the user contact point moves across the touch sensitive display.
  • the user contact point 292 includes a tip of a finger of the user. In an embodiment, the user contact point includes a tip of a handheld stylus held by the user. In an embodiment, the path 280 is defined by the user contact point moving across and touching the touch sensitive display 210. [0039] In an embodiment, the motion analysis circuit 224 is further configured to analyze an aspect of the movement of the user contact point 292 across the detected segment 282 of the path 280, and to determine a parameter descriptive of a motion 294 of the user contact point during its movement across a detected segment of the path based on the analyzed aspect. In an embodiment, the motion parameter is descriptive of an aspect of the motion of the user contact point.
  • the motion parameter is descriptive of the motion of the user contact point during a portion of its movement across the detected segment of the path.
  • the motion parameter includes a velocity parameter of the user contact point.
  • a velocity parameter may include a parameter responsive to a linear or rotation motion of the user contact point.
  • a velocity parameter may involve a projection of 3D motion onto the plane of the touchscreen.
  • a change in motion may be due to changes in direction.
  • a motion parameter may indicate angular velocity, angular acceleration, or the like.
  • a motion parameter may be based upon a time history of the contact point.
  • a motion parameter may be inferred in response to a proximity of the user contact point to an outer perimeter of the touch sensitive display.
  • a motion parameter may be based upon data received from the touchscreen's digitizer.
  • the motion parameter includes a two-dimensional velocity parameter of the user contact point.
  • the motion parameter includes an acceleration parameter of the user contact point.
  • An acceleration parameter may indicate a change in speed, either speeding up or slowing down.
  • the motion parameter includes a two-dimensional acceleration parameter of the user contact point.
  • the motion parameter includes an orientation or motion of the user contact point relative to the touch sensitive display.
  • the motion may include a linear or an angular motion.
  • the motion parameter includes a difference between a detected motion and a previously made prediction of the motion.
  • the motion parameter includes a curvature of the path.
  • the motion parameter includes (i) a motion parameter of the user contact point and (ii) a motion parameter of a finger or a hand of the user forming the contact point, or of a handheld stylus forming the contact point.
  • the motion analysis circuit 224 is further configured to determine a parameter descriptive of a motion of the user contact point 292 defined by a tip of a handheld stylus during its movement across the detected segment 282 of the path 280.
  • the determination is responsive to a signal generated by the handheld stylus and indicative of a sensed parameter descriptive of a motion of the handheld stylus during its movement across detected segment of the path.
  • the signal includes data indicative of a velocity or acceleration of the handheld stylus.
  • the data may include data acquired using accelerometers carried by the handheld stylus having a known distance from the tip.
  • the data may include data indicative of a stylus orientation, stylus angular motion, or the like.
  • the sensed parameter includes a sensed parameter indicative of a two- dimensional velocity of the tip of the handheld stylus.
  • the sensed parameter may be indicated by a vector.
  • the sensed parameter includes a linear or angular motion of the tip of the handheld stylus.
  • the sensed parameter includes a projection of 3D motion onto the plane of the touchscreen.
  • the sensed parameter includes a sensed parameter indicative of a two- dimensional acceleration of the tip of the handheld stylus.
  • the sensed parameter includes a sensed parameter indicative of an orientation or [linear, angular] motion of the handheld stylus relative to the touch sensitive display.
  • the motion may include a linear or an angular motion.
  • the sensed parameter includes a sensed parameter indicative of a motion of the tip of the handheld stylus and a sensed parameter of a motion of another portion of the handheld stylus.
  • the motion analysis circuit 224 is further configured to determine a parameter descriptive of a motion of the tip of the handheld stylus during its movement across the detected segment of the path. The determination is responsive to (i) a signal generated by the handheld stylus and indicative of a sensed parameter descriptive of a motion of the tip of the handheld stylus during its movement across detected segment of the path, and (ii) an aspect of the movement of the tip of the handheld stylus across the detected segment of the path.
  • the predictive filter 226 is configured to predict in response to the detected motion parameter a next contiguous segment 282 of the path 280 of the user contact point 292 likely to occur during a time interval.
  • the time interval is a function of the latency period of the apparatus.
  • the latency period of the apparatus may be considered as a touchscreen lag of the apparatus, sometimes referred to as touch screen latency or delay.
  • the latency period of the apparatus may include a delay imposed by the whole computing device.
  • the latency period may include a delay in displayed content between a user touch and the touch being displayed.
  • the time interval is specified by a manufacturer of a computing device into which the touch sensitive display is incorporated or by a human user.
  • the predictive filter is further configured to determine the time interval based upon an analysis of the motion parameter.
  • the predictive filter is further configured to determine the time interval based at least partially upon a weighted error rate.
  • a weighted error rate can be based upon past prediction errors. For example, errors can be weighted with respect to time, so that preference is given to longer predictions.
  • the predictive filter is further configured to determine an optimum update schedule usable by the updating circuit 232 in response to a historical iterative convergence between the predicted likely next segment and the actual detected next segment.
  • an update schedule may be considered a refresh rate.
  • the update schedule may be subject to limitations otherwise inherent in the device 205.
  • the predictive filter is further configured to dynamically determine an optimized update schedule usable by the updating circuit.
  • the predictive filter is configured to predict in response to the motion parameter of the user contact point and in response to a motion parameter of the touch sensitive display a next contiguous segment of the path of the user contact point moving across the touch sensitive display.
  • the prediction may involve projection of 3D motion onto the plane of the touchscreen.
  • the prediction may involve subtraction of touchscreen acceleration.
  • the predictive filter includes a Kalman filter.
  • the predictive filter includes a model-based filter.
  • the motion prediction may combine a motion parameter extension with course- prediction (e.g., prediction of the letter, symbol, word, screen destination).
  • the predictive filter includes a high-speed digitizer configured to obtain sufficient sample points for the predictive filter to predict in response to the motion parameter a next contiguous segment of the path defined by the user-contact point moving across the touch sensitive display.
  • the updating circuit 232 is configured to initiate an update of the detected segment of the path and the predicted next contiguous segment of the path as the user contact point 292 moves across the touch sensitive display 210 based on a schedule.
  • the schedule includes an optimization schedule determined by the predictive filter.
  • the schedule includes initiating an update at least once during a latency period of the apparatus.
  • the updating circuit is configured to initiate an update of the detected segment of the path 282 and the predicted next contiguous segment 284P of the path 280 as the user contact point moves across the touch sensitive display based on a length of the detected segment of the path.
  • the updating circuit is further configured to initiate updates while a handheld stylus moves across the touch sensitive display.
  • the updating circuit is configured to initiate a display by the touch sensitive display of the detected segment of the path and the predicted next segment of the path concurrent with the movement across the touch sensitive display by the user contact point. In an embodiment, the updating circuit is configured to initiate a display by the touch sensitive display of the detected segment of the path using a first visual representation and of the predicted next segment of the path using a second visual representation that is humanly distinguishable from the first visual representation. For example, a first visual representation and of the predicted next segment of the path using a second visual representation that is humanly distinguishable from the first visual representation. For example, a first visual
  • a representation of the detected segment may include a solid black line
  • a second visual representation of the predicted next segment may include a dashed black line
  • a first visual representation of the detected segment may include a black line
  • a second visual representation of the predicted next segment may include a red line.
  • the apparatus 220 further comprises the touch sensitive display 210.
  • the apparatus includes the device 205 that includes the touch sensitive display 210.
  • the apparatus includes a receiver circuit 234 configured to receive a signal generated by a handheld stylus.
  • the receiver circuit may include a wireless receiver circuit 263.
  • the apparatus includes a learning circuit 236 configured to adaptively learn a motion parameter associated with a specific user based upon a history of at least two motion parameters determined in response to the path defined by a user contact point moving across the touch sensitive display.
  • the learning circuit is further configured to store in a computer readable storage media 240 the adaptively learned motion parameter in an association with an identifier of the specific user.
  • the learning circuit is configured to adaptively learn a motion parameter associated with a specific software application running on the apparatus and based upon a history of at least two motion parameters determined in response to a path defined by the user contact point moving across the touch sensitive display.
  • the learning circuit is further configured to store in a computer readable storage media the learned motion parameter in an association with an identification of the specific software application running on the apparatus.
  • the apparatus includes a non- transitory computer readable storage media.
  • FIG. 5 illustrates an example operational flow 300 implemented in a computing device.
  • the computing device may include the thin computing device 20 illustrated in the computing environment 19 described in conjunction with FIG. 1.
  • the device may include the general purpose computing device 110 described in conjunction with the general purpose computing environment 100 described in conjunction with FIG. 2.
  • the operational flow includes a tracking operation 310.
  • the tracking operation includes detecting a segment of a path defined by a user contact point moving across a touch sensitive display.
  • the tracking operation may be implemented using the touch tracking circuit 222 described in conjunction with FIG. 3.
  • An analysis operation 320 includes determining a parameter descriptive of a motion of the user contact point during its movement across the detected segment of the path (hereafter "motion parameter").
  • the analysis operation may be implemented using the motion analysis circuit 224 described in conjunction with FIG. 3.
  • a prediction operation 330 includes predicting in response to the motion parameter a next contiguous segment of the path of the user contact point moving across the touch sensitive display.
  • the prediction operation may be implemented using the predictive filter 226 described in conjunction with FIG. 3.
  • a display operation 340 includes displaying a human-perceivable rendering of the detected segment of the path and the predicted next segment of the path.
  • the display operation may be initiated by the latency compensation circuit 228 initiating the displaying by the touch sensitive display 210 as described in conjunction with FIG. 3.
  • An update operation 350 includes updating the detected segment of the path and the predicted next contiguous segment of the path as the user contact point moves across the touch sensitive display.
  • the updating may include a continuously updating the detected segment of the path and the predicted next contiguous segment of the path.
  • the updating may include an incrementally updating the detected segment of the path and the predicted next contiguous segment of the path.
  • the updating may include a repeatedly updating the detected segment of the path and the predicted next contiguous segment of the path.
  • the update operation may be implemented using the updating circuit 232 described in conjunction with FIG. 3.
  • the operational flow includes an end operation.
  • the analysis operation 330 includes analyzing an aspect of the movement of the user contact point across the detected segment of the path, and determining a parameter descriptive of a motion of the user contact point during its movement across the detected segment of the path based on the analyzed aspect.
  • the analysis operation includes determining a parameter descriptive of a motion of the user contact point during its movement across the detected segment of the path based on a signal generated by the handheld stylus and indicative of a sensed parameter descriptive of a motion of the user contact point during its movement across the detected segment of the path.
  • FIG. 6 illustrates an example operational flow 400 implemented in a computing device.
  • the computing device may include the thin computing device 20 illustrated in the computing environment 19 described in conjunction with FIG. 1.
  • the device may include the general purpose computing device 110 described in conjunction with the general purpose computing environment 100 described in conjunction with FIG. 2.
  • the operational flow includes a first tracking operation 410.
  • the first tracking operation includes detecting a first segment of a path defined by a user contact point moving across a touch sensitive display of the computing device.
  • a first analysis operation 420 includes determining a first parameter descriptive of a first motion of the user contact point during its movement across the detected first segment of the path (hereafter "first motion parameter").
  • a first prediction operation 430 includes predicting in response to the first motion parameter a second contiguous segment of the path of the user contact point moving across the touch sensitive display.
  • a first display operation 440 includes displaying on the touch sensitive display the detected first segment of the path and the predicted second segment of the path.
  • a second tracking operation 450 includes detecting a second segment of the path defined by the user contact point moving across the touch sensitive display of the computing device.
  • the first and second tracking operations may be implemented using the touch tracking circuit 222 described in conjunction with FIG. 3.
  • a second analysis operation 460 includes determining a second parameter descriptive of a second motion of the user contact point during its movement across detected second segment of the path (hereafter "second motion parameter").
  • the first and second analysis operations may be implemented using the motion analysis circuit 224 described in conjunction with FIG. 3.
  • a second prediction operation 470 includes predicting in response to the second motion parameter a third contiguous segment of the path defined by user contact point moving across the touch sensitive display.
  • the first and second prediction operations may be implemented using the predictive filter 226 described in conjunction with FIG. 3.
  • a second display operation 480 includes displaying on the touch sensitive display the detected first segment, the detected second segment, and the predicted third segment of the path.
  • the first and second display operations may be implemented using the touch tracking circuit 222 described in conjunction with FIG. 3.
  • the display operation may be initiated by the latency compensation circuit 228 initiating the displaying by the touch sensitive display 210 as described in conjunction with FIG. 3.
  • the operational flow includes an end operation.
  • the first detection operation 410 includes detecting a first segment of a continuing path of the user contact point moving across the touch sensitive display.
  • the first prediction operation 430 includes analyzing an aspect of the movement of the user contact point across the detected first segment of the path, and determining a first parameter descriptive of a motion of the user contact point during its movement across detected first segment of the path based on the analyzed aspect.
  • the first prediction operation 430 includes determining a first parameter descriptive of a motion of a tip of a stylus held by the user during its movement across the detected first segment of the path based on a first signal generated by the handheld stylus and indicative of a parameter descriptive of a sensed motion of the tip of the handheld stylus during its movement across detected first segment of the path.
  • the first signal is indicative of a parameter descriptive of a sensed motion of the tip of the handheld stylus relative to the touch sensitive display device during its movement across a portion of the detected first segment of the path.
  • the operational flow 400 may include receiving the first signal generated by the handheld stylus and indicative of a sensed motion parameter of the user contact point during a portion of the detected first segment of the path.
  • the first prediction operation includes detecting a first segment of a continuing path of the user contact point moving across the touch sensitive display.
  • the first determining operation includes determining a parameter descriptive of a motion of the user contact point during a portion of the movement of the user contact point across the detected first segment of the path.
  • the portion of movement may include movement over a portion of the detected first segment, such as middle 50%, last 25%, or last 10%>.
  • the portion of the movement is a movement during a time interval equal to or less than a detection latency period of the touch sensitive display. In an embodiment, the portion of the movement is a movement during a time interval less than one-half of the detection latency period of the touch sensitive display. In an embodiment, the portion of the movement is less than one-half of the linear length of the detected first segment of the path. In an embodiment, the first segment and the second segment are contiguous portions of the path of the user contact point. In an embodiment, the predicted second segment has a time interval at least equal to a detection latency period of the touch sensitive display. In an embodiment, the predicted second segment has a time interval specified by a manufacturer of the computing device or by a human user of the computing device.
  • the predicted second segment has an optimized time interval selected in response to an analysis of the movement of the handheld stylus across the touch sensitive display. In an embodiment, the predicted second segment has a length approximately equal to a length of the first segment. In an embodiment, the predicted second segment includes a segment of the path of the user contact point moving across the touch sensitive display formed subsequent to the formation of the first segment and not yet detected. In an embodiment, the predicted second segment includes a predicted second segment responsive to a forward projection of the first sensed motion parameter. For example, the forward projection of the first sensed motion parameter may be a speed, acceleration, direction change parameter. In an embodiment, the predicted second segment includes a predicted second segment responsive to an extension of the sensed motion parameter combined with a course-prediction.
  • FIG. 7 schematically illustrates an example environment 500 in which embodiments may be implemented.
  • the environment includes a device 505, illustrated as a computing device, and the user 290.
  • the device may include the thin computing device 20 illustrated in the computing environment 19 described in conjunction with FIG. 1.
  • the device may include the general purpose computing device 110 described in conjunction with the general purpose computing environment 100.
  • the device includes the touch sensitive display 210 and an apparatus 520.
  • the apparatus 520 includes a touch tracking circuit 522 configured to detect a segment 282 of the path 280 defined by a user contact point 292 moving across the touch sensitive display 210.
  • FIG. 4 illustrates previously described features and associated reference numbers of the path.
  • a motion analysis circuit 524 is configured to determine a parameter descriptive of a motion 294 of the user contact point during its movement across the detected segment of the path (hereafter "motion parameter").
  • An interval selection circuit 526 is configured to select responsive to the motion parameter a time-interval forecasted to improve a correspondence between a predicted next contiguous segment of the path defined by the user contact point and a subsequently detected 284D next contiguous segment of the path.
  • the time -interval may be selected to improve accuracy in predicting the next contiguous segment with respect to the display lag.
  • a predictive filter 528 is configured to predict in response to the motion parameter and the selected time-interval the next contiguous segment 284P of the path defined by the user contact point.
  • a latency compensation circuit 532 is configured to initiate a display by the touch sensitive display 210 of the detected segment of the path and the predicted next contiguous segment of the path.
  • An updating circuit 534 is configured to initiate an update of the detected segment of the path and the predicted next contiguous segment of the path as the user contact point moves across the touch sensitive display.
  • the interval selection circuit 526 is configured to select an increased time-interval in response to a motion parameter indicative of a hesitating motion or pausing motion of the user contact point. In an embodiment, the interval selection circuit is configured to select a decreased time-interval in response to a motion parameter indicative of an increasing speed of the user contact point across the touch sensitive display or forward jerking motion of the user contact point. In an embodiment, the interval selection circuit is configured to update the time-interval in response to a change in an aspect of the motion parameter. In an embodiment, the interval selection circuit is configured to update the time -interval in response to each instance of an updating of the detected segment of the path.
  • the interval selection circuit configured to select the time -interval responsive to the motion parameter and to available computing resources. In an embodiment, the interval selection circuit configured to select the time-interval responsive to the motion parameter, available computing resources, and an aspect of a user experience related to the touch screen display.
  • the interval selection circuit is configured to update the time -interval based on a time schedule.
  • a schedule may be every 2 seconds, 5 seconds, or 10 seconds.
  • the interval selection circuit is configured to update the time-interval based on a schedule responsive to a specified number of instances of updating the detected segment of the path.
  • the time- interval may be each 10th update of the detected segment, or each 25th update of the detected segment.
  • the interval selection circuit is configured to update the time -interval in response a change of a user of the apparatus.
  • the interval selection circuit is configured to update the time-interval in response to a start of a new session on the apparatus by a user.
  • the interval selection circuit is configured to update the time-interval in response to a particular elapsed usage time of the touch sensitive display. For example, an elapsed time may be 1 minute, 2 minutes, or 5 minutes.
  • the interval selection circuit is configured to retrieve a stored time-interval associated with a particular user of the apparatus. For example, stored-time interval may be retrieved from a computer readable storage media 540.
  • the interval selection circuit is configured to retrieve a stored time -interval associated with a handheld stylus currently being used to form the contact point. In an embodiment, the interval selection circuit is configured to retrieve a time-interval stored in the handheld stylus. In an embodiment, the updating circuit includes an updating circuit configured to initiate an update of the selected time-interval, the detected segment of the path, and the predicted next contiguous segment of the path as the user contact point moves across the touch sensitive display.
  • FIG. 8 illustrates an example operational flow 600 implemented in a computing device. After a start operation, the operational flow includes a tracking operation 610. The tracking operation includes detecting a segment of a path defined by a user contact point moving across a touch sensitive display.
  • the tracking operation may be implemented using the touch tracking circuit 522 described in conjunction with FIG. 7.
  • An analysis operation 620 includes determining a parameter descriptive of a motion of the user contact point during its movement across detected segment of the path (hereafter "motion parameter").
  • the analysis operation may be implemented using the motion analysis circuit 524 described in conjunction with FIG. 7.
  • An interval selection operation 630 includes selecting responsive to the motion parameter a time-interval forecasted to improve a correspondence between a predicted next contiguous segment of the path defined by the user contact point and a subsequently detected next contiguous segment of the path.
  • the interval selection operation may be implemented using the interval selection circuit 526 described in conjunction with FIG. 7.
  • a prediction operation 640 includes predicting in response to the motion parameter and the selected time -interval a next contiguous segment of the path defined by the user contact point.
  • the prediction operation may be implemented using the predictive filter 528 described in conjunction with FIG. 7.
  • a display operation 650 includes initiating a display by the touch sensitive display of the detected segment of the path and the predicted next segment of the path.
  • the display operation may be implemented by the latency compensation circuit 532 initiating the displaying by the touch sensitive display 210 described in conjunction with FIG. 7.
  • An update operation 660 includes initiating an update of the detected segment of the path, and the predicted next contiguous segment of the path as the user contact point moves across the touch sensitive display.
  • the update operation may be implemented using the updating circuit 534 described in conjunction with FIG. 7.
  • the operational flow includes an end operation.
  • the selecting of the interval selection operation 630 includes selecting an updated time -interval in response to a change in an aspect of the motion parameter. In an embodiment, the selecting includes selecting an updated time- interval in response to each instance of an updating of the detected segment of the path. In an embodiment, the selecting includes selecting an updated time-interval based on a schedule. In an embodiment, the interval selection circuit is configured to update the time-interval in response to a change of a user of the apparatus. In an embodiment, the initiating of the display operation 650 further includes initiating an update of the selected time-interval setting, the detected segment of the path, and the predicted next contiguous segment of the path as the user contact point moves across the touch sensitive display.
  • FIG. 9 illustrates an example apparatus 700.
  • the apparatus includes means 710 for detecting a segment of a path defined by a user contact point moving across a touch sensitive display.
  • the apparatus includes means 720 for determining a parameter descriptive of a motion of the user contact point during its movement across the detected segment of the path (hereafter "motion parameter").
  • the apparatus includes means 730 for selecting responsive to the motion parameter a time-interval forecasted to improve a correspondence between a predicted next contiguous segment of the path defined by the user contact point and a subsequently detected next contiguous segment of the path.
  • the apparatus includes means 740 for predicting in response to the motion parameter and the selected time-interval a next contiguous segment of the path defined by the user contact point.
  • the apparatus includes means 750 for initiating a display by the touch sensitive display of the detected segment of the path and the predicted next segment of the path.
  • the apparatus includes means 760 for initiating an update of the detected segment of the path, and the predicted next contiguous segment of the path as the user contact point moves across the touch sensitive display.
  • FIG. 10 schematically illustrates an example environment 800 in which embodiments may be implemented.
  • the environment includes a device 805, illustrated as a computing device, and the user 290.
  • the device may include the thin computing device 20 illustrated in the computing environment 19 described in conjunction with FIG. 1.
  • the device may include the general purpose computing device 110 described in conjunction with the general purpose computing environment 100.
  • the device includes the touch sensitive display 210 and an apparatus 820.
  • the apparatus 820 includes a touch tracking circuit 822 configured to detect a segment of the path 280 defined by the user contact point 292 moving across the touch sensitive display 210.
  • a motion analysis circuit 824 is configured to determine (i) a parameter descriptive of a motion of the user contact point during its movement across the detected segment of the path (hereafter "motion parameter"), and an indicator of an impending change in the motion of the user contact point occurring during its movement across the detected segment of the path (hereafter "indicator parameter").
  • a predictive filter 826 is configured to predict in response to the motion parameter and the indicator parameter a next contiguous segment of the path defined by the user contact point.
  • a latency compensation circuit 828 is configured to initiate a display by the touch sensitive display 210 of the detected segment of the path and the predicted next segment of the path.
  • An updating circuit 832 is configured to update the detected segment of the path, the motion parameter, the indicator parameter, and the predicted next contiguous segment of the path as the user contact point moves across the touch sensitive display.
  • the apparatus includes a computer readable storage media 840.
  • the touch tracking circuit 822 is configured to detect a segment of a path 280 described or formed by the user contact point 292 moving across the touch sensitive display 210.
  • the indicator of an impending change includes a change in a tilt of a finger or of a handheld stylus forming the user contact point.
  • the change in tilt may be relative to the touch sensitive display.
  • the change in tilt may be relative to the earth's horizon.
  • the indicator of an impending change includes a flexing of a finger forming the user contact point, or of a flexing of one or more fingers holding a handheld stylus forming the user contact point.
  • the indicator of an impending change includes a twisting of a finger forming the user contact point, or of a twisting of a handheld stylus forming the user contact point relative to the touch sensitive display.
  • the indicator of an impending change includes a change in a user's hand grip on a handheld stylus forming the user contact point.
  • the change may include a change in position of a user's hand grip.
  • the indicator of an impending change includes a change in a force applied by the user to the touch sensitive display at the contact point.
  • the predictive filter 826 is further configured to adjust a technique of the predictive filter in response to the indicator parameter.
  • the predictive filter is further configured to adjust or change a parameter of a motion prediction system of the predictive filter in response to the indicator parameter.
  • the adjust or change of a parameter of a motion prediction system includes shortening a sampling interval, or decreasing a prediction model's inertia.
  • the adjust or change of a parameter of a motion prediction system includes changing a weight given the motion parameter.
  • the adjust or change of a parameter of a motion prediction system includes adjusting or changing a type or value of a parameter employed by a motion prediction system.
  • the adjust or change of a parameter of a motion prediction system includes adjusting or changing a weighting of one type of motion compared to another by the motion prediction system.
  • FIG. 11 illustrates an example operational flow 900 implemented in a computing device.
  • the computing device may include the thin computing device 20 illustrated in the computing environment 19 described in conjunction with FIG. 1.
  • the device may include the general purpose computing device 110 described in conjunction with the general purpose computing environment 100 described in conjunction with FIG. 2.
  • the operational flow includes a tracking operation 910.
  • the tracking operation includes detecting a segment of a path defined by a user contact point moving across a touch sensitive display.
  • the tracking operation may be implemented using the touch tracking circuit 822 described in conjunction with FIG. 10.
  • An analysis operation 920 includes determining a parameter descriptive of a motion of the user contact point during its movement across the detected segment of the path (hereafter "motion parameter”).
  • the analysis operation includes determining an indicator of an impending change in the motion of the user contact point occurring during its movement across the detected segment of the path (hereafter "indicator parameter”).
  • the analysis operation may be implemented using the motion analysis circuit 824 described in conjunction with FIG. 10.
  • a prediction operation 930 includes predicting in response to the motion parameter and the indicator parameter a next contiguous segment of the path defined by the user contact point.
  • the prediction operation may be implemented using the predictive filter 826 described in conjunction with FIG. 10.
  • a display operation 940 includes displaying with the touch sensitive display the detected segment of the path and the predicted next segment of the path.
  • the display operation may be implemented by the latency compensation circuit 828 initiating a display by the touch sensitive display 210 as described in conjunction with FIG. 10.
  • An update operation 950 includes updating the detected segment of the path, the motion parameter, the indicator parameter, and the predicted next contiguous segment of the path as the user contact point moves across the touch sensitive display.
  • the update operation may be implemented using the updating circuit 832 described in conjunction with FIG. 10.
  • the operational flow includes an end operation.
  • the predicting of the prediction operation 930 further includes adjusting a prediction technique in response to the indicator parameter. In an embodiment, the predicting further includes adjusting or changing a parameter of a motion prediction technique of the predictive filter in response to the indicator parameter.
  • FIG. 12 schematically illustrates an example environment 1000 in which embodiments may be implemented.
  • the environment includes a device 1005, illustrated as a computing device, and the user 290.
  • the device may include the thin computing device 20 illustrated in the computing environment 19 described in conjunction with FIG. 1.
  • the device may include the general purpose computing device 110 described in conjunction with the general purpose computing environment 100.
  • the device includes the touch sensitive display 210 and an apparatus 1020.
  • the apparatus 1020 includes a touch tracking circuit 1022 configured to detect a segment of the path 280 defined by the user contact point 292 moving across the touch sensitive display 210.
  • the apparatus includes a predictive filter 1024 configured to predict a next contiguous segment of the path defined by the user contact point in response to an adaptively learned motion parameter.
  • the adaptively learned motion parameter is based on at least two previous instances of the determined motion parameters respectively descriptive of a motion of a user contact point during its movement across the touch sensitive display.
  • the apparatus includes a latency compensation circuit 1026 configured to initiate a display by the touch sensitive display of the detected segment of the path and the predicted next contiguous segment of the path.
  • the apparatus includes an updating circuit 1028 configured to update the detected segment of the path and the predicted next contiguous segment of the path as the user contact point moves across the touch sensitive display.
  • the adaptively learned motion parameter includes an adaptively learned motion parameter associated with a specific human user.
  • the adaptively learned motion parameter includes an adaptively learned motion parameter associated with a specific human user and based upon a history of at least two motion parameters determined in response to the path 280 defined by the user contact point 290 moving across the touch sensitive display 210 and formed by the specific human user.
  • the adaptively learned motion parameter associated with a specific human user comprises a motion parameter learned during a previous usage session involving the user, and can be retrieved from a computer readable storage media 1040 having stored thereupon the previously learned motion parameter.
  • the previously learned motion parameters were learned during a previous usage session involving device 1005, while in other embodiments the previously learned motion parameters were learned during a previous usage session involving a different device.
  • the user contact point is formed by the specific human user.
  • the adaptively learned motion parameter includes an adaptively learned motion parameter associated with a specific software application running on the apparatus 1005.
  • the adaptively learned motion parameter includes an adaptively learned motion parameter associated with a specific software application running on the apparatus, and based upon a history of at least two motion parameters determined in response to a path defined by the user contact point moving across a touch sensitive display in conjunction with a user interaction with the specific software application.
  • the predictive filter 1024 is configured to predict a next contiguous segment of the path 280 defined by the user contact point 292 in response to a learned motion parameter associated with a specific user and in response to a learned motion parameter associated with a specific software application running on the apparatus 1005.
  • the apparatus further includes a motion analysis circuit 1032 configured to determine a parameter descriptive of a motion of the user contact point during its current movement across detected segment of the path (hereafter "current motion parameter").
  • the predictive filter is further configured to predict a next contiguous segment of the path defined by the user contact point in response to the learned motion parameter and the current motion parameter.
  • the apparatus includes a learning circuit 1034 configured to adaptively learn the motion parameter.
  • the apparatus includes a computer readable storage media 1040 having stored thereupon the adaptively learned motion parameter.
  • the computer readable storage media includes a non-transitory computer readable storage media.
  • the apparatus includes a communication circuit configured to transmit the adaptively learned motion parameter to a remote device.
  • the apparatus includes a communication circuit configured to receive the adaptively learned motion parameter from a remote device.
  • FIG. 13 illustrates an example operational flow 1100 implemented in a computing device.
  • the computing device may include the thin computing device 20 illustrated in the computing environment 19 described in conjunction with FIG. 1.
  • the computing device may include the general purpose computing device 110 described in conjunction with the general purpose computing environment 100 described in conjunction with FIG. 2.
  • the operational flow includes a tracking operation 1110.
  • the tracking operation includes detecting a segment of a path defined by a user contact point moving across a touch sensitive display.
  • the tracking operation may be implemented using the touch tracking circuit 1022 described in conjunction with FIG. 12.
  • a prediction operation 1120 includes predicting a next contiguous segment of the path defined by the user contact point in response to an adaptively learned motion parameter.
  • the adaptively learned motion parameter is based on at least two previous instances of the determined motion parameters respectively descriptive of a motion of a user contact point during its movement across the touch sensitive display.
  • the prediction operation may be implemented using the predictive filter 1024 described in conjunction with FIG. 12.
  • a display operation 1130 includes displaying with the touch sensitive display the detected segment of the path and the predicted next contiguous segment of the path.
  • the display operation may be implemented by the latency compensation circuit 1026 initiating the displaying by the touch sensitive display 210 as described in conjunction with FIG. 12.
  • An update operation 1140 includes updating the detected segment of the path and the predicted next contiguous segment of the path as the user contact point moves across the touch sensitive display.
  • the update operation may be implemented using the updating circuit 1028 described in conjunction with FIG. 12.
  • the operational flow includes an end operation. [0068] In an embodiment, the operational flow 1100 includes adaptively learning the motion parameter.
  • the operational flow includes determining a parameter descriptive of a motion of the user contact point during its current movement across detected segment of the path (hereafter "current motion parameter").
  • the prediction operation 1120 further includes predicting a next contiguous segment of the path defined by the user contact point in response to the adaptively learned motion parameter and the current motion parameter.
  • "configured” includes at least one of designed, set up, shaped, implemented, constructed, or adapted for at least one of a particular purpose, application, or function.
  • “open” terms For example, the term “including” should be interpreted as “including but not limited to.”
  • the term “having” should be interpreted as “having at least.”
  • the term “has” should be interpreted as “having at least.”
  • the term “includes” should be interpreted as “includes but is not limited to,” etc.
  • any of these phrases would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B, and C together, and may further include more than one of A, B, or C, such as A ls A 2 , and C together, A, B ls B 2 , Ci, and C 2 together, or Bi and B 2 together).
  • any two components herein combined to achieve a particular functionality can be seen as “associated with” each other such that the desired functionality is achieved, irrespective of architectures or intermedial components.
  • any two components so associated can also be viewed as being “operably connected,” or “operably coupled,” to each other to achieve the desired functionality.
  • Any two components capable of being so associated can also be viewed as being “operably couplable” to each other to achieve the desired functionality.
  • operably couplable include but are not limited to physically mateable or physically interacting components or wirelessly interactable or wirelessly interacting components.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

Conformément à des modes de réalisation, l'invention concerne un appareil et un procédé. Dans un appareil, un circuit de suivi de toucher détecte un segment d'un trajet défini par un point de contact d'utilisateur se déplaçant à travers un dispositif d'affichage tactile. Un circuit d'analyse de mouvement détermine un paramètre descriptif d'un mouvement du point de contact d'utilisateur durant son déplacement à travers le segment détecté du trajet (dénommé ci-après "paramètre de mouvement"). Un filtre prédit, en réponse au paramètre de mouvement, un segment contigu suivant du trajet défini par le point de contact d'utilisateur se déplaçant à travers le dispositif d'affichage tactile. Un circuit de compensation initie un affichage par le dispositif d'affichage tactile du segment détecté du trajet et du segment suivant prédit du trajet. Un circuit de mise à jour initie une mise à jour du segment détecté du trajet et du segment contigu suivant prédit du trajet au fur et à mesure que le point de contact d'utilisateur se déplace à travers le dispositif d'affichage tactile.
PCT/US2014/067366 2013-12-03 2014-11-25 Compensation d'une latence lors de l'affichage d'une partie d'un mouvement initié par la main WO2015084644A1 (fr)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US14/095,641 US20150153855A1 (en) 2013-12-03 2013-12-03 Display latency compensation responsive to an indicator of an impending change in a hand-initiated movement
US14/095,623 2013-12-03
US14/095,641 2013-12-03
US14/095,612 2013-12-03
US14/095,612 US20150153890A1 (en) 2013-12-03 2013-12-03 Compensating for a latency in displaying a portion of a hand-initiated movement
US14/095,623 US20150153898A1 (en) 2013-12-03 2013-12-03 Latency compensation in a display of a portion of a hand-initiated movement

Publications (1)

Publication Number Publication Date
WO2015084644A1 true WO2015084644A1 (fr) 2015-06-11

Family

ID=53273989

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2014/067366 WO2015084644A1 (fr) 2013-12-03 2014-11-25 Compensation d'une latence lors de l'affichage d'une partie d'un mouvement initié par la main

Country Status (1)

Country Link
WO (1) WO2015084644A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019172829A1 (fr) * 2018-03-08 2019-09-12 Flatfrog Laboratories Ab Appareil tactile
US11893189B2 (en) 2020-02-10 2024-02-06 Flatfrog Laboratories Ab Touch-sensing apparatus

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080309636A1 (en) * 2007-06-15 2008-12-18 Ricoh Co., Ltd. Pen Tracking and Low Latency Display Updates on Electronic Paper Displays
JP2011125517A (ja) * 2009-12-18 2011-06-30 Kyoraku Sangyo Kk 遊技機
US20120306768A1 (en) * 2011-06-03 2012-12-06 Microsoft Corporation Motion effect reduction for displays and touch input
KR20130034777A (ko) * 2011-09-29 2013-04-08 삼성전자주식회사 휴대단말기의 펜 입력장치 및 방법
US20130181908A1 (en) * 2012-01-13 2013-07-18 Microsoft Corporation Predictive compensation for a latency of an input device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080309636A1 (en) * 2007-06-15 2008-12-18 Ricoh Co., Ltd. Pen Tracking and Low Latency Display Updates on Electronic Paper Displays
JP2011125517A (ja) * 2009-12-18 2011-06-30 Kyoraku Sangyo Kk 遊技機
US20120306768A1 (en) * 2011-06-03 2012-12-06 Microsoft Corporation Motion effect reduction for displays and touch input
KR20130034777A (ko) * 2011-09-29 2013-04-08 삼성전자주식회사 휴대단말기의 펜 입력장치 및 방법
US20130181908A1 (en) * 2012-01-13 2013-07-18 Microsoft Corporation Predictive compensation for a latency of an input device

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019172829A1 (fr) * 2018-03-08 2019-09-12 Flatfrog Laboratories Ab Appareil tactile
US11106312B2 (en) 2018-03-08 2021-08-31 Flatfrog Laboratories Ab Touch apparatus
US11893189B2 (en) 2020-02-10 2024-02-06 Flatfrog Laboratories Ab Touch-sensing apparatus

Similar Documents

Publication Publication Date Title
CN109891491B (zh) 控制交互式显示器的方法和装置
US11740724B2 (en) Deep machine learning to perform touch motion prediction
US10509560B2 (en) Electronic device having flexible display and method for operating the electronic device
EP3191922B1 (fr) Classification d'entrée tactile comme étant non intentionnelle ou intentionnelle
US20190318261A1 (en) System and method for active machine learning
US20210263620A1 (en) Standardizing User Interface Elements
KR20210022498A (ko) 반복 신경망들을 통한 포즈 예측
EP3625664A1 (fr) Lancement d'application dans un dispositif à affichage multiple
US20140035829A1 (en) Adjusting a displayed widget or delineated touch-selectable area of a touch screen dispaly in response to a predicted touch-contact site of an approaching user-appendage
US9268405B2 (en) Adaptive gesture-based method, system and computer program product for preventing and rehabilitating an injury
US20220206646A1 (en) Contextual action mechanisms in chat user interfaces
CN104054043A (zh) 可更换皮肤的触摸设备抓握模式
KR102577571B1 (ko) 로봇 장치 및 로봇 장치의 감정 표현 방법
US20160147388A1 (en) Electronic device for executing a plurality of applications and method for controlling the electronic device
EP3830672A1 (fr) Entrée prévue dans une interface utilisateur à partir de positions gestuelles détectées
US11430137B2 (en) Electronic device and control method therefor
US20150153855A1 (en) Display latency compensation responsive to an indicator of an impending change in a hand-initiated movement
US20150153898A1 (en) Latency compensation in a display of a portion of a hand-initiated movement
WO2015084644A1 (fr) Compensation d'une latence lors de l'affichage d'une partie d'un mouvement initié par la main
US20150153890A1 (en) Compensating for a latency in displaying a portion of a hand-initiated movement
US11782522B1 (en) Methods and systems for multimodal hand state prediction
US11743574B2 (en) System and method for AI enhanced shutter button user interface
US11423225B2 (en) On-device lightweight natural language understanding (NLU) continual learning
CN115079832A (zh) 虚拟现实场景的显示处理方法及虚拟现实设备
US11775815B2 (en) System and method for deep memory network

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14867742

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14867742

Country of ref document: EP

Kind code of ref document: A1