WO2017107086A1 - Touch gesture detection assessment - Google Patents

Touch gesture detection assessment Download PDF

Info

Publication number
WO2017107086A1
WO2017107086A1 PCT/CN2015/098381 CN2015098381W WO2017107086A1 WO 2017107086 A1 WO2017107086 A1 WO 2017107086A1 CN 2015098381 W CN2015098381 W CN 2015098381W WO 2017107086 A1 WO2017107086 A1 WO 2017107086A1
Authority
WO
WIPO (PCT)
Prior art keywords
gestures
gesture
detection
touch
computing device
Prior art date
Application number
PCT/CN2015/098381
Other languages
English (en)
French (fr)
Inventor
Fai Angus Yeung
Jinkui REN
Tingqian LI
Tong Chen
Gregory Meunier
Original Assignee
Intel Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Intel Corporation filed Critical Intel Corporation
Priority to PCT/CN2015/098381 priority Critical patent/WO2017107086A1/en
Priority to DE112015007219.5T priority patent/DE112015007219T5/de
Priority to US15/774,667 priority patent/US10488975B2/en
Priority to CN201580085074.2A priority patent/CN108292165B/zh
Publication of WO2017107086A1 publication Critical patent/WO2017107086A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/0416—Control or interface arrangements specially adapted for digitisers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/0416—Control or interface arrangements specially adapted for digitisers
    • G06F3/0418—Control or interface arrangements specially adapted for digitisers for error correction or compensation, e.g. based on parallax, calibration or alignment
    • G06F3/04186—Touch location disambiguation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/0414—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using force sensing means to determine a position

Definitions

  • Embodiments described herein generally relate to information processing equipment and associated methodology and, more particularly, to the detection of user input using touch sensing.
  • Finger gesture recognition also presents some critical challenges for mobile devices since users tend to have different physical characteristics and habitual behavior. For example, users may have different skin textures, affix a wearable device differently, apply a gesture with different degrees of force, or tap the device in different patterns. These kinds of variations complicate the design of mobile devices for accurate gesture recognition.
  • touch-sensitive devices offer a calibration, or optimization algorithm to tune their responsiveness to user-applied touch gestures for specific users. While a fully-tuned touch gesture detection algorithm may accommodate the majority, say 90%, of the user population reasonably well, there remains a small but very vocal group of users who struggle with incorrectly-interpreted gestures. Properly addressing the user experience issues, particularly from a disgruntled group, becomes a high priority in managing the brand image for a wearable device.
  • FIG. 1A is a block diagram illustrating some of the components of an example computing device according to some embodiments.
  • FIG. 1B is a diagram illustrating a wearable computing device according to some embodiments.
  • FIG. 2 is a block diagram illustrating an exemplary system architecture of a computing devices such as the devices of FIGs. 1A and 1B, according to an embodiment.
  • FIG. 3 is a diagram illustrating an exemplary hardware and software architecture of a computing device such as the one depicted in FIG. 2, in which various interfaces between hardware components and software components are shown.
  • FIG. 4 is a block diagram illustrating a set of engines that may be implemented using the computing hardware of a computing device that may be configured, or configurable according to executing, or installed, software or firmware program instructions according to some embodiments.
  • FIG. 5 is a time series diagram illustrating example outputs of a multi-axis motion sensor that observes a double-tapping gesture according to some embodiments.
  • FIG. 6 is a flow diagram illustrating an example process of detecting a double-tapping gesture according to some embodiments.
  • FIG. 7 is a table listing various example parameter values for tap gesture detection according to an embodiment.
  • FIG. 8 is a state diagram illustrating various potential sequences of tapping gestures according to some embodiments.
  • FIG. 9 is a table illustrating the various counters corresponding to each sequential tapping event depicted in FIG. 8 according to some embodiments.
  • FIG. 10 is a graph diagram illustrating sequential relationships between gesture events, including the temporal ordering of those events according to some embodiments.
  • FIG. 11 is a table associating the edges of the diagram of FIG. 10 with the event sequences shown in the diagram of FIG. 8 according to an example embodiment.
  • FIG. 12 is a diagram illustrating a reduced graph representing tapping events of interest for recalibrating double-tapping gesture detection according to an example embodiment.
  • FIG. 13 is a flow diagram illustrating a general process performing gesture recognition assessment according to some embodiments.
  • gestures are directed to touch-input gesture detection in computing devices.
  • touch-input gestures by the user as inputs to one or more sensors of a computing device, may be referred to simply as gestures.
  • One aspect of the embodiments is directed to automation of the re-calibration process for gesture recognition in computing devices.
  • the re-calibration is automatically initiated, meaning initiated without a specific user command in every instance, after an initial calibration process that is administered at an initial setup session with the user.
  • the calibration operation for gesture recognition is adaptive in nature. It detects a user’s gesture usage and pattern, and adaptively switches to another calibration profile if the new profile is more appropriate for the user.
  • Some embodiments provide an opportunity for a gesture recognition algorithm to fine-tune or modify calibrated configuration settings when the algorithm detects a change in user’s gesture pattern.
  • the gesture recognition re-calibration initiation is based on a recognition that a user tends to exhibit certain behaviors when the existing gesture recognition functionality fails to give the user the correct response as expected. The user will attempt to correct the gesture, for example, by applying the same gesture again.
  • This kind of behavior-based monitoring may advantageously reduce the occurrence of gesture recognition failures or false gesture detections.
  • a computing device in the present context may take any of a variety of device types. For instance, it may be a multi-functional device such as a smartphone, tablet, laptop, desktop PC, smartwatch, wearable form factor (e.g., smart glasses, or a device embedded in garment) , etc.
  • a computing device may have several integrated data capture devices.
  • FIG. 1A is a block diagram illustrating some of the components of an example computing device 100 according to an embodiment.
  • Computing device 100 is illustrated as a smartphone in this example, through it will be understood that computing device 100 is representative of other types of computing devices, which may have more or fewer data capture devices or other features than exemplary computing device 100.
  • Computing device 100 has a housing 102 that encloses the interior components. Housing 102 may provide access to the interior of device 100 to some degree. For instance, in devices with a user-replaceable battery, flash memory card, or subscriber identity module (SIM) card, housing 102 may include a user-removable cover. In devices having a design that does not facilitate user access to the interior, housing 102 may nonetheless have a provision for permitting access to technicians so that certain components may be repaired or replaced if needed.
  • SIM subscriber identity module
  • Computing device 100 further includes touchscreen 104, which may form a part of the overall enclosure of device 100 in cooperation with housing 102.
  • Touchscreen 104 includes hardware that functions as an output device (e.g., an LED screen for visual display, power and controller circuitry, etc. ) , and an input device generally layered over the visual display and formed from a suitable touch or proximity-sensitive technology (e.g., capacitive, resistive, optical, ultrasonic, etc. ) , along with the corresponding detection and power circuitry.
  • computing device 100 includes a user input device, which in this example represents one or more user-operable input devices, such as button (s) , keypad, keyboard, trackpad, mouse, etc.
  • computing device 100 may have several data capture devices, such as sensing transducers, the physical stimulation of which produces signaling that may be sampled, digitized, and stored as captured data.
  • Accelerometer 110 may include a multi-axis sensor that produces signaling in response to changes in motion, and electronics to sample and digitize that signaling.
  • the sensor may be a micro electro-mechanical system (MEMS) -based device utilizing capacitive, piezoelectric, or other suitable technology to produce electrical signaling, that is then amplified and processed with suitable analog and digital circuitry.
  • MEMS micro electro-mechanical system
  • Other sensors may include deformation or strain sensors, acoustic sensors, and the like.
  • FIG. 1B is a diagram illustrating a computing device that has a wearable form factor.
  • smartwatch 150 may also include a touchscreen 104 and an accelerometer 110.
  • FIG. 2 is a block diagram illustrating an exemplary system architecture 200 of computing devices 100 or 150 according to some embodiments.
  • Central processor unit (CPU) 202 includes one or more microprocessors on which the overall functionality of computing device 100, 150 is executed.
  • CPU 202 is formed from hardware that is electrically interfaced with system link 203, which carries data and control signaling between the various components. As illustrated, system link 203 is similarly interfaced with each of the other components of system architecture 200.
  • Memory 204 includes working memory space, and is constructed from suitable high-speed memory devices such as synchronous dynamic random access memory (SDRAM) . In the embodiment illustrated, CPU 202 may access memory 204 using high-speed interface 205.
  • SDRAM synchronous dynamic random access memory
  • Non-volatile memory 206 is constructed using read-only memory (ROM) , electrically-erasable programmable read-only memory (EEPROM) , flash memory or other suitable non-volatile storage technology.
  • ROM read-only memory
  • EEPROM electrically-erasable programmable read-only memory
  • flash memory or other suitable non-volatile storage technology.
  • Non-volatile memory 206 stores system and application software that is executed by CPU 202 and, in some cases, by processors present in one or more other components.
  • External non-volatile memory 207 includes an interface such as a secure digital (SD) card slot, which may accept removable storage media to be used as additional non-volatile data storage.
  • SD secure digital
  • Display 208 includes display 104 and circuitry for interfacing the display 104 with the system, as well as video driving circuity.
  • Sound 210 contains circuitry for driving the audio output to a speaker or headphones, and the circuitry for interfacing with the system.
  • User input 212 contains the circuitry for interfacing with input devices such as touchscreen input device 104.
  • Communications block 214 represents communications circuitry and circuitry for interfacing the communications circuitry with the system. Communications block 214 may include a radio for communicating over a cellular network such as a network designed according to the Long-Term Evolution (LTE) , LTE-Advanced, 5G or Global System for Mobile Communications (GSM) families of standards.
  • LTE Long-Term Evolution
  • GSM Global System for Mobile Communications
  • communications circuitry 214 may include a Wi-Fi communications radio according to the IEEE 801.11 family of standards, or a Bluetooth radio circuit according to the IEEE 802.15 family of standards.
  • Real-time clock 216 includes circuitry that provides a clock that maintains the current date and time, and that interfaces the clock to the system.
  • Data capture devices 220 are integrated with computing device 200. According to various embodiments, data capture devices 220 include a plurality of different types of sensing transducers and their associated processing and interface circuitry, such as a accelerometer 110, as well as a camera, GPS, and biometric sensors according to various embodiments.
  • the processing circuitry associated with each corresponding transducer may include amplification, buffering, filtering, or other signal-conditioning circuitry to receive the raw analog signal from the corresponding transducer and prepare the analog signaling for digitization, analog-to-digital conversion circuitry to perform sampling, quantization, and digital encoding, and, in some cases, further processing to produce a digital signal representing the physical phenomenon being measured by the transducer in a form that is readable by CPU 202.
  • FIG. 3 is a diagram illustrating an exemplary hardware and software architecture of a computing device such as the one depicted in FIG. 2, in which various interfaces between hardware components and software components are shown. As indicated by HW, hardware components are represented below the divider line, whereas software components denoted by SW reside above the divider line.
  • processing devices 302 which may include one or more microprocessors, digital signal processors, etc., each having one or more processor cores, are interfaced with memory management device 304 and system interconnect 306.
  • Memory management device 304 provides mappings between virtual memory used by processes being executed, and the physical memory. Memory management device 304 may be an integral part of a central processing unit which also includes the processing devices 302.
  • Interconnect 306 includes a backplane such as memory, data, and control lines, as well as the interface with input/output devices, e.g., proprietary bus, PCI, USB, etc.
  • Memory 308 e.g., dynamic random access memory-DRAM
  • non-volatile memory 309 such as flash memory (i.e., electrically-erasable read-only memory–EEPROM, NAND Flash, NOR Flash, etc. ) are interfaced with memory management device 304 and interconnect 306 via memory controller 310.
  • This architecture may support direct memory access (DMA) by peripherals in some embodiments.
  • DMA direct memory access
  • I/O devices including video and audio adapters, non-volatile storage, external peripheral links such as USB, Bluetooth, etc., as well as network interface devices such as those communicating via Wi-Fi or LTE-family interfaces, are collectively represented as I/O devices and networking 312, which interface with interconnect 306 via corresponding I/O controllers 314.
  • pre-OS pre-operating system
  • BIOS system basic input/output system
  • UEFI unified extensible firmware interface
  • OS 318 provides a kernel that controls the hardware devices, manages memory access for programs in memory, coordinates tasks and facilitates multi-tasking, organizes data to be stored, assigns memory space and other resources, loads program binary code into memory, initiates execution of the application program which then interacts with the user and with hardware devices, and detects and responds to various defined interrupts.
  • operating system 318 provides device drivers, and a variety of common services such as those that facilitate interfacing with peripherals and networking, that provide abstraction for application programs so that the applications do not need to be responsible for handling the details of such common operations.
  • Operating system 318 additionally provides a graphical user interface (GUI) that facilitates interaction with the display, sound, and touch and motion input devices.
  • GUI graphical user interface
  • Runtime system 320 implements portions of an execution model, including such operations as putting parameters onto the stack before a function call, the behavior of long-term storage input/output (I/O) , and parallel execution-related behaviors. Runtime system 320 may also perform support services such as type checking, debugging, or code generation and optimization.
  • Libraries 322 include collections of program functions that provide further abstraction for application programs. These include shared libraries, dynamic linked libraries (DLLs) , for example. Libraries 322 may be integral to the operating system 318, runtime system 320, or may be added-on features, or even remotely-hosted. Libraries 322 define an application program interface (API) through which a variety of function calls may be made by application programs 324 to invoke the services provided by the operating system 318. Application programs 324, oftentimes referred to as apps, are those programs that perform useful tasks for users, beyond the tasks performed by lower-level system programs that coordinate the basis operability of the computing device itself.
  • API application program interface
  • FIG. 4 is a block diagram illustrating a set of engines that may be implemented using the computing hardware of a computing device that may be configured, or configurable according to executing, or installed, software or firmware program instructions according to some embodiments.
  • Engines may be hardware, software, or firmware communicatively coupled to one or more processors in order to carry out the operations described herein.
  • Engines may be hardware engines, and as such engines may be considered tangible entities capable of performing specified operations and may be configured or arranged in a certain manner.
  • circuits may be arranged (e.g., internally or with respect to external entities such as other circuits) in a specified manner as an engine.
  • the whole or part of one or more computer systems e.g., a standalone, client or server computer system
  • one or more hardware processors may be configured by firmware or software (e.g., instructions, an application portion, or an application) as an engine that operates to perform specified operations.
  • the software may reside on a machine-readable medium.
  • the software when executed by the underlying hardware of the engine, causes the hardware to perform the specified operations.
  • the term hardware engine is understood to encompass a tangible entity, be that an entity that is physically constructed, specifically configured (e.g., hardwired) , or temporarily (e.g., transitorily) configured (e.g., programmed) to operate in a specified manner or to perform part or all of any operation described herein.
  • an engine may include one, or any combination, of the blocks depicted, so long as at least one block from the HW side is included.
  • each of the engines need not be instantiated at any one moment in time.
  • the engines comprise a general-purpose hardware processor configured using software; the general-purpose hardware processor may be configured as respective different engines at different times.
  • Software may accordingly configure a hardware processor, for example, to constitute a particular engine at one instance of time and to constitute a different engine at a different instance of time.
  • gesture detection engine 402 is configured with hardware and software components, and operative, to monitor an accelerometer, deformation/strain sensor, display touchscreen, or other suitable touch detection input by the user, and to apply a set of defined criteria to discriminate between user gesture input, or other events picked up by the sensor (s) .
  • one, or a combination, of different sensors may be monitored simultaneously (or nearly-simultaneously within the limits of the computing device) and integrated to produce a diverse multi-modal input.
  • Profile assignment engine 404 is configured with hardware and software components, and operative, to analyze the detected gesture (s) by gesture detection engine 402 and assign one or more gesture characteristic profiles according to defined profile criteria.
  • Sequential event logging engine 406 is configured with hardware and software components, and operative, to collect, tabulate, and count one or more series of sequential gestures according to each gesture’s profile characteristic, and according to each gesture’s temporal position in a sequence of gestures.
  • the result of this tabulation and counting may be a data structure populated with counts of various gestures (or non-gestures) data that may be indicative of user behavior, in terms of the collection of input gestures.
  • Gesture sequence analysis engine 408 is configured with hardware and software components, and operative, to analyze the data collected by sequential event logging engine to assess any failed detections of gestures, any false detection of gestures, and to detect circumstances for invoking re-calibration of gesture detection. The gesture detection engine 402 may then utilize the re-calibration result to improve its gesture-recognition functionality.
  • FIG. 5 is a time series diagram illustrating example outputs of a multi-axis motion sensor that observes a double-tapping gesture according to some embodiments. As depicted in the timing diagram, each time-series trace represents a corresponding axis of measurement.
  • Mag1 502 representing a magnitude of a first peak
  • Range1_max 504 representing the magnitude of the greatest portion of the residual signal following the first peak, also referred to herein as the tail
  • Mag2 506 representing a second peak representing the second tap (occurring close in time to the first tap)
  • Range2_max 508 representing the maximum magnitude of the second tap signal’s tail.
  • mag1_threshold and Mag2_threshold are the minimum acceptance thresholds for the first and second taps, respectively. For example, if Mag1 502 exceeds Mag1_threshold, a first tapping is possibly recognized.
  • Another threshold, P is the qualifying cut-off percentage of the signal magnitude, e.g., Mag1 502, in proportion to the maximum tail magnitude, e.g., Range1_max 504.
  • the signal magnitudes Mag1 502 and Mag2 506 are indicative of the force of each respective tap, for example, how heavily the user taps on the device.
  • the values of Mag1 502, Mag2 506, Mag1_threshold and Mag2_threshold may be varied for each user to improve tapping detection.
  • the tail magnitudes Range1_max 504 and Range2_max 508, and the qualifying cut-off percentage P are indicative of user characteristics, such as the flexibility and other properties of the user’s skin, as well as representing how the device may be held, worn, or fastened, with respect to the user.
  • FIG. 6 is a flow diagram illustrating an example process of detecting a double-tapping gesture according to some embodiments.
  • the first tap minimum threshold, Mag1_threshold is applied, meaning that Mag1 502 is compared against Mag1_threshold.
  • the measured peak of the first tap should meet or exceed Mag1_threshold to qualify as a tap.
  • the first tail maximum threshold, P is applied, where Range1_max 504 is compared against threshold P, which is determined specifically for this application based on the Mag1 502 value.
  • the threshold P may be defined as a percentage of Mag1_threshold, for example.
  • a maximum tail magnitude range1_max that falls below threshold P may qualify the tap as valid.
  • the second tap minimum threshold, Mag2_threshold is applied to the second potential tap signal peak.
  • the tail threshold P is applied to the second peak’s tail.
  • the threshold P may be defined as a percentage of Mag2_threshold.
  • FIG. 7 is a table listing various example parameter values for tap gesture detection according to an embodiment. Each row represents variations in the parameter values according to assigned profiles of strong, medium, and weak. Profile assignment engine 404 may make the profiling determinations according to an example embodiment.
  • FIG. 8 is a state diagram illustrating various potential sequences of tapping gestures according to some embodiments.
  • the transitions of tapping events are illustrated as arrows.
  • the sequences are as illustrated as first tap, second tap, and third (or subsequent tap) for weak, medium, and strong profile characteristics.
  • N and M represent the sequential advance from the Nth to the Mth sequential tap.
  • each connecting arrow between two events is associated with a counter registering the number of transitions between the two specific tapping event states.
  • the statistics of all tapping events may be compiled from these counters by sequential event logging engine 406.
  • FIG. 9 is a table illustrating the various counters corresponding to each sequential tapping event depicted in FIG. 8 according to some embodiments.
  • all the counters are initialized to zero.
  • the tapping event will be fed to each of the gesture recognition algorithms, e.g., strong, medium, and weak profiles for profile assignment engine 404. If the input is qualified as a tapping event, it will be registered independently for that profile and the relevant counter will increment by one by sequential event logging engine 406. For example, if a tapping event is qualified according to both weak and medium profiles but not the strong profile, the following edge counters will increment by one: Mtap01++; Wtap01++; if three or more tapping events are detected consecutively, the counter in Tap3->3 column will increment recurrently.
  • FIG. 10 is a graph diagram illustrating sequential relationships between gesture events, including the temporal ordering of those events according to some embodiments.
  • FIG. 11 is a table associating the edges of the diagram of FIG. 10 with the event sequences shown in the diagram of FIG. 8 according to an example embodiment.
  • a priori user feedback information is used to recognize the false negative and false positive cases.
  • Table 1 below illustrates example false negative and false positive scenarios.
  • failed detections may be assessed based on the tabulated and counted series of gestures by detection of successful gesture attempts in close succession following events that fail to meet gesture detection criteria.
  • any false detections of gestures may be assessed based on the tabulated and counted series of gestures by detection of undo actions (e.g., use of the “back” touch command, swipe-away gesture, etc. ) in close succession to detected gestures.
  • undo actions e.g., use of the “back” touch command, swipe-away gesture, etc.
  • gesture sequence analysis engine 408 applies these criteria to determine a need for recalibration. If a tabulated quantity of false positives or false negatives are observed within some defined time period, gesture sequence analysis engine 408 may initiate a call for recalibration. For example, if there are four occurrences of erroneous tapping detection on the same day of usage, an attempt for recalibration will be initiated. A likely outcome of the recalibration is a new tapping profile will be chosen. No adjustment will be performed if a better tapping profile cannot be found.
  • gesture sequence analysis engine 408 may adjust the tapping profile.
  • the problem of selecting tapping profile is solved as a single-source shortest-path problem according to an example algorithm that may be carried out by gesture sequence analysis engine 408.
  • FIG. 12 is a diagram illustrating a reduced graph representing tapping events of interest for recalibrating double-tapping gesture detection according to an example embodiment.
  • the cost associated with each path may be expressed as:
  • the approach is reduced to finding the single-source shortest-path with the weighted path length.
  • a greedy method such as Dijkstra’s Algorithm may be applied to solve the best path (with the lowest cost) , for example. If the path is different from the previously used one, the new tapping profile corresponding to the shortest path will be used in the double tapping algorithm.
  • the edge counter values may be used directly as the weighted path length in the single-source shortest-path problem solving.
  • FIG. 13 is a flow diagram illustrating a general process performing gesture recognition assessment according to some embodiments.
  • touch-based input by the user is monitored, e.g., based on reading an output from the touch sensor.
  • gestures are detected from among the touch-based input.
  • gesture characteristic profiles are assigned to the detected gestures according to profiling criteria.
  • a series of gestures are tabulated and counted based on assigned characteristic profiles and on temporal sequencing of the gestures.
  • circumstances for invocation of gesture detection re-calibration are assessed based on the tabulated series of gestures.
  • a re-calibration process is initiated in response to the assessment.
  • Example 1 is a system for gesture recognition in a computing device, the system comprising: computing hardware, including a processor, a data store, and an input device including a touch sensor, the computing hardware containing instructions, that when executed on the processor, cause the computing hardware to implement: a gesture detection engine to monitor touch-based input by the user based on an output from the touch sensor, and to detect gestures from among the touch-based input; a profile assignment engine to assign gesture characteristic profiles to the detected gestures according to profiling criteria; a sequential event logging engine to tabulate and count series of gestures based on assigned characteristic profiles and on temporal sequencing of the gestures; and an analysis engine to assess circumstances for invocation of gesture detection re-calibration based on the tabulated series of gestures
  • Example 2 the subject matter of Example 1 optionally includes, wherein the gestures include a double-tapping gesture.
  • Example 3 the subject matter of any one or more of Examples 1–2 optionally include, wherein the gestures include at least one gesture selected from the group consisting of: a double-tap, device rotation, a swipe, and a device shake.
  • Example 4 the subject matter of any one or more of Examples 1–3 optionally include, wherein the touch sensor includes a multi-axial accelerometer.
  • Example 5 the subject matter of any one or more of Examples 1–4 optionally include, wherein the touch sensor includes at least one sensor selected from the group consisting of: a device orientation sensor, an acoustic sensor, a strain sensor, and a material deformation sensor.
  • the touch sensor includes at least one sensor selected from the group consisting of: a device orientation sensor, an acoustic sensor, a strain sensor, and a material deformation sensor.
  • Example 6 the subject matter of any one or more of Examples 1–5 optionally include, wherein the computing hardware includes a plurality of diverse touch sensors, and wherein the gesture detection engine is to monitor and integrate output from the plurality of diverse touch sensors.
  • Example 7 the subject matter of any one or more of Examples 1–6 optionally include, wherein the gesture detection engine is to apply a set of criteria to discriminate between a user gesture input, and other events sensed by the touch sensor.
  • Example 8 the subject matter of any one or more of Examples 1–7 optionally include, wherein the gesture characteristic profiles include profiles representing a plurality of force-of-touch measures.
  • Example 9 the subject matter of any one or more of Examples 1–8 optionally include, wherein the sequential event logging engine is to produce a data structure populated with counts of various gestures and non-gesture motion-based events.
  • Example 10 the subject matter of any one or more of Examples 1–9 optionally include, wherein the analysis engine is to assess any failed detections of gestures based on the tabulated and counted series of gestures by detection of successful gesture attempts in close succession following events that fail to meet gesture detection criteria.
  • Example 11 the subject matter of any one or more of Examples 1–10 optionally include, wherein the analysis engine is to assess any false detections of gestures based on the tabulated and counted series of gestures by detection of undo actions in close succession to detected gestures.
  • Example 12 the subject matter of any one or more of Examples 1–11 optionally include, wherein the analysis engine is to assess any repeated gesture attempts following a non-detection of a gesture based on the tabulated and counted series of gestures.
  • Example 13 the subject matter of any one or more of Examples 1–12 optionally include, wherein the analysis engine is to initiate a call for gesture detection recalibration in response to a tabulated quantity of false detection/failed detection events having occurred during a monitoring period.
  • Example 14 the subject matter of any one or more of Examples 1–13 optionally include, wherein the computing device is a wearable device.
  • Example 15 the subject matter of any one or more of Examples 1–14 optionally include, wherein the computing device is a mobile computing device.
  • Example 16 is at least one computer-readable medium comprising instructions that, when executed on a computing device that includes a processor, a data store, and an input device including a touch sensor, cause the computing device to implement: a gesture detection engine to monitor touch-based input by the user based on an output from the touch sensor, and to detect gestures from among the touch-based input; a profile assignment engine to assign gesture characteristic profiles to the detected gestures according to profiling criteria; a sequential event logging engine to tabulate and count series of gestures based on assigned characteristic profiles and on temporal sequencing of the gestures; and an analysis engine to assess circumstances for invocation of gesture detection re-calibration based on the tabulated series of gestures
  • Example 17 the subject matter of Example 16 optionally includes, wherein the gestures include a double-tapping gesture.
  • Example 18 the subject matter of any one or more of Examples 16–17 optionally include, wherein the gestures include at least one gesture selected from the group consisting of: a device rotation, a swipe, and a device shake.
  • Example 19 the subject matter of any one or more of Examples 16–18 optionally include, wherein the gesture detection engine is to apply a set of criteria to discriminate between a user gesture input, and other events sensed by the touch sensor.
  • Example 20 the subject matter of any one or more of Examples 16–19 optionally include, wherein the gesture characteristic profiles include profiles representing a plurality of force-of-touch measures.
  • Example 21 the subject matter of any one or more of Examples 16–20 optionally include, wherein the sequential event logging engine is to produce a data structure populated with counts of various gestures and non-gesture motion-based events.
  • Example 22 the subject matter of any one or more of Examples 16–21 optionally include, wherein the analysis engine is to assess any failed detections of gestures based on the tabulated and counted series of gestures by detection of successful gesture attempts in close succession following events that fail to meet gesture detection criteria.
  • Example 23 the subject matter of any one or more of Examples 16–22 optionally include, wherein the analysis engine is to assess any false detections of gestures based on the tabulated and counted series of gestures by detection of undo actions in close succession to detected gestures.
  • Example 24 the subject matter of any one or more of Examples 16–23 optionally include, wherein the analysis engine is to assess any repeated gesture attempts following a non-detection of a gesture based on the tabulated and counted series of gestures.
  • Example 25 the subject matter of any one or more of Examples 16–24 optionally include, wherein the analysis engine is to initiate a call for gesture detection recalibration in response to a tabulated quantity of false detection/failed detection events having occurred during a monitoring period.
  • Example 26 is a method for autonomously assessing gesture recognition in a computing device that includes a processor, a data store, and an input device including a touch sensor, the method comprising: monitoring, by the computing device, touch-based input by the user based on an output from the touch sensor; detecting, by the computing device, gestures from among the touch-based input; assigning, by the computing device, gesture characteristic profiles to the detected gestures according to profiling criteria; tabulating and counting, by the computing device, series of gestures based on assigned characteristic profiles and on temporal sequencing of the gestures; and assessing, by the computing device, circumstances for invocation of gesture detection re-calibration based on the tabulated series of gestures
  • Example 27 the subject matter of Example 26 optionally includes, wherein the gestures include a double-tapping gesture.
  • Example 28 the subject matter of any one or more of Examples 26–27 optionally include, wherein the gestures include at least one gesture selected from the group consisting of: a device rotation, a swipe, and a device shake.
  • Example 29 the subject matter of any one or more of Examples 26–28 optionally include, further comprising: applying a set of criteria to discriminate between a user gesture input, and other events sensed by the touch sensor.
  • Example 30 the subject matter of any one or more of Examples 26–29 optionally include, wherein the gesture characteristic profiles include profiles representing a plurality of force-of-touch measures.
  • Example 31 the subject matter of any one or more of Examples 26–30 optionally include, wherein tabulating and counting the series of gestures includes producing a data structure populated with counts of various gestures and non-gesture motion-based events.
  • Example 32 the subject matter of any one or more of Examples 26–31 optionally include, wherein assessing the circumstances for invocation of gesture detection re-calibration includes assessing any failed detections of gestures based on the tabulated and counted series of gestures by detecting successful gesture attempts in close succession following events that fail to meet gesture detection criteria.
  • Example 33 the subject matter of any one or more of Examples 26–32 optionally include, wherein assessing the circumstances for invocation of gesture detection re-calibration includes assessing any false detections of gestures based on the tabulated and counted series of gestures by detecting undo actions in close succession to detected gestures.
  • Example 34 the subject matter of any one or more of Examples 26–33 optionally include, wherein assessing the circumstances for invocation of gesture detection re-calibration includes assessing any repeated gesture attempts following a non-detection of a gesture based on the tabulated and counted series of gestures.
  • Example 35 the subject matter of any one or more of Examples 26–34 optionally include, wherein assessing the circumstances for invocation of gesture detection re-calibration produces a call for gesture detection recalibration in response to a tabulated quantity of false detection/failed detection events having occurred during a monitoring period.
  • Example 36 is a system for autonomously assessing gesture recognition in a computing device that includes processing means and touch sensing means, the system comprising: means for monitoring touch-based input by the user based on an output from the touch sensing means; means for detecting gestures from among the touch-based input; means for assigning gesture characteristic profiles to the detected gestures according to profiling criteria; means for tabulating and counting series of gestures based on assigned characteristic profiles and on temporal sequencing of the gestures; and means for assessing circumstances for invocation of gesture detection re-calibration based on the tabulated series of gestures
  • Example 37 the subject matter of Example 36 optionally includes, wherein the gestures include a double-tapping gesture.
  • Example 38 the subject matter of any one or more of Examples 36–37 optionally include, wherein the gestures include at least one gesture selected from the group consisting of: a device rotation, a swipe, and a device shake.
  • Example 39 the subject matter of any one or more of Examples 36–38 optionally include, further comprising: means for applying a set of criteria to discriminate between a user gesture input, and other events sensed by the touch sensing means.
  • Example 40 the subject matter of any one or more of Examples 36–39 optionally include, wherein the gesture characteristic profiles include profiles representing a plurality of force-of-touch measures.
  • Example 41 the subject matter of any one or more of Examples 36–40 optionally include, wherein the means for tabulating and counting the series of gestures includes means for producing a data structure populated with counts of various gestures and non-gesture motion-based events.
  • Example 42 the subject matter of any one or more of Examples 36–41 optionally include, wherein the means for assessing the circumstances for invocation of gesture detection re-calibration includes means for assessing any failed detections of gestures based on the tabulated and counted series of gestures by detecting successful gesture attempts in close succession following events that fail to meet gesture detection criteria.
  • Example 43 the subject matter of any one or more of Examples 36–42 optionally include, wherein the means for assessing the circumstances for invocation of gesture detection re-calibration includes means for assessing any false detections of gestures based on the tabulated and counted series of gestures by detecting undo actions in close succession to detected gestures.
  • Example 44 the subject matter of any one or more of Examples 36–43 optionally include, wherein the means for assessing the circumstances for invocation of gesture detection re-calibration includes assessing any repeated gesture attempts following a non-detection of a gesture based on the tabulated and counted series of gestures.
  • Example 45 the subject matter of any one or more of Examples 36–44 optionally include, wherein the means for assessing the circumstances for invocation of gesture detection re-calibration produces a call for gesture detection recalibration in response to a tabulated quantity of false detection/failed detection events having occurred during a monitoring period.
  • Example 46 the subject matter of any one or more of Examples 36–45 optionally include, wherein the touch sensing means includes a multi-axial accelerometer.
  • Example 47 the subject matter of any one or more of Examples 36–46 optionally include, wherein the touch sensing means includes at least one sensor selected from the group consisting of: a device orientation sensor, an acoustic sensor, a strain sensor, and a material deformation sensor.
  • the touch sensing means includes at least one sensor selected from the group consisting of: a device orientation sensor, an acoustic sensor, a strain sensor, and a material deformation sensor.
  • Example 48 the subject matter of any one or more of Examples 36–47 optionally include, wherein the computing hardware includes a plurality of diverse touch sensing meanss, and wherein the gesture detection engine is to monitor and integrate output from the plurality of diverse touch sensing meanss.
  • Example 49 the subject matter of any one or more of Examples 36–48 optionally include, wherein the computing device is a wearable device.
  • Example 50 the subject matter of any one or more of Examples 36–49 optionally include, wherein the computing device is a mobile computing device.
  • At least one computer-readable medium comprises instructions that, when executed on a computing device that includes a processor, a data store, and an input device including a touch sensor, cause the computing device to implement the subject matter of any one or more of Examples 26–35.
  • Example 52 a system for autonomously assessing gesture recognition in a computing device that includes processing means and touch sensing means comprises means for implementing the subject matter of any one or more of Examples 26–35.
  • the terms “a” or “an” are used, as is common in patent documents, to include one or more than one, independent of any other instances or usages of “at least one” or “one or more. ”
  • the term “or” is used to refer to a nonexclusive or, such that “Aor B” includes “Abut not B, ” “B but not A, ” and “A and B, ” unless otherwise indicated.
  • the terms “including” and “in which” are used as the plain-English equivalents of the respective terms “comprising” and “wherein.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)
PCT/CN2015/098381 2015-12-23 2015-12-23 Touch gesture detection assessment WO2017107086A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
PCT/CN2015/098381 WO2017107086A1 (en) 2015-12-23 2015-12-23 Touch gesture detection assessment
DE112015007219.5T DE112015007219T5 (de) 2015-12-23 2015-12-23 Berührungsgestenerkennungs-Bewertung
US15/774,667 US10488975B2 (en) 2015-12-23 2015-12-23 Touch gesture detection assessment
CN201580085074.2A CN108292165B (zh) 2015-12-23 2015-12-23 触摸姿势检测评估

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2015/098381 WO2017107086A1 (en) 2015-12-23 2015-12-23 Touch gesture detection assessment

Publications (1)

Publication Number Publication Date
WO2017107086A1 true WO2017107086A1 (en) 2017-06-29

Family

ID=59088813

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/098381 WO2017107086A1 (en) 2015-12-23 2015-12-23 Touch gesture detection assessment

Country Status (4)

Country Link
US (1) US10488975B2 (zh)
CN (1) CN108292165B (zh)
DE (1) DE112015007219T5 (zh)
WO (1) WO2017107086A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10488975B2 (en) 2015-12-23 2019-11-26 Intel Corporation Touch gesture detection assessment

Families Citing this family (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10430894B2 (en) 2013-03-21 2019-10-01 Khoros, Llc Gamification for online social communities
US10902462B2 (en) 2017-04-28 2021-01-26 Khoros, Llc System and method of providing a platform for managing data content campaign on social networks
US11050704B2 (en) 2017-10-12 2021-06-29 Spredfast, Inc. Computerized tools to enhance speed and propagation of content in electronic messages among a system of networked computing devices
US11470161B2 (en) 2018-10-11 2022-10-11 Spredfast, Inc. Native activity tracking using credential and authentication management in scalable data networks
US10999278B2 (en) 2018-10-11 2021-05-04 Spredfast, Inc. Proxied multi-factor authentication using credential and authentication management in scalable data networks
US10346449B2 (en) 2017-10-12 2019-07-09 Spredfast, Inc. Predicting performance of content and electronic messages among a system of networked computing devices
US11570128B2 (en) 2017-10-12 2023-01-31 Spredfast, Inc. Optimizing effectiveness of content in electronic messages among a system of networked computing device
US10785222B2 (en) 2018-10-11 2020-09-22 Spredfast, Inc. Credential and authentication management in scalable data networks
US10601937B2 (en) * 2017-11-22 2020-03-24 Spredfast, Inc. Responsive action prediction based on electronic messages among a system of networked computing devices
US10594773B2 (en) 2018-01-22 2020-03-17 Spredfast, Inc. Temporal optimization of data operations using distributed search and server management
US11061900B2 (en) 2018-01-22 2021-07-13 Spredfast, Inc. Temporal optimization of data operations using distributed search and server management
US10855657B2 (en) 2018-10-11 2020-12-01 Spredfast, Inc. Multiplexed data exchange portal interface in scalable data networks
US10931540B2 (en) 2019-05-15 2021-02-23 Khoros, Llc Continuous data sensing of functional states of networked computing devices to determine efficiency metrics for servicing electronic messages asynchronously
CN111338516B (zh) * 2020-02-26 2022-05-10 业成科技（成都）有限公司 手指触控的检测方法和装置、电子设备、存储介质
US11128589B1 (en) 2020-09-18 2021-09-21 Khoros, Llc Gesture-based community moderation
US11438289B2 (en) 2020-09-18 2022-09-06 Khoros, Llc Gesture-based community moderation
US11924375B2 (en) 2021-10-27 2024-03-05 Khoros, Llc Automated response engine and flow configured to exchange responsive communication data via an omnichannel electronic communication channel independent of data source
US11438282B2 (en) 2020-11-06 2022-09-06 Khoros, Llc Synchronicity of electronic messages via a transferred secure messaging channel among a system of various networked computing devices
US11627100B1 (en) 2021-10-27 2023-04-11 Khoros, Llc Automated response engine implementing a universal data space based on communication interactions via an omnichannel electronic data channel
US11714629B2 (en) 2020-11-19 2023-08-01 Khoros, Llc Software dependency management

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100306710A1 (en) * 2009-05-29 2010-12-02 Microsoft Corporation Living cursor control mechanics
CN102317977A (zh) * 2009-02-17 2012-01-11 奥美可互动有限责任公司 用于姿势识别的方法和系统
CN102985897A (zh) * 2010-07-13 2013-03-20 英特尔公司 高效姿势处理
US20140009378A1 (en) * 2012-07-03 2014-01-09 Yen Hsiang Chew User Profile Based Gesture Recognition
CN104380248A (zh) * 2012-06-29 2015-02-25 英特尔公司 用于基于手势的管理的系统和方法

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6029214A (en) * 1995-11-03 2000-02-22 Apple Computer, Inc. Input tablet system with user programmable absolute coordinate mode and relative coordinate mode segments
US8460103B2 (en) * 2004-06-18 2013-06-11 Igt Gesture controlled casino gaming system
US8745541B2 (en) * 2003-03-25 2014-06-03 Microsoft Corporation Architecture for controlling a computer using hand gestures
US8684839B2 (en) * 2004-06-18 2014-04-01 Igt Control of wager-based game using gesture recognition
US7847787B1 (en) * 2005-11-12 2010-12-07 Navisense Method and system for directing a control action
US9400548B2 (en) * 2009-10-19 2016-07-26 Microsoft Technology Licensing, Llc Gesture personalization and profile roaming
US8593418B2 (en) * 2010-08-08 2013-11-26 Qualcomm Incorporated Method and system for adjusting display content
US9710154B2 (en) * 2010-09-03 2017-07-18 Microsoft Technology Licensing, Llc Dynamic gesture parameters
US20130135218A1 (en) * 2011-11-30 2013-05-30 Arbitron Inc. Tactile and gestational identification and linking to media consumption
US20130194173A1 (en) * 2012-02-01 2013-08-01 Ingeonix Corporation Touch free control of electronic systems and associated methods
US9170676B2 (en) * 2013-03-15 2015-10-27 Qualcomm Incorporated Enhancing touch inputs with gestures
JP6264665B2 (ja) * 2013-04-17 2018-01-24 パナソニックＩｐマネジメント株式会社 画像処理方法および画像処理装置
CN103637807B (zh) * 2013-12-30 2015-04-22 四川大学 一种人体三维姿态和行为状态传感监测方法
US10488975B2 (en) 2015-12-23 2019-11-26 Intel Corporation Touch gesture detection assessment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102317977A (zh) * 2009-02-17 2012-01-11 奥美可互动有限责任公司 用于姿势识别的方法和系统
US20100306710A1 (en) * 2009-05-29 2010-12-02 Microsoft Corporation Living cursor control mechanics
CN102985897A (zh) * 2010-07-13 2013-03-20 英特尔公司 高效姿势处理
CN104380248A (zh) * 2012-06-29 2015-02-25 英特尔公司 用于基于手势的管理的系统和方法
US20140009378A1 (en) * 2012-07-03 2014-01-09 Yen Hsiang Chew User Profile Based Gesture Recognition

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10488975B2 (en) 2015-12-23 2019-11-26 Intel Corporation Touch gesture detection assessment

Also Published As

Publication number Publication date
US10488975B2 (en) 2019-11-26
DE112015007219T5 (de) 2021-09-09
US20180329565A1 (en) 2018-11-15
CN108292165A (zh) 2018-07-17
CN108292165B (zh) 2021-06-22

Similar Documents

Publication Publication Date Title
US10488975B2 (en) Touch gesture detection assessment
US20170344120A1 (en) User-input interaction for movable-panel mobile device
US10776463B2 (en) Active authentication of users
US9454454B2 (en) Memory leak analysis by usage trends correlation
EP2940555B1 (en) Automatic gaze calibration
US20140306903A1 (en) Methods of evaluating touch procesing
EP3133473A1 (en) Method of sensing pressure by touch sensor and electronic device adapted thereto
US10914773B2 (en) Resolution adjustment for capacitive touch sensor
JP2014502763A5 (zh)
US9280284B2 (en) Method, apparatus and computer readable medium for polygon gesture detection and interaction
CN108475136A (zh) 一种指纹识别方法以及电子设备
CN110737355A (zh) 双击事件检测设备、系统和方法
WO2017028491A1 (zh) 触控显示设备及触控显示方法
US20240192806A1 (en) Diffusion-based handedness classification for touch-based input
US20150293598A1 (en) Method for processing information and electronic device
CN106445698B (zh) 计步数据的获取方法及装置
WO2017112122A1 (en) Distinctive value based on true random input
CN105260044B (zh) 电子设备及触控操作识别方法
US9781475B2 (en) Information processing method, system and electronic device
WO2013192025A1 (en) Touch intensity based on accelerometer readings
WO2018053936A1 (zh) 一种交互方法及电子设备
US9547775B2 (en) Device and method for data privacy management
US9274703B2 (en) Method for inputting instruction and portable electronic device and computer readable recording medium
US10401968B2 (en) Determining digit movement from frequency data
TWI507920B (zh) 電子裝置及避免多個輸入裝置互相干擾的方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15911085

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 15774667

Country of ref document: US

WWE Wipo information: entry into national phase

Ref document number: 112015007219

Country of ref document: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15911085

Country of ref document: EP

Kind code of ref document: A1