Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
  • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
  • G06F3/0416—Control or interface arrangements specially adapted for digitisers
  • G06F3/0418—Control or interface arrangements specially adapted for digitisers for error correction or compensation, e.g. based on parallax, calibration or alignment
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
  • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
  • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
  • G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N7/00—Computing arrangements based on specific mathematical models
  • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
  • G06F2203/041—Indexing scheme relating to G06F3/041 - G06F3/045
  • G06F2203/04108—Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
  • G06F2203/048—Indexing scheme relating to G06F3/048
  • G06F2203/04808—Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen

Definitions

• the present disclosure relates to signal processing for tracking of moving objects such as signal processing for systems for detecting gestures performed by an object such as a hand or finger of a user.
• Touch detection systems usually use a capacitive system with horizontal and vertical electrodes arranged in a matrix. Mutual and self-capacitance measurement can be used to determine one or more touch points.
• a controller can be configured to track movements and evaluate gestures.
• An exemplary touch controller is the MTCH6303 manufactured by the assignee of the present application. The application note "MTCH6303 Projected Capacitive Touch Controller Data Sheet", DS40001803, available from Microchip Technology Inc. is hereby incorporated by reference in its entirety.
• mapping measured touch positions to contact identifiers is a combinatorial assignment problem that is solved by calculating distances between measured touch positions and predicted positions of known contacts and determining optimum assignments based on those distances. While in the context of touch sensor systems, the terms 'contact' and 'contact identifiers' may be used interchangeably. Furthermore, the present disclosure may also use the terms 'object' and 'object identifier' as synonyms to the use of 'contact' and 'contact identifiers'.
• the cost function for the mapping of a newly measured position to a contact is derived from the state of a stochastic model for the contact's movement, in contrast to a trivial distance measurement.
• the cost function can account for speed and direction of the movement that is estimated by the self-adjusting model, as well as the likelihood of acceleration parallel and perpendicular to the direction of movement.
• the task is encountered to solve the combinatorial assignment problem of measured positions to objects.
• a cost matrix is made up of the individual 'costs' for the assignment of each measured position to each object.
• the optimum assignment then can be determined by means of, e.g., the Hungarian (Kuhn-Munkres) algorithm [Harold W. Kuhn, "The Hungarian Method for the assignment problem", Naval Research Logistics Quarterly, 2: 83-97, 1955].
• the optimum assignment typically may include the assignment with minimum total cost.
• the cost for each assignment may be a simple metric like the Manhattan distance or the (squared) Euclidean distance between the newly measured touch positions and the recent (or predicted) positions of the contacts. In our new approach, this metric is derived from a movement model.
• Embodiments of the present disclosure include a position tracking system.
• the system may include a processor and a memory coupled to the processor.
• the memory may include instructions that, when loaded and executed by the processor, cause the processor to receive a plurality of measurements from a sensor and map an estimated location of an object to a measurement using a stochastic model.
• the mapping may be based upon a cost relating to the location and the measurement.
• the stochastic model may be based on the location.
• embodiments of the present disclosure include a non-transitory computer- readable medium including instructions.
• the instructions when loaded and executed by a processor, cause the processor to receive a plurality of measurements from a sensor and map an estimated location of an object to a measurement using a stochastic model.
• the mapping may be based upon a cost relating to the location and the measurement.
• the stochastic model may be based on the location.
• embodiments of the present disclosure include a method.
• the method may include receiving a plurality of measurements from a sensor and mapping an estimated location of an object to a measurement using a stochastic model.
• the mapping may be based upon a cost relating to the location and the measurement.
• the stochastic model may be based on the location.
• FIGURE 1 illustrates an example embodiment of a system for tracking the movement of objects
• FIGURE 2 illustrates measurements made for tracking the movement of objects
• FIGURE 3 illustrates an example analysis of the movement of objects using Euclidean distances
• FIGURE 4 illustrates an example embodiment of a cost function obtained from a movement model
• FIGURE 5 illustrates an example analysis of the movement of objects using a stochastic model
• FIGURE 6 illustrates an example embodiment of a method for tracking the movement of objects.
• FIGURE 1 is an illustration of an example embodiment of a system 100 for tracking movement of objects.
• System 100 may track objects such as fingers, hands, other body parts, or entire objects. The objects may be tracked as they interface with or operate near an electronic device, such as electronic device 104. While objects to be tracked are not shown in FIGURE 1, these objects to be tracked may touch, nearly touch, or otherwise be within sensing range of a sensor or a set of sensors.
• system 100 may track movement of objects by predicting where an object will be located in the future. Given a measurement from a sensor, system 100 may determine whether the measurement is noise, from an object, or from which object of more than one objects that might be tracked by electronic device 104.
• system 100 may make such a determination based upon a probabilistic model that relates the new measurement to a prior measurement or calculated trace of an object's movement. In another embodiment, system 100 may make such a determination by assigning a measurement a higher probability that it is related to prior linear movement, acceleration, or deceleration of an object rather than perpendicular movement of the object.
• sensor 106 may be implemented as a touch sensor or proximity sensor, such as capacitive, optical, infra-red, ultrasound, or other sensors to detect and trace a movement of an object such as a user's hand.
• System 100 may include any suitable mechanism for analyzing measurements from sensor 106 and determining whether such measurements are to be assigned to a particular object that has been sensed. Furthermore, system 100 may include any suitable mechanism for mapping traces of movement of such objects. For example, system 100 may include movement analysis module (MAM) 102. MAM 102 may be implemented in any suitable manner. For example, MAM 102 may be implemented in hardware, software, analog circuitry, digital circuitry, reconfigurable logic, or any suitable combination thereof. In one embodiment, MAM 102 may be implemented by instructions in a computer-readable medium such as memory 110, which, when executed by a processor 108, may configured MAM 102 to perform the operations described in this disclosure.
• MAM movement analysis module
• MAM 102 may be implemented by one or more functions, code, processes, scripts, executables, applications, objects, or other entities for execution on processor 108.
• Processor 108 may be implemented in any suitable manner, such as by a field programmable gate array (FPGA), application specific interface circuit (ASIC), microcontroller, or general purpose reconfigurable processor.
• FPGA field programmable gate array
• ASIC application specific interface circuit
• MAM 102 may include or be communicatively coupled to movement models 112 and cost matrix 114.
• MAM 102 may be configured to track any suitable number of objects or dimensions. In the examples below, MAM 102 is shown tracking two fingers with respect to a trackpad, display input, or other similar input to electronic device 104. However, the operations and configuration of MAM 102 may be expanded from these examples into tracking any suitable number and kind of objects, depending upon the availability of sensors. For example, MAM 102 may be configured to track multiple objects in positioning systems in consumer electronics, two or three dimensional spaces, medicine (such as tracking of moving particles in blood vessels), military target tracking, robotics, video surveillance, indoor and/or outdoor positioning systems. In some instances, an object may be tracked that might not be of interest to an end user. For example, a water drop on a two-dimensional touch pad may cause measurements similar to the touch of a finger. However, in the end such an object should be ignored for higher-level applications. These objects could have different movement models than target objects.
• FIGURE 2 is an illustration of example touch data. Given two objects (such as fingers) IDO and IDl, such objects may have previously been determined to be in respective positions shown in FIGURE 2 by the circle marks. Such a determination may have itself have been a prediction or estimation. While being able to determine and track two-dimensional or three- dimensional movements, MAM 102 may thus map these reference points to object traces.
• MAM 102 may be configured to determine whether each of these measurements belongs to the already-tracked objects, to newly detected objects, or to no objects at all. For example, such measurements might have been noise or inadvertent touches.
• measurement #0 and #1 may assign measurements #0 and #1 to either of objects IDO and IDl based upon (squared) Euclidean distance, Manhattan distance, or other distance from the respective objects. For example, measurement #0 is three cost units from IDl and two cost units from IDO. Thus, based upon this simple comparison, measurement #0 might be assigned to IDO. However, at the same or nearly the same time, measurement #1 was made. It may be determined that measurement #1 is four units from IDl and thee units from IDO, and thus, without any other considerations, measurement #1 would also be assigned to IDO. However, both measurements cannot both contemporaneously be attributed to the same object. As a result, some tracking models may apply a combinatorial assignment algorithm to determine which measurement belongs to which object, if any. Even with such combinatorial assignment algorithms, errors may be made.
• FIGURE 3 illustrates the mapping of measurements to tracked objects that may be made as a result of the squared Euclidean distance from the measurements to the last known or calculated positions of the objects.
• FIGURE 3 illustrates an X-Y plot of the movement of two objects (IDO and IDl) from the top of the graph moving towards the bottom of the graph. Each cross "x" in the figure represents a measurement. Previous measurements may have been attributed to one of the traces of the objects as they moved through the graph.
• measurement position b 2 might be attributed to IDO. This may be because the distance between bi and the predicted contact position of IDl plus the distance b 2 and the predicted contact position of IDO is smaller than the distance between bi and the predicted contact position of IDO plus the distance between b 2 and the predicted contact position of IDl .
• the distance calculation determination might thus cause a crisscrossed trace which is likely to be incorrect.
• mapping of measured touch positions to contact identifiers is a combinatorial assignment problem, which could be solved (incorrectly) by calculating the distances between all measured touch positions and the predicted positions of all known contacts, then determining the optimum assignment based on those distances.
• the assignment of measurements from the cost matrix may be made to individual objects using, for example, the Hungarian (Kuhn-Munkres) algorithm [see Harold W. Kuhn, "The Hungarian Method for the assignment problem", Naval Research Logistics Quarterly, 2: 83- 97, 1955)].
• this may be an unlikely correct mapping as, for example, two fingers are likely to have maintained their previous parallel trace operation.
• MAM 102 may utilize different cost functions for the mapping of a newly measured position to an object. Such cost functions may differ from other distance-only calculations. In a further embodiment, the cost functions may be based upon stochastic models of the objects' movement.
• FIGURE 4 is an illustration of cost function values with a stochastic model that may be applied by MAM 102, in accordance with embodiments of the present disclosure.
• the stochastic model may implement, fully or in part, movement models 112.
• MAM 102 may apply the stochastic model to the measurements received from sensor 106 to yield a cost matrix 114, from which the optimum mapping may be determined.
• the stochastic model may take into account previous positions and, from such positions, speed, acceleration, or jerk.
• the stochastic model may take into account direction of the movement, including angle, arc, and derivatives thereof.
• the stochastic model may identify, given current or predicted position and one or more of these other parameters, the probability that the next measurement of the object will be at any given position within the domain space (XY or additional dimensions) space.
• Various portions of the domain space may be defined as representative of a probability.
• the portions may be defined with isolines as shown in FIGURE 4.
• the portions containing the most recent position may be the highest probability.
• the portions moving away from this first portion may represent decreasing probability. Any suitable initial probability value for the first portion and any gradient probability may be used.
• a stochastic model may be generated or applied for each object that is being tracked.
• MAM 102 may apply the stochastic model of each object to the new measurement to determine the probability that the new measurement belongs to each respective object.
• the results of this analysis may be stored in a matrix such as cost matrix 114.
• MAM 102 may apply an optimization routine to determine, for one or more measurements, the best fit for the measurements as mapped to the various objects or determined to be noise.
• the particular shape of the isolines representing the various cost function values that a next measurement will be related to the present position estimate may be determined according to any suitable criteria. For example, the faster that the object has been moving, the greater the probability that future measurements within the trajectory of the object will map to the object.
• the object's movement may be used to determine the stochastic model.
• the cost function determined by the stochastic model may take into account speed and direction of the movement of the object being modeled.
• the stochastic model may be self-adjusting. As the object moves and changes direction, the parameters and variables of the stochastic model may be recalculated.
• the stochastic model may define that it is more likely that the object will move with linear, rather than perpendicular, acceleration or deceleration. Thus, it is more likely that the object that is being tracked will speed up or down in the preexisting trajectory (linear or otherwise) rather than change direction.
• the oval isolines shown in FIGURE 4 reflect such probabilities.
• the stochastic model may assume that an object will tend to keep moving at constant speed in constant direction (Newtonian movement), such as shown by the arrow in FIGURE 4. Some measurements may include random elements, due to noise or intentional movement of the object.
• the probability distribution defining the isolines and probability values may be made according to a distribution, such as Gaussian distribution, applied to Newtonian movement of the object. The probability may be assigned assuming that movement is constant. Accelerations parallel to the direction of movement may be predicted to be more likely than accelerations perpendicular to the direction of movement.
• the probability distribution for deviation has a greater variance in linear or parallel direction of movement than in the direction orthogonal to it as shown in the contours of FIGURE 4.
• the stochastic model may maintain a set of parameters for each tracked object.
• the variances of the probability distribution may be constant or adapted to the current speed, acceleration, jerk, angle, or trajectory of the object's movement.
• the metric for assigning a measurement to an object is the probability (or log-likelihood) that a measurement position is obtained with the respective object's movement model.
• the stochastic model may be adapted from the model upon which Kalman filters are based.
• a Kalman filter includes a prediction stage, wherein a new position estimate is predicted from the old model state, deterministic input, and an auto-adjusted estimate of the variances of random process noise. The position estimate would typically directly be contained in the filter state.
• the stochastic model may be based upon the position estimate of the object, control input from the object's presumed constant movement, and the process noise may be implemented by random deviation from the predicted position. Changes to other uses of the Kalman filters may include distinguishing process noise variances in the direction of movement and orthogonal to the direction of movement, rather than only to distinguishing movement in x- and y-directions.
• the stochastic model may be based on Gaussian distributions with parameters determined in a suitable manner.
• a Gaussian distributed random variable z with a mean value ⁇ , standard deviation ⁇ , and a probability distribution
• the (log) likelihood function may be defined as
• ⁇ ( ⁇ ) : -( ⁇ - ⁇ ) 2 / ⁇ 2
• the joint probability distribution of two such random variables— if they are uncorrected— is the product of their individual probability distributions, similar to a bivariate Gaussian probability distribution with uncorrected components.
• Li(b) - Wb - a ⁇ /o ,
• FIGURE 5 illustrates example operation of MAM 102, in accordance with embodiments of the present disclosure.
• MAM 102 may have created traces according to the movement model prior to the detection of the measured touch positions. Superimposed over each trace are the probability models showing how likely for each tracked object that a measurement within given regions will be attributable to that object.
• the measured touch positions may be the same as shown in FIGURE 3.
• using the stochastic model may have traced the trajectory of object ID1 and applied the stochastic model from the trace.
• the model as shown on the domain in FIGURE 5 shows that MAM 102 expects future touches from ID1 to likely occur within various portions of the domain in a series of isolines on the right, while future touches from ID0 are likely to occur within portions of the domain in a series of isolines on the left.
• the positions which the prediction of the new positions are based on are already on the smoothened trace. Thus, rather than crossing over each other, the traces of IDO and IDl remain in parallel.
• MAM 102 may track positions and map positions a; from a set A to a position bj from set B (or vice-versa) based on a cost or distribution function Fi(3 ⁇ 4, bj). Fi may be changed or generated based upon position 3 ⁇ 4.
• A may be a set of position estimates associated with tracked objects.
• B may be a set of position estimates obtained from measurement data.
• Fi may return a likelihood (or inversely: cost, distance) measure for a; and bj. The return value may be high/low when the likelihood of the pair (a;, bj) is high/low.
• the function Fi may depend upon the history of a;, wherein the likelihood changes differently in direction of movement of a; and in a direction orthogonal to the movement of a;. Typically, the likelihood decreases slower in direction of movement of a; than it decreases in direction orthogonal to the movement of 3 ⁇ 4.
• the function Fi may provide a cost output value for input position pair (3 ⁇ 4 ,bj) that can be used to compute the total cost.
• Fi may be a multivariate Gaussian function with a mean value a;. Fi may be centered around a;. Kalman filtering may be applied to track the object positions, wherein a; is the position resulting from the Kalman prediction stage for a given tracked object.
• the Gaussian function may have different variances in different directions, including different variances in the direction of movement of a; and orthogonal directions of the movement of a;.
• the variances of the Gaussian function may be adapted or constant and, moreover, may be taken from a Kalman filter used to track the object position.
• FIGURE 6 is an illustration of an example embodiment of a method 600 for tracking the position of objects.
• method 600 may be implemented in software.
• Method 600 may be implemented by any suitable mechanism, such as system 100.
• the method is set up to track up to N objects.
• objects may be tracked through, for example, two or three different dimensions.
• the objects may be tracked, for example, by a touch or proximity sensor.
• Stochastic movement models may be initialized for each object.
• measurements b may be received from the sensors.
• a stochastic model for each object may be evaluated for each measurement bj.
• the stochastic model may be based upon the position, angle, movement, speed, acceleration, jerk, or other suitable parameters of the object. These parameters may arise from prior execution of method 600.
• the results of these evaluations may be the probability, likelihood, or cost function to associate a measurement with an object.
• the results of the cost function may be used to populate a cost matrix C.
• an entry (i,f) of cost matrix C is assigned a cost of mapping bj to ⁇ 3 ⁇ 4 according to a model for a The model for each instance of may be different.
• the problem of assigning measurements b to objects a may be solved with cost matrix C according to a cost optimization algorithm.
• variables of the movement models may be updated, for example, the traces of objects, angle, speed, acceleration, jerk, and the probabilities of movement in direction of movement and perpendicular to it. Thereby, the respective newly assigned measurement b may be considered.
• Method 600 may optionally terminate or repeat as necessary.
• Method 600 may be performed multiple times. The methods may be performed beginning at any suitable initialization point.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• General Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Human Computer Interaction (AREA)
• Mathematical Analysis (AREA)
• Computing Systems (AREA)
• Artificial Intelligence (AREA)
• Computational Mathematics (AREA)
• Data Mining & Analysis (AREA)
• Evolutionary Computation (AREA)
• Probability & Statistics with Applications (AREA)
• Mathematical Optimization (AREA)
• Pure & Applied Mathematics (AREA)
• Algebra (AREA)
• Mathematical Physics (AREA)
• Software Systems (AREA)
• Position Fixing By Use Of Radio Waves (AREA)
• Traffic Control Systems (AREA)
• Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
• Position Input By Displaying (AREA)
• User Interface Of Digital Computer (AREA)
• Image Analysis (AREA)
• Length Measuring Devices With Unspecified Measuring Means (AREA)

Priority Applications (4)

Applications Claiming Priority (4)

Publications (1)

Family

ID=57121219

Family Applications (1)

Country Status (7)

Families Citing this family (2)

Citations (4)

Family Cites Families (10)

• 2016
  • 2016-09-28 US US15/278,534 patent/US10234990B2/en active Active
  • 2016-09-29 EP EP16778734.0A patent/EP3356924A1/en not_active Ceased
  • 2016-09-29 WO PCT/EP2016/073371 patent/WO2017055517A1/en not_active Ceased
  • 2016-09-29 CN CN201680043710.XA patent/CN107850977B/zh active Active
  • 2016-09-29 JP JP2017568198A patent/JP6933584B2/ja active Active
  • 2016-09-29 TW TW105131403A patent/TWI731879B/zh active
  • 2016-09-29 KR KR1020177037508A patent/KR102501658B1/ko active Active

Patent Citations (4)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events