WO2006043058A1 - Automated gesture recognition - Google Patents

Automated gesture recognition Download PDF

Info

Publication number
WO2006043058A1
WO2006043058A1 PCT/GB2005/004029 GB2005004029W WO2006043058A1 WO 2006043058 A1 WO2006043058 A1 WO 2006043058A1 GB 2005004029 W GB2005004029 W GB 2005004029W WO 2006043058 A1 WO2006043058 A1 WO 2006043058A1
Authority
WO
WIPO (PCT)
Prior art keywords
gesture
vector
trajectory
library
vectors
Prior art date
Application number
PCT/GB2005/004029
Other languages
English (en)
French (fr)
Inventor
Jocelyn Elgoyhen
John Payne
Paul Anderson
Paul Keir
Tom Kenny
Original Assignee
Glasgow School Of Art
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Glasgow School Of Art filed Critical Glasgow School Of Art
Priority to DE602005009568T priority Critical patent/DE602005009568D1/de
Priority to EP05794750A priority patent/EP1810217B1/en
Priority to US11/577,694 priority patent/US20080192005A1/en
Publication of WO2006043058A1 publication Critical patent/WO2006043058A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017Gesture based interaction, e.g. based on a set of recognized hand gestures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/014Hand-worn input/output arrangements, e.g. data gloves
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20Movements or behaviour, e.g. gesture recognition
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/033Indexing scheme relating to G06F3/033
    • G06F2203/0331Finger worn pointing device

Definitions

  • the present invention relates to computer-based motion tracldng systems and particularly, though not exclusively, to a system capable of tracldng and identifying gestures or traj ectories made by a person.
  • Gesture recognition systems have been identified in the art as being potentially valuable in this re -Oga"rd.
  • WO 03/001340 describes a gesture recognition system which classifies gestures into one of two possible classes, namely (i) planar translation motion, and (ii) angular motion without translation. This enables separate gesture discriminators to work on the interpretation improving the chances of correct gesture discrimination.
  • WO '340 proposes applying different classes of gestures to different functions, such as reciprocal actions for commands, tilt actions for positional (e.g. cursor) control and planar translational motions for handwriting.
  • US 6681031 describes a gesture- controlled interface which uses recursive 'best fit' type operations attempting to find the best fit between all points on a projection of a sampled gesture to all points on candidate gestures.
  • US 2204/0068409 describes a system for analysing gestures based on signals acquired from muscular activity.
  • US 2004/0037463 describes a system for recognising symbols drawn by pen strokes on a sketch-based user interface by dividing the strokes into a number of sub-frames and deriving a signature for each sub-frame that is expressed as a vector quantity.
  • US 6473690 describes a system for comparing and matching data represented as three-dimensional space curves, e.g. for checking geographic database accuracy.
  • US 2004/0037467 describes a system for determining the presence of an object of interest from a template image in an acquired target image.
  • a significant problem in gesture recognition S3 ⁇ stems is how to accurately, reliably and speedily detect a gesture or trajectory being made and compare it to a library of candidate gestures stored in a database.
  • the present invention provides a gesture recognition method comprising the steps of: a) receiving input data related to a succession of positions, velocities, accelerations and/or orientations of at least one object, as a function of time, which input defines a trajectory of the at least one object; b) performing a vector analysis on the trajectory data to determine a number N of vectors making up the object trajectory, each vector having a length and a direction relative to a previous or subsequent vector or to an absolute reference frame, the vectors defining a gesture signature; c) on a vector by vector basis, comparing the object trajectory with a plurality of library gestures stored in a database, each library gesture also being defined by a succession of such vectors; and d) identifying a library gesture that corresponds with the trajectory of the at least one object.
  • the present invention provides a gesture recognition engine comprising: an input for receiving input data related to a succession of positions, velocities, accelerations and/or orientations of at least one object, as a function of time, which input defines a trajectory of the at least one object; a gesture analysis process module for performing a vector analysis on the trajectory data to determine a number N of vectors making up the object trajectory, each vector having a length and a direction relative to a previous or subsequent vector or to an absolute reference frame, the vectors defining a gesture signature; and a gesture comparator module for comparing, on a vector by vector basis, the object trajectory with a plurality of library gestures stored in a database, each librae gesture also being defined by a succession of such vectors and identifying a library gesture that corresponds with the trajectory of the at least one object.
  • Figure Ia is a perspective view of an exemplary motion tracking sensor arrangement
  • Figure Ib is a perspective view of an alternative exemplary motion tracking sensor arrangement
  • Figure 2 is a schematic diagram of a module for pre-processing accelerometer sensor outputs
  • Figure 3 shows illustrations useful in explaining deployment of relative spherical coordinates in gesture definition, in winch figure 3 a shows a tracked gesture defined by absolute points in a Cartesian coordinate system and figure 3b shows the tracked gesture defined by points in a relative spherical coordinate system;
  • Figure 4 is a schematic diagram of a gesture recognition system
  • Figure 5 is a flowchart illustrating steps taken by a gesture analysis module during a gesture recognition process
  • Figure 6 is a flowchart illustrating steps taken by a gesture comparator module during a gesture matching process.
  • Figure 7 is a schematic diagram of a module for pre-processing accelerometer and angular rate sensor outputs.
  • the expression 'gesture' is used to encompass a trajectory or motion behaviour of an object or of a selected part of an object in space.
  • the object could, for example, be a person's hand, or an object being held in a person's hand.
  • the object could be a person.
  • the object may even be a part of a sensor device itself, e.g. a joystick control as guided by a user's hand.
  • the trajectory which encompasses any motion behaviour, generally defines movement of an object or of part of an object relative to a selected stationary reference frame, relative to a moving reference frame, or even relative to another part of the object.
  • a gesture may include a series of positions of the object or part of the object as a function of time, including the possibility that the object does not move over a period of time, which will generally be referred to as a 'posture' or 'stance.
  • a posture or stance is to be included as a special case of a 'gesture', e.g. a fixed gesture.
  • the expression 'object' used herein in connection with defining a gesture is intended to include part of a larger object.
  • a wearable sensor 10 comprising an inertial sensor 11 is housed in a finger cap 12.
  • the inertial sensor 11 is coupled, by wiring 13, to a processor (not shown in the drawing) contained in a strap assembly 14 that may be bound to the user's hand 15.
  • the strap assembly 14 may also include a further inertial sensor (not shown) to provide position data of the user's hand relative to the finger, if desired.
  • the strap assembly 14 preferably includes a telemetry system for streaming output data from the inertial sensor(s) to a computer system to be described.
  • the telemetry system preferably communicates with the computer system over a wireless communication channel, although a wired link is also possible.
  • the wearable sensor 10 preferably also includes one or more switches for signalling predetermined events by the user.
  • a touch switch 16 may be incorporated into the finger cap 12 that is actuated by tapping the finger against another object, e.g. the thumb or desk.
  • a thumb or finger operated function switch 17 may be located on or near the palm side of the strap assembly 14.
  • the at least one inertial sensor 11 comprises three orthogonal linear accelerometers that determine rate of change of velocity as a function of time in three orthogonal directions as indicated by the straight arrows of figure Ia, together with three angular speed sensors that determine rotation rate about the three orthogonal axes, hi combination, these accelerometers and angular speed sensors are capable of providing information relating to the movement of the finger according to the six degrees of freedom.
  • any sensor type and combination may be used that is capable of generating data relating to a succession of relative or absolute positions, velocities, accelerations and/or orientations of at least one object.
  • a number of different types of such sensor are known in the art.
  • FIG. 10 Another example of a sensor arrangement is now described in connection with figure Ib.
  • This sensor arrangement may be described as a handheld sensor 10', rather than a wearable sensor as shown in figure Ia.
  • the sensor 10' comprises an inertial sensor 11 ' is a housing 12' that may conveniently be held in one hand 15'.
  • the inertial sensor 11 ' is coupled to a processor (not shown) contained within the housing 12'.
  • a telemetry system communicates with a remote computer system 18 over a wireless communication channel, although a wired link is also possible.
  • the sensor 10' preferably includes one or more switches 17' for signalling predetermined events by the user.
  • touch switch 17' is incorporated into the housing 12' and is actuated by squeezing or applying pressure to the housing 12'.
  • the at least one inertial sensor 11 ' comprises three orthogonal linear accelerometers that determine rate of change of velocity as a function of time in three orthogonal directions x, y, z.
  • these accelerometers are capable of providing information relating to the movement of the object according to the three degrees of freedom. Roll and pitch can be deduced in relation to the earth's gravitational force, hence providing an additional two degrees of freedom for this embodiment.
  • an object being tracked ma ⁇ ' include one or more markers identifying predetermined locations on the object that are to be tracked by suitable remote sensors.
  • the markers may be optical, being remotely detectable by an imaging system or photocell arrangement.
  • the markers ma ⁇ ' be active in the sense of emitting radiation to be detected by suitable passive sensors.
  • the markers may be passive in the sense of reflecting radiation from a remote illumination source, which reflected radiation is then detected by suitable sensors.
  • the radiation may be optical or may lie in another range of the electromagnetic spectrum. Similarly, the radiation may be acoustic.
  • the object being tracked need not be provided with specific markers, but rely on inherent features (e.g. shape) of the object that can be identified and tracked b ⁇ ' a suitable tracking system.
  • the object may have predetermined profile or profiles that are detectable by an imaging system in a field of view, such that the imaging system can determine the position and/or orientation of the object.
  • any tracking system may be used that is capable of generating data relating to a succession of relative or absolute positions, velocities, accelerations and/or orientations of the object.
  • a number of such tracking systems are available to the person skilled in the art.
  • Figure 2 provides an overview of a data collection operation sensing motion of an object and pre-processing the data to obtain an acceleration signature that may be used by the gesture recognition system of the present invention.
  • the outputs 22x, 22y, 22z from just three linear accelerometers 2Ox, 20 ⁇ ' and 2Oz are used.
  • the linear accelerometers are preferably arranged in orthogonal dispositions to provide three axes of movement labelled x, y, and z. Movement of the object on which the accelerometers are positioned will induce acceleration forces on the accelerometers in addition to the earth gravitational field.
  • the raw signals from the three orthogonal linear accelerometers are pre- processed in order to generate a set of data samples that can be used to identify gesture signatures.
  • the outputs 22x, 22y, 22z of accelerometers 2Ox, 2Oy and 2Oz are preferably digitised using an appropriate A/D converter (not shown), if the outputs 22x, 22y, 22z therefrom are not already in digital form.
  • the digitisation is effected at a sampling frequency and spatial resolution that is sufficient to ensure that the expected gestures can be resolved in time and space. More particularly, the sampling frequency is sufficiently high to enable accurate division of a gesture into a number N of portions or vectors as will be described later.
  • the user marks the start of a gesture by activating a switch 21 (e.g. one of the possible switches 16, 17, 17' of figures Ia and Ib).
  • This switch 21 could generally be in the form of a physical button, a light sensor or a flex sensor. More generally, manual activation of any type of electronic, electromechanical, optoelectronic or other physical switching device may be used.
  • the user could mark the start of a gesture by means of another simple gesture, posture or stance that is readily detected by the s ⁇ f stem.
  • the system ma ⁇ ' continuously monitor input data for a predetermined pattern or sequence that corresponds to a predetermined trajectory indicative of a 'start gesture' signal.
  • the user could indicate the start of a gesture by any means of marking or referencing to a point in time to begin gesture recognition.
  • the gesture recognition system could itself initiate a signal that indicates to the user that a time capture window has started in which the gesture should be made.
  • Each of the three output signals 22x, 22y and 22z of the accelerometers 2Ox, 2Oy and 2Oz has a DC offset and a low frequency component comprising the sensor zero-g levels plus the offset generated by the earth's gravitational field, defined by the hand orientation.
  • DC blockers 23x, 23y and 23z relocate the output signals around the zero acceleration mark.
  • the resulting signals 26x, 26y, 26z are passed to low-pass filters 24x, 24y and 24z that smooth the signals for subsequent processing.
  • the outputs 27x, 27y. 27z of filters 24x, 24y 5 24z are passed to respective integrators 28x. 28y 5 28z which can be started and reset by the switch 21.
  • the output of this pre-processing stage comprises data 25 representing the trajectory or motional behaviour of the object, preferably in at least two dimensions.
  • the start and end of the gesture, posture or stance may be indicated by operation of the switch 21.
  • DC blockers 23 any or all of the functions of DC blockers 23.
  • low-pass filters 24 and integrators 28 can be carried out in either the analogue domain or the digital domain depending upon the appropriate positioning of an analogue to digital converter.
  • the accelerometers would provide analogue outputs 22 and the output data 25 would be digitised. Conversion may take place at a suitable point in the data path therebetween.
  • the gesture recognition system operates on sequences of the two or three- dimensional values or samples gathered from the input devices as described above.
  • the gesture defined by the motion behaviour curve or 'trajectory' of the object may describe a shape that has the same geometric structure as another gesture curve, yet appear unalike due to having a different orientation or position in space.
  • the gesture recognition system preferably first converts the input 'Cartesian' value sequence to one of relative spherical coordinates. This form describes each gesture sequence independently of its macroscopic orientation in space.
  • each three-dimensional value (x ⁇ , y n , Z 11 ) referenced against Cartesian axes 30 is described by a Cartesian three-tuple. Taken together as a sequence of position values they represent a gesture 31 - the path from (xj, y 1; zi) through to (X 4 , y 4 , Z 4 ). Translation, rotation or scaling of this shape will result in a new and different set of Cartesian values. However, for gesture comparison, it is desirable to make comparison of the input data for a tracked gesture at least partly independent of one or more of translation, rotation and scaling.
  • a gesture is recognised even allowing for variation in the magnitude of the gesture (scaling), variation in position in space that the gesture is made (translation), and even the attitude of the gesture relative to a fixed reference frame (rotation). This is particularly important in recognising, for example, hand gestures made by different persons where there is considerable variation in size, shape, speed, orientation and other parameters between different persons' version of the same gesture and indeed between the same person's repetition of the same gesture.
  • the azimuth angle ⁇ represents the angle between the vector pair in the plane defined by the vector pah"
  • the zenith angle ⁇ represents the angle of that plane relative to the plane of the preceding vector pair.
  • zenith angle 6 2 , 3 is the angle that the perpendicular of the V 2 , V 3 plane makes relative to the perpendicular of the V 1 , V 2 plane.
  • the recognition process perceives the data as geometrical, and the data input values handled by the gesture recognition sj'stem may be absolute position in space, relative position in space, or any derivatives thereof with respect to time, e.g. velocity or acceleration.
  • the data effectively define a gesture signature either in terms of a path traced in space, a velocity sequence or an acceleration sequence. In this manner, the process of the gesture recognition system can work effectively with many different types of sensor using the same basic algorithm.
  • the gesture recognition system first performs pre-processing steps as discussed above in order to convert the input data into a useful data stream that can be manipulated to derive the values R, ⁇ and ⁇ above for any one of position, velocity or acceleration.
  • the gesture recognition system 40 includes a module 41 for detecting or deterrnining the nature of the sensors 11 or 20 (figures 1 and 2) from which data is being received. This may be carried out explicitly by exchange of suitable data between the sensors 11 or 20 and the detection module 41. Alternatively, module 41 may be operative to determine sensor type implicitly from the nature of data being received.
  • the detection module 41 controls a conversion module 42 that converts the input data using the pre-processing steps as discussed above, e.g. identification of start and end points of a gesture, removal of DC offsets, filtering to provide smoothing of the sensor output and analogue to digital conversion.
  • a conversion module 42 that converts the input data using the pre-processing steps as discussed above, e.g. identification of start and end points of a gesture, removal of DC offsets, filtering to provide smoothing of the sensor output and analogue to digital conversion.
  • a gesture recognition process receives (step 501) the input relating to a succession of positions, velocities or accelerations (or further derivatives) of the object as a function of time that define the gesture signature, or trajectory of the object being sensed.
  • a gesture analysis process module 43 then performs steps to define the gesture signature in terms of the coordinate system described in connection with figure 3b. Firstly, a sampling rate r is selected (step 502). In a preferred embodiment, a default sampling rate is at least 60 samples per second, and more preferably 100 samples per second or higher. However, this may be varied either by the user, or automatically by the gesture analysis process module 43 according to a sensed length of gesture, speed of movement or sensor type.
  • the process module 43 determines (step 503) whether analysis is to be carried out on the basis of position, velocity or acceleration input values, e.g. by reference to the determined sensor type.
  • the process module 43 selects a number N of values to resample each gesture signature sequence into, i.e. the gesture signature is divided into TV " portions (step 504).
  • the value for N " is 10.
  • any suitable value may be used depending upon, for example, the length of gesture signature and the number of portions of gesture signatures in a library against which the input gesture signature must be matched.
  • the JV portions preferably represent ⁇ ⁇ portions of equal temporal duration.
  • the gesture signature is defined on the basis of JV equal time intervals or TV equal number of input data sample points.
  • the N portions may be of equal length.
  • the JV portions may be of unequal time and length, being divided by reference to points on the trajectory having predetermined criteria- such as points corresponding to where the trajectory has a curvature that exceeds a predetermined threshold.
  • portions of the trajectory that have a low curvature may be of extended length, while portions of the trajectory that have high curvature may be of short length.
  • Plural curvature thresholds may be used to determine portions of differing lengths.
  • the process module 43 also determines the dimensional format of the data (step 505), i.e. how many dimensions the input values relate to. This also may affect the selection of candidates in a library of gesture signatures against which the input gesture signature may be potentially matched. For example, two or three dimensional samples may be taken depending upon sensor type, context etc.
  • the N gesture signature portions are converted into N vectors V n hi the spherical coordinate sj'stem (step 506).
  • the vectors V n are then normalised for each vector pair, to derive the ⁇ ectors in the relative spherical coordinate system described in connection with figure 4 (step 507). More specifically, R,,, ⁇ n and ⁇ ,, are determined where R,, is the ratio of the length of the 77.Hi vector to the preceding vector; ⁇ ,, is the angle between the nth vector and the preceding vector; and ⁇ , is the angle between the perpendicular of the plane defined by vectors ⁇ n, n-1 ⁇ and the perpendicular of the plane defined by the vectors ⁇ 77-1, n-2 ⁇ .
  • the first vector will have a length and direction only.
  • the direction of the first vector vi relative to a reference frame may be ignored if the gesture signature recognition is to be orientation insensitive.
  • the direction of the first vector may be referenced against another frame, e.g. that of the object or other external reference.
  • the direction of any vector in the sequence of N vectors may be used to reference against an external frame if absolute orientation is to be established.
  • the first vector is selected for convenience, one or more vectors anywhere in the sequence may be used.
  • the second vector will have an R value and a ⁇ value only, unless the plane of the first vector pair vi and V 2 is to be referenced against an external reference frame.
  • the gesture signature has been defined as a sequence of R, ⁇ and ⁇ values for each of a plurality of portions or segments thereof (step 508).
  • gesture recognition system 40 further includes a database or library 44 containing a number of gesture signatures, each gesture signature also being defined as a sequence of R, ⁇ and ⁇ values.
  • the gesture signatures in the library will each have a type specification indicating a class of gestures to which they belong.
  • the type specification may include a sensor type specification indicating the type of sensor from which the signature was derived, thereby indicating whether the signature specifies position data, velocity data or acceleration data.
  • the type specification may also indicate a spatial dimension of the signature.
  • the type specification may also indicate a size dimension of the signature, i.e. the number of portions (vectors) into which the signature is divided.
  • Other type specifications may be included, providing a reference indicating how the library gesture signature should be compared to an input gesture or whether the library gesture signature is eligible for comparison with an input gesture.
  • the gesture library 44 may be populated with gesture signatures using the gesture analysis module 43 when operating in a 'learn' mode. Thus, a user ma ⁇ ' teach the system a series of gesture signatures to be stored in the Library for comparison with later input gesture signatures. Alternatively or in addition, the library 44 may be populated with a collection of predetermined gesture signatures from another source.
  • the gesture recognition system 40 further includes a gesture comparator module 45 for effecting a comparison of an input gesture signature with a plurality of previously stored library gesture signatures in the database library 44.
  • the gesture comparator module 45 performs the following steps.
  • a group or subset of library gesture signatures which are potentially eligible for matching with an input gesture signature is selected (step 601).
  • the group may comprise one library of many libraries; a subset of the library 44; all available library gestures or some other selection.
  • the group may be selected according to the type specification stored with each library gesture signature.
  • a threshold for degree of match is determined (step 602).
  • This may be a simple default parameter, e.g. 90%.
  • the default parameter could be overruled by the user according to predetermined preferences.
  • the default parameter could be selected by the system according to the gesture type specification. For example, three dimensional gesture signatures could have a different threshold than two dimensional gesture signatures, and acceleration signatures could have a different threshold than velocity signatures. Further, individual users may be provided with different threshold values to take into account a learned user variability.
  • the threshold degree of match may be used by the gesture comparator module 45 to determine which library gestures to identify as successful matches against an input gesture signature.
  • the gesture comparator module 45 may operate on a 'best match' basis, to determine the library gesture signature that best matches the input gesture signature.
  • the threshold degree of match may then be used to provide a lower level cut-off below which library gestures will not even be regarded as potential matches and thus will not be considered for best match status.
  • the next step carried out by the gesture comparator module 45 is to compare each of the N-I vector pairs of the input gesture signature with a corresponding vector ' pair of one of the group of library gestures selected for comparison, and to compute a difference value in respect of the length ratios (R,,), azimuth angles ( ⁇ ,,) and zenith angles ( ⁇ w ). These difference values are referred to respectively as dR n , d ⁇ n and d ⁇ n .
  • the mean square error for each of the respective difference values for all portions of the signature is calculated, i.e. to find the mean square error for each of dR n , d ⁇ n and d ⁇ n in the signature comparison (step 604).
  • This single error value may then be checked (step 606) to see if it is inside the threshold degree of match selected in step 602. If it is not, it can be discarded (step 607). If it is within the threshold degree of match, then the identity of the library gesture signature compared may be stored in a potential match list (step 608). The gesture comparator module 45 may then check to see if further library gesture signatures for comparison are still available (step 609), and if so, return to step 603 to repeat the comparison process with a new library gesture signature.
  • the comparator module 45 may select the library gesture signature having the lowest error value from the potential match list.
  • the comparator module 45 may alternatively present as a 'match' the first library gesture that meets the threshold degree of match criteria. Alternatively, the comparator 45 may output a list of potential matches including all gesture signatures that meet the threshold degree of match criteria. A number of other selection criteria will be apparent to those skilled in the art.
  • the gesture comparator module 45 then outputs a list of potential matches, or outputs a single best match if the threshold degree of match criteria are met, or outputs a 'no match' signal if no library gestures reach the threshold degree of match criteria.
  • the output module 46 may comprise a display output a printed output, or a control output for issuing an appropriate command or signal to another computer system or automated device to initiate a predetermined action based on the gesture identified by the match.
  • the gesture recognition system 40 may be incorporated into another system to provide a user interface with that system, such that the system may be controlled at least in part by user gestures.
  • gesture recognition system 40 perform gesture analysis based on a motion behaviour of a single 'track', e.g. the motion behaviour of a single point through or in space. It will be recognised that more complex object behaviour may also constitute a gesture signature, e.g. considering the motion behaviour of several points on the object in space, so that the gesture signature effectively comprises more than one 'track'. In another example, it may be desirable also to take into account rotational behaviour of a tracked point, i.e. rotation of the object about its own axes or centre of gravity.
  • the sensor inputs may provide data for two or more tracked points on the object.
  • these data may be considered as providing data for a 'compound signature', or signature having two or more tracks.
  • Each of these tracked points may be analysed by the gesture analysis process module 43 in the manner already described.
  • the gesture comparator module 45 may then average together the error values for each of the tracks in order to determine a final error value which can be used for the match criteria.
  • multiple tracked points may be inferred from rotation data of the motion behaviour of the object if a sensor system that provided rotation behaviour is used.
  • gesture signature recognition may be obtained by using signatures comprising two or more of position data, velocity data and acceleration data.
  • the gesture analysis module 43 may separately determine R n , ⁇ ;? and O 7 , for position as a function of time, for velocity as a function of time and/or for acceleration as a function of time.
  • the gesture comparator module 45 then separately compares positional R n , ⁇ ⁇ and ⁇ ,,, velocity R n , ⁇ ,, and O n and/or acceleration R n , ⁇ ⁇ and Q n of the gesture signature with corresponding values from the gesture library 44 in order to determine match.
  • each of N vectors during gesture matching may be performed in respect of values of R, ⁇ and ⁇ for successive vectors, relative to a preceding vector. It is also possible to compare N vectors in respect of ⁇ and ⁇ values referenced to a fixed reference frame. For example, for a fixed reference frame having conventional Cartesian x, 3' and z axes, the values compared may be an azimuth angle ⁇ of the vector relative to the x axis within the x-y plane, and a zenith angle ⁇ of the vector relative to the z-axis (steps 507 and 508, figure 5).
  • the ⁇ and ⁇ values of the 77th vector of the input gesture are compared with the corresponding ⁇ and ⁇ values of the 7?th vector of a library gesture, and similarly for all n from 1 to N " .
  • the lengths / of the vectors are compared such that the length / of the /?th vector of the input gesture is compared with the length / of the corresponding ;?th vector of a library gesture, and similarly for all n from 1 to N.
  • the comparisons may be on a difference basis or a ratio basis, e.g. and ⁇ ,, >mj3a/ / ⁇ , 7 ,/, ⁇ ,- ⁇ o . or ⁇ n ,mp ut ⁇ ⁇ njibrwy 3XiO. u n,mpui ' " n.hbrary Ol " n.mput ⁇ ⁇ " nj ⁇ rary-
  • comparison step 603 is modified to include a transformation first applied to bring the input gesture signature vector data as close as possible to the current one of the library gestures being compared, the transformation being a combination of one or more of rotation, scale and translation. Then, in a modification to step 604, the root mean square error sum is calculated for all the N transformed input vectors compared to the respective N vectors of the library gesture signature. A zero error value would be a perfect match.
  • the best transformation to apply may be determined according to any suitable method. One such method is that described by Berthold K P Horn in "Closed form solution of absolute orientation using unit quaternions", J. Opt. Soc. of America A, Vol. 4, p. 629 et seg, April 1987.
  • Horn describes that the best translational offset is the difference between the centroid of the coordinates in one system and the rotated and scaled centroid of the coordinates in the other system.
  • the best scale is equal to the ratio of the root-mean-square deviations of the coordinates in the two systems from their respective centroids.
  • FIG 7 a further sensor arrangement and pre-processing module for providing velocity data input and positional data input is shown.
  • Three orthogonal accelerometers 70 provide acceleration signals a x , a y , a 2 ; and three angular rate sensors 72 provide angular rotation rate signals ⁇ x , ⁇ y and ⁇ z .
  • a switch or sensor 71 provides a gesture start / stop indication, similar to that described in connection with switch 21 of figure 2.
  • the angular rate sensor data is passed to an attitude vector processing module 73 which determines a current attitude vector.
  • TMs is used in conjunction with the three orthogonal acceleration signals a x . a y , a z to derive motion behaviour information for the six degrees of freedom by axis transformation module 74.
  • This information is then processed by the integrator module 75 to derive velocity signals and position signals relative to a predetermined axis, e.g. the earth's gravitational field. These velocity and position signals may then be used as input to the gesture analysis process module 43.
  • the gesture recognition system may also be provided with a calibration module.
  • a user may be asked to perform certain specified gestures which are tracked by the sensors and analysed by the gesture analysis process module 43. These gestures are then added to the gesture library 44 for future comparison.
  • the library gestures may include in their type specification, a user for which these gestures represent a valid subset for comparison.
  • an output display may be provided to display a rendered image of the user's hand, or other object being tracked. This display may be overlaid with the gesture signature being tracked and/or identified.
  • the system may be used to control that object.
  • a handheld device such as a mobile telephone may be adapted to interface with the user by moving the mobile phone itself through predetermined gestures in order to instruct the phone to perform certain commands . , e.g. for menu access.
  • a joystick may have the gesture recognition engine inbuilt to detect certain patterns of movement which can then be interpreted in a special way.
  • the gesture recognition engine has many applications in computer gaming, e.g. for tracking the head, hand, limb or whole body movement of a game player to implement certain gaming input.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Social Psychology (AREA)
  • Psychiatry (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • User Interface Of Digital Computer (AREA)
  • Image Analysis (AREA)
  • Measuring Pulse, Heart Rate, Blood Pressure Or Blood Flow (AREA)
PCT/GB2005/004029 2004-10-20 2005-10-19 Automated gesture recognition WO2006043058A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
DE602005009568T DE602005009568D1 (de) 2004-10-20 2005-10-19 Automatische gestik-erkennung
EP05794750A EP1810217B1 (en) 2004-10-20 2005-10-19 Automated gesture recognition
US11/577,694 US20080192005A1 (en) 2004-10-20 2005-10-19 Automated Gesture Recognition

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
GB0423225A GB2419433A (en) 2004-10-20 2004-10-20 Automated Gesture Recognition
GB0423225.2 2004-10-20

Publications (1)

Publication Number Publication Date
WO2006043058A1 true WO2006043058A1 (en) 2006-04-27

Family

ID=33484828

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/GB2005/004029 WO2006043058A1 (en) 2004-10-20 2005-10-19 Automated gesture recognition

Country Status (6)

Country Link
US (1) US20080192005A1 (pt-PT)
EP (1) EP1810217B1 (pt-PT)
AT (1) ATE407409T1 (pt-PT)
DE (1) DE602005009568D1 (pt-PT)
GB (1) GB2419433A (pt-PT)
WO (1) WO2006043058A1 (pt-PT)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109816699A (zh) * 2019-01-30 2019-05-28 华通科技有限公司 一种基于背景抑制帧间差分法的云台角度计算方法

Families Citing this family (191)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8352400B2 (en) 1991-12-23 2013-01-08 Hoffberg Steven M Adaptive pattern recognition based controller apparatus and method and human-factored interface therefore
US7966078B2 (en) 1999-02-01 2011-06-21 Steven Hoffberg Network media appliance system and method
US20090131151A1 (en) * 2006-09-01 2009-05-21 Igt Automated Techniques for Table Game State Tracking
DE102004057157A1 (de) * 2004-08-21 2006-04-27 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Verfahren und Vorrichtung zum Erfassen einer eigenhändig ausgeführten Unterschrift oder Signatur und zum Erkennen der Echtheit der Unterschrift oder Signatur
US7882435B2 (en) 2005-12-20 2011-02-01 Sony Ericsson Mobile Communications Ab Electronic equipment with shuffle operation
US8930834B2 (en) * 2006-03-20 2015-01-06 Microsoft Corporation Variable orientation user interface
US8144125B2 (en) 2006-03-30 2012-03-27 Cypress Semiconductor Corporation Apparatus and method for reducing average scan rate to detect a conductive object on a sensing device
US8139059B2 (en) * 2006-03-31 2012-03-20 Microsoft Corporation Object illumination in a virtual environment
US8059015B2 (en) 2006-05-25 2011-11-15 Cypress Semiconductor Corporation Capacitance sensing matrix for keyboard architecture
US8001613B2 (en) * 2006-06-23 2011-08-16 Microsoft Corporation Security using physical objects
US8086971B2 (en) * 2006-06-28 2011-12-27 Nokia Corporation Apparatus, methods and computer program products providing finger-based and hand-based gesture commands for portable electronic device applications
US20080040692A1 (en) * 2006-06-29 2008-02-14 Microsoft Corporation Gesture input
US8040321B2 (en) 2006-07-10 2011-10-18 Cypress Semiconductor Corporation Touch-sensor with shared capacitive sensors
US20080020733A1 (en) * 2006-07-21 2008-01-24 Tomas Karl-Axel Wassingbo Mobile electronic device with motion detection authentication
US8058937B2 (en) 2007-01-30 2011-11-15 Cypress Semiconductor Corporation Setting a discharge rate and a charge rate of a relaxation oscillator circuit
WO2008092527A1 (en) * 2007-01-31 2008-08-07 International Business Machines Corporation Deliberate access permission to data on contactless devices
US20080229255A1 (en) * 2007-03-15 2008-09-18 Nokia Corporation Apparatus, method and system for gesture detection
US8144126B2 (en) 2007-05-07 2012-03-27 Cypress Semiconductor Corporation Reducing sleep current in a capacitance sensing system
US20080291160A1 (en) * 2007-05-09 2008-11-27 Nintendo Co., Ltd. System and method for recognizing multi-axis gestures based on handheld controller accelerometer outputs
US8532871B2 (en) * 2007-06-05 2013-09-10 Mitsubishi Electric Company Multi-modal vehicle operating device
US20100292007A1 (en) * 2007-06-26 2010-11-18 Nintendo Of America Inc. Systems and methods for control device including a movement detector
US8258986B2 (en) 2007-07-03 2012-09-04 Cypress Semiconductor Corporation Capacitive-matrix keyboard with multiple touch detection
US8225343B2 (en) * 2008-01-11 2012-07-17 Sony Computer Entertainment America Llc Gesture cataloging and recognition
US8384661B2 (en) * 2008-03-04 2013-02-26 Namco Bandai Games Inc. Program, information storage medium, determination device, and determination method
US8170186B2 (en) 2008-04-07 2012-05-01 Sony Mobile Communications Ab Electronic device with motion controlled functions
US8462996B2 (en) * 2008-05-19 2013-06-11 Videomining Corporation Method and system for measuring human response to visual stimulus based on changes in facial expression
US8418084B1 (en) * 2008-05-30 2013-04-09 At&T Intellectual Property I, L.P. Single-touch media selection
US9200901B2 (en) 2008-06-19 2015-12-01 Microsoft Technology Licensing, Llc Predictive services for devices supporting dynamic direction information
US20100009662A1 (en) 2008-06-20 2010-01-14 Microsoft Corporation Delaying interaction with points of interest discovered based on directional device information
US8996332B2 (en) * 2008-06-24 2015-03-31 Dp Technologies, Inc. Program setting adjustments based on activity identification
JP2010015238A (ja) * 2008-07-01 2010-01-21 Sony Corp 情報処理装置、及び補助情報の表示方法
US8370106B2 (en) * 2008-07-07 2013-02-05 Keynetik, Inc. Spatially aware inference logic
EP2169517A1 (en) 2008-09-25 2010-03-31 Research In Motion Limited System and method for analyzing movements of an electronic device
US8744799B2 (en) 2008-09-25 2014-06-03 Blackberry Limited System and method for analyzing movements of an electronic device
KR102339297B1 (ko) * 2008-11-10 2021-12-14 구글 엘엘씨 멀티센서 음성 검출
EP2199948A1 (en) * 2008-12-18 2010-06-23 Koninklijke Philips Electronics N.V. Method of plotting a 3D movement in a 1D graph and of comparing two arbitrary 3D movements
US8030914B2 (en) * 2008-12-29 2011-10-04 Motorola Mobility, Inc. Portable electronic device having self-calibrating proximity sensors
US8275412B2 (en) * 2008-12-31 2012-09-25 Motorola Mobility Llc Portable electronic device having directional proximity sensors based on device orientation
US7996793B2 (en) 2009-01-30 2011-08-09 Microsoft Corporation Gesture recognizer system architecture
US20100199231A1 (en) * 2009-01-30 2010-08-05 Microsoft Corporation Predictive determination
CA2748037C (en) * 2009-02-17 2016-09-20 Omek Interactive, Ltd. Method and system for gesture recognition
US8339367B2 (en) * 2009-02-27 2012-12-25 Research In Motion Limited System and method for analyzing movements of an electronic device using rotational movement data
EP2224314A1 (en) * 2009-02-27 2010-09-01 Research In Motion Limited System and method for analyzing movements of an electronic device using rotational movement data
US9052710B1 (en) 2009-03-20 2015-06-09 Exelis Inc. Manipulation control based upon mimic of human gestures
US20100271331A1 (en) * 2009-04-22 2010-10-28 Rachid Alameh Touch-Screen and Method for an Electronic Device
US20100271312A1 (en) * 2009-04-22 2010-10-28 Rachid Alameh Menu Configuration System and Method for Display on an Electronic Device
US8542186B2 (en) 2009-05-22 2013-09-24 Motorola Mobility Llc Mobile device with user interaction capability and method of operating same
US8294105B2 (en) * 2009-05-22 2012-10-23 Motorola Mobility Llc Electronic device with sensing assembly and method for interpreting offset gestures
US8269175B2 (en) * 2009-05-22 2012-09-18 Motorola Mobility Llc Electronic device with sensing assembly and method for detecting gestures of geometric shapes
US8619029B2 (en) 2009-05-22 2013-12-31 Motorola Mobility Llc Electronic device with sensing assembly and method for interpreting consecutive gestures
US8344325B2 (en) * 2009-05-22 2013-01-01 Motorola Mobility Llc Electronic device with sensing assembly and method for detecting basic gestures
US8391719B2 (en) * 2009-05-22 2013-03-05 Motorola Mobility Llc Method and system for conducting communication between mobile devices
US8788676B2 (en) 2009-05-22 2014-07-22 Motorola Mobility Llc Method and system for controlling data transmission to or from a mobile device
US8304733B2 (en) * 2009-05-22 2012-11-06 Motorola Mobility Llc Sensing assembly for mobile device
US8682608B2 (en) * 2009-05-27 2014-03-25 Panasonic Corporation Behavior recognition apparatus
US8872767B2 (en) * 2009-07-07 2014-10-28 Microsoft Corporation System and method for converting gestures into digital graffiti
US8319170B2 (en) * 2009-07-10 2012-11-27 Motorola Mobility Llc Method for adapting a pulse power mode of a proximity sensor
US20110019105A1 (en) * 2009-07-27 2011-01-27 Echostar Technologies L.L.C. Verification of symbols received through a touchpad of a remote control device in an electronic system to allow access to system functions
US20110054833A1 (en) * 2009-09-02 2011-03-03 Apple Inc. Processing motion sensor data using accessible templates
US8341558B2 (en) * 2009-09-16 2012-12-25 Google Inc. Gesture recognition on computing device correlating input to a template
FR2950713A1 (fr) * 2009-09-29 2011-04-01 Movea Sa Systeme et procede de reconnaissance de gestes
US9174123B2 (en) * 2009-11-09 2015-11-03 Invensense, Inc. Handheld computer systems and techniques for character and command recognition related to human movements
KR101647722B1 (ko) * 2009-11-13 2016-08-23 엘지전자 주식회사 영상표시장치 및 그 동작방법
US8622742B2 (en) * 2009-11-16 2014-01-07 Microsoft Corporation Teaching gestures with offset contact silhouettes
US20110117526A1 (en) * 2009-11-16 2011-05-19 Microsoft Corporation Teaching gesture initiation with registration posture guides
US8665227B2 (en) * 2009-11-19 2014-03-04 Motorola Mobility Llc Method and apparatus for replicating physical key function with soft keys in an electronic device
US8436821B1 (en) 2009-11-20 2013-05-07 Adobe Systems Incorporated System and method for developing and classifying touch gestures
US20130328770A1 (en) * 2010-02-23 2013-12-12 Muv Interactive Ltd. System for projecting content to a display surface having user-controlled size, shape and location/direction and apparatus and methods useful in conjunction therewith
US9880619B2 (en) * 2010-02-23 2018-01-30 Muy Interactive Ltd. Virtual reality system with a finger-wearable control
US8228292B1 (en) 2010-04-02 2012-07-24 Google Inc. Flipping for motion-based input
US8963845B2 (en) 2010-05-05 2015-02-24 Google Technology Holdings LLC Mobile device with temperature sensing capability and method of operating same
US9103732B2 (en) 2010-05-25 2015-08-11 Google Technology Holdings LLC User computer device with temperature sensing capabilities and method of operating same
US8751056B2 (en) 2010-05-25 2014-06-10 Motorola Mobility Llc User computer device with temperature sensing capabilities and method of operating same
US9285983B2 (en) * 2010-06-14 2016-03-15 Amx Llc Gesture recognition using neural networks
US8639020B1 (en) 2010-06-16 2014-01-28 Intel Corporation Method and system for modeling subjects from a depth map
US20120016641A1 (en) * 2010-07-13 2012-01-19 Giuseppe Raffa Efficient gesture processing
US20120095575A1 (en) * 2010-10-14 2012-04-19 Cedes Safety & Automation Ag Time of flight (tof) human machine interface (hmi)
JP2012098988A (ja) * 2010-11-04 2012-05-24 Sony Corp 画像処理装置および方法、並びにプログラム
US9870141B2 (en) * 2010-11-19 2018-01-16 Microsoft Technology Licensing, Llc Gesture recognition
US12101354B2 (en) * 2010-11-29 2024-09-24 Biocatch Ltd. Device, system, and method of detecting vishing attacks
US10621585B2 (en) 2010-11-29 2020-04-14 Biocatch Ltd. Contextual mapping of web-pages, and generation of fraud-relatedness score-values
US10970394B2 (en) 2017-11-21 2021-04-06 Biocatch Ltd. System, device, and method of detecting vishing attacks
US10685355B2 (en) 2016-12-04 2020-06-16 Biocatch Ltd. Method, device, and system of detecting mule accounts and accounts used for money laundering
US10949514B2 (en) 2010-11-29 2021-03-16 Biocatch Ltd. Device, system, and method of differentiating among users based on detection of hardware components
US11210674B2 (en) * 2010-11-29 2021-12-28 Biocatch Ltd. Method, device, and system of detecting mule accounts and accounts used for money laundering
US10404729B2 (en) 2010-11-29 2019-09-03 Biocatch Ltd. Device, method, and system of generating fraud-alerts for cyber-attacks
US10298614B2 (en) * 2010-11-29 2019-05-21 Biocatch Ltd. System, device, and method of generating and managing behavioral biometric cookies
US10586036B2 (en) 2010-11-29 2020-03-10 Biocatch Ltd. System, device, and method of recovery and resetting of user authentication factor
US10728761B2 (en) 2010-11-29 2020-07-28 Biocatch Ltd. Method, device, and system of detecting a lie of a user who inputs data
US10776476B2 (en) 2010-11-29 2020-09-15 Biocatch Ltd. System, device, and method of visual login
US10069837B2 (en) * 2015-07-09 2018-09-04 Biocatch Ltd. Detection of proxy server
US10949757B2 (en) 2010-11-29 2021-03-16 Biocatch Ltd. System, device, and method of detecting user identity based on motor-control loop model
US10037421B2 (en) * 2010-11-29 2018-07-31 Biocatch Ltd. Device, system, and method of three-dimensional spatial user authentication
US10917431B2 (en) * 2010-11-29 2021-02-09 Biocatch Ltd. System, method, and device of authenticating a user based on selfie image or selfie video
US10897482B2 (en) 2010-11-29 2021-01-19 Biocatch Ltd. Method, device, and system of back-coloring, forward-coloring, and fraud detection
US10262324B2 (en) 2010-11-29 2019-04-16 Biocatch Ltd. System, device, and method of differentiating among users based on user-specific page navigation sequence
US10474815B2 (en) 2010-11-29 2019-11-12 Biocatch Ltd. System, device, and method of detecting malicious automatic script and code injection
US11269977B2 (en) 2010-11-29 2022-03-08 Biocatch Ltd. System, apparatus, and method of collecting and processing data in electronic devices
US10747305B2 (en) 2010-11-29 2020-08-18 Biocatch Ltd. Method, system, and device of authenticating identity of a user of an electronic device
US20190158535A1 (en) * 2017-11-21 2019-05-23 Biocatch Ltd. Device, System, and Method of Detecting Vishing Attacks
US10834590B2 (en) 2010-11-29 2020-11-10 Biocatch Ltd. Method, device, and system of differentiating between a cyber-attacker and a legitimate user
US11223619B2 (en) 2010-11-29 2022-01-11 Biocatch Ltd. Device, system, and method of user authentication based on user-specific characteristics of task performance
US9569002B2 (en) * 2010-12-17 2017-02-14 Blackberry Limited Portable electronic device having a sensor arrangement for gesture recognition
US8786547B2 (en) * 2010-12-23 2014-07-22 Microsoft Corporation Effects of gravity on gestures
US20150019459A1 (en) * 2011-02-16 2015-01-15 Google Inc. Processing of gestures related to a wireless user device and a computing device
ES2693060T3 (es) * 2011-03-17 2018-12-07 Ssi Schäfer Automation Gmbh Control y supervisión de una instalación de almacén y preparación de pedidos mediante movimiento y voz
KR20140033025A (ko) * 2011-03-25 2014-03-17 오블롱 인더스트리즈, 인크 비전 기반 손 추적기를 초기화하기 위한 고속 손가락끝 검출
US10223743B2 (en) * 2011-03-29 2019-03-05 Blackberry Limited Communication system providing near field communication (NFC) transaction features and related methods
US8863275B2 (en) * 2011-03-30 2014-10-14 Elwha Llc Access restriction in response to determining device transfer
US20120254809A1 (en) * 2011-03-31 2012-10-04 Nokia Corporation Method and apparatus for motion gesture recognition
US8873841B2 (en) 2011-04-21 2014-10-28 Nokia Corporation Methods and apparatuses for facilitating gesture recognition
JP6074170B2 (ja) 2011-06-23 2017-02-01 インテル・コーポレーション 近距離動作のトラッキングのシステムおよび方法
US11048333B2 (en) 2011-06-23 2021-06-29 Intel Corporation System and method for close-range movement tracking
CN103034324A (zh) * 2011-09-30 2013-04-10 德信互动科技(北京)有限公司 人机互动系统和方法
JP2013101526A (ja) * 2011-11-09 2013-05-23 Sony Corp 情報処理装置、表示制御方法、およびプログラム
US8963885B2 (en) 2011-11-30 2015-02-24 Google Technology Holdings LLC Mobile device for interacting with an active stylus
US9063591B2 (en) 2011-11-30 2015-06-23 Google Technology Holdings LLC Active styluses for interacting with a mobile device
US8958631B2 (en) 2011-12-02 2015-02-17 Intel Corporation System and method for automatically defining and identifying a gesture
JP2013125373A (ja) * 2011-12-14 2013-06-24 Sony Corp 情報処理装置および方法、並びにプログラム
US8693731B2 (en) 2012-01-17 2014-04-08 Leap Motion, Inc. Enhanced contrast for object detection and characterization by optical imaging
US10691219B2 (en) 2012-01-17 2020-06-23 Ultrahaptics IP Two Limited Systems and methods for machine control
US8638989B2 (en) 2012-01-17 2014-01-28 Leap Motion, Inc. Systems and methods for capturing motion in three-dimensional space
US11493998B2 (en) 2012-01-17 2022-11-08 Ultrahaptics IP Two Limited Systems and methods for machine control
US12260023B2 (en) 2012-01-17 2025-03-25 Ultrahaptics IP Two Limited Systems and methods for machine control
US9070019B2 (en) 2012-01-17 2015-06-30 Leap Motion, Inc. Systems and methods for capturing motion in three-dimensional space
US9679215B2 (en) 2012-01-17 2017-06-13 Leap Motion, Inc. Systems and methods for machine control
US9501152B2 (en) 2013-01-15 2016-11-22 Leap Motion, Inc. Free-space user interface and control using virtual constructs
GB201203832D0 (en) * 2012-03-05 2012-04-18 Elliptic Laboratories As User input system
WO2013175484A2 (en) * 2012-03-26 2013-11-28 Tata Consultancy Services Limited A multimodal system and method facilitating gesture creation through scalar and vector data
US9477303B2 (en) 2012-04-09 2016-10-25 Intel Corporation System and method for combining three-dimensional tracking with a three-dimensional display for a user interface
DE202013100255U1 (de) * 2012-06-20 2013-04-12 Samsung Electronics Co. Ltd. Anzeigevorrichtung, Fernbedienungsvorrichtung sowie Bedienungsfunktion derselben
US9696547B2 (en) * 2012-06-25 2017-07-04 Microsoft Technology Licensing, Llc Mixed reality system learned input and functions
US10158898B2 (en) 2012-07-26 2018-12-18 Comcast Cable Communications, Llc Customized options for consumption of content
KR101978967B1 (ko) 2012-08-01 2019-05-17 삼성전자주식회사 제스처의 방향에 기초하여 상기 제스처를 인식하는 제스처 인식 장치와 제스처 인식 방법
US9285893B2 (en) 2012-11-08 2016-03-15 Leap Motion, Inc. Object detection and tracking with variable-field illumination devices
US10609285B2 (en) 2013-01-07 2020-03-31 Ultrahaptics IP Two Limited Power consumption in motion-capture systems
US9626015B2 (en) 2013-01-08 2017-04-18 Leap Motion, Inc. Power consumption in motion-capture systems with audio and optical signals
US10241639B2 (en) 2013-01-15 2019-03-26 Leap Motion, Inc. Dynamic user interactions for display control and manipulation of display objects
US9459697B2 (en) 2013-01-15 2016-10-04 Leap Motion, Inc. Dynamic, free-space user interactions for machine control
US20140198040A1 (en) * 2013-01-16 2014-07-17 Lenovo (Singapore) Pte, Ltd. Apparatus, system and method for self-calibration of indirect pointing devices
US9442570B2 (en) 2013-03-13 2016-09-13 Google Technology Holdings LLC Method and system for gesture recognition
WO2014200589A2 (en) 2013-03-15 2014-12-18 Leap Motion, Inc. Determining positional information for an object in space
US10620709B2 (en) 2013-04-05 2020-04-14 Ultrahaptics IP Two Limited Customized gesture interpretation
US9916009B2 (en) 2013-04-26 2018-03-13 Leap Motion, Inc. Non-tactile interface systems and methods
US9747696B2 (en) 2013-05-17 2017-08-29 Leap Motion, Inc. Systems and methods for providing normalized parameters of motions of objects in three-dimensional space
US9221170B2 (en) * 2013-06-13 2015-12-29 GM Global Technology Operations LLC Method and apparatus for controlling a robotic device via wearable sensors
US9020194B2 (en) * 2013-06-14 2015-04-28 Qualcomm Incorporated Systems and methods for performing a device action based on a detected gesture
US10846942B1 (en) 2013-08-29 2020-11-24 Ultrahaptics IP Two Limited Predictive information for free space gesture control and communication
TWI567592B (zh) * 2013-09-03 2017-01-21 緯創資通股份有限公司 手勢辨識方法及穿戴式裝置
US9405375B2 (en) 2013-09-13 2016-08-02 Qualcomm Incorporated Translation and scale invariant features for gesture recognition
US9632572B2 (en) 2013-10-03 2017-04-25 Leap Motion, Inc. Enhanced field of view to augment three-dimensional (3D) sensory space for free-space gesture interpretation
US10168873B1 (en) 2013-10-29 2019-01-01 Leap Motion, Inc. Virtual interactions for machine control
US9996638B1 (en) 2013-10-31 2018-06-12 Leap Motion, Inc. Predictive information for free space gesture control and communication
US9996797B1 (en) 2013-10-31 2018-06-12 Leap Motion, Inc. Interactions with virtual objects for machine control
US9613262B2 (en) 2014-01-15 2017-04-04 Leap Motion, Inc. Object detection and tracking for providing a virtual device experience
JP6273872B2 (ja) * 2014-02-03 2018-02-07 富士通株式会社 プログラム、情報処理方法、情報処理システム、ウェアラブル装置
EP2911089B1 (en) * 2014-02-25 2018-04-04 Karlsruher Institut für Technologie Method and system for handwriting and gesture recognition
US20150286279A1 (en) * 2014-04-07 2015-10-08 InvenSense, Incorporated Systems and methods for guiding a user during calibration of a sensor
US9741169B1 (en) 2014-05-20 2017-08-22 Leap Motion, Inc. Wearable augmented reality devices with object detection and tracking
US9958946B2 (en) * 2014-06-06 2018-05-01 Microsoft Technology Licensing, Llc Switching input rails without a release command in a natural user interface
JP2016038889A (ja) 2014-08-08 2016-03-22 リープ モーション, インコーポレーテッドLeap Motion, Inc. モーション感知を伴う拡張現実
EP3180675B1 (en) * 2014-08-16 2021-05-26 Google LLC Identifying gestures using motion data
US10660039B1 (en) 2014-09-02 2020-05-19 Google Llc Adaptive output of indications of notification data
US11308928B2 (en) 2014-09-25 2022-04-19 Sunhouse Technologies, Inc. Systems and methods for capturing and interpreting audio
EP3198247B1 (en) 2014-09-25 2021-03-17 Sunhouse Technologies, Inc. Device for capturing vibrations produced by an object and system for capturing vibrations produced by a drum.
JP6396186B2 (ja) * 2014-11-17 2018-09-26 ラピスセミコンダクタ株式会社 半導体装置、携帯端末装置および操作検知方法
AU2015375530B2 (en) * 2015-01-09 2021-04-15 Razer (Asia-Pacific) Pte. Ltd. Gesture recognition devices and gesture recognition methods
US10039975B2 (en) 2015-01-13 2018-08-07 Disney Enterprises, Inc. Techniques for representing imaginary participants in an immersive play environment
US9855497B2 (en) 2015-01-20 2018-01-02 Disney Enterprises, Inc. Techniques for providing non-verbal speech recognition in an immersive playtime environment
US10265621B2 (en) 2015-01-20 2019-04-23 Disney Enterprises, Inc. Tracking specific gestures relative to user movement
US9577992B2 (en) * 2015-02-04 2017-02-21 Aerendir Mobile Inc. Data encryption/decryption using neuro and neuro-mechanical fingerprints
US10429923B1 (en) 2015-02-13 2019-10-01 Ultrahaptics IP Two Limited Interaction engine for creating a realistic experience in virtual reality/augmented reality environments
US9696795B2 (en) 2015-02-13 2017-07-04 Leap Motion, Inc. Systems and methods of creating a realistic grab experience in virtual reality/augmented reality environments
US9936128B2 (en) 2015-05-20 2018-04-03 Google Llc Automatic detection of panoramic gestures
GB2539705B (en) 2015-06-25 2017-10-25 Aimbrain Solutions Ltd Conditional behavioural biometrics
US9804679B2 (en) 2015-07-03 2017-10-31 Google Inc. Touchless user interface navigation using gestures
US10599919B2 (en) * 2015-12-31 2020-03-24 Microsoft Technology Licensing, Llc Detection of hand gestures using gesture language discrete values
US20170199586A1 (en) * 2016-01-08 2017-07-13 16Lab Inc. Gesture control method for interacting with a mobile or wearable device utilizing novel approach to formatting and interpreting orientation data
US10345914B2 (en) 2016-01-26 2019-07-09 Infinity Augmented Reality Israel Ltd. Method and system for generating a synthetic database of postures and gestures
GB2552032B (en) 2016-07-08 2019-05-22 Aimbrain Solutions Ltd Step-up authentication
US10540491B1 (en) 2016-10-25 2020-01-21 Wells Fargo Bank, N.A. Virtual and augmented reality signatures
US10579784B2 (en) 2016-11-02 2020-03-03 Biocatch Ltd. System, device, and method of secure utilization of fingerprints for user authentication
TWI620097B (zh) * 2016-12-30 2018-04-01 漳州立達信光電子科技有限公司 輸入方法及使用該輸入方法的觸控裝置、手勢偵測裝置、電腦可讀取紀錄媒體、及電腦程式產品
US9785250B1 (en) * 2017-05-15 2017-10-10 Newtonoid Technologies, L.L.C. Intelligent gesture based security system and method
US10397262B2 (en) 2017-07-20 2019-08-27 Biocatch Ltd. Device, system, and method of detecting overlay malware
US11771283B2 (en) * 2017-12-06 2023-10-03 BISSELL , Inc. Method and system for manual control of autonomous floor cleaner
US11875012B2 (en) 2018-05-25 2024-01-16 Ultrahaptics IP Two Limited Throwable interface for augmented reality and virtual reality environments
EP3841497B1 (en) 2018-08-24 2025-02-26 Truu, Inc. Machine learning-based platform for user identification
US20230266831A1 (en) * 2020-07-10 2023-08-24 Telefonaktiebolaget Lm Ericsson (Publ) Method and device for obtaining user input
US11606353B2 (en) 2021-07-22 2023-03-14 Biocatch Ltd. System, device, and method of generating and utilizing one-time passwords
US11604512B1 (en) * 2022-01-05 2023-03-14 City University Of Hong Kong Fingertip-motion sensing device and handwriting recognition system using the same

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0516862A1 (en) * 1990-12-19 1992-12-09 Kabushiki Kaisha Yaskawa Denki Apparatus for inputting multidimensional information
US20020118880A1 (en) * 2000-11-02 2002-08-29 Che-Bin Liu System and method for gesture interface
US20030076293A1 (en) * 2000-03-13 2003-04-24 Hans Mattsson Gesture recognition system
US20030156756A1 (en) * 2002-02-15 2003-08-21 Gokturk Salih Burak Gesture recognition system using depth perceptive sensors
US20040037463A1 (en) * 2002-01-28 2004-02-26 Calhoun Christopher L. Recognizing multi-stroke symbols

Family Cites Families (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS59117509A (ja) * 1982-12-24 1984-07-06 Mitsui Petrochem Ind Ltd オレフインの重合方法
US4525555A (en) * 1983-01-14 1985-06-25 Nippon Oil Company, Limited Process for preparing polyolefins
US5454043A (en) * 1993-07-30 1995-09-26 Mitsubishi Electric Research Laboratories, Inc. Dynamic and static hand gesture recognition through low-level image analysis
JP3630712B2 (ja) * 1994-02-03 2005-03-23 キヤノン株式会社 ジェスチャー入力方法及びその装置
US5741182A (en) * 1994-06-17 1998-04-21 Sports Sciences, Inc. Sensing spatial movement
US6045364A (en) * 1997-05-19 2000-04-04 Dugan; Brian M. Method and apparatus for teaching proper swing tempo
KR19990011180A (ko) * 1997-07-22 1999-02-18 구자홍 화상인식을 이용한 메뉴 선택 방법
US6162123A (en) * 1997-11-25 2000-12-19 Woolston; Thomas G. Interactive electronic sword game
US6249606B1 (en) * 1998-02-19 2001-06-19 Mindmaker, Inc. Method and system for gesture category recognition and training using a feature vector
US6154558A (en) * 1998-04-22 2000-11-28 Hsieh; Kuan-Hong Intention identification method
US6375572B1 (en) * 1999-10-04 2002-04-23 Nintendo Co., Ltd. Portable game apparatus with acceleration sensor and information storage medium storing a game progam
WO2003001340A2 (en) * 2001-06-22 2003-01-03 Motion Sense Corporation Gesture recognition system and method
US7038659B2 (en) * 2002-04-06 2006-05-02 Janusz Wiktor Rajkowski Symbol encoding apparatus and method
US20030214481A1 (en) * 2002-05-14 2003-11-20 Yongming Xiong Finger worn and operated input device and method of use
US20040001113A1 (en) * 2002-06-28 2004-01-01 John Zipperer Method and apparatus for spline-based trajectory classification, gesture detection and localization
EP1408443B1 (en) * 2002-10-07 2006-10-18 Sony France S.A. Method and apparatus for analysing gestures produced by a human, e.g. for commanding apparatus by gesture recognition
JP4007899B2 (ja) * 2002-11-07 2007-11-14 オリンパス株式会社 運動検出装置
JP4142460B2 (ja) * 2003-01-31 2008-09-03 オリンパス株式会社 運動検出装置
US20050168443A1 (en) * 2004-01-29 2005-08-04 Ausbeck Paul J.Jr. Method and apparatus for producing one-dimensional signals with a two-dimensional pointing device
US7301529B2 (en) * 2004-03-23 2007-11-27 Fujitsu Limited Context dependent gesture response

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0516862A1 (en) * 1990-12-19 1992-12-09 Kabushiki Kaisha Yaskawa Denki Apparatus for inputting multidimensional information
US20030076293A1 (en) * 2000-03-13 2003-04-24 Hans Mattsson Gesture recognition system
US20020118880A1 (en) * 2000-11-02 2002-08-29 Che-Bin Liu System and method for gesture interface
US20040037463A1 (en) * 2002-01-28 2004-02-26 Calhoun Christopher L. Recognizing multi-stroke symbols
US20030156756A1 (en) * 2002-02-15 2003-08-21 Gokturk Salih Burak Gesture recognition system using depth perceptive sensors

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
DAVIS J ET AL: "Recognizing hand gestures", COMPUTER VISION - ECCV'94. THIRD EUROPEAN CONFERENCE ON COMPUTER VISION. PROCEEDINGS. VOL.I SPRINGER-VERLAG BERLIN, GERMANY, 1994, pages 331 - 340, XP002360860, ISBN: 3-540-57956-7 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109816699A (zh) * 2019-01-30 2019-05-28 华通科技有限公司 一种基于背景抑制帧间差分法的云台角度计算方法
CN109816699B (zh) * 2019-01-30 2021-07-27 国网智能科技股份有限公司 一种基于背景抑制帧间差分法的云台角度计算方法

Also Published As

Publication number Publication date
DE602005009568D1 (de) 2008-10-16
EP1810217A1 (en) 2007-07-25
GB0423225D0 (en) 2004-11-24
GB2419433A (en) 2006-04-26
ATE407409T1 (de) 2008-09-15
EP1810217B1 (en) 2008-09-03
US20080192005A1 (en) 2008-08-14

Similar Documents

Publication Publication Date Title
EP1810217B1 (en) Automated gesture recognition
Wang et al. Controlling object hand-over in human–robot collaboration via natural wearable sensing
US8010911B2 (en) Command input method using motion recognition device
JP5631535B2 (ja) ジェスチャベースの制御システムのためのシステムおよび方法
Shukor et al. A new data glove approach for Malaysian sign language detection
Bernardin et al. A sensor fusion approach for recognizing continuous human grasping sequences using hidden markov models
CN102246125B (zh) 具有运动姿态识别的移动设备
JP6064280B2 (ja) ジェスチャを認識するためのシステムおよび方法
US20100023314A1 (en) ASL Glove with 3-Axis Accelerometers
TWI569176B (zh) 手寫軌跡識別方法與系統
EP2304527A1 (en) Gesture-based control system for vehicle interfaces
US11175729B2 (en) Orientation determination based on both images and inertial measurement units
Noh et al. A decade of progress in human motion recognition: A comprehensive survey from 2010 to 2020
JP5788853B2 (ja) ジェスチャベースの制御システムのためのシステムおよび方法
Romero et al. Modeling and evaluation of human-to-robot mapping of grasps
Park et al. Real-time 3D pointing gesture recognition in mobile space
Hoshino Hand gesture interface for entertainment games
Panduranga et al. Dynamic hand gesture recognition system: a short survey
Bernardin et al. A hidden markov model based sensor fusion approach for recognizing continuous human grasping sequences
Abdullah et al. Research and development of IMU sensors-based approach for sign language gesture recognition
Das et al. Hand Gesture-Based Recognition System for Human–Computer Interaction
CN114327042B (zh) 检测手套、手势追踪方法、ar设备及按键方法
Ousmer A Systematic Procedure for Comparative Testing of 3D Hand Gesture Template-Based Recognizers in Multiple Contexts of Use
Dutta et al. Intelligent Gesture Analysis, Recognition and Computation–Wizard
Wook et al. Gesture based Input Device: An All Inertial Approach

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KM KP KR KZ LC LK LR LS LT LU LV LY MA MD MG MK MN MW MX MZ NA NG NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SM SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): BW GH GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU LV MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2005794750

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 2005794750

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 11577694

Country of ref document: US

WWG Wipo information: grant in national office

Ref document number: 2005794750

Country of ref document: EP