JP2015524977A - Automatic gesture recognition for sensor systems - Google Patents

Abstract

Automatic detection to detect one or more gesture related signals using a plurality of associated detection sensors and to determine whether the gesture corresponds to one of a predetermined set of gestures. Evaluating a gesture detected from one or more gesture-related signals using a recognition technique. In one embodiment, evaluating the gesture includes determining the start and stop of the gesture. In one embodiment, each gesture is represented by one or more hidden Markov models (HMMs).

Description

(Citation of related application)
This application claims priority from US Provisional Patent Application No. 61 / 684,039 (filed Aug. 16, 2012). The application is incorporated by reference in its entirety as if fully set forth herein.

(Technical field)
The present disclosure relates to methods and systems for sensor systems, and in particular, to automatic gesture recognition for such systems. More specifically, the present disclosure relates to a gesture recognition system that is invariant to gesture transformation and / or scaling and for a relatively large distance between a hand / finger and a sensor system.

  Systems for non-contact detection and recognition of gestures are known. Such systems may be based on capacitive (eg, surface capacitive, projected capacitive, mutual capacitive, or self-capacitive), infrared, optical imaging, distributed signal, ultrasound, or acoustic pulse recognition sensor technology. .

  A capacitive sensor system can be realized, for example, by generating an alternating electric field and measuring the potential difference (ie, voltage) obtained at the sensor electrodes in the electric field. Depending on the implementation, a single electrode can be used, or a transmission electrode and one or more receiving electrodes can be used. The voltage at the sensor electrode is a measure of the capacitance between the sensor electrode and its electrical environment. That is, it is affected by an object such as a human finger or hand that can specifically make a gesture within the detection space provided by the electrode array. Furthermore, from this voltage, for example, finger distance or gesture can be inferred. This information can be used for a human-machine interface.

  Considering a 3D positioning system, a straightforward and highest level approach for gesture detection takes an x / y position estimate as input to an automatic gesture recognition system and uses z-distance for start / stop criteria It is to be. The position estimate is the result of one or more stages in which the sensor data is processed (calibration, non-linear relationship between calibrated sensor values and distance, position that is a trigonometric function of distance), each stage with extra uncertainty Using x / y / z estimates would be more error-prone than using data from earlier processing stages.

  These and other aspects of the present disclosure will be further appreciated and understood when considered in conjunction with the following description and the accompanying drawings. However, it is to be understood that the following description illustrates various embodiments of the present disclosure and numerous specific details thereof, but is provided by way of illustration and not limitation. Many substitutions, modifications, additions, and / or rearrangements may be made within the scope of the present disclosure without departing from the spirit thereof, and the disclosure includes all such substitutions, modifications, additions, And / or including rearrangement.

  A method for gesture recognition according to an embodiment uses a plurality of associated detection sensors to detect one or more gesture related signals, and the gesture is one of a predetermined set of gestures. Evaluating a gesture detected from one or more gesture-related signals using an automatic recognition technique to determine whether it corresponds. In some embodiments, the initiation of the gesture decreases the distance between the target object and the at least one sensor, increases the distance between the target object and the at least one sensor, and the predetermined plurality of Determined when short-term variation over signal samples or equivalent rating measure is less than threshold. In some embodiments, cessation of a gesture reduces the distance between the target object and all sensors at a given point in time and / or short-term variation or equivalent assessment across a given plurality of signal samples. A measure is determined if it is less than a threshold and / or the signal change is less than a predetermined threshold over a predetermined number of signal samples after a given time point.

  In some embodiments, each gesture is represented by one or more hidden Markov models (HMMs). In some embodiments, evaluating the gesture includes evaluating a probability rating measure for one or more HMMs. In some embodiments, the features with which the HMM observation matrix is associated are unquantized or quantized sensor signal levels, x / y / z position, distance, direction, orientation, angle, and / or their time. Is a first, second, or higher order derivative, or any combination thereof. In some embodiments, the feature is the first derivative of the sensor signal level that is quantized to two quantization levels.

  A system for gesture recognition according to an embodiment includes a sensor array for detecting one or more gesture-related signals using a plurality of associated detection sensors, and a gesture of a predetermined set of gestures. A module for evaluating gestures detected from one or more gesture-related signals using automatic recognition techniques to determine whether one is supported.

  In some embodiments, the initiation of the gesture decreases the distance between the target object and the at least one sensor, increases the distance between the target object and the at least one sensor, and the predetermined plurality of Determined when short-term variation over signal samples or equivalent rating measure is less than threshold. In some embodiments, cessation of a gesture reduces the distance between the target object and all sensors at a given point in time and / or short-term variation or equivalent assessment across a given plurality of signal samples. Determined when the measure is less than a threshold and / or the signal change is less than a predetermined threshold over a predetermined number of signal samples after a given time point.

  A computer-readable medium according to an embodiment uses a plurality of detection sensors to receive one or more gesture-related signals and whether the gesture corresponds to one of a predetermined set of gestures. To determine, includes one or more non-transitory machine readable program instructions for evaluating gestures detected from one or more gesture related signals using automatic recognition techniques.

  In some embodiments, the initiation of the gesture decreases the distance between the target object and the at least one sensor, increases the distance between the target object and the at least one sensor, and the predetermined plurality of Determined when short-term variation over signal samples or equivalent rating measure is less than threshold. In some embodiments, cessation of a gesture reduces the distance between the target object and all sensors at a given point in time and / or short-term variation or equivalent assessment across a given plurality of signal samples. Determined when the measure is less than a threshold and / or the signal change is less than a predetermined threshold over a predetermined number of signal samples after a given time point.

The accompanying drawings, which form a part of this specification, are included to depict certain aspects of the present disclosure. It should be noted that the features illustrated in the drawings are not necessarily drawn to scale. A more complete understanding of the present disclosure and its advantages may be obtained by reference to the following description, considered in conjunction with the accompanying drawings, wherein like reference numerals indicate like features.
FIG. 1 is a schematic diagram that schematically illustrates a keyboard, including an exemplary capacitive sensing system. FIG. 2 is a schematic diagram illustrating the electrode layout of an exemplary capacitive sensing system. FIG. 3 illustrates the relationship between the distance from the finger to the electrode and the measured value of the electrode signal. FIG. 4 illustrates an exemplary defined gesture. FIG. 5A illustrates an exemplary hidden Markov model observation probability matrix for a check gesture. FIG. 5B illustrates an exemplary check gesture. FIG. 6A illustrates an exemplary determination of a start event. FIG. 6B illustrates an exemplary clockwise circular gesture. FIG. 7 illustrates an exemplary stop criterion. FIG. 8 illustrates a state diagram and state transition probability matrix of an exemplary linear and circular hidden Markov model. FIG. 9 illustrates the initial state distribution and transition matrix of an exemplary linear and circular hidden Markov model. FIG. 10 is a flowchart illustrating a process flow according to an embodiment. FIG. 11 illustrates an exemplary circular gesture and the resulting feature vector. FIG. 12 illustrates exemplary test results for gesture recognition according to an embodiment. FIG. 13 illustrates an exemplary system according to an embodiment.

  The present disclosure and its various features and advantageous details are illustrated in the accompanying drawings and detailed in the following description, and are therefore more fully described with reference to non-limiting embodiments. Explained. However, it should be understood that the detailed description and specific examples, while indicating the preferred embodiment, are given by way of illustration only and not limitation. Descriptions of known programming techniques, computer software, hardware, operating platforms, and protocols may be omitted so as not to unnecessarily obscure the details of the present disclosure. Various substitutions, modifications, additions, and / or rearrangements within the spirit and / or scope of the underlying concepts of the present invention will be apparent to those skilled in the art from this disclosure.

  According to an embodiment, the system is provided with reliable automatic recognition of predefined hand gestures performed in front of a sensor system with two or more sensor electrodes providing measurement data. According to embodiments, recognition is invariant to gesture transformation and / or scaling and is for a wide range of z-axis distances between the hand / finger and the respective sensor system. According to various embodiments, a hidden Markov recognizer with improved feature extraction and gesture detection for hand gesture recognition can be provided.

  For convenience, embodiments are described in the context of a capacitive sensor system, but any suitable sensor system that can provide distance or depth information may be employed. Furthermore, although the embodiments are described in the context of a three-dimensional (x / y / z) sensor system, the proposed method for gesture recognition also applies to a two-dimensional sensor system (z = 0). Is possible. Examples of suitable sensor systems are capacitive (eg, surface capacitive, projected capacitive, mutual capacitive, or self-capacitive), surface acoustic waves, infrared, optical or video imaging, one or more photodiodes, distributed Including, but not limited to, those based on signal, ultrasound, or acoustic pulse recognition sensor technology. Accordingly, the figures are merely exemplary.

  Referring now to the drawings and with particular attention to FIG. 1, the sensor configuration is shown and generally identified by reference numeral 100. More specifically, FIG. 1 illustrates a user input object such as a human finger 102 on a PC keyboard 103 with a built-in sensor system such as a capacitive sensor system. Regions 104a-104d identify the position of the sensor electrode under the key. A line 106 between the finger 102 and the rectangular areas 104a-104d indicates the shortest path from the fingertip to the sensor electrode. The rectangular area 107 limited in the x / y dimension by the areas 104a to 104d is represented as “active area”, and the cubic space 108 above the active area is represented as “active space”.

  Although illustrated as a sensor system incorporated into a stand-alone keyboard, embodiments include cell phones, MP3 players, PDAs, tablet computers, computers, remote controls, radios, computer mice, touch-sensitive displays, and televisions, etc. Note that it may be employed with a sensor system that is itself a user interface or associated with an electronic device that has a user interface. The user input object may be something like a stylus (eg, a small pen-shaped instrument) or a digital pen. The user input object can also be the user's hand or finger in this sense.

  FIG. 2 illustrates the keyboard 103 with some of the keys removed, showing the sensor system in more detail. Specifically, a basic printed circuit board is shown that holds the sensor system with bottom (EB) 104a, left (EL) 104d, top (ET) 104c, and right (ER) 104b electrodes.

  FIG. 3 qualitatively shows the magnitude of the sensor measurement (vertical axis) as a function of the distance between the sensor electrode and the fingertip (horizontal axis). As the distance increases, the measured value decreases. For equivalent embodiments of the invention, the sensor value increases with distance. The asymptotic offset 302 for distances increasing towards infinity is generally unknown and may change over time.

  In some embodiments, the sensor system provides discrete time measurements for each sensor electrode at a predetermined sampling rate f_s. Typically, in the pre-processing stage, the sensor signal is a low-pass filter to match the hand gesture frequency range, which is typically up to f_max = <15 Hz to 20 Hz, depending on the application. And f_s> 2 * f_max. Considering these signals for one or more electrodes, the goal is to provide reliable automatic recognition of hand gestures made on the keyboard or active area, respectively.

  In FIG. 4, according to an embodiment, specific examples of eight gestures are defined. However, according to other embodiments, other gestures may be defined. In the illustrated embodiment, the gesture has four “flicks”: fast linear finger movements in four major directions (right 402, left 404, top 406, bottom 408), clockwise 410 and counterclockwise. A clockwise 412 circle, a “confirm” or “check” gesture 414, and a cancel gesture 416 that is a series of continuous short left <-> right movements.

  Common approaches in pattern recognition, such as automatic speech recognition and handwritten character recognition, each use a hidden Markov model to estimate the most likely one of a predefined set of phonemes, words, or characters. (HMM) and trellis calculation. HMMλ: = (A, B, π) is a finite stochastic state machine with N states that can generate M output symbols or “features”. This is represented by three parameters: a state transition probability distribution matrix A of size N × N, an N × M symbol probability distribution matrix B containing the probability of each state feature map, and an N × 1 initial state distribution vector π. Is done. A trellis is a graph that adds the dimension of time to a state diagram and enables efficient computation. The embodiments discussed herein relate to primary HMMs, but in other embodiments higher order HMMs may be used.

  In gesture recognition, for example, each HMM represents a gesture. A gesture can be divided into gesture fragments whose features are represented by states in their assigned HMM. Features are extracted from sensor signals detected during a gesture, typically in equidistant discrete time instances. Such features may include, for example, first, second, or higher order derivatives with respect to unquantized or quantized signal levels, x / y / z position, distance, direction, orientation, angle, and / or their time. Contains a function, or any combination thereof. For example, if the feature utilized is the first derivative of x / y position with respect to time, i.e., x / y velocity, then the feature sequence obtained while detecting the gesture is the x / y velocity vector This is the sequence. The length of this sequence depends on the time duration of the gesture. When different people make the same gesture, eg, a “check” gesture, in front of the sensor system, the duration of the gesture, the speed of movement, and / or the shape of the gesture may be slightly different. Therefore, there will also be variations in the corresponding feature sequence.

  The HMM can incorporate these variations because it represents not only a single feature at each time instance, but also a probability distribution over all possible feature vectors. In order to obtain these probability distribution matrices B, as well as the probability of transitions between their states, each HMM needs to be trained using a feature sequence of a so-called training gesture set. During training, the HMM probability distribution is optimized to maximize the conditional probability of the training sequence taking this HMM into account. Thus, the HMM represents the feature sequence of all gestures for which it has been trained. The number N of states required for the HMM depends inter alia on the complexity of the corresponding gesture. For each state, it belongs to a single observation probability distribution (rows in matrix B), the more different features in the gesture feature sequence, the more states are needed to represent it. Is done. Additional details regarding the use of hidden Markov models in gesture recognition are incorporated herein by reference in their entirety as if fully set forth herein, “System and Method for Multidimensional”. No. 8,280,732 by the same applicant entitled “Gesture Analysis”.

  Depending on the application, different HMM model topologies, for example linear models (often with state transitions to the current or next state used in speech recognition) or circular models as shown in FIG. (Often used in image processing, the last state has a non-zero transition probability to the first state).

  According to embodiments, linear models are preferred for gestures with distinct start and end points, such as flicks 402, 404, 406, 408, and confirmation gesture 414 (FIG. 4). For gestures with unclear start and end points such as circles 410, 412, and cancel gesture 414, a circular model is preferred. In addition, fully connected models with non-zero transition probabilities between all states can be used, for example, for so-called “garbage” HMMs for gestures to be rejected.

  A frequently used HMM initialization prior to training is shown in FIG. 9, which reveals linear and circular properties in state transition matrix A. During HMM training, the non-zero inputs of the A and B matrices will change. However, the zero input will remain as such and therefore the model topology will also remain.

  The inputs to both the training algorithm that optimizes the matrices A and B for each gesture, as well as the recognition algorithm, are temporal sequences of features, each sequence corresponding to one execution of the gesture.

  Knowledge of gesture start and stop times may be important for recognition performance. And various embodiments are very simple for use with (but not limited to) HMM, but are invariant to scaling and transitions, and therefore very robust, start / stop Use a combination of criteria and signal features.

  FIGS. 5A and 5B show a feature sequence of a “check” gesture and examples of exemplary sensors EB, EL, ET, ER, where the feature vector is linear with respect to the time of distance between the fingertip and the four sensor electrodes. Consists of derivatives. Each column in the feature sequence matrix of FIG. 5A corresponds to a discrete time instance. For example, the value “−4.1” in matrix position (2, 2) indicates a relatively fast movement away from electrode E2 at time instance 2 and at the same time the value “0” in row 4. .4 "indicates a relatively slow movement toward electrode E4.

  For gesture start and stop detection, the following working assumptions can be made: A) A gesture begins either when entering the active space from the outside (or the active region if z = 0) or at the start of movement after a rest period in the active space (ie, the finger is No need to leave the active space in between). B) Terminate when a gesture rests in or leaves the active space, or when the probability that the gesture made is one of a predetermined set of gestures exceeds a threshold .

  As mentioned above, from these assumptions, the criteria for start and stop detection are inferred directly.

(Start detection :)
A1) The distance between the fingertip and at least one sensor is increased and the distance between the fingertip and at least one sensor is decreased (or the signal level of at least one electrode is increased and at least one The signal level of the other electrode is reduced). (A hand movement in active space at a constant z-altitude always leads to an increase in distance to at least one electrode and a decrease in distance to at least one other electrode.) For Z = 0 The 3D recognition problem is a 2D touch gesture recognition problem. Detecting the increase or decrease can include detecting a threshold distance or signal level change.

  A2) For example, a short-term variation in signal level over 10 signal samples or a similar evaluation measure must exceed a threshold (finger moving fast enough). If both criteria are met simultaneously, it is assumed that the start of the gesture is detected. Immediately after the detected gesture start, the validity of the start detection can be verified by checking the above (or similar) criteria again. If the validation is negative, gesture recognition is interrupted.

(Stop detection :)
B1) The distance from the hand to the electrode increases for all electrodes (or decreases the signal level of all sensors), for example over nD = 10 samples. (This is not possible for movement in active space at a constant z distance.)
B2) For example, the short-time variation of the sensor signal over nV = 10 signal samples or a similar rating scale should not exceed the threshold (the finger is moving slowly enough).

  B3) If the criterion B1) or B2) is satisfied with a discrete time instance T_E, the end of the gesture is detected as being at time T_E-nD or T_E-nV, respectively.

  If the time between the start and stop of a gesture is not within a predefined time interval, the gesture can be excluded from the set of possible gestures.

  FIG. 6A shows the determination of the start event of the clockwise circle gesture shown in FIG. 6B, with the following criteria applied: A1) The signal level of> = 1 electrode increases and the signal level of> = 1 other electrode decreases (corresponding to a finger moving in the active space). A2) Short-term fluctuations in sensor signal or similar rating scale exceeds threshold (finger moving fast enough).

  FIG. 7 shows gesture stop detection such that criterion B1) is applied. The signal level of all electrodes is reduced (this corresponds to a finger leaving the active space).

  FIG. 10 is a flowchart 1000 illustrating the overall gesture recognition process, including start / stop detection. Depending on the start of the process 1004, for each input signal sample (consisting of sensor values from all electrodes), first of all, gesture recognition with calculation of HMM probability is already active (ACTIVE = true), It is checked whether it is not operating (ACTIVE = false) (step 1004). In the latter case, the start criteria are evaluated (step 1008). If the start criteria are met (determined at step 1022), the ACTIVE flag is set to "true" (step 1024) and actual gesture recognition begins (step 1016). The probability of exceeding the threshold is determined (step 1020), and if the criteria are met, the recognized gesture is output (step 1014) and ACTIVE is set to false (step 1018). Otherwise, the process ends (step 1026).

  If ACTIVE is true for the input sample (step 1004), the stop criterion is evaluated (step 1006). If the stop criterion is met (step 1010), gesture recognition is complete, the result is evaluated (step 1012), output (step 1014), and ACTIVE is set to false (step 1018). Otherwise, gesture recognition continues (step 1016).

  FIG. 11 shows an example of the resulting feature vector, which is a finger drawing a clockwise circle at a constant z distance above the active area, and the signal level change for all electrodes. In this time step 1, the finger moves towards the electrodes ER and EB and away from the electrodes EL and ET.

  This leads to an increase in the signal level of the electrodes ER and EB and a decrease in the signal level of the electrodes EL and ET. At time step 3, the finger still moves away from the ET and towards the EL, but here it moves away from the ER and towards the EL. This leads to an increase in the signal level of the electrodes EB and EL and a decrease in the signals of the electrodes ET and ER.

  According to some embodiments, for performance evaluation, the HMM (ie, each gesture) was trained with data from a set of people who made each gesture a defined number of times. For HMM training, see L.C. R. Rabiner: “A tutu on Hidden Markov Models unselected applications in spec recognition”. Proceedings of the IEEE, Vol. 77 No. 2, Feb. See 1989.

  In order to objectively assess recognition rates, additional test gesture databases were obtained from another distinctly different set of people. The achieved gesture recognition rate (in percent) is shown in FIG. The rows of the matrix indicate the gestures made and the columns indicate the recognized gestures. The column “Reject” indicates a rejected gesture. The garbage model represents all unintentional undefined hand movements. This is trained using all training data of all people in the first set of people, ie all their gestures. The column “Sum” shows the sum of the probabilities of all recognized gestures and should therefore be 100. A recognition rate of at least 95% can be achieved for all defined gestures. Many of the unrecognized gestures are rejected, which is advantageous because false detection of different gestures is considered more inconvenient than rejection.

  The proposed method is not limited to just a sensor system with exactly four electrodes / sensors or an electrode layout as in FIG. This applies to systems with any number, i.e. from 2 up to an infinite number of sensors. For example, it may also be a system with only two electrodes, a left electrode EL and a right electrode ER, which are circular or another shape.

  Instead of comparing short-term variations in the sensor signal with the threshold for each channel, compare the weighted sum of these variances with a single threshold, or use any other measure of the signal that represents active movement It is also possible.

  Various embodiments are applicable to any sensor system in which the sensor provides distance dependent measurement data. Such systems include capacitive and resistive touch sensors, ultrasonic sensors, radar, surface acoustic sensors. The disclosed embodiments are not limited to gesture recognition using HMMs, but can also be used with dynamic time stretching, neural networks, or other automatic recognition techniques.

  Referring now to FIG. 13, a block diagram illustrates a specific implementation of a sensor system 1300 for gesture recognition including start / stop detection, according to an embodiment. The system of FIG. 13 may be particularly useful in capacitive sensing systems. System 1300 includes a sense controller 1301, a sense electrode 1302, and a host system 1303. The sensing electrode 1302 may implement a configuration as shown in FIG. The host 3103 can be any system that can utilize touch sensor signals, such as a mobile phone, laptop computer, input / output device, optical switch, coffee maker, medical input device, and the like.

In the illustrated embodiment, the TX signal generator 1304 provides the transmitter signal V _TX to the transmission electrode TXD. The reception electrodes RX0 to RX4 are received by a signal conditioning module 1306 for implementing filtering and the like. The output of the signal conditioning is provided to ADC 1307 and is provided to signal processing unit 1308 through bus 1308. The signal processing unit 1308 may implement gesture recognition functionality. The resulting output can be provided to the host 1303 via the input / output unit 1318.

  The system may further include various additional modules such as an internal clock 1309, memory such as flash memory 1312, voltage reference 1310, power management 1314, low power start 1316, reset control 1322, and communication control 1320.

The following references provide additional information about the use of hidden Markov models.
L. E. Baum, et al .: “A maximization technique occurring in the statistic analysis of probabilistic functions of Markov chains”, Ann. Math. Statist. . vol. 41. no. 1. pp. 164-171, 1970.
E. Behrends: Introduction to Markov Chains, Viehweg Verlag, 2000.
L. R. Rabiner: A tutor on Hidden Markov Models unselected applications in specification recognition, Proceedings of the IEEE, Vol. 77 No. 2, Feb. 1989.
A. J. et al. Viterbi: 'Error bounds for convolutional codes and an asymptotically optimal decoding algorithm ", IEEE Transactions on Information Theory, Vol.
Welch: “Hidden Markov Models and the Baum-Welch Algorithm”. IEEE Information Theory Society Newsletter, Dec. 2003.
Although the invention has been described with reference to specific embodiments thereof, these embodiments are merely illustrative and not limiting of the invention. The description herein of the illustrated embodiments of the invention, including the summary and summary descriptions, is intended to be exhaustive or to limit the invention to the precise form disclosed herein. (And in particular, inclusion of any particular embodiment, feature, or function in the summary or summary is not intended to limit the scope of the invention to such embodiment, feature, or function). Rather, the description is not intended to limit the invention to any specifically described embodiments, features, or functions, including any such embodiment features or functions described in the Summary or Summary to those skilled in the art, In order to provide a background for understanding the present invention, it is intended to describe exemplary embodiments, features, and functions. While specific embodiments of the present invention and examples thereof are described herein for purposes of illustration only, various equivalent modifications will be recognized and understood by those skilled in the art. It is possible within the spirit and scope. As indicated, these modifications may be made to the invention in light of the foregoing description of illustrated embodiments of the invention and are intended to be within the spirit and scope of the invention. Thus, although the invention is described herein with reference to specific embodiments thereof, various modifications, various changes, and substitutions are contemplated within the foregoing disclosure, and in some cases Without departing from the scope and spirit of the invention as described, certain features of embodiments of the invention may be employed without the corresponding use of other features. Will be understood. Accordingly, many modifications may be made to adapt a particular situation or material to the essential scope and spirit of the present invention.

  Throughout this specification "one embodiment", "an embodiment", or "a specific embodiment", or similar terminology reference, It means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment and is not necessarily present in all embodiments. Accordingly, the phrases “in one embedment” (in one embodiment), “in an embodiment” (in one embodiment), or “in a specific embodiment” (in a specific embodiment), or throughout the specification. Each representation of a similar terminology in various places does not necessarily refer to the same embodiment. Furthermore, the particular features, structures, or characteristics of any particular embodiment may be combined with one or more other embodiments in any suitable manner. It is understood that other variations and modifications of the embodiments described and illustrated herein are possible in light of the teachings herein and should be considered as part of the spirit and scope of the present invention. I want to be.

  In the description herein, numerous specific details are provided to provide a thorough understanding of embodiments of the invention, such as examples of components and / or methods. However, one of ordinary skill in the art appreciates that embodiments may be described without one or more of the specific details, or with other devices, systems, assemblies, methods, components, materials, parts, and / or the like. You will recognize that it can be practicable. In other instances, well-known structures, components, systems, materials, or operations are not specifically shown or described in detail to avoid obscuring aspects of embodiments of the present invention. Although the present invention may be illustrated by using specific embodiments, this is not any specific embodiment and is not intended to limit the present invention, and those skilled in the art will recognize additional embodiments, It will be appreciated that it is readily understandable and is part of the present invention.

  Use any suitable programming language, including C, C ++, Java, assembly language, etc., to implement the routines, methods, or programs of the embodiments of the invention described herein. be able to. Different programming techniques such as procedural or object oriented can be employed. Any particular routine may be executed on a single computer processing device or multiple computer processing devices, a single computer processor or multiple computer processors. Data may be stored in a single storage medium or distributed across multiple storage media and may reside in a single database or multiple databases (or other data storage techniques). The steps, actions, or calculations may be presented in a specific order, but this order may be changed in different embodiments. In some embodiments, to the extent that multiple steps are shown sequentially herein, some combinations of such steps in alternative embodiments may be performed simultaneously. The sequence of operations described herein can be interrupted, suspended, or otherwise controlled by another process such as an operating system, kernel, or the like. The routine can operate within the operating system environment or as an independent routine. The functions, routines, methods, steps, and operations described herein may be performed in hardware, software, firmware, or any combination thereof.

  The embodiments described herein may be implemented in the form of control logic in software, hardware, or a combination of both. The control logic is stored in an information storage medium, such as a computer readable medium, as a plurality of instructions adapted to instruct the information processing device to perform a set of steps disclosed in various embodiments. obtain. Based on the disclosure and teachings provided herein, one of ordinary skill in the art will appreciate other techniques and / or methods for implementing the invention.

  It is also within the spirit and scope of the present invention to implement any of the steps, operations, methods, routines, or portions thereof described herein in software programming or code. Software programming or code can be stored in a computer-readable medium and causes a processor to perform any of the steps, operations, methods, routines, or portions thereof described herein. Can be operated. The present invention may be implemented by using application specific integrated circuits, programmable logic devices, field programmable gate arrays, etc., by using software programming or code in one or more general purpose digital computers. Optical, chemical, biological, quantum mechanical, or nanofabrication systems, components and mechanisms can be used. In general, the functions of the present invention can be achieved by any means as is known in the art. For example, distributed or networked systems, components, and circuits can be used. In another example, the communication or transfer of data (or otherwise moving from one location to another) may be by wire, wireless, or any other means.

  “Computer-readable medium” is any medium that can contain, store, communicate, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, system, or device. It can be a medium. The computer readable medium is not limiting and may be, by way of example only, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, system, device, propagation medium, or computer memory. Such computer readable media is generally intended to include machine programming or code that may be machine readable and may be human readable (eg, source code) or machine readable (eg, object code). Examples of non-transitory computer readable media are random access memory, read only memory, hard drive, data cartridge, magnetic tape, floppy diskette, flash memory drive, optical data storage device, compact disk read Dedicated memory can be included, as well as other suitable computer memory and data storage devices. In the illustrative embodiment, some or all of the software components may reside on any combination of single server computers or separate server computers. As can be appreciated by one skilled in the art, a computer program product that implements the embodiments disclosed herein stores one or more computer instructions that are translatable by one or more processors in a computing environment. Non-transitory computer readable media.

  A “processor” includes any hardware system, mechanism, or component that processes data, signals, or other information. The processor can include a system with a general purpose central processing unit, multiple processing units, dedicated circuitry to achieve functionality, or other systems. Processing need not be limited to geographic locations or have time restrictions. For example, the processor can perform its functions in “real time”, “offline”, “batch mode”, and the like. Some of the processing can be performed by different (or the same) processing systems at different times and at different locations.

  As used herein, the terms “comprise”, “comprising”, “include”, “including”, “has”, “ “having”, or any other variation thereof, is intended to cover non-exclusive inclusions. For example, a process, product, article, or device comprising a list of elements is not necessarily limited to only those elements, is not explicitly listed, or such process, process, article, or Other elements specific to the device may be included.

  Further, the term “or” as used herein is generally intended to mean “and / or” unless otherwise indicated. For example, condition A or B is satisfied by any one of: A is true (or present), B is false (or absent), A is false (Or absent), B is true (or present), and both A and B are true (or present). As used herein, including the following claims, preceded by “a” or “an” (and “the” when the antecedent is “a” or “an”) Terms include both the singular and plural of such terms (ie, reference to “a” or “an” includes only the singular or the plural unless specifically stated otherwise in the claims.) Only clearly). Also, as used throughout the description and the following claims, the meaning of “in” means “in” unless the context clearly indicates otherwise. ”And“ on ”.

  One or more of the elements depicted in the drawings / diagrams may also be implemented in a more separate or integrated manner, or in some cases removed or inoperable, as useful according to the particular application. It will be understood that In addition, any signal arrows in the drawings / drawings should be regarded as illustrative rather than restrictive unless specifically stated otherwise.

Claims (31)

A method for gesture recognition,
Detecting one or more gesture related signals using a plurality of associated detection sensors;
Determining whether the gesture corresponds to one of a predetermined set of gestures by evaluating gestures detected from the one or more gesture-related signals using automatic recognition techniques; And a method comprising:   The method of claim 1, wherein evaluating the gesture includes determining a start and stop of the gesture.   The method of claim 1, wherein evaluating the gesture includes determining to stop the gesture.   In determining the start of the gesture, the start is a decrease in the distance between the target object and the at least one sensor, an increase in the distance between the target object and the at least one sensor, and a predetermined The method of claim 1, wherein the method is determined when a short-term variation across multiple signal samples or an equivalent rating measure is less than a threshold.   The cessation of the gesture means that, at a given time, the distance between the target object and all sensors decreases and / or short-term variation over a given number of signal samples or an equivalent rating scale is below the threshold. The method of claim 1, wherein the method is determined to be small and / or if the signal change is less than a predetermined threshold over a predetermined number of signal samples after the given time point.   The method of claim 1, wherein each gesture is represented by one or more hidden Markov models (HMMs).   The method of claim 6, wherein evaluating a gesture includes evaluating a probability rating scale for one or more HMMs.   The features associated with the observation matrix of the HMM are: non-quantized or quantized sensor signal level, x / y / z position, distance, direction, orientation, angle, and / or primary, secondary with respect to these times Or a higher order derivative or any combination thereof.   9. The method of claim 8, wherein the feature is a first derivative of the sensor signal level that is quantized to two quantization levels.   8. The method of claim 7, wherein the probability of each hidden Markov model is updated for each new signal sample or feature.   The method according to claim 10, wherein the recognition is stopped if the probability of a hidden Markov model exceeds a predefined threshold.   A system for gesture recognition using an alternating electric field generated by a sensor array and associated detection electrodes, wherein the electrode signal is evaluated using a hidden Markov model, and start and stop criteria for gesture determination Has been determined, the system.   13. The system of claim 12, wherein the feature sequence used to evaluate the probability of the hidden Markov model is a first derivative of the sensor signal level that is quantized to two quantization levels. A system for gesture recognition,
A sensor array for detecting one or more gesture-related signals using a plurality of associated detection sensors;
For evaluating the gesture detected from the one or more gesture-related signals using an automatic recognition technique to determine whether a gesture corresponds to one of a predetermined set of gestures. A system comprising a module and   The start of the gesture is a short period of time over a predetermined plurality of signal samples, with a decrease in the distance between the target object and the at least one sensor, an increase in the distance between the target object and the at least one sensor. 15. The system of claim 14, wherein the system is determined if the variation or equivalent rating scale is less than a threshold value.   The cessation of the gesture means that, at a given time, the distance between the target object and all sensors decreases and / or short-term variation over a given number of signal samples or an equivalent rating scale is below the threshold. 15. The system of claim 14, wherein the system is small and / or is determined if the signal change is less than a predetermined threshold over a predetermined number of signal samples after the given time point.   The system of claim 14, wherein each gesture is represented by one or more hidden Markov models.   The system of claim 17, wherein evaluating a gesture includes evaluating a probability rating scale for one or more hidden Markov models.   The features associated with the observation matrix of the HMM are: non-quantized or quantized sensor signal level, x / y / z position, distance, direction, orientation, angle, and / or primary, secondary with respect to these times The system of claim 17, or a higher order derivative, or any combination thereof.   The system of claim 19, wherein the feature is a first derivative of the sensor signal level that is quantized to two quantization levels.   The system of claim 18, wherein the probability of each hidden Markov model is updated for each new signal sample or feature.   The system of claim 21, wherein the recognition is stopped if the probability of a hidden Markov model exceeds a predefined threshold. A computer readable medium comprising one or more non-transitory machine readable program instructions, the program instructions comprising:
Using one or more detection sensors to receive one or more gesture related signals;
Evaluating the gesture detected from the one or more gesture-related signals using an automatic recognition technique to determine whether a gesture corresponds to one of a predetermined set of gestures; A computer-readable medium for performing   Initiating a gesture reduces the distance between the target object and the at least one sensor, increases the distance between the target object and the at least one sensor, and short-term variation over a given plurality of signal samples 24. The computer readable medium of claim 23, or determined if the equivalent rating scale is less than a threshold value.   The cessation of the gesture means that, at a given time, the distance between the target object and all sensors decreases and / or short-term variation over a given number of signal samples or an equivalent rating scale is below the threshold. 24. The computer readable medium of claim 23, wherein the computer readable medium is small and / or is determined when a signal change is less than a predetermined threshold over a predetermined plurality of signal samples after the given time point.   24. The computer readable medium of claim 23, wherein each gesture is represented by one or more hidden Markov models (HMMs).   27. The computer-readable medium of claim 26, wherein evaluating a gesture includes evaluating a probability rating measure for one or more HMMs.   The features associated with the observation matrix of the HMM are: non-quantized or quantized sensor signal level, x / y / z position, distance, direction, orientation, angle, and / or primary, secondary with respect to these times 27. The computer readable medium of claim 26, wherein the computer readable medium is a higher order derivative, or any combination thereof.   29. The computer readable medium of claim 28, wherein the feature is a first derivative of the sensor signal level that is quantized to two quantization levels.   28. The computer readable medium of claim 27, wherein the probability of each hidden Markov model is updated for each new signal sample or feature.   31. The computer readable medium of claim 30, wherein the recognition is stopped if the probability of a hidden Markov model exceeds a predefined threshold.

Images