CN105807923A - Ultrasonic wave based volley gesture identification method and system - Google Patents
Ultrasonic wave based volley gesture identification method and system Download PDFInfo
- Publication number
- CN105807923A CN105807923A CN201610127516.1A CN201610127516A CN105807923A CN 105807923 A CN105807923 A CN 105807923A CN 201610127516 A CN201610127516 A CN 201610127516A CN 105807923 A CN105807923 A CN 105807923A
- Authority
- CN
- China
- Prior art keywords
- movement tendency
- gesture
- palm movement
- model
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
Abstract
The invention provides an ultrasonic wave based volley gesture identification method and system. The method comprises the steps of by utilizing a pre-trained palm motion trend model, identifying palm motion trends for acquired ultrasonic signals reflected back through a human hand to obtain a palm motion trend time-sequential sequence containing a series of palm motion trends, wherein the palm motion trend model is a model used for identifying the palm motion trends and obtained by training according to acoustic features of the ultrasonic signals reflected back through the human hand; and by utilizing a pre-trained gesture identification model, performing gesture identification on the obtained palm motion trend time-sequential sequence, wherein the gesture identification model is a model used for identifying gestures and obtained by training according to a training data set consisting of the palm motion trend time-sequential sequence. The method and system are suitable for an intelligent mobile terminal and can realize high precision and high robustness of gesture identification.
Description
Technical field
The present invention relates to field of human-computer interaction, more particularly, to based on hyperacoustic Gesture Recognition high up in the air, especially towards the ultrasound wave of intelligent mobile terminal Gesture Recognition high up in the air.
Background technology
In recent years, along with the development of intelligent mobile terminal Yu wearable technology, occur in that many novel man-machine interaction scenes and form.The man-machine interaction mode based on touch screen of current main flow requires that user contacts with touch screen, affect universality and the naturality of man-machine interaction, in particular for special scenes (as driven vehicle processes, cooking food process etc.) and particular device form (such as intelligent glasses, Intelligent bracelet etc.), user is difficult to touch, it is impossible to fully meet the diversified demand of man-machine interaction.Therefore, in the urgent need to a kind of gesture input method high up in the air suitable in intelligent mobile terminal.
In traditional man-machine interactive system, for providing the prior art of gesture identification high up in the air mainly to have following three kinds of modes:
(1) based on computer vision.The hand of user is identified based on the deep image information of the color image information of camera collection or the depth camera collection of specialty, and identifying the gesture of user further, this type of method can be the Chinese patent application of CN104360742A, CN103136541A, CN103176605A, CN103472916A, CN104915010A etc. referring to publication number.But, this type of method is more sensitive to light, and algorithm complex is higher, and the identification range of gesture is limited, it is necessary to more system resource, and needs to be equipped with special camera sensing device, is unsuitable for carrying out popularization and application under general environment.
(2) based on data glove.Gathering hand motion by data glove, thus identifying the gesture of user, this type of method can be the Chinese patent application of CN104392237A, CN105127973A, CN204765652U, CN204740561U etc. referring to publication number.The method needs user to wear position tracker and data glove, the complicated operation of complexity, is unfavorable for natural man-machine interaction, and price comparison is expensive, is unsuitable for carrying out popularization and application equally under general environment.
(3) based on myoelectric sensor.The mode adopting wrist strap measures the electromyographic signal of forearm, and due to the electromyographic signal that different gesture motion performances is different, thus identifying different static gestures, this type of method can be the Chinese patent application of CN105138133A, CN105139038A etc. referring to publication number.The method needs user to wear the myoelectricity wrist strap of specialty, it is desirable to wrist strap and user's forearm fit tightly, it is possible to cause the discomfort of user, and relatively costly.
Additionally, three of the above mode is not all suitable for intelligent mobile terminal.
Summary of the invention
For solving above-mentioned problems of the prior art, according to one embodiment of present invention, it is provided that a kind of based on hyperacoustic gesture identification method high up in the air, including:
Step 1), utilize the palm movement tendency model that training in advance is good, to the ultrasonic signal identification palm movement tendency being reflected back via staff gathered, obtain comprising the palm movement tendency time series of a series of palm movement tendency；Wherein, described palm movement tendency model is that the acoustic features of the ultrasonic signal being reflected back according to staff trains the model for identifying palm movement tendency obtained；
Step 2), utilize the gesture identification model that training in advance is good, to step 1) the palm movement tendency time series that obtains carries out gesture identification；Wherein, described gesture identification model is to train, according to the training dataset being made up of palm movement tendency time series, the model for identifying gesture obtained.
The step 1 of said method) including:
Feature is extracted from the ultrasonic signal being reflected back via staff gathered, utilize the palm movement tendency model that training in advance is good for the feature identification palm movement tendency extracted, the identified palm movement tendency obtained of buffer memory is until gesture terminates, and the palm movement tendency of buffer memory is formed palm movement tendency time series, wherein:
For gather one frame ultrasonic signal perform following steps, wherein a frame ultrasonic signal by L sampling number of continuous print according to forming:
Step 11), from a described frame ultrasonic signal extract feature, obtain input data；
Step 12), judge current state, if current state instruction be in gesture process; perform step 14), otherwise perform step 13)；
Step 13), utilize palm movement tendency model that training in advance is good for described input data identification palm movement tendency, if recognition result instruction is without palm movement tendency, return step 11 for the next frame ultrasonic signal gathered), otherwise current state is arranged in gesture process, palm movement tendency that buffer memory identification obtains and for the next frame ultrasonic signal return step 11 gathered)；
Step 14), judge whether gesture terminates, if terminating, the palm movement tendency of buffer memory being formed palm movement tendency time series and performs step 2), otherwise return step 11 for the next frame ultrasonic signal gathered).
Said method is in step 2) after also include:
Empty the palm movement tendency of buffer memory, current state is set to non-gesture process, and returns step 1).
In said method, also include training palm movement tendency model according to following steps:
Step a), from the every frame ultrasonic signal being reflected back via staff gathered, extract feature obtain every frame training sample, and be every frame training sample corresponding classification of imparting, form the first training dataset；
Step b), according to described first training dataset, train described palm movement tendency model.
In said method, step b) including: utilizes the requirement of ELM models coupling cross validation to train described palm movement tendency model.
In said method, also include training gesture identification model according to following steps:
Step c), described first training dataset is carried out Hand Gesture Segmentation via the result that obtains of described palm movement tendency model after training, form the training sample that different gesture is corresponding, and it is that training sample corresponding to each gesture gives corresponding classification, forms the second training dataset；
Step d), according to described second training dataset, for each gesture train corresponding gesture identification model.
In said method, step d) including: utilizes HMM model pin in conjunction with the requirement of cross validation, trains corresponding gesture identification model for each gesture.
In said method, the speaker that intelligent mobile terminal is intrinsic launch ultrasonic signal and the ultrasonic signal being reflected back via staff by the mike collection that described intelligent mobile terminal is intrinsic constantly.
In said method, described hyperacoustic frequency is 18kHz-22kHz.
According to one embodiment of present invention, also provide for a kind of based on hyperacoustic gesture recognition system high up in the air, including:
Palm movement tendency identification device, for utilizing the palm movement tendency model that training in advance is good, to the ultrasonic signal identification palm movement tendency being reflected back via staff gathered, obtains comprising the palm movement tendency time series of a series of palm movement tendency；Wherein, described palm movement tendency model is that the acoustic features of the ultrasonic signal being reflected back according to staff trains the model for identifying palm movement tendency obtained；
Gesture identifying device, for utilizing the gesture identification model that training in advance is good, to step 1) the palm movement tendency time series that obtains carries out gesture identification；Wherein, described gesture identification model is to train, according to the training dataset being made up of palm movement tendency time series, the model for identifying gesture obtained.
In said system, the speaker that intelligent mobile terminal is intrinsic launch ultrasonic signal and the ultrasonic signal being reflected back via staff by the mike collection that described intelligent mobile terminal is intrinsic constantly.
Provided by the invention is the Hierarchical Approach of a kind of acoustic features having merged acoustic signals and temporal aspect based on hyperacoustic gesture identification method high up in the air: at the first level, the ultrasonic signal data gathered for each moment (hereinafter referred to as frame) extract acoustic features, identify the palm movement tendency of each frame, obtain the time series of palm movement tendency therein；At the second level, the palm movement tendency sequence for identifying is identified classification, thus realizing gesture identification high up in the air.The method performs further gesture classification identifying on the basis of palm movement tendency, it is achieved that the gesture identification high up in the air of high accuracy and high robust.
In addition, compared with above-mentioned prior art, in the present invention, data to be dealt with are relatively simple, without using extra data glove, myoelectric sensor etc., suitable in intelligent mobile terminal, the mike that intelligent mobile terminal can be used intrinsic carries out hyperacoustic transmitting and reception with speaker, therefore less costly, it is easy to popularization and application.
Accompanying drawing explanation
Fig. 1 is the flow chart of model off-line training method according to an embodiment of the invention；
Fig. 2 is flow chart based on hyperacoustic gesture identification method high up in the air according to an embodiment of the invention；
Fig. 3 is schematic diagram based on the stratification identification process in hyperacoustic gesture identification method high up in the air according to an embodiment of the invention.
Detailed description of the invention
Below in conjunction with the drawings and specific embodiments, the present invention is illustrated.Should be appreciated that specific embodiment described herein is only in order to explain the present invention, is not intended to limit the present invention.
According to one embodiment of present invention, it is provided that a kind of based on hyperacoustic gesture identification method high up in the air.
Generally, the method is by launching, receive ultrasound wave, and the ultrasonic signal received is carried out the movement tendency identification of (two-layer) palm and gesture identification, catches user's hand motion, thus realizing the identification of gesture high up in the air.Wherein, the palm movement tendency model utilizing training in advance good carries out palm movement tendency identification, and the gesture identification model utilizing training in advance good carries out gesture identification.It should be noted that ultrasound wave here refers to the unheard audio frequency of normal adult (being namely not less than 18kHz)；And gesture high up in the air includes but not limited to: extension, post-tensioning, click, double-click etc..
Hereafter will point two stages describing provided by the invention based on hyperacoustic gesture identification method high up in the air, the two stage be model training stage and gesture identification stage respectively.
One. the model training stage
Before carrying out gesture identification (including palm movement tendency identification and gesture identification) high up in the air, it is necessary to the good palm movement tendency model of precondition and gesture identification model.Referring to Fig. 1, this model training method comprises the following steps:
1, ultrasound wave gesture data collection high up in the air
In this step, ultrasonic transmission device ultrasound wave and the ultrasound wave being reflected back via staff by ultrasonic probe, ultrasonic receiver collection are play constantly.
According to one embodiment of present invention, it is possible to use the intrinsic speaker of intelligent mobile terminal plays ultrasound wave.When using intelligent mobile terminal to launch and receiving ultrasound wave (referring to next section), the hyperacoustic frequency range adopted can for 18kHz to 22kHz, it is preferred to 18kHz.Intelligent mobile terminal involved herein includes but not limited to: the wearable smart machines such as smart mobile phone, panel computer and intelligent watch.
Simultaneously, it is possible to use intelligent mobile terminal (namely launches hyperacoustic intelligent mobile terminal) ultrasound wave that intrinsic mike is reflected back via staff with predetermined sample frequency collection.For the present invention, staff can be positioned at any position relative to this intelligent mobile terminal, it is preferable that is positioned at and the scope of this intelligent mobile terminal distance half meter.Additionally, in acoustic sampling, despite the presence of 11.025kHz, 22.05kHz and the sample frequency of the standard such as 44.1kHz, but for the ultrasound wave of 18kHz, ultrasonic probe, ultrasonic receiver herein preferably employs the sample frequency of 44.1kHz and carries out the collection of periodic sound wave signal (ultrasound wave being namely reflected back).
2, the signal data collected is carried out pretreatment
First, with the slip hamming window that length is L, the ultrasonic signal data gathered are divided into continuous print multiframe (namely the time series of every L sampled point constitutes a frame), and the data between adjacent two frames do not have overlap.Wherein, L can be 2048 (or other n power of 2, to be applicable to feature extraction hereafter), thus for the sample frequency of 44.1kHz, the time of every frame is about 46.44ms, and every frame data can be expressed as: Ai={ a1,a2,…,a2048, wherein aj(can be called time-domain signal in frame) represents the analog voltage of sampled point j integer type numerical value after A/D changes, and continuous print multiframe data form acoustic signals sequence A={ A1,A2,…An}。
Then, the frame data collected are carried out pretreatment.Pretreatment includes preliminary smoothing processing, for improving final recognition accuracy.It is for instance possible to use time-domain signal in frame is filtered processing by N rank moving average filter method (MovingAverageFilter), its receptance function is expressed as follows:
After smoothing processing, these frame data are represented by: Ai={ a1’,a2’,…,a2048’}。
3, feature extraction
In this step, from pretreated frame data, feature, the training sample set of composition palm movement tendency model are extracted.
Specifically, pretreated every frame data are extracted each frame characteristic of correspondence (such as acoustic features) as a unit, thus forming the training sample set of palm movement tendency model.Wherein, the feature extracted can be spectrum peak, spectrum peak position, frequency collection of illustrative plates, average, zero-crossing rate, standard deviation etc..It will be understood by those skilled in the art that the feature extracted here is not limited to several typical characteristics listed above.It addition, one of them or its combination of features above may be incorporated for realizing the training of palm movement tendency model.
For example, when every frame data include 2048 sampled points, following manner can be taked to extract every frame characteristic of correspondence:
Obtained the frequency domain data of 1025 points (i.e. 1025 frequency ranges) by FFT time-frequency convert, extract the spectrum value of each n (n is positive integer) frequency range before and after spectrum peak therein and spectral peak vectorial as this frame characteristic of correspondence.Wherein it is determined that the frequency range at spectral peak place includes following manner: 1, at the high-frequency region (> 17000Hz of obtained frequency domain data), using spectrum value maximum as spectral peak place frequency range；2, selecting the frequency range at carrier frequency place as spectral peak place frequency range, wherein under the premise of 18kHz carrier wave (i.e. above-mentioned 18kHz ultrasound wave), 44.1kHz sample frequency and 2048 sampled points/frame, the subscript of carrier frequency place frequency range is about 835.
4, the demarcation of palm movement tendency data
Classification according to palm movement tendency is (such as, including forward, backward etc.), corresponding palm movement tendency category label is given respectively (such as every frame data, palm movement tendency category label forward is 1, palm movement tendency category label backward is 2 etc.), thus the characteristic vector extracted by every frame data (training sample set of above-mentioned palm movement tendency model) and this frame category form the training dataset TrainDataSet_1 of the first level.
5, training palm movement tendency model
Herein, the output of palm movement tendency model can include but not limited to: " null ", " forward " and " backward ", indicates without palm movement tendency respectively, relative to mike near and relative to mike away from.
In this step, the training dataset TrainDataSet_1 of the first level is divided into m part, for cross validation；Disaggregated model (such as, naive Bayesian, SVM, DNN etc.) is utilized to be trained.
Preferably, the requirement utilizing ELM (very fast learning machine, ExternLearningMachine) models coupling cross validation carries out the training of palm movement tendency model.Specifically, first random setting model parameter, then with m-1 number according to carrying out training pattern, with another 1 number according to carrying out test model precision, whole process Repeated m time, using the meansigma methods of precision as final accuracy value, and output model parameter.
For example, it is possible to training sample is equally divided into 10 (i.e. m=10) group, select wherein 1 group as test data, all the other 9 groups as training data.Use the training sample set structure palm movement tendency model that 9 groups of training samples are formed by ELM model, test data are tested by the model set up, obtain training accuracy rate and test accuracy rate；Repeating said process 9 times, choose different 1 group as test data every time, all the other 9 groups, as training data, obtain training accuracy rate and test accuracy rate accordingly.Take the parameter of the average of 10 Experiment Training accuracys rate and the test accuracy rate accuracy rate as final mask output model.
6, the extraction of gesture data and demarcation
First, the result (time series of palm movement tendency) that TrainDataSet_1 palm movement tendency model after training is obtained, carry out Hand Gesture Segmentation, to form the training sample set for different gestures；Wherein, sequence length normalized can be carried out in training sample concentration and repeat gesture removal process.
Then, training sample for each gesture gives corresponding class number (such as respectively, the category label " clicked " is 1, the category label of " double-click " is 2 etc.), thus by the training dataset TrainDataSet_2 forming the second level being used for gesture identification model for the training sample set of different gestures and respective classes.
7, training gesture identification model
In this step, for each gesture, such as CRF, HMM etc. are adopted to utilize the model of temporal aspect to carry out model training.
Preferably, HMM model (hidden Markov model, HiddenMarkovModel) is utilized to be trained.Specifically, the training dataset TrainDataSet_2 of the second level is divided into m part, for cross validation (for example with ten folding cross validation methods)；For gesture collection, (this gesture collection includes but not limited to: " forward ", " backward ", " click ", " double-click ", " reversely click ") in each gesture (it should be noted that, HMM model is in the training process, it is train a HMM model for each gesture, so, model corresponding to one palm each gesture of movement tendency sequence pair has a likelihood probability, the gesture that wherein maximum of probability is corresponding is the gesture of identification), HMM model is utilized to carry out the training of gesture identification model in conjunction with the requirement (process of this cross validation is similar to the 5th step) of cross validation, training obtains the output parameter of model after terminating.
Thus, each gesture concentrated for gesture establishes corresponding gesture identification model respectively.
Two. the gesture identification stage
Referring to Fig. 2, what the present embodiment provided comprises the steps of based on hyperacoustic gesture identification method high up in the air:
The first step: persistently play ultrasound wave and receive the ultrasound wave being reflected back via staff
1, ultrasonic transmission device plays ultrasound wave constantly.
Identical with the 1st step in model training stage, in order to make environment exists ultrasonic signal, ultrasonic transmission device (speaker as intrinsic in intelligent mobile terminal) to be utilized to play the ultrasound wave of 18～20kHz, it is preferable that to launch the ultrasound wave of 18kHz.
2, simultaneously, the ultrasound wave that ultrasonic probe, ultrasonic receiver (such as, launching the mike that hyperacoustic intelligent mobile terminal is intrinsic) receives and is reflected back via staff with the sample frequency collection of 44.1kHz.Often gather a number of ultrasonic signal, then enter second step and these ultrasonic signals are carried out subsequent treatment.
Hereinbefore, with the slip hamming window that length is L (such as 2048), the ultrasonic signal data gathered are divided into continuous print multiframe, in this step, often gather a frame of L sampled point composition, enter second step for these frame data.
Second step: the signal collected is carried out pretreatment and extracts feature from pretreated signal to form the input data of palm movement tendency model
Similar with the model training stage, the frame data gathered in the first step are carried out pretreatment and feature extraction by this step, including:
1, above-mentioned formula (1) is utilized to be filtered processing to the frame data gathered；
2 is identical with feature extraction mode given above, from pretreated extracting data feature, constitutes the input data of palm movement tendency model.
3rd step: based on input data identification palm movement tendency
This step includes following sub-step:
1, current state is judged
Whether current state is used for indicating and is currently in gesture process (that is, doing in the process of gesture).If being in gesture process, then current state is expressed as " gesture " and is otherwise expressed as " wait " (that is, non-gesture process).If being in gesture process (" gesture "), then performing sub-step 3, otherwise (i.e. " wait ") performs sub-step 2.Herein, original state is " wait ".
2, for input data identification palm movement tendency
Palm movement tendency is identified for above-mentioned entering data to according to the palm movement tendency model that training in advance is good.As described above, the recognition result of this palm movement tendency model includes but not limited to: " forward ", " backward ", " null ", represent respectively relative to mike close, away from and without palm movement tendency.
Specifically, input data are inputted palm movement tendency model and is identified, be identified result；According to recognition result judge current palm movement tendency be whether " null " (namely, movement tendency without palm), if being " null ", return the first step, to perform the subsequent processes such as pretreatment according to the next frame gathered, otherwise current state is set to " gesture ", and by the current palm movement tendency that identifies (such as, " backward " or " forward ") carry out buffer memory, it is then returned to the first step, to perform the subsequent processes such as pretreatment according to the next frame gathered.
3, judge whether gesture terminates, if gesture terminates, then enter the 4th step, otherwise return the first step, to perform the subsequent processes such as pretreatment according to the next frame gathered.
In one embodiment of the invention, it is possible to the mark continuous three " null " terminated as gesture, if palm movement tendency model identifies three " null " recently continuously, the 4th step is entered；Otherwise return the first step.
4th step: the palm movement tendency of buffer memory is carried out gesture identification
After entering the 4th step, (namely the palm movement tendency of buffer memory can form the time series of palm movement tendency before, palm movement tendency sequence), this sequence is identified by the gesture identification model adopting training in advance good, and exports gesture identification result.It is identified wherein it is possible to first the time series of palm movement tendency to be carried out length normalization method process again through gesture identification model.Following present gesture motion trend sequence a example:
(forward,forward,…,backward,backward)
As described above, the gesture identification model corresponding with each gesture is adopted to be identified.Palm movement tendency sequence obtains a likelihood probability after by each gesture identification model, takes gesture corresponding to wherein maximum of probability as the gesture identified.
5th step: response gesture operation
In this step, (such as, intelligent mobile device) responds with the operation corresponding with recognition result.Empty buffer memory simultaneously, current state is set to " wait " (i.e. non-gesture process), and returns the first step, to perform the subsequent processes such as pretreatment according to the next frame gathered.
Said method is a kind of Hierarchical Approach, and gesture is decomposed into the time series of multiple palm movement tendency by the degree of freedom according to hand natural motion.In the first level (referring to the 3rd step), carry out the identification of palm movement tendency；In the second level (referring to the 4th step), a series of palm movement tendencies according to identifying carry out gesture identification.Fig. 3 illustrates the schematic flow sheet by this Hierarchical Approach identification gesture.
According to one embodiment of present invention, also provide for a kind of based on hyperacoustic gesture recognition system high up in the air, including:
Palm movement tendency identification device, for utilizing the palm movement tendency model that training in advance is good, to the ultrasonic signal identification palm movement tendency being reflected back via staff gathered, obtains comprising the palm movement tendency time series of a series of palm movement tendency；Wherein, palm movement tendency model is that the acoustic features of the ultrasonic signal being reflected back according to staff trains the model for identifying palm movement tendency obtained；
Gesture identifying device, for utilizing the gesture identification model that training in advance is good, to step 1) the palm movement tendency time series that obtains carries out gesture identification；Wherein, gesture identification model is to train, according to the training dataset being made up of palm movement tendency time series, the model for identifying gesture obtained.
In one embodiment, this gesture recognition system high up in the air also includes: ultrasonic transmission device, for launching ultrasonic signal constantly；Ultrasonic probe, ultrasonic receiver, for gathering the ultrasonic signal being reflected back via staff.Wherein, ultrasonic transmission device can be the speaker of intelligent mobile terminal, and ultrasonic probe, ultrasonic receiver can be the mike of intelligent mobile terminal.Intelligent mobile terminal can be the intelligence panel computer under general environment, smart mobile phone etc..
For verifying the effectiveness of gesture identification method high up in the air provided by the invention and system, inventors performed following experiment:
1, experiment porch
● experimental situation: normal working environment (～30dB)
● experimental facilities: MacBookPro13-inch；DellInsprion1545
● speaker: the speaker that experimental facilities is intrinsic
● mike: the mike that experimental facilities is intrinsic
2, gesture data source
4 users (2 man users and 2 Ms users) implement " forward " at random in 1 point of half, " backward " and " clicking " hands gesture, acquire the acoustic signals that 7600 frame palm movement tendencies are corresponding altogether, then (frequency range is [17677 to the artificial spectrum value that every frame acoustic signals (i.e. previously described every frame data) extracts 15 frequency ranges of spectrum peak and front and back thereof, 18323]), and every frame data are demarcated, training data and test data as initial palm movement tendency model, and select the palm movement tendency sequence (i.e. effective palm movement tendency sequence hereinafter) wherein constituting gesture as the training data of gesture identification model and test data.
3, palm movement tendency model
The 7600 frame palm movement tendency data obtained are divided into 10 groups, the excitation function of ELM is set to Sigmoid, the number of hidden nodes is set to 90, then the method adopting ten folding cross validations, utilize ELM algorithm that palm movement tendency model of cognition is trained and is tested, and contrast with rule-based palm movement tendency recognition methods, should be as shown in the table based on the experimental result of the palm movement tendency model of ELM.
Table 1
4, gesture identification model
On the basis of 7600 frame palm movement tendency data, obtain effective palm movement tendency sequence by palm movement tendency model and Hand Gesture Segmentation, and it is carried out length normalization method, gesture-type demarcation and repeats gesture sequence removal process.Then the method adopting ten folding cross validations, the characteristic vector that the palm movement tendency sequence demarcated is trained as HMM model, HMM algorithm is utilized gesture model of cognition to be trained and tests, should be as shown in the table based on the experimental result of the gesture identification model of HMM.
Table 2
According to above experimental result it can be seen that method and system provided by the invention is applicable to intelligent mobile terminal, and it is simultaneously achieved high-precision gesture identification high up in the air.
It should be noted that and understand, when without departing from the spirit and scope of the present invention required by appended claims, it is possible to the present invention of foregoing detailed description is made various amendment and improvement.It is therefore desirable to the scope of the technical scheme of protection is not by the restriction of given any specific exemplary teachings.
Claims (12)
1., based on a hyperacoustic gesture identification method high up in the air, described method includes:
Step 1), utilize the palm movement tendency model that training in advance is good, to the ultrasonic signal identification palm movement tendency being reflected back via staff gathered, obtain comprising the palm movement tendency time series of a series of palm movement tendency；Wherein, described palm movement tendency model is that the acoustic features of the ultrasonic signal being reflected back according to staff trains the model for identifying palm movement tendency obtained；
Step 2), utilize the gesture identification model that training in advance is good, to step 1) the palm movement tendency time series that obtains carries out gesture identification；Wherein, described gesture identification model is to train, according to the training dataset being made up of palm movement tendency time series, the model for identifying gesture obtained.
2. method according to claim 1, wherein, step 1) including:
Feature is extracted from the ultrasonic signal being reflected back via staff gathered, utilize the palm movement tendency model that training in advance is good for the feature identification palm movement tendency extracted, the identified palm movement tendency obtained of buffer memory is until gesture terminates, and the palm movement tendency of buffer memory is formed palm movement tendency time series.
3. method according to claim 2, wherein, step 1) including:
For gather one frame ultrasonic signal perform following steps, wherein a frame ultrasonic signal by L sampling number of continuous print according to forming:
Step 11), from a described frame ultrasonic signal extract feature, obtain input data；
Step 12), judge current state, if current state instruction be in gesture process; perform step 14), otherwise perform step 13)；
Step 13), utilize palm movement tendency model that training in advance is good for described input data identification palm movement tendency, if recognition result instruction is without palm movement tendency, return step 11 for the next frame ultrasonic signal gathered), otherwise current state is arranged in gesture process, palm movement tendency that buffer memory identification obtains and for the next frame ultrasonic signal return step 11 gathered)；
Step 14), judge whether gesture terminates, if terminating, the palm movement tendency of buffer memory being formed palm movement tendency time series and performs step 2), otherwise return step 11 for the next frame ultrasonic signal gathered).
4. method according to claim 3, wherein, step 2) also include afterwards:
Empty the palm movement tendency of buffer memory, current state is set to non-gesture process, and returns step 1).
5., according to the method that in claim 1-4, any one is described, also include training palm movement tendency model according to following steps:
Step a), from the every frame ultrasonic signal being reflected back via staff gathered, extract feature obtain every frame training sample, and be every frame training sample corresponding classification of imparting, form the first training dataset；
Step b), according to described first training dataset, train described palm movement tendency model.
6. method according to claim 5, wherein, step b) including:
Utilize the requirement of ELM models coupling cross validation to train described palm movement tendency model.
7. method according to claim 5, also includes training gesture identification model according to following steps:
Step c), described first training dataset is carried out Hand Gesture Segmentation via the result that obtains of described palm movement tendency model after training, form the training sample that different gesture is corresponding, and it is that training sample corresponding to each gesture gives corresponding classification, forms the second training dataset；
Step d), according to described second training dataset, for each gesture train corresponding gesture identification model.
8. method according to claim 7, wherein, step d) including:
Utilize HMM model pin in conjunction with the requirement of cross validation, train corresponding gesture identification model for each gesture.
9. according to the method that in claim 1-4, any one is described, wherein, the speaker that intelligent mobile terminal is intrinsic launch ultrasonic signal and the ultrasonic signal being reflected back via staff by the mike collection that described intelligent mobile terminal is intrinsic constantly.
10. according to the method that in claim 1-4, any one is described, wherein, described hyperacoustic frequency is 18kHz-22kHz.
11. based on a hyperacoustic gesture recognition system high up in the air, including:
Palm movement tendency identification device, for utilizing the palm movement tendency model that training in advance is good, to the ultrasonic signal identification palm movement tendency being reflected back via staff gathered, obtains comprising the palm movement tendency time series of a series of palm movement tendency；Wherein, described palm movement tendency model is that the acoustic features of the ultrasonic signal being reflected back according to staff trains the model for identifying palm movement tendency obtained；
Gesture identifying device, for utilizing the gesture identification model that training in advance is good, to step 1) the palm movement tendency time series that obtains carries out gesture identification；Wherein, described gesture identification model is to train, according to the training dataset being made up of palm movement tendency time series, the model for identifying gesture obtained.
12. system according to claim 11, wherein, the speaker that intelligent mobile terminal is intrinsic launch ultrasonic signal and the ultrasonic signal being reflected back via staff by the mike collection that described intelligent mobile terminal is intrinsic constantly.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610127516.1A CN105807923A (en) | 2016-03-07 | 2016-03-07 | Ultrasonic wave based volley gesture identification method and system |
| PCT/CN2016/085475 WO2017152531A1 (en) | 2016-03-07 | 2016-06-12 | Ultrasonic wave-based air gesture recognition method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610127516.1A CN105807923A (en) | 2016-03-07 | 2016-03-07 | Ultrasonic wave based volley gesture identification method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105807923A true CN105807923A (en) | 2016-07-27 |
Family
ID=56466783
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610127516.1A Pending CN105807923A (en) | 2016-03-07 | 2016-03-07 | Ultrasonic wave based volley gesture identification method and system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN105807923A (en) |
| WO (1) | WO2017152531A1 (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106446801A (en) * | 2016-09-06 | 2017-02-22 | 清华大学 | Micro-gesture identification method and system based on ultrasonic active detection |
| CN106951861A (en) * | 2017-03-20 | 2017-07-14 | 上海与德科技有限公司 | Ultrasonic wave gesture identification method and device |
| CN107450724A (en) * | 2017-07-31 | 2017-12-08 | 武汉大学 | A kind of gesture identification method and system based on dual-channel audio Doppler effect |
| CN107943300A (en) * | 2017-12-07 | 2018-04-20 | 深圳大学 | A kind of gesture identification method and system based on ultrasonic wave |
| CN108562890A (en) * | 2017-12-29 | 2018-09-21 | 努比亚技术有限公司 | Calibration method, device and the computer readable storage medium of ultrasonic wave characteristic value |
| CN108959866A (en) * | 2018-04-24 | 2018-12-07 | 西北大学 | A kind of lasting identity identifying method based on high frequency sound wave frequency |
| WO2019105376A1 (en) * | 2017-11-30 | 2019-06-06 | 腾讯科技（深圳）有限公司 | Gesture recognition method, terminal and storage medium |
| CN110031827A (en) * | 2019-04-15 | 2019-07-19 | 吉林大学 | A kind of gesture identification method based on ultrasonic distance measurement principle |
| CN110780741A (en) * | 2019-10-28 | 2020-02-11 | Oppo广东移动通信有限公司 | Model training method, application running method, device, medium and electronic equipment |
| CN111124108A (en) * | 2019-11-22 | 2020-05-08 | Oppo广东移动通信有限公司 | Model training method, gesture control method, device, medium and electronic equipment |
| CN111562843A (en) * | 2020-04-29 | 2020-08-21 | 广州美术学院 | Positioning method, device, equipment and storage medium for gesture capture |
| CN112740219A (en) * | 2018-11-19 | 2021-04-30 | 深圳市欢太科技有限公司 | Method and device for generating gesture recognition model, storage medium and electronic equipment |
| CN113450537A (en) * | 2021-06-25 | 2021-09-28 | 北京小米移动软件有限公司 | Fall detection method and device, electronic equipment and storage medium |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109947249B (en) * | 2019-03-15 | 2024-03-19 | 努比亚技术有限公司 | Interaction method of wearable device, wearable device and computer storage medium |
| CN111625087A (en) * | 2020-04-28 | 2020-09-04 | 中南民族大学 | Gesture collection and recognition system |
| CN112883849A (en) * | 2021-02-02 | 2021-06-01 | 北京小米松果电子有限公司 | Gesture recognition method and device, storage medium and terminal equipment |
| CN113095386B (en) * | 2021-03-31 | 2023-10-13 | 华南师范大学 | Gesture recognition method and system based on triaxial acceleration space-time feature fusion |
| CN115884067A (en) * | 2021-09-28 | 2023-03-31 | 华为技术有限公司 | Equipment networking and sound channel configuration method and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102982315A (en) * | 2012-11-05 | 2013-03-20 | 中国科学院计算技术研究所 | Gesture segmentation recognition method capable of detecting non-gesture modes automatically and gesture segmentation recognition system |
| US20130182539A1 (en) * | 2012-01-13 | 2013-07-18 | Texas Instruments Incorporated | Multipath reflection processing in ultrasonic gesture recognition systems |
| CN104751141A (en) * | 2015-03-30 | 2015-07-01 | 东南大学 | ELM gesture recognition algorithm based on feature image full pixel gray values |
| CN104766038A (en) * | 2014-01-02 | 2015-07-08 | 株式会社理光 | Palm opening and closing action recognition method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104616028B (en) * | 2014-10-14 | 2017-12-12 | 北京中科盘古科技发展有限公司 | Human body limb gesture actions recognition methods based on space segmentation study |
-
2016
- 2016-03-07 CN CN201610127516.1A patent/CN105807923A/en active Pending
- 2016-06-12 WO PCT/CN2016/085475 patent/WO2017152531A1/en active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130182539A1 (en) * | 2012-01-13 | 2013-07-18 | Texas Instruments Incorporated | Multipath reflection processing in ultrasonic gesture recognition systems |
| CN102982315A (en) * | 2012-11-05 | 2013-03-20 | 中国科学院计算技术研究所 | Gesture segmentation recognition method capable of detecting non-gesture modes automatically and gesture segmentation recognition system |
| CN104766038A (en) * | 2014-01-02 | 2015-07-08 | 株式会社理光 | Palm opening and closing action recognition method and device |
| CN104751141A (en) * | 2015-03-30 | 2015-07-01 | 东南大学 | ELM gesture recognition algorithm based on feature image full pixel gray values |
Non-Patent Citations (1)
| Title |
|---|
| 杨晓东等: "面向可穿戴设备的超声波手势识别方法", 《计算机科学》 * |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106446801B (en) * | 2016-09-06 | 2020-01-07 | 清华大学 | Micro-gesture recognition method and system based on ultrasonic active detection |
| CN106446801A (en) * | 2016-09-06 | 2017-02-22 | 清华大学 | Micro-gesture identification method and system based on ultrasonic active detection |
| CN106951861A (en) * | 2017-03-20 | 2017-07-14 | 上海与德科技有限公司 | Ultrasonic wave gesture identification method and device |
| CN107450724A (en) * | 2017-07-31 | 2017-12-08 | 武汉大学 | A kind of gesture identification method and system based on dual-channel audio Doppler effect |
| WO2019105376A1 (en) * | 2017-11-30 | 2019-06-06 | 腾讯科技（深圳）有限公司 | Gesture recognition method, terminal and storage medium |
| CN109857245A (en) * | 2017-11-30 | 2019-06-07 | 腾讯科技（深圳）有限公司 | A kind of gesture identification method and terminal |
| CN109857245B (en) * | 2017-11-30 | 2021-06-15 | 腾讯科技（深圳）有限公司 | Gesture recognition method and terminal |
| CN107943300A (en) * | 2017-12-07 | 2018-04-20 | 深圳大学 | A kind of gesture identification method and system based on ultrasonic wave |
| CN108562890A (en) * | 2017-12-29 | 2018-09-21 | 努比亚技术有限公司 | Calibration method, device and the computer readable storage medium of ultrasonic wave characteristic value |
| CN108959866A (en) * | 2018-04-24 | 2018-12-07 | 西北大学 | A kind of lasting identity identifying method based on high frequency sound wave frequency |
| CN108959866B (en) * | 2018-04-24 | 2020-10-23 | 西北大学 | Continuous identity authentication method based on high-frequency sound wave frequency |
| CN112740219A (en) * | 2018-11-19 | 2021-04-30 | 深圳市欢太科技有限公司 | Method and device for generating gesture recognition model, storage medium and electronic equipment |
| CN110031827A (en) * | 2019-04-15 | 2019-07-19 | 吉林大学 | A kind of gesture identification method based on ultrasonic distance measurement principle |
| CN110031827B (en) * | 2019-04-15 | 2023-02-07 | 吉林大学 | Gesture recognition method based on ultrasonic ranging principle |
| CN110780741A (en) * | 2019-10-28 | 2020-02-11 | Oppo广东移动通信有限公司 | Model training method, application running method, device, medium and electronic equipment |
| CN110780741B (en) * | 2019-10-28 | 2022-03-01 | Oppo广东移动通信有限公司 | Model training method, application running method, device, medium and electronic equipment |
| CN111124108A (en) * | 2019-11-22 | 2020-05-08 | Oppo广东移动通信有限公司 | Model training method, gesture control method, device, medium and electronic equipment |
| CN111124108B (en) * | 2019-11-22 | 2022-11-15 | Oppo广东移动通信有限公司 | Model training method, gesture control method, device, medium and electronic equipment |
| CN111562843A (en) * | 2020-04-29 | 2020-08-21 | 广州美术学院 | Positioning method, device, equipment and storage medium for gesture capture |
| CN113450537A (en) * | 2021-06-25 | 2021-09-28 | 北京小米移动软件有限公司 | Fall detection method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2017152531A1 (en) | 2017-09-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105807923A (en) | Ultrasonic wave based volley gesture identification method and system | |
| CN110717339B (en) | Semantic representation model processing method and device, electronic equipment and storage medium | |
| US9720515B2 (en) | Method and apparatus for a gesture controlled interface for wearable devices | |
| CN103035135B (en) | Children cognitive system based on augment reality technology and cognitive method | |
| Amma et al. | Airwriting: Hands-free mobile text input by spotting and continuous recognition of 3D-space handwriting with inertial sensors | |
| CN110853617B (en) | Model training method, language identification method, device and equipment | |
| Amma et al. | Airwriting: a wearable handwriting recognition system | |
| Bukhari et al. | American sign language translation through sensory glove; signspeak | |
| CN110853618A (en) | Language identification method, model training method, device and equipment | |
| CN107369439B (en) | Voice awakening method and device | |
| CN107221320A (en) | Train method, device, equipment and the computer-readable storage medium of acoustic feature extraction model | |
| CN103294199B (en) | A kind of unvoiced information identifying system based on face's muscle signals | |
| McIntee | A TaskModel of Free-SpaceMovement-Based Gestures | |
| Sadek et al. | A new approach for designing a smart glove for Arabic Sign Language Recognition system based on the statistical analysis of the Sign Language | |
| CN105844216A (en) | Detection and matching mechanism for recognition of handwritten letters using WiFi signals | |
| Das et al. | Smart glove for sign language communications | |
| CN105760825A (en) | Gesture identification system and method based on Chebyshev feed forward neural network | |
| CN110443113A (en) | A kind of virtual reality Writing method, system and storage medium | |
| CN112733609A (en) | Domain-adaptive Wi-Fi gesture recognition method based on discrete wavelet transform | |
| CN117390409A (en) | Method for detecting reliability of answer generated by large-scale language model | |
| CN101901348B (en) | Normalization based handwriting identifying method and identifying device | |
| CN109766951A (en) | A kind of WiFi gesture identification based on time-frequency statistical property | |
| CN111913575B (en) | Method for recognizing hand-language words | |
| Sabri et al. | Option for optimal extraction to indicate recognition of gestures using the self-improvement of the micro genetic algorithm | |
| CN110621038B (en) | Method and device for realizing multi-user identity recognition based on WiFi signal detection gait |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20160727 |
|
| WD01 | Invention patent application deemed withdrawn after publication |