CN106648068A - Method for recognizing three-dimensional dynamic gesture by two hands - Google Patents
Method for recognizing three-dimensional dynamic gesture by two hands Download PDFInfo
- Publication number
- CN106648068A CN106648068A CN201610998758.8A CN201610998758A CN106648068A CN 106648068 A CN106648068 A CN 106648068A CN 201610998758 A CN201610998758 A CN 201610998758A CN 106648068 A CN106648068 A CN 106648068A
- Authority
- CN
- China
- Prior art keywords
- mobile terminal
- gesture
- data
- master
- identification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention provides a method for recognizing a three-dimensional dynamic gesture by two hands. In the method, two mobile terminals are utilized; the two mobile terminals adopt a master/slave mode; one mobile terminal is used as a master mobile terminal, and the other mobile terminal is used as a slave mobile terminal; the two mobile terminals respectively track gestures of a left hand and a right hand; the master mobile terminal scans connectable external equipment in the outside and establishes wireless data connection with the slave mobile terminal; the master mobile terminal collects gesture data; the slave mobile terminal collects gesture data and sends the gesture data to the master mobile terminal in real time through wireless data connection for data processing, synchronization and gesture recognition of two hands; and the master mobile terminal sends results recognized by the gesture data obtained by the master mobile terminal and the slave mobile terminal to to-be-controlled equipment. According to the method, the hands of people are directly used as an input end of intelligent equipment, interaction between human and a computer does not need other medium any more, the interaction efficiency is improved and various error recognition caused by intermediate mediums is avoided.
Description
Technical field
The present invention relates to human-computer interaction technique field, more particularly to a kind of dynamic gesture identification method and system.
Background technology
With the progress of each side's surface technology, it is desirable to obtain more preferable man-machine interaction experience, to interactive real-time, know
The not requirement of the rigid index such as rate is also increasingly harsher, or even starts to wish by some small information of itself, such as finger
Information, information of acoustic wave, eyeball information, brain wave information, muscle information etc. are realizing the communication with machine.This promotes many experts
Scholar constantly searches for excavating man-machine interaction mode more convenient, more rapidly, more accurate, more natural, and begins to focus on and utilize people
The body language of class itself realizing apparatus control, and wherein gesture as variation pattern at most, be best able to express people's wish
One of mode, become the emphasis of concern.Before the epoch that brain wave controls the world arrive, gesture is most possible change
The research point of penetration of conventional human's interactive mode.Because both hands be body posture it is most changeable, most expressivity one, it can
To express the idea and wish of people most possibly.Not by any medium, directly using a simple gesture motion just
Life can be manipulated, is the preferable man-machine interaction mode that current people are pursued.This brand-new mode of operation so that Ren Lei
It is this thing thing connected interconnection intelligence epoch, real to realize controlling the life of oneself with the hand of oneself.Gesture also can will be man-machine
Interactive mode extend to space three-dimensional technology of identification from traditional planar technology of identification, and that realizes the development of more technologies can
Can property.The both hands Three-Dimensional Dynamic Gesture Recognition based on IOS mobile terminals to be done of the invention, using more next in people's life
The instrument that the IOS intelligent slidings moved end more popularized gathers directly as gesture and recognizes, realizes friendly with PC ends or other-end
Man-machine interaction mode.
The solution of several main flows was produced in the evolution of Gesture Recognition, for example：
1) scheme based on data glove：Starting is earliest, is also a kind of scheme of most direct process problem.The program has
Data volume is little, speed is high, it is affected by environment less and can direct access gesture data many advantages, such as.But require that user wears
The gloves of complexity are worn, Consumer's Experience is greatly reduced, and the protective layer of the sensor coating higher hardness to attachment thereon exists
Technically acquire a certain degree of difficulty, and relatively fragile equipment there is also inconvenience in carrying.
2) processing scheme based on machine vision：Current most popular Gesture Recognition solution, that is, image
Head image procossing, such as the Kinect of Microsoft.Maximum advantage is that user need not wear any equipment, and man-machine interaction is good.
But this technology is more restricted than larger for space, environment have, therefore except some large-scale somatic sensation television games are used, is giving birth to
Popularization degree in work is not high.
3) scheme based on SEMG：The field of multi-crossed disciplines is in, except needing user as data glove scheme
Wear outside electrode, similar with the scheme of machine vision, the program is also easily disturbed by factors such as external environment condition and users.It is outside
The interference of environment includes the change of the measuring electrode contact resistance that the interference of electromagnetic environment and ambient humidity change are caused.Meanwhile,
The body ＆ mind state of user has in different situations different a reference values, causes system to be difficult to demarcate.
Existing human-computer interaction technology is suffered from the drawback that：
(1) mechanical keyboard interaction is most ripe, but while be also most poorly efficient interactive mode, because of its cost, loss, poorly efficient
Deng a variety of causes, just slowly eliminate in mobile device；
(2) touch screen operation is used as man-machine interaction mode emerging in recent years, operation is succinct with it, meet user's custom,
Learning cost is low, can multi-point touch the advantages of, become the effective interactive mode of mobile device of new generation.But, touch technology is not
It is evitable to need user to be operated on touchpad, limit the application scenarios and scope of touch technology.
(3) speech recognition is also at present a kind of highly developed interactive mode, and discrimination is higher, especially with iPhone
The appearance of middle Siri, causes interactive voice mode to step and has gone up a new step based on the technology of identification of natural-sounding.However, language
Sound interactive mode can be affected by locale language difference and personal phonetic difference, meanwhile, natural-sounding identification needs the moment
Networking, the shortcoming of high energy consumption, greatly limit application of the voice technology in wearable device.
In view of the above problems, the inventor of this case carries in patent document 1 (China Patent Publication No. CN105929940A)
A kind of quick three-dimensional dynamic gesture identification method based on subdivision method of characteristic and system are gone out, present in prior art
Problem, devises a compact hardware platform of outward appearance, and 3 axle accelerations of user gesture, gyroscope are gathered with accelerometer
To gather 3 axis angular rates of user gesture, magnetometer gathering 3 axle magnetic induction intensity of user gesture, using filtering algorithm
Eliminate and carry out attitude algorithm again after data error, obtain real-time three attitude angles (course angle, roll angle, the angle of pitch), so
Afterwards these information datas are passed through into proposed feature analysis al, on the basis of certain accuracy and real-time is guaranteed,
Respectively to movement, rotation, make hook, picture and the Three-Dimensional Dynamic gesture itself with certain discrimination such as pitch, rock, tapping and know
Not, so as to realizing corresponding application.But, only for singlehanded gesture identification after patent document 1, therefore, need a kind of both hands three badly
Dimension dynamic gesture identification method.
The content of the invention
It is an object of the invention to the problems such as overcoming bimanual input identification inaccurate, inconvenient, there is provided a kind of both hands are three-dimensional
Dynamic gesture identification method and system.
It is that, up to above-mentioned purpose, the present invention is achieved through the following technical solutions：
A kind of both hands Three-Dimensional Dynamic gesture identification method, using two mobile terminals, two mobile terminal devices are using master
Slave pattern, will wherein one mobile terminal as main frame, referred to as master mobile terminal, another mobile terminal claims as slave
Be from mobile terminal；Two mobile terminals track respectively the gesture of left hand and the right hand；Master mobile terminal is outwardly scanned can be even
The external equipment for connecing, is connected with wireless data is set up from mobile terminal；Master mobile terminal gathers gesture data；Adopt from mobile terminal
Collection gesture data, and gesture data is sent in real time by wireless data connection for master mobile terminal carries out data processing, same
Step and bimanual input identification；The result that master mobile terminal is recognized master mobile terminal with the gesture data obtained from mobile terminal
It is sent to control equipment to be controlled.
Further, 9 axle sensor modules are integrated with the mobile terminal device, wherein, the measurement of 3 axis accelerometers adds
Speed, 3 axle gyroscopes measurement angular speed and 3 axle magnetometer measures magnetic fields.
Further, the synchronous process includes：KVO is detected using the key assignments of mobile terminal operating system, two are monitored
Whether mobile terminal device completes the collection and identification of gesture, if a side completes, into wait state, external operation is to its nothing
Effect；When the opposing party also completes to gather identification process, data command is transmitted to control equipment to be controlled, then make two shiftings
Dynamic terminal device equipment enters ready state, waits bimanual input next time.
Further, when gesture is intercepted, by a stronger threshold values Ats come the motion of detection gesture, here basis
On the beginning and end of gesture is judged by threshold values Atb, Atf of two smaller values, wherein, Ats>Atf、Ats>Atb；Need
Ats and Atf, Atb and the size of both time differences are limited, so as to avoid for the continuous action of a gesture being divided into multiple hands
Gesture；Furthermore, it is necessary to limit the size of Atf and Atb time differences, it is to avoid the accidental shake of user's hand is identified as into opening for gesture
Begin；In order to ensure the integrality of gesture data, the time series data intercepted at Atb and Atf two needs toward two ends suitably to extend
Time span Te.
Specifically, first it is detected that the time series point of Ats, then passes through to search forward, backward on the basis of Ats
Rope, finds the time series point of Atf and Atb, and on here, appropriate expansion time Te length obtains the complete acceleration of gesture
Data.
On the other hand, the invention allows for a kind of both hands Three-Dimensional Dynamic gesture recognition system, the system is including two
Mobile terminal and control equipment to be controlled；Each of the mobile terminal includes that gesture data collecting unit, wireless data lead to
Letter unit, two mobile terminal devices adopt master slave mode, will wherein one mobile terminal as main frame, referred to as based on it is mobile eventually
End, another mobile terminal as slave, referred to as from mobile terminal；Two mobile terminals track respectively the hand of left hand and the right hand
Gesture；Master mobile terminal outwardly scans attachable external equipment, is connected with wireless data is set up from mobile terminal；It is main mobile whole
The gesture data collecting unit collection gesture data at end；Gesture data is gathered from the gesture data collecting unit of mobile terminal, and
Gesture data is sent in real time master mobile terminal by wireless data communication unit carries out data processing, synchronization and both hands hand
Gesture is recognized；Master mobile terminal is by wireless data communication unit by master mobile terminal and the gesture data institute that obtains from mobile terminal
The result of identification is sent to control equipment to be controlled.
Further, the wireless data communication unit is bluetooth module, WiFi module or RF modules.
Further, the control equipment to be controlled is PC, game machine, unmanned plane or the VR helmets.
Further, the master mobile terminal be smart mobile phone, it is described from mobile terminal be wearable device.Or, institute
State master mobile terminal and from mobile terminal all be smart mobile phone.Or, the master mobile terminal and from mobile terminal all can to wear
Wear equipment.
The invention has the beneficial effects as follows：The bimanual input recognition methods of the present invention is directly using staff as to smart machine
Input, the interaction of between humans and machines no longer needs other media, improves interactive efficiency and avoids because intermediary is led
The various misrecognitions for causing；Extend the application scenario of smart machine so that man-machine interaction is no longer limited to the privileged site of equipment,
User can be allowed to be inconvenient to talk or be inconvenient to that in the case of taking out equipment certain operation can be carried out, such as in laundry clothes
When by certain gesture come answering cell phone incoming call；The application scenarios of smart machine are enriched, such as bimanual input can be operated
In combination with virtual scene, the operational, recreational of handheld device is enriched；Bimanual input input is shown there is provided a kind of robot
The new approaches of model study, an important ring is exactly User Defined gesture in gesture input；9 axles in IOS intelligent slidings moved end are passed
Sensor effectively increases the precision of identification, enriches the function of realization；Data are carried out using wireless communication technologys such as bluetooths
Transmission, it is low in energy consumption, and multi-user is supported while using.
Description of the drawings
Fig. 1 is the hardware block diagram of double gesture recognition systems that the method for the present invention is based on；
Fig. 2 is bimanual input data syn-chronization schematic diagram；
Fig. 3 is the gesture recognition system implementation flow chart of mobile device；
Fig. 4 (a) is that the gesture data based on FBGD intercepts schematic diagram；
Fig. 4 (b) is the data after the gesture data based on FBGD is intercepted.
Specific embodiment
Accompanying drawing is combined below by specific embodiment to be described in further detail the present invention.
Intelligent mobile end equipment is popularized comprehensively, and based on mobile-terminal platform gesture identification is developed, it will allow more
People uses the facility of upper gesture identification.9 axle sensor modules are integrated with mobile terminal device, and (measurement of 3 axis accelerometers accelerates
Degree, 3 axle gyroscopes measurement angular speed and 3 axle magnetometer measures magnetic fields), and possess higher accuracy of identification, it is possible to achieve it is right
In effective identification of complicated gesture.The present invention is interacted using mobile phone with the cooperation of both wrist-watch or bracelet, realizes the interaction of data, real
Existing bimanual input interaction, greatly enriches the function of single gesture.
The hardware block diagram of double gesture recognition systems that the method for the present invention is based on as shown in Figure 1, including two movements
Terminal device and control equipment to be controlled, such as PC, game machine, unmanned plane.Communication between the parties passes through Bluetooth technology reality
It is existing.Two mobile terminal devices adopt master slave mode, will wherein one mobile terminal (such as iPhone) as main frame, referred to as based on
Mobile terminal, another mobile terminal (such as iWatch) as slave, referred to as from mobile terminal；Two mobile terminals respectively with
The gesture (for example, left hand holds iPhone, and the right hand wears iWatch) of track left hand and the right hand.Master mobile terminal is outwardly scanned can
The external equipment of connection, is connected with blue-teeth data is set up from mobile terminal；Gesture data is gathered from mobile terminal, and by bluetooth
Gesture data is sent in real time master mobile terminal carries out the identification of data processing, synchronization and bimanual input；Then, by main movement
Terminal is sent to the Bluetooth receptions module of control equipment to be controlled with the result that the gesture data obtained from mobile terminal is recognized；Most
Afterwards, control equipment to be controlled carries out corresponding control operation according to the recognition result.
Because bimanual input has the order of action priority, when a side collects gesture, perhaps the opposing party does not have started,
Or yet in identification.Thus cause nonsynchronous problem of two hand data, it is therefore necessary to formulate a sets of data and complete
Agreement is controlling the asynchronous problem of gesture.The present invention is proposed using key assignments detection (KVO) of mobile terminal operating system, monitoring two
Whether equipment completes the collection and identification of gesture.If a side completes, into wait state, external operation is invalid to its.Until another
When one side also completes to gather identification process, data command is transmitted to control equipment to be controlled, then make equipment enter ready shape
State, waits bimanual input next time.Its operation logic is as shown in Figure 2.
The essence of gesture identification is according to gesture model gesture motion to be classified using Gesture Recognition Algorithm.Gesture identification side
The quality of method is directly connected to the efficiency and precision of gesture identification.Conventional gesture identification method includes：
(1) DTW algorithms are a kind of non-linear regular technologies that Time alignment and distance measure calculations incorporated are got up, and are had
Nonlinear Time normalizes effect.The fluctuation on time shaft is approximately built using the non-linear warping function of certain specified attribute
Mould, is allowed to reach farthest overlap with another pattern by the time shaft of one of pattern of stretching, and makes residual error distance
Minimum, so as to eliminate the time difference between two space-time intermediate schemes.Actually it be, the simplification of hidden Markov model,
For fairly simple time series, both they are of equal value.Method allows to have between test pattern and reference model to fill
The elasticity divided, so as to realize classification.
(2) hidden Markov model is the extension of Markov model.Markov model describes a random mistake
Transfer between journey and state.Hidden Markov model is described between two random processes, one random process description output and state
Probabilistic relation, i.e. output is the transfer relationship that a functional of a stochastic process of state another random process is described between state.See
The person of examining is it can be seen that output, but cannot see that the transfer between state, that is, the transfer between state is implicit.Due to hidden
The particularity of Markov model topological structure, causes it excessively complicated when hand signal is analyzed so that trains and recognizes
It is computationally intensive, especially in continuous hidden Markov model, need to calculate substantial amounts of state probability density, need what is estimated
Number of parameters is more so that training and the speed for recognizing are relatively slow.
(3) artificial neural network, in gesture identification field, artificial neural network is a kind of application instrument widely.
Artificial neural network has self-organizing and self-learning capability, and noise resisting ability is strong, with very strong fault-tolerance and robustness.Manually
Neutral net is that a kind of Information Processing Network of complexity is constituted by the way that substantial amounts of simple process unit is extensively coupled together, its
Middle processing unit and its be connected with each other pattern be use for reference people's brain neuron structure and connection mechanism design.This network has
The ability of learning and memory similar with human brain, knowledge are summarized and input information feature extraction ability.Through development for many years, manually
Neutral net has had many models, such as fuzzy neural network and BP neural network.At present application be more widely with
BP neural network based on back propagation learning algorithm.
(4) machine learning, it is common that the closest methods of K_ and SVMs are had based on the sorting technique of machine learning
Method.The basic thought of the closest methods of K_ is, according to traditional vector space model, content of text formalization to be characterized into sky
Between in weighted feature vector, for a test object, calculate its each Sample Similarity concentrated with training sample, find out
Individual K most like text, the classification according to belonging to Weighted distance judges test object.SVMs is using training error as excellent
The constraints of change problem, is minimized as target using fiducial range value, is a kind of based on empirical risk minimization
Learning method.The computation complexity of SVMs depends on the number of supporting vector, rather than the dimension of sample space, and this is at certain
Plant and avoided " dimension space " in meaning.And in the method, additions and deletions non-supporting vector sample does not affect on model, to letter
Several selections are nor very sensitive.Additionally, supporting vector sample set has certain robustness.
In addition, the method that also HMM and threshold values compare combination, methods of the HMM in combination with neutral net, Bayes
The method that network and SVMs combine, using the recognition methods of the features such as gesture slope characteristic.
Because higher to the requirement of gesture recognition speed, the present invention is thick with feature using E_DTW dynamic times template matching method
The method that classification be combined with each other.Therefore when gesture motion is selected, need to carry out substantial amounts of sample view and analysis to gesture, lead to
Sample collection and waveform observation are crossed, the standard form of gesture identification is obtained, while self-defined gesture identification database realizing is built,
Realize dynamic gesture template matches.8 following class gestures are temporarily adopted at present as the gesture of system default.As shown in table 1：
The gesture collection of table 1 is defined
There is certain discrimination, the characteristic quantity that the sensor information of each of which is constituted between above-mentioned gesture motion
Between there is relatively higher identifiability, therefore different graders can be set up according to the characteristic value of different gestures.Root first
Eight big class gestures are carried out with the pre- classification for being divided into 3 classes, identifying its place according to the characteristic value calculated, redesign algorithm is directed to
Rotation class and mobile class action are carefully divided, and identify the direction of motion, fast using subdivision eigenvalue Method so as to be intended to
Speed efficiently realizes the classification and identification of gesture.
The gesture recognition system implementation flow process of mobile device is as shown in Figure 3.
DTW template matches technology of identification, its core concept is to carry out the initial data of input with the template for prestoring
Matching, by measuring the similarity between two templates identification mission is completed.Conventional Distance conformability degree computational methods have plus
Power Euclidean distance method, correlation coefficient process and logarithm Furthest Neighbor.Template matches need to solve input data and prestore template
The inconsistent problem of length of time series, because even being same gesture, the change that its duration all can be random.To understand
Certainly this time calibration problem, typical sequential template matches are DTW.
Presorted for the length characteristic of gesture data, according to the length and the size of its energy of different gestures, come
Realize for the gesture for differing greatly is presorted.Therefore the premise is intercepting effective to gesture data, i.e., from all readings
The paragraph from the origin-to-destination of gesture is intercepted in the initial data got, then again coordinate is carried out to the data segment that gets of intercepting
The conversion of system.Having carried out cutting and the gesture data section after coordinate system conversion could carry out gesture identification for extracting feature,
Next, will describe in detail to gesture cutting, coordinate system conversion and feature recognition method.
Because user moves or hand inevitably shake and the impact of sensor precision itself during gesture,
The gesture motion data of acceleration transducer collection are inevitably subject to noise jamming, situation about shaking up and down occur.Steadily
Denoising is mainly processed the interference such as ambient noise data, will affect to minimize as far as possible.I adopts, simple rolling average
The method of line filter carries out steady denoising to the acceleration information for obtaining.SMA can be on the basis of quick response be kept
Filter off random noise.The derivation formula of SMA wave filters is as follows:
SMAnow=(Xi+Xi-1+....+Xi-n+1)/n；N=1,2,3,4 ....
N represents the length of data sequence in above formula.The magnitude relationship of n is to smooth effect.N is too little, and flattening effect is failed to understand
Aobvious n is excessive, and flattening effect is good, but is easily caused gesture information loss.With reference to experience, depending on different situations, n typically take 5 to
15。
When calculating, following formula can be directly used
SMAnow=SMAprevious-Xi-n/n+Xi/n。
The judgement of the beginning and end of gesture motion plays vital effect in gesture identification.Part described previously herein
Gesture identification of the great majority already mentioned above based on acceleration transducer needs the extra behaviour such as button during gesture execution
Make to inform the beginning and end of gesture recognition system gesture.Beginning and the knot of gesture are judged by touching the button this mode
Beam, can ensure that gesture recognition system to the accurate sampling of gesture motion really, but its have the following disadvantages one be user by
During button, due to the shake of hand, it is user in the mistake for touching the button that noise jamming two will necessarily be carried out to gesture data band
Cheng Zhong, needs the position come confirming button by eyes, has disperseed the notice of user, reduces the Experience Degree of user, unfavorable
It is that, for blind or dumb person or the inflexible people of finger, the operation such as touch the button is compared it in freer, natural man-machine interaction three
It is more difficult, it is unfavorable for popularizing for gesture recognition system.
In the gesture motion based on threshold value judges, the setting of threshold values has been largely fixed the accuracy for judging.
If threshold values is too low, the deliberate action between user is careless can all be judged to the beginning of gesture, if causing erroneous judgement threshold values mistake
Height, then can miss the judgement to normal gesture.For this purpose, with reference to the characteristics of gesture motion, invention adopts the gesture judgement side of FBGD
Method.By a stronger threshold values Ats come the motion of detection gesture, on here basis by the threshold values Atf of two smaller values,
Judging the beginning and end of gesture, Ats, Atf, Atb be in time T Atb, the maximum of gesture acceleration information sequence and most
The threshold values of the difference of little value.Meanwhile, we limit Ats and Atf, Atb and the size of both time differences, so as to avoid a gesture
Continuous action be divided into multiple gestures.Furthermore, it is necessary to limit the size of Atf and Atb time differences, it is to avoid user's hand is accidental
Shake be identified as the beginning of gesture.In order to ensure the integrality of gesture data, the time series number intercepted at Atb and Atf two
According to needs toward appropriate expansion time length Te in two ends.It is as shown such as accompanying drawing 4 (a) and accompanying drawing 4 (b), after the data of gesture are smoothed,
When sending into gestures detection, first it is detected that the time series point of Ats, then by searching forward, backward on the basis of Ats
Rope, finds the time series point of Atf and Atb, and on here, appropriate expansion time Te length obtains the complete acceleration of gesture
Data.
Different gesture data data lengths are different, and amplitude is also differed.The even same gesture of same person
Between, there is also difference, it is therefore necessary to by data normalization to same amplitude range, sample identical length, such advantage
Have：1. gesture quantitative criteria is unified, it is to avoid the excessive or too small gesture of amplitude is misjudged；2. the undistorted premise of waveform is being ensured
Under, fixed sample point actually reduces computational complexity, improves recognition speed.
Gesture Recognition Algorithm of the present invention includes but is not limited to above-mentioned algorithm, it would however also be possible to employ in patent document 1
The quick three-dimensional dynamic gesture identification method based on subdivision method of characteristic recorded.
Embodiment 1
Traditional UAV Flight Control, usually using RF RF remotes.Possess a long antenna outside it, it is distant
Control instruction be all by casing outside controlling switch and button, modulation through internal circuit, coding, then by high-frequency signal
Amplifying circuit is gone out electromagnetic radiation by antenna.At present conventional telecontrol transmitter has three types：Boxlike button hand-held,
Portable rod type remote-control, hand-held gun-type remote control.Such as remote control rod-type emitter has two control sticks, and left side bar is used for controlling unmanned plane
Raising and lowering, the right bar control unmanned plane during flying direction.It is furnished with LCD Panel, display working condition and work(in central authorities
Energy.Its advantage is that communication distance is remote, and functional integration is high, adapts to unmanned plane flight control under various circumstances.But its is right
Human users' requirement is higher, and generally requiring carries out professional training.
With the further investigation of gesture identification, can by gesture identification in combination with unmanned aerial vehicle (UAV) control, realize gesture control without
Man-machine flight path.Space gesture track is identified by mobile end equipment, by recognition result by wireless device to nothing
Man-machine transmission control instruction.On the premise of the stable safe flight of unmanned plane is ensured, user can be with the flight of self-defined unmanned plane
Track.The present invention is based on the dynamic gesture track identification of MEMS sensor.Effective track sample is trained, and to defeated
The centroid trajectory for entering gesture is identified, so as to reach the demand of control unmanned plane during flying track.It is by gesture and aircraft
System is integrated, and the gesture control to unmanned plane during flying track is realized in systems.
Embodiment 2
In the game of first person, with reference to gesture identification, gesture identification equipment platform is made as game paddle
With.Additionally, the somatic sensation television game sensitivity based on machine vision popular on the market at present is still not enough, experience effect is not good.If
Integrated camera using vision and hardware platform, can combine on intelligent terminal, just can determine that the position of hardware platform, then enter
One step carries out finely positioning by data such as the gyroscopes on hardware platform, and accuracy of identification and sensitive is improve to a great extent
Degree.Due to the identification of both hands, during the game of large-scale sports fistfight of many can be applied to, there is provided better man-machine interaction body
Test.
Embodiment 3
Both hands equipment, with reference to the VR helmets, realizes user's game experiencing of immersion, and using the helmet 3D images are projected out, knot
Close mobile device, it is possible to the game experiencing that realization is personally on the scene.Because the autgmentability of both hands, numerous interaction sides can be realized
Formula, can give the general operation handle of the user abundant function to be provided.
Embodiment 4
PC ends are widely present the operation for needing directionality to control, the such as upper downslide of the broadcasting of video, the switching of picture, webpage
Move, traditional mouse control limits larger in distance, handling low, affect people to interact with the close friend at PC ends.
F.F. and retrogressing when at present video is seen at PC ends, are substantially realized by mouse and keyboard, but for user
For be not very to facilitate, therefore propose use, portable mobile device controls video using gesture identification
Play, or the scaling of Photo Viewer, the function of switching.Four kinds of gestures can be defined, it is relative with these four orders respectively
Should, the purpose of control player plays has just been reached eventually through gesture, and due to the popularization of gesture identification equipment, further
Meet most user.
Above content is to combine specific preferred embodiment further description made for the present invention, it is impossible to assert
The present invention be embodied as be confined to these explanations.For general technical staff of the technical field of the invention,
On the premise of without departing from present inventive concept, some simple deduction or replace can also be made, should all be considered as belonging to the present invention's
Protection domain.
Claims (10)
1. a kind of both hands Three-Dimensional Dynamic gesture identification method, it is characterised in that：Methods described utilizes two mobile terminals, two shiftings
Dynamic terminal device adopts master slave mode, will wherein one mobile terminal as main frame, referred to as master mobile terminal, another movement
Terminal as slave, referred to as from mobile terminal；Two mobile terminals track respectively the gesture of left hand and the right hand；Master mobile terminal
Attachable external equipment is outwardly scanned, is connected with wireless data is set up from mobile terminal；Master mobile terminal gathers gesture number
According to；Gesture data is gathered from mobile terminal, and gesture data is sent in real time by wireless data connection for master mobile terminal
Carry out the identification of data processing, synchronization and bimanual input；Master mobile terminal is by master mobile terminal and the gesture that obtains from mobile terminal
The result that data are recognized is sent to control equipment to be controlled.
2. method according to claim 1, it is characterised in that：9 axle sensor moulds are integrated with the mobile terminal device
Block, wherein, 3 axle acceleration measures accelerations, 3 axle gyroscopes measurement angular speed and 3 axle magnetometer measures magnetic fields.
3. method according to claim 1, it is characterised in that：The synchronous process includes：It is using mobile terminal operation
The key assignments detection KVO of system, monitors collection and identification that whether two mobile terminal devices complete gesture, if a side completes, enters
Wait state, external operation is invalid to its；Until the opposing party also complete gather identification process when, by data command transmit to
Control equipment to be controlled, then makes two mobile terminal device equipment enter ready state, waits bimanual input next time.
4. method according to claim 1, it is characterised in that：When gesture is intercepted, by a stronger threshold values Ats come
The motion of detection gesture, judges the beginning and end of gesture by threshold values Atb, Atf of two smaller values on here basis,
Wherein, Ats>Atf、Ats>Atb；Need to limit Ats and Atf, Atb and the size of both time differences, so as to avoid a hand
The continuous action of gesture is divided into multiple gestures；Furthermore, it is necessary to limit the size of Atf and Atb time differences, it is to avoid by user's hand idol
Right shake is identified as the beginning of gesture；In order to ensure the integrality of gesture data, the time series intercepted at Atb and Atf two
Data are needed toward appropriate expansion time length Te in two ends.
5. method according to claim 4, it is characterised in that：Specifically, first it is detected that the time series point of Ats, so
Afterwards the time series point of Atf and Atb is found by searching for forward, backward on the basis of Ats, on here, appropriate extension
Time Te length, obtains the complete acceleration information of gesture.
6. method according to claim 1, it is characterised in that：The wireless data is connected as bluetooth, Wifi or RF.
7. a kind of both hands Three-Dimensional Dynamic gesture recognition system, it is characterised in that：The system includes two mobile terminals and treats
Control device；Two mobile terminal devices adopt master slave mode, will wherein one mobile terminal as main frame, referred to as based on move
Terminal, another mobile terminal as slave, referred to as from mobile terminal；Two mobile terminals track respectively left hand and the right hand
Gesture；Master mobile terminal outwardly scans attachable external equipment, is connected with wireless data is set up from mobile terminal；Main movement
Terminal gathers gesture data；Gesture data is gathered from mobile terminal, and is in real time sent out gesture data by wireless data connection
Giving master mobile terminal carries out the identification of data processing, synchronization and bimanual input；Master mobile terminal is by master mobile terminal and from movement
The result that the gesture data that terminal is obtained is recognized is sent to control equipment to be controlled.
8. system according to claim 7, it is characterised in that：9 axle sensor moulds are integrated with the mobile terminal device
Block, wherein, 3 axle acceleration measures accelerations, 3 axle gyroscopes measurement angular speed and 3 axle magnetometer measures magnetic fields.
9. system according to claim 7, it is characterised in that：The synchronous process includes：It is using mobile terminal operation
The key assignments detection KVO of system, monitors collection and identification that whether two mobile terminal devices complete gesture, if a side completes, enters
Wait state, external operation is invalid to its；Until the opposing party also complete gather identification process when, by data command transmit to
Control equipment to be controlled, then makes two mobile terminal device equipment enter ready state, waits bimanual input next time.
10. system according to claim 7, it is characterised in that：The master mobile terminal is smart mobile phone, described from movement
Terminal is wearable device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610998758.8A CN106648068A (en) | 2016-11-11 | 2016-11-11 | Method for recognizing three-dimensional dynamic gesture by two hands |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610998758.8A CN106648068A (en) | 2016-11-11 | 2016-11-11 | Method for recognizing three-dimensional dynamic gesture by two hands |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106648068A true CN106648068A (en) | 2017-05-10 |
Family
ID=58806616
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610998758.8A Pending CN106648068A (en) | 2016-11-11 | 2016-11-11 | Method for recognizing three-dimensional dynamic gesture by two hands |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106648068A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107346207A (en) * | 2017-06-30 | 2017-11-14 | 广州幻境科技有限公司 | A kind of dynamic gesture cutting recognition methods based on HMM |
| CN107592422A (en) * | 2017-09-20 | 2018-01-16 | 上海交通大学 | A kind of identity identifying method and system based on gesture feature |
| CN108052237A (en) * | 2018-01-05 | 2018-05-18 | 上海昶音通讯科技有限公司 | 3D projection touch device and touch method thereof |
| CN108375913A (en) * | 2018-03-28 | 2018-08-07 | 山东大学 | It is a kind of based on the smart home things system and its operation method of NAO robots and application |
| CN109313455A (en) * | 2017-11-16 | 2019-02-05 | 深圳市大疆创新科技有限公司 | Intelligent glasses and its method, holder, control method and the unmanned plane for controlling holder |
| CN109445464A (en) * | 2019-01-08 | 2019-03-08 | 深圳市道通智能航空技术有限公司 | A kind of flight control method and flight control system |
| CN113268143A (en) * | 2020-09-29 | 2021-08-17 | 中国人民解放军军事科学院国防科技创新研究院 | Multimodal man-machine interaction method based on reinforcement learning |
| CN113574496A (en) * | 2019-07-12 | 2021-10-29 | Oppo广东移动通信有限公司 | Method for recognizing gesture for operating terminal device and terminal device |
| CN114115537A (en) * | 2021-11-22 | 2022-03-01 | 中国电子科技集团公司第五十四研究所 | Gesture control method and system for unmanned system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104134060A (en) * | 2014-08-03 | 2014-11-05 | 上海威璞电子科技有限公司 | Sign language interpreting, displaying and sound producing system based on electromyographic signals and motion sensors |
| CN104601795A (en) * | 2014-11-03 | 2015-05-06 | 中国科学技术大学苏州研究院 | Convenient and efficient smartphone user hand identification method |
| CN105160323A (en) * | 2015-09-07 | 2015-12-16 | 哈尔滨市一舍科技有限公司 | Gesture identification method |
| CN105929940A (en) * | 2016-04-13 | 2016-09-07 | 哈尔滨工业大学深圳研究生院 | Rapid three-dimensional dynamic gesture recognition method and system based on character value subdivision method |
-
2016
- 2016-11-11 CN CN201610998758.8A patent/CN106648068A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104134060A (en) * | 2014-08-03 | 2014-11-05 | 上海威璞电子科技有限公司 | Sign language interpreting, displaying and sound producing system based on electromyographic signals and motion sensors |
| CN104601795A (en) * | 2014-11-03 | 2015-05-06 | 中国科学技术大学苏州研究院 | Convenient and efficient smartphone user hand identification method |
| CN105160323A (en) * | 2015-09-07 | 2015-12-16 | 哈尔滨市一舍科技有限公司 | Gesture identification method |
| CN105929940A (en) * | 2016-04-13 | 2016-09-07 | 哈尔滨工业大学深圳研究生院 | Rapid three-dimensional dynamic gesture recognition method and system based on character value subdivision method |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107346207A (en) * | 2017-06-30 | 2017-11-14 | 广州幻境科技有限公司 | A kind of dynamic gesture cutting recognition methods based on HMM |
| CN107346207B (en) * | 2017-06-30 | 2019-12-20 | 广州幻境科技有限公司 | Dynamic gesture segmentation recognition method based on hidden Markov model |
| CN107592422B (en) * | 2017-09-20 | 2019-07-02 | 上海交通大学 | A kind of identity identifying method and system based on gesture feature |
| CN107592422A (en) * | 2017-09-20 | 2018-01-16 | 上海交通大学 | A kind of identity identifying method and system based on gesture feature |
| CN109313455B (en) * | 2017-11-16 | 2021-09-28 | 深圳市大疆创新科技有限公司 | Intelligent glasses, method for controlling holder of intelligent glasses, holder, control method and unmanned aerial vehicle |
| CN109313455A (en) * | 2017-11-16 | 2019-02-05 | 深圳市大疆创新科技有限公司 | Intelligent glasses and its method, holder, control method and the unmanned plane for controlling holder |
| CN108052237A (en) * | 2018-01-05 | 2018-05-18 | 上海昶音通讯科技有限公司 | 3D projection touch device and touch method thereof |
| CN108052237B (en) * | 2018-01-05 | 2022-01-14 | 上海昶音通讯科技有限公司 | 3D projection touch device and touch method thereof |
| CN108375913A (en) * | 2018-03-28 | 2018-08-07 | 山东大学 | It is a kind of based on the smart home things system and its operation method of NAO robots and application |
| CN109445464A (en) * | 2019-01-08 | 2019-03-08 | 深圳市道通智能航空技术有限公司 | A kind of flight control method and flight control system |
| CN113574496A (en) * | 2019-07-12 | 2021-10-29 | Oppo广东移动通信有限公司 | Method for recognizing gesture for operating terminal device and terminal device |
| CN113268143A (en) * | 2020-09-29 | 2021-08-17 | 中国人民解放军军事科学院国防科技创新研究院 | Multimodal man-machine interaction method based on reinforcement learning |
| CN114115537A (en) * | 2021-11-22 | 2022-03-01 | 中国电子科技集团公司第五十四研究所 | Gesture control method and system for unmanned system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106648068A (en) | Method for recognizing three-dimensional dynamic gesture by two hands | |
| WO2021082749A1 (en) | Action identification method based on artificial intelligence and related apparatus | |
| US11493993B2 (en) | Systems, methods, and interfaces for performing inputs based on neuromuscular control | |
| CN105929940B (en) | Quick three-dimensional dynamic gesture identification method and system based on subdivision method of characteristic | |
| Wang et al. | Hear sign language: A real-time end-to-end sign language recognition system | |
| US11567573B2 (en) | Neuromuscular text entry, writing and drawing in augmented reality systems | |
| US10061389B2 (en) | Gesture recognition system and gesture recognition method | |
| CN112739254A (en) | Neuromuscular control of augmented reality systems | |
| WO2013139181A1 (en) | User interaction system and method | |
| US11262851B2 (en) | Target selection based on human gestures | |
| CN205721628U (en) | A kind of quick three-dimensional dynamic hand gesture recognition system and gesture data collecting device | |
| US9619024B2 (en) | Virtual input device and virtual input method | |
| CN104410883A (en) | Mobile wearable non-contact interaction system and method | |
| KR102089002B1 (en) | Method and wearable device for providing feedback on action | |
| Linqin et al. | Dynamic hand gesture recognition using RGB-D data for natural human-computer interaction | |
| CN110412996A (en) | It is a kind of based on gesture and the unmanned plane control method of eye movement, device and system | |
| CN206411610U (en) | A kind of bimanual input identifying system based on IOS terminals | |
| CN107450672B (en) | Wrist type intelligent device with high recognition rate | |
| CN109547706A (en) | Glasses device and system | |
| CN112488157A (en) | Dialog state tracking method and device, electronic equipment and storage medium | |
| CN115171196B (en) | Face image processing method, related device and storage medium | |
| CN114662606A (en) | Behavior recognition method and apparatus, computer readable medium and electronic device | |
| CN111610857A (en) | Gloves with interactive installation is felt to VR body | |
| KR102308970B1 (en) | System and method for inputting touch signal by wearable glass device | |
| CN115793855A (en) | Display interface interaction method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170510 |