CN107589782A - Method and apparatus for the ability of posture control interface of wearable device - Google Patents
Method and apparatus for the ability of posture control interface of wearable device Download PDFInfo
- Publication number
- CN107589782A CN107589782A CN201710543994.5A CN201710543994A CN107589782A CN 107589782 A CN107589782 A CN 107589782A CN 201710543994 A CN201710543994 A CN 201710543994A CN 107589782 A CN107589782 A CN 107589782A
- Authority
- CN
- China
- Prior art keywords
- signal
- user
- sensor
- detected
- known poses
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 68
- 230000005062 synaptic transmission Effects 0.000 claims abstract description 25
- 230000007830 nerve conduction Effects 0.000 claims abstract description 16
- 230000033001 locomotion Effects 0.000 claims description 45
- 210000000707 wrist Anatomy 0.000 claims description 38
- 230000006854 communication Effects 0.000 claims description 27
- 238000004891 communication Methods 0.000 claims description 26
- 238000013527 convolutional neural network Methods 0.000 claims description 22
- 238000004422 calculation algorithm Methods 0.000 claims description 21
- 210000005036 nerve Anatomy 0.000 claims description 16
- 230000007935 neutral effect Effects 0.000 claims description 15
- 238000012549 training Methods 0.000 claims description 10
- 238000002567 electromyography Methods 0.000 claims description 6
- 230000007787 long-term memory Effects 0.000 claims description 5
- 239000000203 mixture Substances 0.000 claims description 4
- 230000007774 longterm Effects 0.000 claims description 2
- 239000000284 extract Substances 0.000 claims 1
- 230000036544 posture Effects 0.000 description 110
- 210000003811 finger Anatomy 0.000 description 50
- 238000001514 detection method Methods 0.000 description 31
- 210000003813 thumb Anatomy 0.000 description 23
- 230000001815 facial effect Effects 0.000 description 19
- 238000010586 diagram Methods 0.000 description 16
- 238000012545 processing Methods 0.000 description 16
- 238000005259 measurement Methods 0.000 description 15
- 230000008859 change Effects 0.000 description 14
- 230000015654 memory Effects 0.000 description 14
- 210000005224 forefinger Anatomy 0.000 description 11
- 238000006243 chemical reaction Methods 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 10
- 230000008569 process Effects 0.000 description 10
- 230000000694 effects Effects 0.000 description 9
- 210000003414 extremity Anatomy 0.000 description 9
- 210000003205 muscle Anatomy 0.000 description 9
- 238000004364 calculation method Methods 0.000 description 7
- 241000446313 Lamella Species 0.000 description 6
- 239000011521 glass Substances 0.000 description 6
- 239000011159 matrix material Substances 0.000 description 6
- 230000003321 amplification Effects 0.000 description 5
- 230000004438 eyesight Effects 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 5
- 238000003199 nucleic acid amplification method Methods 0.000 description 5
- 238000007637 random forest analysis Methods 0.000 description 5
- 230000006399 behavior Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 4
- 238000000605 extraction Methods 0.000 description 4
- 230000008713 feedback mechanism Effects 0.000 description 4
- 238000007689 inspection Methods 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 230000000638 stimulation Effects 0.000 description 4
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 4
- 230000001133 acceleration Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000010224 classification analysis Methods 0.000 description 3
- 230000005057 finger movement Effects 0.000 description 3
- 210000001061 forehead Anatomy 0.000 description 3
- 230000001771 impaired effect Effects 0.000 description 3
- 210000004932 little finger Anatomy 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 238000002310 reflectometry Methods 0.000 description 3
- 238000000611 regression analysis Methods 0.000 description 3
- 230000035807 sensation Effects 0.000 description 3
- 210000002435 tendon Anatomy 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 206010011878 Deafness Diseases 0.000 description 2
- 230000004913 activation Effects 0.000 description 2
- 238000005452 bending Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 230000001149 cognitive effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000009977 dual effect Effects 0.000 description 2
- 230000005611 electricity Effects 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 210000004247 hand Anatomy 0.000 description 2
- 210000003128 head Anatomy 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000003183 myoelectrical effect Effects 0.000 description 2
- 230000002232 neuromuscular Effects 0.000 description 2
- 238000012805 post-processing Methods 0.000 description 2
- 238000000513 principal component analysis Methods 0.000 description 2
- 230000005855 radiation Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 230000001953 sensory effect Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 239000000758 substrate Substances 0.000 description 2
- 230000009897 systematic effect Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- RZVAJINKPMORJF-UHFFFAOYSA-N Acetaminophen Chemical compound CC(=O)NC1=CC=C(O)C=C1 RZVAJINKPMORJF-UHFFFAOYSA-N 0.000 description 1
- 241000208340 Araliaceae Species 0.000 description 1
- 240000002791 Brassica napus Species 0.000 description 1
- 101000739754 Homo sapiens Semenogelin-1 Proteins 0.000 description 1
- 206010049816 Muscle tightness Diseases 0.000 description 1
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 1
- 235000003140 Panax quinquefolius Nutrition 0.000 description 1
- 102100037550 Semenogelin-1 Human genes 0.000 description 1
- 230000036982 action potential Effects 0.000 description 1
- 230000003213 activating effect Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000033228 biological regulation Effects 0.000 description 1
- 210000004204 blood vessel Anatomy 0.000 description 1
- 210000000988 bone and bone Anatomy 0.000 description 1
- 210000004027 cell Anatomy 0.000 description 1
- 150000001875 compounds Chemical class 0.000 description 1
- 239000004020 conductor Substances 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 210000000256 facial nerve Anatomy 0.000 description 1
- 210000003195 fascia Anatomy 0.000 description 1
- 238000010304 firing Methods 0.000 description 1
- 238000009472 formulation Methods 0.000 description 1
- 230000005021 gait Effects 0.000 description 1
- 235000008434 ginseng Nutrition 0.000 description 1
- 230000003370 grooming effect Effects 0.000 description 1
- 230000030214 innervation Effects 0.000 description 1
- 238000012417 linear regression Methods 0.000 description 1
- 230000004118 muscle contraction Effects 0.000 description 1
- 210000000107 myocyte Anatomy 0.000 description 1
- 230000008035 nerve activity Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000149 penetrating effect Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 238000002203 pretreatment Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004043 responsiveness Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000003238 somatosensory effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 230000004936 stimulating effect Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 238000002560 therapeutic procedure Methods 0.000 description 1
- 210000001519 tissue Anatomy 0.000 description 1
- 210000000623 ulna Anatomy 0.000 description 1
- 230000001720 vestibular Effects 0.000 description 1
- 208000029257 vision disease Diseases 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/16—Constructional details or arrangements
- G06F1/1613—Constructional details or arrangements for portable computers
- G06F1/163—Wearable computers, e.g. on a belt
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
- G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B21/00—Teaching, or communicating with, the blind, deaf or mute
- G09B21/02—Devices for Braille writing
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
- G09G5/003—Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2370/00—Aspects of data communication
- G09G2370/16—Use of wireless transmission of display information
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Hardware Design (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Dermatology (AREA)
- Neurology (AREA)
- Neurosurgery (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Business, Economics & Management (AREA)
- Educational Administration (AREA)
- Educational Technology (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses the method and apparatus of the ability of posture control interface for wearable device.Ability of posture control interface arrangement includes one or more biopotential sensors and a processor.One or more biopotential sensors may be worn on the body of user, for detecting one or more bioelectrical signals from user's body, wherein, one or more biopotential sensors include at least one surface nerve conduction (SNC) sensor, for detecting at least one surface neuro-transmission signal.At least one surface neuro-transmission signal that processor is configured as detecting is compared with corresponding to the data of multiple reference signals of multiple known poses, each reference signal is substantially associated with one of known poses, to identify the known poses corresponding with least one surface neuro-transmission signal from multiple known poses, and the known poses identified are communicated to computerized equipment.
Description
The cross reference of related application
The application is the U.S. Patent application No.14/588 submitted on January 2nd, 2015,592 part continuation application,
And the U.S. Patent application No.14/588,592 priority on January 2nd, 2015 submitted of request on December 31st, 2015
The international patent application No.PCT/IL2015/051273 of submission part continuation application, both are both incorporated herein by reference.
Technical field
The present invention relates to wearable device.More particularly, the present invention relate to wearable device based on biopotential
Interface.
Background technology
Natural language is intuitively for human communication.It dependent on spoken language, but it be subconsciousness be based on body and hands
Gesture, wherein providing constant feedback by onlooker, such as via delicate body language, speaker is made a response.The opposing party
Face, mankind's computer interface are not based on delicate mankind's technical ability, therefore compared with spoken human and body language, this right and wrong
It is often troublesome and non-intuitive.Naturally another example intuitively exchanged is the musical instrument of such as violin or piano, and musician uses
The posture such as moved produces the sound used also as audio feedback.In such a system, well-trained musician can
With not having eye contact shoegaze, such feedback add pace of learning.
For many years, man-machine interaction is mainly carried out using " QWERTY " keyboard of standard, and screen provides the user keyboard (and mouse
Mark) input visual feedback.With the constantly improve of computerized equipment technology, these keyboards turn into cumbersome communication now
Means.At present, the most important development in intelligence interface field is based on the computer vision using camera and video analysis.But
Due to the complexity of vision data, this method is restricted.
In recent years, touch screen interface has become most common for typically being instructed to computer input text or provide
One of solution, thus touch-screen instead of QWERTY keyboard and mouse.But needed using touch-screen eyes and finger are complete
Portion is concentrated on screen, does not need the interface of direct viewing screen or unavailable currently.
In order to seek the more intuitive means for man-machine interaction, such as (Mike built in use can have been obtained in recent years
Wind and/or camera) speech recognition and gesture recognition other solutions；But these solutions can not be provided to input
Accurate explanation.Speech recognition is based on the signal that can not be interpreted easily (in the case of no one group of additional signal),
And gesture recognition is based on computer vision, thus it is extremely sensitive to many ambient parameters.
It has been changed into the additional solution of universal human interface from medical applications (such as prosthese biomethanics solution)
Scheme is the equipment based on surface electromyography (sEMG), there is provided to the rough gesture of basic command (such as to control that prosthese grasps
System) identification, wherein sEMG sensors are located near ancon.But such device can not easily detect the delicate shifting of hand
It is dynamic, such as the movement of single finger, therefore the interface of wider range posture can not be effectively served as.In addition, such device needs
SEMG sensor arrays are wanted to be located at the slightly lower position of ancon, this is inconvenient for most of users, therefore in medical profession
It is not widely accepted in addition.Other equipment is applied to visually impaired person, and is shown with physics braille, but they are not provided based on sEMG
Interface, therefore posture can not be detected.US8,447,704 describes a kind of for being based on predefined one group of sEMG signal identifications
The interface of general posture.
Therefore, it is necessary to which a kind of be used for the efficient of computerization machine and intuitively user interface, it can be based on EMG signal and know
Different types of delicate posture (is not defined by the user).In addition, with the development of Internet of Things (IoT) suitable device, it is particularly
Wearable intelligent watch, based on the computer interface of screen due to the feedback being difficult between closed user and computerized equipment
Loop, the interaction for complexity become less and less, more and more inconvenient.
The content of the invention
Therefore, according to some embodiments of the present invention, there is provided including one or more biopotential sensors and processing
The ability of posture control interface arrangement of device.One or more of biopotential sensors may be worn on the body of user, for examining
One or more bioelectrical signals from user's body are surveyed, wherein, one or more of biopotential sensors are included extremely
Few surface nerve conduction (SNC) sensor, for detecting at least one surface neuro-transmission signal.The processor by with
It is set at least one surface neuro-transmission signal and the number of multiple reference signals corresponding to multiple known poses that will be detected
According to being compared, each reference signal is substantially associated with one of known poses, to know from the multiple known poses
The not known poses corresponding with least one surface neuro-transmission signal, and the known poses identified are communicated to meter
Calculation machine equipment.
According to some embodiments of the present invention, the device is configured to be assembled in the wrist of user, wherein at least one
SNC sensors are configured as detecting the electric signal of the nerve tract in wrist.
According to some embodiments of the present invention, the device includes at least one motion sensor, is configured as detecting body
Movement, and the processor movement that is configured with detecting identifies known poses.
According to some embodiments of the present invention, the device includes tactile actuator, is configured as the known appearance identified
When gesture is registered in computerized equipment, touch feedback is activated on the body of user.
According to some embodiments of the present invention, the processor is configured as by using one or more bioelectrical signals
To train the data of the body for user, by least one surface neuro-transmission signal with it is each in multiple known poses
It is individual associated.
According to some embodiments of the present invention, one or more biopotential sensors are selected from by surface electromyography (sEMG)
The group of sensor, electric capacity electromyogram (cEMG) sensor and skin conductance sensors composition.
According to some embodiments of the present invention, the processor is configured as by that will have surface nerve conduction (SNC)
The wavelet transform (DWT) of morther wavelet is applied to the one or more bioelectrical signals detected, from the detection
To one or more bioelectrical signals filter out electromyogram (EMG) noise signal.
According to some embodiments of the present invention, the known poses of the identification include forcing together at least two fingers,
And the processor is configured as by assessing including the proportional pressure with being applied between at least two finger
Amplitude and at least one surface neuro-transmission signal detected of frequency identify that at least two finger is pressed in one
Rise.
According to some embodiments of the present invention, the processor is configured as estimation and applied between at least two finger
The pressure added.
According to some embodiments of the present invention, it is further provided a kind of method, by ability of posture control interface arrangement and based on
Communication between calculation machine equipment, methods described include：Detection is from the one or more biopotentials being placed on user's body
One or more bioelectrical signals of sensor, wherein, one or more of biopotential sensors include at least one table
Facial nerve conducts (SNC) sensor, for detecting at least one surface neuro-transmission signal.Using processor, by the detection
To at least one surface neuro-transmission signal with corresponding to multiple known poses multiple reference signals data compared with,
Each reference signal is substantially associated with known poses one of them described.From the multiple known poses identification with it is described
The corresponding known poses of at least one surface neuro-transmission signal.The known poses of the identification are communicated into computerization to set
It is standby.
According to some embodiments of the present invention, identify that the known poses are included at least one surface detected
Nerve conduction (SNC) signal carries out denoising, detects the event at least one SNC signals, using segmentation with described in determining
One or more frames of the event detected, the statistical nature in one or more of frames is extracted, and the number will be based on
According to sorting algorithm be applied to the extraction statistical nature, to determine the known poses.
According to some embodiments of the present invention, the known poses include forcing together at least two fingers, and its
In, identify that at least two finger forces together including assessing the pressure included with being applied between at least two finger
Proportional amplitude and at least one surface neuro-transmission signal detected of frequency.
According to some embodiments of the present invention, methods described is included by by one or more of biological electricity detected
Signal is applied to the proportional control pipeline including convolutional neural networks (CNN) and shot and long term memory (LSTM) neutral net
Estimation is applied to the pressure between at least two finger.
According to some embodiments of the present invention, this method includes training LSTM neutral nets by using auxiliary signal.
Brief description of the drawings
For a better understanding of the present invention and in order to understand its practical application, it is provided below and with reference to the following drawings.Should
Work as attention, accompanying drawing only provides as example, is in no way intended to limit the scope of the present invention.Identical component is by identical reference number table
Show.
Figure 1A schematically shows the front view of the flexible interface according to some embodiments of the present invention；
Figure 1B is schematically shown according to the flexible PCB interfaces around user's wrist of some embodiments of the present invention
Cross-sectional view；
Fig. 2 depicts the letter between the user interface and computerized equipment shown according to some embodiments of the present invention
Cease the block diagram of the attitude control systems of stream；
Fig. 3 is depicted according to the attitude control systems with additional heart rate sensor of some embodiments of the present invention
Block diagram, show the information flow between user interface and computerized equipment；
Fig. 4 A depict the block diagram of the attitude control systems according to some embodiments of the present invention, wherein all processing are being counted
Performed in the embedded device of calculation machine；
Fig. 4 B depict the frame of the attitude control systems with input/output interface according to some embodiments of the present invention
Figure；
Fig. 5 depicts the flow chart that text is write using attitude control systems according to some embodiments of the present invention；
Fig. 6 A schematically show the hand of the user according to some embodiments of the present invention；
Fig. 6 B schematically show the symbol of the letter " C " in the braille according to some embodiments of the present invention；
Fig. 7 A show according to some embodiments of the present invention the signal as caused by moving forefinger；
Fig. 7 B show according to some embodiments of the present invention the signal as caused by moving middle finger；
Fig. 7 C show according to some embodiments of the present invention the signal as caused by moving thumb；
Fig. 7 D show the signal according to caused by moving the fist as holding of some embodiments of the present invention；
Fig. 8 A show the three kinds of appearances classified according to the different characteristic of sEMG signals according to some embodiments of the present invention
The drawing of gesture；
Fig. 8 B show three of the prominent features for including measuring for three kinds of postures according to some embodiments of the present invention
Dimension is drawn；
Fig. 9 schematically shows the facial pose control system according to some embodiments of the present invention；
Figure 10 A schematically show the combination sensor and tactile feedback actuators according to some embodiments of the present invention
Exemplary circuit；
Figure 10 B schematically show showing according to the combination sensors with concentric ring of some embodiments of the present invention
The cross-sectional view of example property circuit；
Figure 11 A are schematically shown such as the finger of control wrist-watch performed in the prior art；
Figure 11 B schematically show the thumb posture of the control wrist-watch according to some embodiments of the present invention；
Figure 11 C are schematically shown such as the thumb of control handheld device performed in the prior art；
Figure 11 D schematically show the thumb posture of the control handheld device according to some embodiments of the present invention；
Figure 11 E schematically show the thumb of the control game console as performed in the prior art；
Figure 11 F schematically show the thumb posture of the control game console according to some embodiments of the present invention；
Figure 12 A are schematically shown is maintained at wrist according to having for some embodiments of the present invention by intelligent spire lamella
On intelligent watch hand back view；
Figure 12 B are schematically shown is maintained at wrist according to having for some embodiments of the present invention by intelligent spire lamella
On intelligent watch hand palm view；
Figure 13 schematically show according to some embodiments of the present invention be configurable for reflectometry around
The intelligent watch that wrist is placed；
Figure 14 A show the thumb mobile gesture identified by intelligent watch according to some embodiments of the present invention；
Figure 14 B show the forefinger mobile gesture identified by intelligent watch according to some embodiments of the present invention；
What Figure 14 C showed two fingers identified according to some embodiments of the present invention by intelligent watch raps appearance
Gesture；
Figure 14 D show being squeezed in two fingers by what intelligent watch 16 identified according to some embodiments of the present invention
Posture together；
Figure 15 is to describe to be used in the flexible user interface of ability of posture control and calculating according to some embodiments of the present invention
The flow chart of the method to be communicated between machine equipment；
Figure 16 is to describe the flow chart for being used to identify the method for known poses according to some embodiments of the present invention；
Figure 17 is schematically shown to be examined according to some embodiments of the present invention when two fingers are pressed together
The biopotential signals measured；
Figure 18 A are the block diagrams according to the data lines for gesture recognition of some embodiments of the present invention；
Figure 18 B are the block diagrams for being used for the data lines that ratio controls according to some embodiments of the present invention；
Figure 19 schematically shows the combination pipeline architecture using neutral net according to some embodiments of the present invention；
Figure 20 is that two fingers for showing to combine pipeline architecture estimation according to the use of some embodiments of the present invention extrude
The curve map of standardization Pressure versus Time frame when together；
Figure 21 A schematically show the first reality of the gloves with touch feedback according to some embodiments of the present invention
Apply example；And
Figure 21 B schematically show the second reality of the gloves with touch feedback according to some embodiments of the present invention
Apply example.
Embodiment
In the following detailed description, many details are elaborated, to provide thorough understanding of the present invention.But
It is, it will be appreciated by those skilled in the art that the present invention can be put into practice in the case of these no details.In other feelings
Under condition, known method, program, component, module, unit and/or circuit are not described in, in order to avoid make the obscure difficulty of the present invention
Understand.
Although embodiments of the invention are unrestricted in this regard, using such as " processing ", " calculating ",
" calculation ", " it is determined that ", " foundation ", " analysis ", the discussion of the term such as " inspection " may refer to computer, calculating platform, computing system or
The operation of other electronic computing devices and/or process, it manipulates and/or will be indicated as the register and/or memory of computer
In the data conversion of physics (such as electronics) quantity into the register and/or memory for being similarly represented as computer or can deposit
Store up the physics in the other information non-transitory storage medium (for example, memory) of the instruction for performing operation and/or process
Other data of quantity.Although embodiments of the invention are unrestricted in this regard, terms used herein " multiple " and
" more several " may include such as " multiple " or " two or more ".Term " multiple " or " more can be used throughout the specification
It is several " two or more components, equipment, element, unit, parameter etc. described.Unless expressly stated otherwise, it is described herein
Embodiment of the method is not limited to specific order or sequence.In addition, some in described embodiment of the method or its element can be
Same time point or concurrent generation simultaneously are carried out.Unless otherwise stated, the use of "or" connection used herein
It should be understood pardon (option any or all).
Figure 1A schematically shows the front view of the flexible user interface 10 according to some embodiments of the present invention.User
Interface 10 includes the printed circuit board (PCB) (PCB) with multiple element, is configured as allowing user and computerized equipment (such as flat
Plate computer) between interface.The PCB of user interface 10 is integrated into elastic base plate 11 so that the user interface 10 can pass through
Connect edge 19 and deform, to realize the cylinder form that can be assembled on the limbs of user (such as the soft of watch
Property watchband).
The PCB of flexible user interface 10 includes multiple biopotential sensors 12 and the array of tactile feedback actuators 14,
Wherein conductive bars 17 have the corresponding wiring for these elements.Each sensor 12 may include the direct skin contact with user
At least two electrodes 16, pass through their detection signals.
Alternatively, biopotential sensor 12 is surface electromyography (sEMG) sensor, and conductive bars 17 have and are used for
Electric power transmission and the several conducting shells for being additionally operable to signal transmission.In certain embodiments, sEMG is replaced using other sensors,
Such as electric capacity electromyogram (cEMG) sensor.It should be noted that electromyography transducer, which can be detected from muscle, moves obtained signal,
Wherein, these signals can transport along limbs.
Preferably, biopotential sensor 12 is surface nerve conduction (SNC) sensor, can detect the god from wrist
Through signal, wherein, these signals are as caused by the movement of user.Specifically, the signal from three major nerves is detected：Just
Middle nerve, ulna nerve and nervus radialis, as performed in diagnosing nerve conduction study in standard medical.It should be noted that wrapping
In the embodiment for including SNC sensors, it may be necessary at least three SNC sensors, to detect exactly from three main god
The nervous activity (that is, each major nerve, a sensor) of warp.
PCB further comprises wireless communication controller 13, there is provided radio communication (for example, with bluetooth transceiver) is to nearby setting
It is standby；And motion sensor 15.These motion sensors 15 are preferably MEMS (MEMS), and may include acceleration
Count any other combination of (detection acceleration), gyroscope (detection direction), magnetometer or proper sensors.
With the system realize correction ratio other systems more accurately solution can be provided for gesture recognition, this be by
The movement that can not possibly be only provided with accelerometer is provided in the combination of the data associated with built-in acceleration meter and gyroscope
Information (has all possible direction).Alternatively, biopotential sensor 12 can be alignd with multipair configuration, to detect
Different electrical activity source, because each nerve produces signal (for example, the sensor of arm dorsal part can not detected in ad-hoc location
Movable signal in front of to arm).
In a preferred embodiment, communication controler 13 is Bluetooth Low Energy (BLE) controller, and reduction is provided for radio communication
Power consumption.
It should be noted that the array of tactile feedback actuators is used as user's tactile feedback mechanism, rather than regarding based on screen
Feedback is felt, so as to produce closed loop feedback.Closed loop feedback is the successful key components of any interface, such as Jiang N. et al.,
“Myoelectric control of artificial limbs-is there a need to change focus”,
IEEE Signal Processing Magazine (2012), Vol.29, No.5, pp.152-150, shown in prosthese control
System, conclusion therein are generally related to man-machine interaction.This closed loop feedback can be practised by any user with minimum consciousness mechanics, and
And adapt to provide important layer in this interface in the mankind.
Sensor 12 can have the property qualitative difference configuration for corresponding to the signal detected, and can be each by measuring
Voltage difference between electrode 16 corresponding at least two of sensor 12 is detected in electronic work caused by these sensor proximities
Potential.Such electrode 16 be typically dry electrode, can with the skin engagement of user, without other material (such as gel)
To improve skin electric conductivity.Therefore, if biopotential sensor 12 is attached to the body of user, due to caused electronic
Make potential, each action can be detected by these sensors.By being carried out to the reception signal of known action (such as clenching one's fists)
Appropriate calibration, can be associated with the movement of body by any signal received by biopotential sensor 12.Alternatively, it is biological
The distance between adjacent electrode pair of electric potential sensor 12 is~1.5cm, corresponding to known signal propagation rate in body most
Good distance.In certain embodiments, at least a portion of user interface is rigid and not perfectly elastic (for example, class
It is similar to wrist-watch).
Alternatively, conductive bars 17 are connected further to the reference drive of the pre-position in elastic base plate 11
18.The reference drive 18 limits electromyogram input voltage within a predetermined range, and can suppress the normal of such as fluorescent radiation
See the 50Hz/60Hz radiation (causing measurement noise) of noise and the standard from power line.It should be noted that come from reference drive
18 single reference signal is used for all biopotential sensors 12, and this is with the typically used as of sensor on the contrary, in sensor
In typically used as, each sensor generally passes through the reference of right leg drive (DRL) circuit drives their own.Therefore, keeping passing
Sensor 12 it is high-precision simultaneously, less element (therefore the less electric power of consumption and space) can be used, this is due to flexibility
The quality of output signal in user interface 10 does not reduce (as configured with this shown in the several tests carried out).Alternatively,
Public DRL mechanisms rather than above-mentioned configuration may be used.
In some embodiments of the invention, each sensor from biopotential sensor 12 is also connected to increase automatically
Benefit control amplifier (AGC), to reduce signal intensity (being discussed further below).Alternatively, all biopotential sensings
Device 12 is activated, but only detection understands the sensor passes of the signal data to be further processed.
Figure 1B schematically shows the flexible PCB user around user's wrist 5 according to some embodiments of the present invention
The cross-sectional view of interface 10.In the configuration, all biopotential sensors 12 and all tactile feedback actuators 14 and wrist 5
Direct skin contact.Therefore, any movement of user's wrist 5 corresponds in nerve and follows hard on the electronic work on myocyte
Potential, and can be detected by biopotential sensor 12.In addition, motion sensor 15 can detect by biopotential sensor 12 not
Some movements of (due to caused by the change of position and direction) detected, because some postures are needed in the seldom of measured zone
Or muscle movement is not needed, therefore do not measure obvious electric action potential.Alternatively, user interface further comprises conduct
The display (such as similar to display on intelligent watch) of the interface of system.
By the signal that is detected according to these sensors come initial calibration as move or posture, can in later phases
Can be associated with posture by the signal received so that can create user-computer interface and by its be tuned to specific user.
This calibration process is described further below.Once calibration is completed, interpretation gesture (such as enclosed in flexible PCB user interface 10
In the case of wrist) it can allow to control and operate the equipment with computerization interface (such as PC, TV or tablet personal computer)
Or other wearable devices (such as intelligent watch), wherein, each posture corresponds to the order received by computer.This feature can
Perfection even substitutes current touch screen interface.
In certain embodiments, this sensor array can be integrated into the wrist strap of existing intelligent watch, or alternative
Ground can be used as autonomous device.The digital signal processing unit used in equipment can be used by handling the data from these sensors
(DSP) real-time " machine learning " is realized.Alternatively, such sensor array can be integrated into the wrist strap of existing standard wrist-watch
In, so as to convert thereof into intelligent watch.
It should be noted that in a similar way, flexible PCB user interface 10 can surround the different piece of user's body (generally
Around a part for limbs), the wherein direct skin contact of biopotential sensor 12 and tactile feedback actuators 14 and user.
Alternatively, the interface is imperceptible for body so that user be freely movable without from the equipment for it
The interference of skin.
Fig. 2 depicts showing between user interface 10 and computerized equipment 29 according to some embodiments of the present invention
Information flow attitude control systems block diagram.The direction of the direction configured information stream of arrow.User interface 10 detects user's
Posture and movement (as described above).The signal detected is adjusted signal processor 22, and label is applied into data set,
So that with such known poses corresponding order of the specific movement of user with being sent to computerized equipment 29 is matched.Cause
This, computerized equipment 29 can by user interface 10, thus computerized equipment 29 can be have such as smart phone, PC,
Any equipment of the computerization interface of tablet personal computer, TV etc..
User interface 10 detects posture (as described above) using biopotential sensor 12 and motion sensor 15.In order to increase
The analog signal received from biopotential sensor 12 by force, additional amplification electron device 21 can be couple to each biopotential
Sensor 12, wherein, amplification electron device 21 can be embedded in the substrate 11 of flexible user interface 10 (as shown in Figure 1A).
Preferably, electronic device 21 may include analogue amplifier and/or analog-digital converter (ADC) so that analog signal is exaggerated, then
Data signal is converted into, further to be handled in the stage later.In addition, each biopotential signals 12 may also coupled to mould
Intend gain controller (AGC) so that the gain of amplification electron device 21 is impartial, to ensure suitable voltage range.
By real-time sampling and special letter is sent to from the information that biopotential sensor 12 and motion sensor 15 receive
Number processor 22, wherein, signal processor 22 can be embedded in the substrate 11 of flexible user interface 10 (shown in Figure 1A).Signal transacting
Device 22 can perform baseband signal regulation processing, then export one group of designator for each signal.Then, signal processor 22
The particular combination of the recognizable signal from these designators, such as reduce method using dimension.
It is stored in and from all data of signal processor 22 in flash memory module 24, to allow institute
There is the database that these data are uploaded in remote computerized device or service based on cloud.For example, needed greatly to develop
The supplementary features of example data are measured, can be used to analyze in data as later phases collection.In certain embodiments, it is not required to
Want single memory module.
In order to identify specific posture, system 20 to memory module 24 by being sampled and being joined using predetermined inside
Number performs classification processing, and so as to which posture to be distributed to one in N+1 symbol, (" N " is known symbol, " NULL " symbol table
Show static schema when user does not assume a position).Once given pose is categorized as into symbol 26, then class symbol 26 is as defeated
Go out to be sent to communication controler 13.Therefore, signal processor only identifies known posture.Alternatively, the immediate feedback of symbol 26
23 can be sent and arrive symbol feedback control 28.
Then, communication controler 13 can regard class symbol 26 as corresponding life via radio communication (being indicated by the use of dotted arrow)
Order is sent to computerized equipment 29.Once computerized equipment 29 receives order, additional signal also can be via channel radio
Letter is sent back to communication controler 13, such as instruction has been carried out the signal of the order.Alternately, not from computerization
Equipment 29 receives signal, and handles and stop herein.
Communication controler 13 can be carried out the corresponding signal from computerized equipment 29 as the input of user interface 10
Send.Then, the signal received is identified that symbol feedback control unit 28 uses touch feedback by symbol feedback control unit 28
Actuator 14 activates the corresponding touch feedback to user.
For example, user makes the gesture (Management Information Base based on calibration) corresponding with order " opening ".The order is divided
Class is symbol 26, and immediate feedback (that is, corresponding to the data of order) 23 is correspondingly produced at symbol feedback control 28.Together
When, the order is sent to the interface of computerized equipment 29 (such as interface of " intelligent television ") via communication controler 13.Once
Perform " opening " order, TV can send back signal user interface 10 so that user obtains touch feedback, without with
Direct eye contact between family and TV.
This can be realized by appropriate calibration, due to the appropriate sense feedback of the gesture identification for identification.Formed
Closed feedback loop so that produce relation between the posture and the feedback that is received of user over time.With this
Mode, user can also be by identifying the touch feedback received come from computerized equipment 29 " reading " symbol sebolic addressing.For example, with
Family receives text message and feels such message using haptic feedback mechanism 14, is set without user and computerization
Direct eye contact between standby 29.Such example may be especially relevant with the driver of driving vehicle or visually impaired person.
In some embodiments of the invention, once computerized equipment 29 receives the order of identification, then the sense of hearing is created
Feedback so that user can hear the order for the requirement for having been received by and/or performing.Alternatively, audio feedback is only carried out
Without the corresponding touch feedback carried out by haptic feedback mechanism 14.
It should be noted that Figure 1A can further comprise add ons to the user interface 10 described in Figure 1B, to increase posture
The accuracy of detection.Some in these elements are described below.
Fig. 3 depicts the attitude control systems 30 with additional heart rate sensor 32 according to some embodiments of the present invention
Block diagram.Fig. 3 shows information flow (wherein, the direction of arrow expression information flow between user interface and computerized equipment 29
Direction).User interface 33 is further equipped with multiple heart rate sensors 32 of the heart rate of detectable user, and (this feature has been made
Can be obtained for the embedded components in some smart phones), such as with the optical sensing for penetrating skin, the light beam to be rebounded from blood vessel
Device.Generally, heart rate does not change during rest, therefore heart rate sensor 32 can provide cognitive effort/strain identification (strain
recognition.)。
The designator that user is absorbed in during the heart rate detected can be used as system calibration, wherein, user is being trained to create
Build given pose and identify tactile feedback patterns.If the instruction user of heart rate sensor 32 is absorbed in, then the posture of calibration can quilt
The higher index of distribution so that the given pose may be differently weighed, so as to improve calibration process.In addition, user's is special
Note can be used for improving the whole communication process between user and computerized equipment 29, because if signal processor 22 not from
Heart rate sensor 32 receives required instruction, then unintentionally posture can be ignored.
In some embodiments of the invention, it is possible to provide at least one skin conductivity of the Skin Resistance of user can be measured
Rate sensor so that the executable calibration process when measuring the predetermined value of Skin Resistance.Alternatively, at least one skin electric conductivity
Sensor can be combined further with pulse, and muscle tension sensor can be directed to the stimulation of user or be absorbed in and provide optimal knowledge
Not.
Fig. 4 A depict the block diagram of the attitude control systems 40 according to some embodiments of the present invention, wherein, all processing
(direction of the direction configured information stream of arrow) is carried out on computerized equipment.In the present embodiment 40, signal processor 22,
Flash memory 24 and class symbol 26 are all the elements of computerized equipment 49.
From user interface 43 (that is, from biopotential sensor 12, motion sensor 15 and/or from sensing heart rate
Device 32) signal from o controller 42 be sent to communication controler 13 as output.Then be able to will be exported via radio communication
Computerized equipment 49 is sent to, to handle these signals (as described above).Class symbol 26 is sent to computerization and set
Standby 49 command processor 44 so that executable desired order.
In the case where signal to be sent back to user (such as in the case of exectorial), it is sent to via radio communication
The signal of communication controler 13 is sent to symbol feedback control unit 28 and is ultimately delivered to tactile feedback actuators 14.Should
Pay attention to, the processing performed in the present embodiment 40 is similar with the process described by previous embodiment, wherein, make it is all processing with
The outside major advantage performed of family interface 43 is to save the space for other application.Further, since set in computerization
All power consumption calculations are remotely performed at standby 49, therefore electric power can be saved at user interface 43 so that the battery of user interface 43
The sustainable longer time.
, it is necessary to the calibration of execution system before the initial use of attitude control systems.Calibration process is used as signal inspection
Mechanism is surveyed, it initially sets the value of still-mode (i.e. NULL postures), wherein, unique input should be noise, have predetermined quantity
Iteration.The signal of biopotential sensor is calibrated by " dual threshold " method, to disappear when carrying out signal of change
Except fake information.This method has been described as the letter successfully occurred with sEMG detections in noise background environment in the following documents
Number：Bonato P.et al.,“A Statistical Method for the Measurement of Muscle
Activation Intervals from Surface Myoelectric Signal During Gait”,IEEE
Transactions on Biomedical Engineering (1998), Vol.45, NO.3, pp.287-299, and
Severini G.et al.,“Novel formulation of a double threshold algorithm for the
estimation of muscle activation intervals designed for variable SNR
environments”,Journal of Electromyography and Kinesiology(2012),Vol.22,
pp.878-885。
In the next step, systematic learning distinguishes different postures (such as specific movement of hand or specific finger).User
Instruction performs given pose, and according to the posture, systematic learning is for the typical predefined one group of inner parameter of specific user.So
Afterwards, these parameters are stored in internal flash by system.User repeats this process NxM times, and wherein N represents that system is examined with low mistake
The posture number measured, M represent the number of repetition of given pose.For example, repeat to represent the posture 20 times of letter " E ", thus not
Different postures can be used to represent same alphabetical " E " in same user.Alternatively, each user be based on predefined training set come
Learn specific posture.
In some embodiments of the invention, initial extension training set is provided to user interface.Due to most people for
Same posture has similar muscle activity, therefore initial training mode may belong to a great number of people.Using this large amount of numbers
According to an example of Efficient gesture forecast model be " deep learning " method.Therefore, expansion can be provided together with specific training set
Open up training set so that user only learns predetermined gesture (in a short time), rather than performs complete calibration process.By using with
Family information is matched user with model, and spread training collection can be applied to each kind of groups.For example, male user can be distributed to
" the male's gesture model " of extension.
Fig. 4 B are depicted according to the attitude control systems 41 with input/output interface of some embodiments of the present invention
Block diagram.The direction of the direction configured information stream of arrow.In the embodiment 41, sensor array 12 and tactile feedback actuators 14
It is used as universal input/output (I/O) interface for sensory substitution together.In the configuration, electric signal can be in signal processor
It is adjusted in 22, and I/O user interfaces 45 is directly fed back to via electric touch and/or vibrating tactile stimulator 48, without
Want discrete classification.Such I/O interfaces can substitute or strengthen various somatosensory abilities completely.For example, as blind person's
Camera, direct tactile feel can be converted into as the Inertial Measurement Unit (IMU) of the people for being lost with vestibular, or conduct
Feel the microphone of the deaf person user of input.Such example is used with other and for example discussed in the following documents：Bach-y-
Rita,P.,“Tactile sensory substitution studies”,ANNALS-NEW YORK ACADEMY OF
SCIENCES(2004),Vol.1013,pp.83-91。
In certain embodiments, all postures and symbol are selected so that most simple and most short posture will be used for expression and use
Most common letter, syllable, word and sentence in the language of family.In this manner, it may be possible to write faster than existing method, because not
Direct eye contact is needed again.One example of this principle can see that it can be represented in alphabet in braille writing
All letters, and conventional English suffix " tion ", " ing " and common word, such as " the ", " and ".Therefore can realize logical
Cross well-trained posture write-in text；Or text is read by touch feedback.
In some embodiments of the invention, attitude control systems can detect when user holds writing implement (such as pen)
It is hand-written.In this embodiment, the system detectio is believed due to during writing caused by the muscle that moving for hand is activated
Number.
In some embodiments of the invention, attitude control systems can make together with the operating system based on special posture
With, wherein, all basic commands are all towards posture and touch feedback.Using such operating system, screen will be optional
, because need not be with the direct eye contact of computerized equipment.The interface of such operating system can be based purely on
Order, without screen or mouse, thus can especially with such as intelligent refrigerator " Internet of Things " hardware compatibility.
Fig. 5 depicts the flow chart that text is write using attitude control systems according to some embodiments of the present invention.Most
Just, user interface is activated 50, and wherein user interface is connected to computerized equipment via radio communication.System is waited until letter
Number activity is detected 52, uses motion sensor and/or biopotential sensor.The 51a when not detecting mobile, system
Return to original state.Once detecting mobile 51b, then whether systems inspection movement is the posture 54 identified.If movement is not
The posture 53a of identification, then system return to original state, until detecting that another activity starts.Otherwise, it is if mobile
It is the posture 53b of identification, in addition to performing order (if such order can apply), related tactile is also sent to user
Feedback 56 so that user knows that correct posture is registered, so as to form " man-machine " backfeed loop of closure.
Next, whether the posture that systems inspection is identified is complete symbols 58, because special symbol (such as alphabetical " C ")
It may include one group of several posture.If the posture identified is not complete symbols 55a, system returns to original state, until
Detect that another starts.Otherwise, if the posture of identification is complete symbols 55b, requirement is performed in computerized device
Order 59.For example, term " and (and) " is written into text message.Alternatively, complete symbols 58 can further include time-out
Mechanism so that if having passed through the time of scheduled volume, all data quilts relevant with such sequence before sequence completion
Erasing.
In some embodiments of the invention, the array of tactile feedback actuators is can create the configuration quilt of different mode
It is positioned at user interface.For example, the watering system of computerization detects that specific water sprinkler has failure.Then, water
System can initially notify user's computerization watering system should be noted via haptic feedback mechanism.Next, computerization
Watering system can be by activating the different elements in tactile feedback actuators array, with direction indication (such as specific actuating
Device combination instruction moves right), the water sprinkler damaged is reached until system identification goes out user, such as using based on normal place
Service, guiding user reach the position of the water sprinkler of damage.
With reference now to Fig. 6 A to 6B, the exemplary use of the braille language for attitude control systems is shown.Fig. 6 A show
Show to meaning property the hand 4 according to the user of some embodiments of the present invention., can be by bending forefinger 1, curved using braille language
Bent middle finger 2, the bending third finger 3 also create posture by rotating 61 hands 4.
Fig. 6 B schematically show the symbol of the letter " C " in the braille 63 according to some embodiments of the present invention.For
The posture corresponding with alphabetical " C " is formed in braille 63, user's needs while 61 hand 4 are rotated bend nameless a3,
And finally bend nameless b3.In a similar way, braille language can be used to represent all letters in alphabet, in order to
With the writing in the case where direct eye contact need not be carried out with screen and/or read text.It should be noted that braille class
Type language can be realized with various posture sequences.Alternatively, generate different types of touch feedback with corresponding to particular letter and
Word.
The major advantage of this attitude control systems is：
Operation freedom-skilled user can be a small amount of cognitive resources distribution operate the said equipment.This causes so
User can perform the operation (such as drive and write) of complexity simultaneously.
The accessibility of modern smart machine is provided for visually impaired person and deaf individual.
Eyes and ear-from seeing screen and listen in speech are discharged.
Protection privacy-avoid damaging when using speech recognition interface or when screen is visible to other people.
In some embodiments of the invention, attitude control systems are used as independent product, so as to which interface be exposed
Application programming interfaces (API) are given, to be usually integrated into original equipment manufacturer (OEM) system, this may save money
Source (electric power, disposal ability etc.).
With reference now to Fig. 7 A to 7D, these figures show that being located at nervus radialis and chi for the different gestures that are performed by user
The curve map for the SNC Signal's behaviors that sensor between bone nerve detects.Fig. 7 A show some implementations according to the present invention
Example from thumb movement caused by signal 72.Fig. 7 B are shown according to caused by some embodiments of the present invention from forefinger movement
Signal 74.Fig. 7 C show the signal 76 according to caused by some embodiments of the present invention from the first little finger of toe movement.Fig. 7 D are shown
According to signal 78 caused by some embodiments of the present invention from the second little finger of toe movement.It can be easily seen that from these figures, no
Same posture provides different Signal's behaviors, and in the typical case detected by SNC sensors, electrically (that is, different postures are drawn for behavior
Play different due to voltage spikes) and both the Typical duration of signal in have differences.But more accurately algorithm can be used
(measurement additional parameter), to identify the different gestures of user's execution.
In all Fig. 7 A into 7D, binary signal 71 indicates when that system identification goes out and has performed posture.Use dual threashold
Value method (as described above), noise is ignored, and system is only reacted to actual posture.
With reference now to Fig. 8 A to 8B, the figures illustrate showing for the posture sorting algorithm using single biopotential sensor
Example property result.In the exemplary algorithm, the length for measuring frame is expressed as Nf, sample of signal xi(x1, x2... xNf).Will be corresponding
Binary signal (instruction detects posture, such as shown in Fig. 7 A to 7D) be expressed as sigdetSo that wherein detect myoelectricity
The sample of the frame of activity is expressed as sigdet=1, similarly, sig is expressed as corresponding to the sample of noisedet=0.Finally, define
Five exemplary statistical natures are classified to the attribute of SNC signals：
Card side and：
Total detection length：
Arc length：
Gini index (dispersion index)：
Wherein μ is average, and n is the quantity of the positive index detected, wherein sigdet=1.
Average absolute value：
Consider that all these features can create signal signature for each measurement so that different postures can be distinguished.
The activity of fisrt feature measurement signal in chi range of distribution.The detection length of second feature measurement signal.Third feature
The activity in time domain is measured using absolute derivative.Fourth feature is modern decentralized measure.Fifth feature is average absolute amplitude.
During measurement, 20 frames including three postures are sampled：For seven postures of forefinger movement 81
Repeat, seven postures that fist holds 82 are repeated, and six postures of little finger of toe movement 83 are repeated (altogether with 20 frames).For
These features are classified exactly, it is necessary to consider all features described above.
Fig. 8 A show the different characteristic f according to SNC signals according to some embodiments of the present invention1To f4The three of classification
The drawing of the posture 81,82,83 of type.From these figures it can be seen that different postures gives different Mode behaviors, but
It is to classify exactly to posture, these features must be embedded in multidimensional feature space so that each posture quilt
It is clearly separated and forms cluster.
Fig. 8 B show the three-dimensional scatter diagram according to some embodiments of the present invention, and it includes the posture for three types
The prominent features of 81,82,83 measurements.The axle of the scatter diagram is the feature f of such " feature space " as defined above1-
f3-f4.Sorting algorithm can be used one group of SNC data characteristics of mark and export segmentation so that each posture cluster is in feature space
It is separated.When being sampled to new posture and its feature (or signature) is calculated, accordingly " in multidimensional feature space
Point " is by allocated section (that is, posture).It should be noted that single SNC sensors are used only in this example, and multiple sensors
More preferable type of gesture identification and improved Generalization Capability can be achieved.
Although it should be noted that provided herein is example use special characteristic, further feature or algorithm can be used
And do not limit the scope of the invention.
Fig. 9 schematically shows the facial pose control system 90 according to some embodiments of the present invention.Except (by enclosing
Around the system detectio of a part for user's limbs) outside above-mentioned posture, EMG sensors can also be used to carry out facial pose identification.
For example, EMG sensors can detect the facial pose of frontalis (being located at forehead) and temporalis (being located at temple), such as documents below discussion
's：Hamedi M.et al.,“EMG-based facial gesture recognition through versatile
elliptic basis function neural network”,BioMedical Engineering OnLine(2013),
Vol.12,NO.73。
Facial pose can not be assembled in detected by the system on the limbs of user, it is therefore desirable to which one different to be
System, it can be worn over the head of user, while similar to the operation of said system, because only that the type change of posture.It is preferred that
Ground, such system are embedded in the wearable device of such as glasses etc.Such system may be provided as stand-alone product
(wherein glasses are not used in eyesight improving), it are couple on existing glasses or are embedded into intelligent glasses so that gesture recognition is eye
The supplementary features of mirror.
Facial pose control system 90, which is embedded into, can be worn in the equipment of user's head (such as glasses), and can with it is above-mentioned
System similarly operates (for example, similar to attitude control systems 20 shown in Fig. 2).Facial pose control system 90 includes corresponding
In frontalis and the forehead EMG sensors 92 of facial pose at forehead can be identified.Facial pose control system 90 is further wrapped
Include corresponding to temporalis and at least one temple EMG sensors 94 of the facial pose at temple can be identified.Alternatively,
At least one temple EMG sensors 94 can be couple at least one tactile feedback actuators so that facial pose control system
90 can be calibrated to identify facial pose, and then user can receive the touch feedback of the posture for identification (at temple).
Such system can it is following it is at least one in it is useful：
Control computer equipment, such as using the intelligent glasses of facial pose.
Well-trained user may can identify facial pose from the feedback received.By this way, two
The user of remote match can obtain the distinct feed-back of mutual facial pose, and can identify and generally only meet face-to-face
When the sensation that is noted and other nuances.This can be felt and body language and text, voice and generally by passing on
The video that uses enriches electronic communication.
There is the user of physical problem for dominating its limbs, use facial pose control computer equipment.
Such system is identified with the facial pose based on image procossing and is combined the user that can help vision disorder
Identify the sensation and facial pose of its partner.
In addition, these systems can be used for cooperating with self-closing disease user, to improve body and mind technical ability.
With reference now to 10A to 10B, these figures are related to the further implementation being embedded into touch feedback in EMG sensors
Example.Although EMG is the reading to nerve action potential caused by muscle, neuromuscular electric stimulation therapy (NMES) is actually opposite
Action, wherein carrying out stimulus movement nerve using electric signal and causing contraction of muscle.Electrotactile stimulation (ETS) is swashed using potential
The nerve fibre living being connected with the tactile sensation recipient under skin.
Recent studies have shown that NMES and electric touch method can relate to touch feedback and tactile display, and NMES leads to
It is usually used in analog force sensing (such as feeling to promote the resistance of weight), and is shown using electric touch to simulate sense of touch, such as line
Reason.Some examples of these researchs can be found in the following literature：Pamungkas D.et al.,“Electro-Tactile
Feedback for Tele-operation of a Mobile Robot”,Proceedings of Australasian
Conference on Robotics and Automation,University of New South Wales,Australia
(2013)；Peruzzini,M.et al.,“Electro-tactile device for material texture
Simulation ", IEEE (2012), pp.178-183 and Kruijff, E., et al., " Using neuromuscular
electrical stimulation for pseudo-haptic feedback”,Proceedings of the ACM
symposium on Virtual reality software and technology(2006),pp.316-319。
In this embodiment, EMG can be sensed and to produce tactile anti-by EMG sensors being combined to produce with NMES and ETS
The individual unit of feedback.In addition to being used together immediately with attitude control systems, the sensor/actuator of this combination can produce
Touch feedback for finger movement receives in order to read and notify.Specifically, this device can be used for field of prosthetic limbs, its
Middle artificial limb arm can be controlled by EMG sensors, then provide a user the feedback on texture and power.Alternatively, the sensing of combination
Device-actuator can be used for the robot of computerization and the remote control field of machine.In certain embodiments, combination
Sensor/actuator further can respond (GSR) sensor combinations with pulse transducer and/or galvanic skin.
Figure 10 A schematically show the combination sensor and tactile feedback actuators according to some embodiments of the present invention
Exemplary circuit.(such as shown in Figure 2) circuit of amplifier right leg drive (DRL) element 21 of biopotential sensor 12
ETS and NMES stimulators are may be used as, to produce touch feedback and read muscle potentials.Because EMG is (for example, sEMG/
CEMG) it is substantially difference amplifier, so the stimulus signal for being added to common-mode signal will not be put by EMG sensors
Greatly, wherein, the common-mode signal arrives body by DRL drivings.
In standard DRL circuits, EMG signal 101 is collected on positive electrode 104 and negative pole 106, to use differential amplification
Device 21 is amplified, to produce the EMG signal of amplification.Meanwhile carry out the EMG signal 105 of self-electrode 106,104 in stimulator 107
Place is averaged to produce common-mode signal 103, and the common-mode signal is then amplified by booster amplifier 109 and by reference to electrode 108
Drive the skin of user.In this embodiment, stimulating current signal closes at stimulator 107 with common mode signal group, by phase
Same path is to reference electrode 108, here, it stimulates cutaneous nerve.
Figure 10 B schematically show showing according to the combination sensors with concentric ring of some embodiments of the present invention
The cross-sectional view of example property circuit.Further embodiment 120 shown in Figure 10 B includes reference electrode 108, and reference electrode 108 wraps
Two concentric rings are included, there is internal electrode 110 and the outer electrode separated by non-conducting material 111 with internal electrode 110
112, wherein stimulus signal is driven to internal electrode 110 and common-mode signal is driven to outer annular electrode 112.In this hair
In some bright embodiments, electric current is driven across by the impedance of skin by using the measurement of biopotential sensor and caused shaken
Width, so as to measure the skin electric conductivity of user's wrist between two electrodes.Because electric current is constant and by the equipment control
System, measured voltage may change according to Skin Resistance.
In some embodiments of the invention, special purpose operating system (OS) operation can be used in attitude control systems.In the reality
Apply in example, OS can be used for control and navigational computer equipment (for example, intelligent watch).With display, have with user's
The display menu of four icons corresponding to different fingers (such as forefinger, middle finger, the third finger and thumb).So that mobile specific finger
Corresponding to special icon, and select that additional menu (for example, mobile forefinger instruction selection alphabetical group " A-G ") can be navigate to.When
When (for example, using intelligent watch) writes text message, the operation can be used to navigate by different letters.In addition, so
Operating system available dedicated language (the braille language such as shown in Fig. 6 A to 6B) operation.
With reference now to Figure 11 A to 11F, calculated the figures illustrate such as performed in the prior art with thumb ability of posture control
The example of machine equipment；By using the operating system based on posture, according to some exemplary embodiments of disclosed theme.
Attitude control systems can couple with that can send the operating system based on posture of order, to control and browse calculating
Machine equipment (for example, intelligent watch).In this embodiment, system detectio is due to wrist domination caused by the specific movement of thumb
Nerve signal.Such operating system can perform operation on computerized equipment, regardless of whether including touch-screen, it is not necessary to see
Examine screen.The interface of this operating system can be based purely on thumb posture, and without touch-screen curtain, keyboard, mouse, its combination
Deng.
Figure 11 A show the commercially available solution of the touch-screen user interface (UI) with thumb control wrist-watch.Show at some
In example property embodiment, according to some embodiments of the present invention, thumb movement is emulated in the case of no real screen, is such as schemed
Shown in 11B, there is provided the UI elements of the operating system based on posture control wrist-watch.
Figure 11 C show the commercially available solution of the touch-screen UI with thumb control handheld device.In some exemplary realities
Apply in example, according to some embodiments of the present invention, thumb movement is emulated in the case of no real screen, as shown in Figure 11 D,
The UI elements for providing the operating system based on posture control handheld device.
Figure 11 E show the commercially available solution of the control stick UI for controlling game console using two thumbs.
In some exemplary embodiments, according to some embodiments of the present invention, thumb shifting is emulated in the case of untrue control stick
It is dynamic, as shown in fig. 11f, there is provided the UI elements of the operating system based on posture control game console.
As the electronic communication equipment for being more used for various platforms is introduced, such as Internet of Things (IoT), virtual reality
(VR), smart home, intelligent television, computerization vehicle etc., for example, with touch-screen or these equipment of Keyboard Control and/or
Person's keyboard be used in particular for for example such as drive when operate or manipulate automobile function when play music it is busy movable when be fiber crops
Tired.Intelligent watch is wearable user interface, and it can be configured as moving to control by using foregoing posture
These communication equipments.
Intelligent watch may include sensor 12, has flexible form and interconnect 17 intelligent spire lamella 10, for performing customization
Processor/computing unit 22, communication component 13 and the tactile actuator 14 of algorithm, for example, such as Figure 1A to 1B (such as intelligent wrist
Band) and Fig. 2 to 4 block diagram shown in.Intelligent watch designs the signal to noise ratio that may include the processed bioelectrical signals detected
(SNR), the unique balance between comfort level and function (trade-off, balance).For example, good design can be bonded well
The wrist of user, so as to increase SNR and reduce for being detected the changes in contact between electrode and user's skin during movement
To bioelectrical signals in motion artifacts.
Figure 12 A schematically show the back view 140 of the hand 150 according to some embodiments of the present invention, and hand has
The intelligent watch 160 kept by intelligent spire lamella 165 in wrist 155.Intelligent watch 160 may include screen 162, wherein hand
The screen elements that 150 known poses made can be used on control screen 162 and/or computerized equipment 29,49, are such as selected
Icon.Figure 12 B schematically show the palm view 145 of the hand 150 according to some embodiments of the present invention, and hand, which has, to be passed through
Intelligent spire lamella 165 is maintained at the intelligent watch 160 in wrist 155.
Illustration 170 shows the bottom view 175 of intelligent watch 160.One embodiment of flexible interface 10 in Figure 1A can
Including the wrist strap 165 with the biopotential electrodes 16 for detecting biopotential signals.In biopotential electrodes 16 at least
One surface nerve conduction (SNC) electrode that may include for detecting surface neuro-transmission signal.As shown in Figure 1A, bioelectrode
16 are couple to biopotential sensor 12.Wrist strap 155 may include tactile actuator 14 and communication controler 13.Intelligent watch 160 can
Including the processor 22 for identifying posture, it is known that association or bioelectrical signals between posture and surface nerve conduction
It is stored in memory 24.
Processor 22 may include one or more processing units of such as one or more computer.Processor 22 may include
Field programmable gate array (FPGA), graphics processing unit (GPU), microcontroller and microprocessor.Processor 22 may include far
Journey computer may include any other suitable treatment technology.
Sensor 180 may include that such as Inertial Measurement Unit (IMU), pressure sensor, photoelectric plethysmogram (PPG) pass
Any sensor of sensor and RF sensors.Sensor 180 can be placed in any position along wrist strap 165.Pressure sensor can
For measuring tendon movement.Inertial Measurement Unit (IMU) can be used for the rough movement of measurement hand 150.Pressure sensor can be used for
Measure the power on the move of tendon in hand 150 and arm.PPG sensors (the sensing such as based on light emitting diode (LED) technology
Device) available for the Volume Changes for measuring wrist during wrist tendon moves.Except the god by biopotential sensor 12 from wrist 155
Outside the bioelectrical signals arrived after testing, the signal from sensor 180 can be used by processor, to increase from biological telecommunications
The possibility of number (for example, SNC signals) identification correct body position.
In some embodiments of the invention, communication controler 13 can be in the processor 22 in intelligent watch 160 and storage
Letter is relayed between biopotential sensor 12, sensor 180 and tactile actuator 14 between device 24 and on intelligent spire lamella 165
Breath.In other embodiments, processor 22 and memory 24 can be also placed on wrist strap 165.Sensor 180 may include above-mentioned technology
Any combinations.As shown in Figure 1A, SNC sensors may include AFE(analog front end) and electrod-array.
Figure 13 schematically show according to some embodiments of the present invention be configurable for reflectometry around
The intelligent watch 160 that wrist 150 is set.Sensor 180 may include the RF sensors of such as wave producer 190 and receiver 195.
The RF pulses as caused by wave producer 190 can be used for measurement due to as pulse from generator 190 travels to receiver 195, no
The change of reflectivity caused by same posture movement in the tissue of wrist 155.
In some embodiments of the invention, examined in gesture recognition algorithm (hereinafter referred to as gesture recognition) detection sensor 12
Event in bioelectrical signals caused by the posture made due to hand 150 measured, and sorting algorithm is applied to what is detected
Event is to identify the posture as described in Fig. 8 A to 8B.
Figure 14 A to 14D show the posture 200 that can recognize that by intelligent watch 160 according to some embodiments of the present invention.
Posture 200 shown in Figure 14 A to 14D is merely for vision definition.Processor 22 can be configured as identifying any suitable hand and
Finger gesture, and the posture 200 being not limited in Figure 14 A to 14D.
Figure 14 A are shown moves 210 appearances according to the thumb that can recognize that by intelligent watch 160 of some embodiments of the present invention
Gesture.For example, the thumb movement 210 made by user can be by the screen 162 and/or computerized equipment 29,49 of intelligent watch 160
Screen on cursor be moved to right side, and/or start by computerized equipment 29,49 perform some functions.
Figure 14 B are shown moves appearance according to the forefinger 215 that can recognize that by intelligent watch 160 of some embodiments of the present invention
Gesture.For example, forefinger movement 215 can be by the light on the screen of the screen 162 of intelligent watch 160 and/or computerized equipment 29,49
Mark is moved to left side, and/or starts some functions of being performed by computerized equipment 29,49.
Figure 14 C are shown kowtows 220 according to two fingers that can recognize that by intelligent watch 160 of some embodiments of the present invention
Posture together.By at least two fingers kowtow 220 can for example select together on the screen 162 of intelligent watch 160 and/or
Item on the screen of computerized equipment 29,49, and/or start some functions of being performed by computerized equipment 29,49.
Figure 14 D show 225 two fingers of the extruding identified by intelligent watch 160 according to some embodiments of the present invention
Posture.By at least two fingers extruding 225 or be pressed together can continuously select on the screen 162 of intelligent watch 160 and/
Or the project on the screen of computerized equipment 29,49, and/or start some functions of being performed by computerized equipment 29,49.
In some embodiments of the invention, intelligent watch 160 can be configured as and (such as the control positioned at fascia
In platform processed) radio of broadcasting music or sound system communicated.User can be by the way that at least two fingers extruding 225 be existed
Come together change radio speaker volume.For example, forefinger and thumb are crowded together available for increasing volume, by middle finger and
Thumb squeezes can be used for reducing volume together.In other embodiments, accelerometer can be placed on wrist strap 165.Accelerometer
It is pressed down against available for whether detection forefinger and thumb refer to upward (for example, increase volume) or refer to together (for example, reducing sound
Amount).
Figure 15 be describe according to some embodiments of the present invention by the flexible user interface 10 of ability of posture control and based on
The flow chart of the method 250 to be communicated between calculation machine equipment 49.Method 250 comes from including detection 225 and is placed in user's body
One or more biopotential sensors (for example, sensor 12) one or more bioelectrical signals, one of them or it is more
Individual biopotential sensor includes being used at least one surface nerve conduction (SNC) sensor for detecting at least one SNC signals.
Method 250 includes (for example, by using processor 22) by least one SNC signals detected with corresponding to multiple known appearances
The data of multiple reference signals of gesture are compared 260, and each reference signal is substantially associated with one of known poses.Number
According to being storable in memory 24.
Method 250 may include that (for example, by using processor 22) corresponds at least one from multiple known poses identification 265
The known poses of individual SNC signals.Method 250 may include that (for example, by using processor 22) will identify via communication equipment 13
Known poses transmission 270 give computerized equipment 49.
Figure 16 is to describe the flow chart for being used to identify the method 300 of known poses according to some embodiments of the present invention.
Gesture recognition may include event detection and sorting algorithm, applied to the bioelectrical signals from the detection of biopotential electrode 16 225.Method
300 may include that (for example, by using processor 22) is gone at least one surface nerve conduction (SNC) signal detected
Make an uproar 305.
In some embodiments of the invention, filtering out noise from bioelectrical signals or carry out denoising 305 to it may include to give birth to
Customization basic function into bioelectrical signals represents.Wavelet transform (DWT) can be used there are efficacious prescriptions as generation rarefaction representation
Formula.Signal in time frame is changed in the following manner：
Wherein,It is wavelet coefficient,It is morther wavelet.Morther wavelet may be selectedSo that represent bioelectrical signals
F (t) be transformed to sparse domain.Small wavelet coefficient can be cleared, so as to realize effective denoising of bioelectrical signals.At it
In his embodiment, the principal component analysis (PCA) of the correlation between bioelectrical signals such as from sensor 16 it is attached
Add conversion by (f1(t), f2(t), f3(t) ...) provide, wherein, index n=1,2,3 ... it is the quantity of sensor.With this side
Formula calculates wavelet coefficient in equation (6)The higher precision to bioelectrical signals denoising is added, because in adjacent sensors
The noise detected in bioelectrical signals between 12 is related, because sensor 12 is placed in wrist 155 with being close together.
In embodiment as described herein, the adjacent sensors of " being close together ", which can refer to, can be placed on identical limbs and close to muscle group
Sensor 12, for example, less than about 15cm.Generally, sensor 12, which can be placed in, is separately less than 1cm, for example, and generally along dynamic
Make the path (for example, along nerve) of electric potential signal.In addition, can be from EMG signal using SNC morther wavelets as sole basis
Extraction SNC signals in (for example, EMG noises).It can be generated based on data caused by the multiple testing experiment of many users female small
Ripple
In addition to EMG noises, other noise signals in the bioelectrical signals detected may include electrode movement, rub
Wipe artifact, 50/60Hz power line noises and other may be mistaken as the noise source of the pressure from innervation.Due to noise
The shape of artifact may be different from morther wavelet, the inner product very little of (6) and these unwanted can be made an uproar providing in equation
Acoustical signal be cleared or ignore in the calculating of effective denoising.
Method 300 may include to detect the event in 310 at least one SNC signals.After to SNC signal denoisings, processing
Incident Detection Algorithm can be used to determine whether that there occurs gesture event in device 22.For example, showing that sensor 12 detects life
For Fig. 7 A of thing electric signal into 7D, processor 22 can detect gesture event, such as the posture thing corresponding to foregoing hand 150
The bioelectrical signals 72,74,76,78 of part.
Method 300 may include to be segmented the event application 315 detected with determine one of wherein raw gesture event hair or
Multiple time frames.In Fig. 7 A into 7D, processor 22 can recognize that section 71, and it indicates start and stop time, or wherein detects
The time frame of the gesture event beginning and end arrived.Implementation shown in Fig. 7 A to 7D exemplifies hard sectoring, wherein beginning and end
Voltage of the frame from 0 to 1 defines rectangle.In other embodiments of the invention, soft sectoring can be used, its stage casing 71 can not be square
Shape, and can be any shape of the envelope for the gesture event that tracing detection arrives.The amplitude of section may include any higher limit, and
It is not limited to the voltage 1 in the case of hard sectoring.In certain embodiments, soft sectoring can will belong to the probability of known poses event
Distribute to each sensor samples.
Method 300 may include to extract 320 statistical natures in one or more frames of the event detected.Fig. 8 A are shown
The four statistical nature f obtained respectively from equation (1) to (4)1、f2、f3、f4, so as to what is detected in this example for three
The attribute of 81,82,83 pairs of bioelectrical signals (for example, SNC signals) of gesture event is classified.As it was previously stated, as shown in Figure 8 B
Feature space by three feature f1、f3And f4Form.
Method 300 may include for the sorting algorithm based on the data related to SNC signals to be applied to extracted statistics spy
Sign, so as to posture known to determining.Fig. 8 B show the feature space with known poses 81,82,83.
In some embodiments of the invention, sorting algorithm may include feature space, such as shown in Figure 8 B, its be based on
The related data of SNC signals.In other embodiments, sorting algorithm can be configured as the side between the cluster in identification feature space
Boundary, to increase the possibility that known poses are determined from SNC signals.
In some embodiments of the invention, soft sectoring can allow weighted feature to extract.For example, except equation (1) to (5) it
Another outer statistical nature may include adopting for the correlation between sensor 12 and/or the bioelectrical signals from sensor 12
Sample frame.The more accurate mode for determining known poses can be provided using weighted correlation metric.
In some embodiments of the invention, posture grader (example is used after feature extraction that can be in step 320
Such as, machine learning).Sorting algorithm may include random forest grader (random forest classifier).It can train more
Individual random forest grader.Denoising may include to use low pass filter.For example, event detection may include to grow with stationary window
One of grader trained in the data flow of degree.Grader can be exported to each sample and voted.Once votes reach threshold value, all
As for example, the half-sample in length of window is classified as belonging to gesture event, then the snapshot of posture can be input into grader.
Snapshot may include the series of frames of all the sensors 12 from multiple data points, until signal (for example, event detection) is completed
Launch (firing), wherein event detection window stops switching between zero and one.Data are input into posture grader and more
Individual snapshot is trained.Above-mentioned random forest grader analysis helps to assess extensive error.Can by polymerize multiple snapshots come
Predict to reduce mistake.
Ratio control is that biopotential sensor reading is converted into continuous control signal, and it can be input into computerization
Device 29.Posture 225, wherein at least two finger can press or press together, wherein the life detected by sensor 12
Thing electric signal can be used for measuring or estimating the pressure between at least two fingers.For example, two fingers are pressed together available
In producing the control signal applied to video equipment, so as to the film of user's F.F. viewing, such as the pressure being applied between finger
Power is bigger, and the fast forward speed for being converted into film is faster.In an identical manner, posture 225 can be used by the driver of automobile, such as
When driving the volume of the radio in automobile is controlled by the way that his finger is pressed together.
Estimation to bioelectrical signals reading to control signal should be smooth, and be consistent, example in time
Such as, so as not to changing the parameter of the speed of such as radio volume or F.F. film too quickly.Classification analysis can be by gesture recognition
Use, as described by Figure 14 and Figure 15 flow chart, and regression analysis controls for ratio.Generally, intelligent watch 160 uses
Classification analysis, for example, to identify posture 225 that hand 150 is made, and applied to the biological electricity thing detected by sensor 12
The regression analysis of part, posture is converted into continuous control signal, as described in following ratio control embodiment.
Figure 17 is schematically shown according to some embodiments of the present invention, when two fingers 405 are pressed together
The biopotential signals 420 detected.Biopotential signals 420 are the signals from the detection of one of multiple sensors 12.In Figure 17
In show the posture 400 that at least two fingers press together together.The finger 405 of hand 415 with pressure P (t) extruding or
It is pressed together, wherein pressure can be represented with any suitable unit, such as such as Pascal, pound per square inch (psi).
The detection sensor voltage 420 of sensor 12 on the wrist strap of intelligent watch 410, the wherein increase of amplitude and frequency in time with
Increased pressure P (t) is directly proportional.In other words, when known posture includes at least two fingers being pressed together, and
Processor 22 is configured as including with being applied at least two by assessing at least one surface neuro-transmission signal detected
The amplitude and frequency of proportional pressure between finger, to identify known poses.Then ratio as described below can be applied to control
Algorithm can be applied to the pressure controling signal of computerized equipment 29 to extract.
In some embodiments of the invention, the regression analysis for the classification analysis of gesture recognition and for ratio control
Similar data lines (data pipeline) can be used to carry out processing data.For example, gesture recognition and ratio control data pipeline
The similar algorithm discussed first herein can be used.
Figure 18 A are the block diagrams 421 according to the data lines for gesture recognition of some embodiments of the present invention.Block diagram
421 include denoising frame 432, event detection frame 434, soft sectoring frame 436, feature extraction frame 438 and classfying frame 440.For posture
Each in these frames in the data lines of identification discusses in Figure 16 flow chart.
Figure 18 B are the block diagrams 431 for being used for the data lines that ratio controls according to some embodiments of the present invention.Block diagram
431 include denoising frame 432, dimension reduces frame 442, conversion frame 444, feedback frame 446 and pressure estimate frame 448.
In some embodiments of the invention, can be used to realize this two pipelines using the machine learning techniques of neutral net
In data flow.Phase can be used in the functional block of the gesture recognition represented in Figure 18 A to 18B and ratio control pipeline as described above
With algorithm or pipeline architecture realize, use wavelet transform (DWT) 422 (as previously described), convolutional neural networks (CNN)
424 and shot and long term memory (LSTM) neutral net 426 handle one or more bioelectrical signals, as will be described later.
CNN 424 is the neutral net for being selected for the bioelectrical signals that management detects from the sensor 12 being placed near wrist.CNN
424 processing detection to related signal in terms of be effective.(for example, time frame) keeps biology to LSTM 426 in time
The memory of electric signal, and can be within short time and long period in detection signal pattern, such as by compound tube below
Discussed in line architecture.
After the ratio control pipeline 431 for estimating the pressure between at least two fingers, to being examined by sensor 12
The one or more bioelectrical signals measured carry out denoising 422 (for example, denoising frame 432) and discrete wavelet transformation can be used
(DWT), as it was previously stated, being used for gesture recognition pipeline.Carrying out denoising to bioelectrical signals may include for example to remove from SNC signals
EMG signal noise.
In some embodiments of the invention, the dimension in pipeline 431 reduces 442 and can be used for reducing data volume, so as to only
Leave the significant data related to posture detection and reduce the complexity of detection.There are various technologies to realize this point：
A. unsupervised dimension is reduced：In some embodiments of the invention, using such as NMS (Non-negative Matrix Factorization) etc
Technology, the dimensions of the data in frame can be reduced, i.e. the detection data from sensor 12, which may decrease to, represents single time sequence
The single frame of row.This reduction can be completed by minimizing cost function：
(8)minW, H||F-W·H||w
According to condition W, the primitive organism electrical signal data after the frame denoising that the expression of H >=0, wherein F is arranged in the matrix form,
H is hidden variable (for example, the pressure applied between finger 405), and W is weight matrix, each sample in one of frame
Help to rebuild F via W.The selection of norm in equation (8) is used for the minimum for adjusting ratio control application.
B. the unlabelled dimension supervised is reduced：In some embodiments of the invention, can be via autocoder nerve
Network reduces data F.The framework may include feedforward neural network, but be that instead of in data set DF(for example, frame FiMultiple realities
Example) on training network so as to prediction label Yi(classification), network can be trained to reconstruct input DF.Constraint can be applied so that automatic
The quantity of hiding node layer in encoder neutral net is less than the quantity of input layer, forces its own tight of e-learning
Gather expression.Hidden layer is reduced available for dimension.
C. the dimension for supervising mark is reduced：In some embodiments of the invention, it is to return that the mark dimension of supervision, which is reduced,
Problem, wherein via certain analytic function establish input output relation.After switch process 444, the technology may be more suitable
With.In addition, this relation may be not necessarily it is linear.Therefore, classical linear regression does not apply to.Random forest return and most
Small absolute retract and Selecting operation symbol (LASSO) return more suitable for it is such the problem of.
The modernism of time of supervision Sequence Learning is periodic neutral net (recurrent neural
Network), specifically LSTM (shot and long term memory).With more " classics " learning method on the contrary, LSTM neutral nets can be with
Consider that the mode of context handles serial data.More specifically, LSTM networks are contemplated that the data from previous frame.It is all its
His method can all handle the frame of fixed size.Provide the feedback from previous frame, but feed back and predefined (pass through fixation
The number of previous frame).Therefore, because its unique hidden state formula, the method that LSTM provides more flexible processing data.
Supervised learning needs the data set marked.In some embodiments of the invention, identified in processor 22 known
After posture 400, in order to which the data of posture 400 derived from the bioelectrical signals from sensor 12 are marked, to user
Auxiliary signal is provided.For example, voice signal can be played to user.Voice signal can change in frequency and/or amplitude.It may indicate that
The auxiliary signal that user and user hear proportionally changes the pressure P (t) between finger 405.Data from sensor 12
It is recorded and is further used as set of tags Y and is supplied to data DF.It is contemplated that user is heard between voice signal and customer responsiveness
Response time, to prevent Y and DFBetween mismatch.
In some embodiments of the invention, the conversion 444 in pipeline 431 can be used for carrying out data by pipeline 431
Pre-process (precondition) or post processing (post-condition).In the case of supervised learning, conversion 444 can be used as
Domain knowledge is introduced into system (for example, right in neural recording by pre-treatment step to realize preferably estimated pressure learning procedure
The sensitivity characteristic of amplitude and frequency).For other dimensions reduce technology, shift step 444 dimension reduce step 442 it
Afterwards, to convert the output into more meaningful signal.
In some embodiments of the invention, conversion 444 may include the filtered Teager- as post processing formula
Kaiser energy calculations accord with.The operator is defined as：
(9)TK[f(ti)]=f (ti)2-f(ti-1)·f(ti+1)
Wherein, f (t) is bioelectrical signals, and TK operators are proportional to the instantaneous frequency and amplitude of signal.
In formula is pre-processed, wave filter includes CNN (convolutional neural networks) 444.This method is proved to learn to have
The partial transformation of effect, similar to engineering wave filter, such as low-pass/high-pass based on Fourier or TK operators.
Control and apply for ratio, auxiliary signal can be converted via filtered TK operators using equation (9).It is auxiliary
Signal noiseless is helped, therefore is preferable for TK conversion.CNN 424 can learn such expression.Simply filter and change
The advantages of data, is that such a neutral net is derived from data, makes it more sane for noise.But this
The network of sample needs substantial amounts of data, and computationally more expensive.
Figure 19 schematically illustrates the combination pipeline architecture using neutral net according to some embodiments of the present invention
450.Combination pipeline architecture 450 may include that DWT 470 and CNN424 for denoising 432, and dimension reduce by 442 frames, with
Combined in the LSTM 426 for realizing 446 frames of conversion 444 and feedback.Pay attention to, combination pipeline architecture 450 can be used for realizing gesture recognition
(GR) pipeline 421 and ratio control (PC) pipeline 431.But realize that the difference between GR and PC pipelines is neutral net (example
Such as, CNN 424 and LSTM 426) it can be trained to realize gesture recognition or realize ratio control.
Combination pipeline architecture 450 may include the n bioelectrical signals detected by n sensor 12, and wherein n is integer.
From sensor1Bioelectrical signals 455, from sensor2Bioelectrical signals 460 and from sensornBioelectrical signals
465 can be input into and be expressed as DWT1、DWT2、……DWTnCorresponding DWT unit or frame 422.Wavelet transform can be used
(DWT) each overlapping fragmentses in the bioelectrical signals 455,460 and 465 from each respective sensor are decomposed.Decompose
Result is time frame and the single matrix of each sensor.Matrix element may include the volume of bioelectrical signals and one group of self-defined small echo
Product, causes super complete sparse base, prepares to be used for denoising (for example, for removing incoherent data).It is many in the formula
Matrix element coefficient can be small and can be neglected, so as to realize the rarefaction representation of shape and trend in data.Due to
Electrode can be placed near each other and can be sampled together, so signal can be relative to each other (for example, both signal and noise).
, can be from the shape (being represented by DWT coefficients) and time trend (DWT coefficients of data by using signal collection as sparse basis representation
Change) infer observation.
In each time frame, DWT 422 each piece has corresponding input node in CNN 424 input layer 482
485.CNN 424 is configured as making hidden layer 484 have the node less than input layer 482, so as to realize that dimension reduces 442.
Because signal is related because of approaching for electrode 16, CNN 424 is configured as reducing data volume.Due to bioelectrical signals space-
Temporal correlation, this is possible.Reduce data dimension and deleting unnecessary composition allows to extract feature interested, with
Just data are adjusted before classification or recurrence.Interconnection between node 485 includes weight.For example, each node may include such as
Logarithm or the conversion of S-shaped conversion.Node 485 in CNN 424 CNN output layers 486 can be used as the LTSM on each time frame
The input vector of 426 machines.
LSTM 426, which has, is expressed as LSTM1、LSTM2……LSTMmM units, wherein m is integer.CNN output layers 486
In CNN nodes 485 be connected to the input of m LSTM unit, as shown in figure 19.It is each with hiding in m LSTM unit
Door connection 491, it provides the storage mechanism that is presented in LSTM Concealed doors.The stackable individual multilayer LSTM units of m ' are (wherein
M ' is integer), to realize that the more abstract of data represents (not shown in Figure 19).Pay attention to, in pipeline architecture 450 is combined, instead
Feedback 446 is integrated into LSTM storage mechanism.LSTM1Output 492, LSTM2Output 494 ... LSTMmOutput
496 be the sample of the estimated pressure P (t) between the finger 405 to be pressed together in continuous time frame.
M LSTM memory cell can according to based on be previously entered the hiding memory that signal is provided input (for example, come
From CNN 424 output) receive input and output decision.During the training period, LSTM units can receive feature (for example, local retouch
State symbol), the result of network data and previous LSTM units decision-making.Using LSTM 426 unique hidden layer, LSTM 426 can be
Long-term and short-term data (for example, unique memory component of variable-length) detection pattern.
In some embodiments of the invention, previously described auxiliary signal can be used to be trained for the machines of LTSM 426.
For example, auxiliary signal can represent pressure, such as user response changes hand in the change for the frequency and volume for hearing voice signal
Refer to the pressure between 405.Similarly, auxiliary signal may include the discrete signal for representing posture.LSTM networks will receive auxiliary letter
Number (supervised learning) simultaneously converges on a scheme.
Figure 20 is to show that combining pipeline architecture 450 according to the use of some embodiments of the present invention estimates two fingers 405
The curve map 500 of standardization Pressure versus Time frame during extruding.When intelligent watch 410 user in posture 400 with increase
Pressure by finger 405 it is pinched together when, from pressure estimation frame 448 estimation normalization pressure increase, until user not
More forcibly finger 405 can be pressed together, therefore, normalized P (t) saturation value is 1.Curve map 500 can be by that will divide
Not from LSTM1、LSTM2……LSTMmThe representative of output 492,494 and 496 draw with time frame link to produce.
In some embodiments of the invention, the input in 446 loops of feedback, which may be such that, has estimated that " instantaneous " pressure.Instead
Feedback 446 provides a kind of stablizes the method exported using input signal and control loop.This, which will potentially contribute to eliminate, to refer to
Momentary fluctuation in the bioelectrical signals that shape thing 405 detects when pressing together, the otherwise fluctuation may be limited for controlling
The use (for example, the volume of auto radio, the speed of F.F. video, such as) of the signal of computerized equipment 29.Control letter
Number not necessarily need to be most accurately, but it is sufficiently stable to reach the intention of user.
In some embodiments of the invention, backfeed loop may include various frameworks.Simplest framework may include to utilize
Pressure from previous time samples is with the weighted average of the pressure of some time rank：
Low pass filter of the above-mentioned formula equivalent to output pressure signal.More complicated framework may include more complicated filtering
Device or control backfeed loop, such as such as proportional plus integral plus derivative controller (PID) controller.
In some embodiments of the invention, may include can for the operation of attitude control systems as illustrated in the flow chart of figure 5
Dress keyboard.Can glove user provides touch feedback to strengthen text write-in study and/or enable text on hand to user
Read.
Figure 21 A schematically show of the gloves 525 with touch feedback according to some embodiments of the present invention
One embodiment 520.The gloves 525 worn by user include tactile actuator 527 and sensor 532, such as Inertial Measurement Unit
(IMU), moved for detecting the finger of user.When gloves can be used as wearable keyboard, by using tactile actuator 527 to
Family provides touch feedback, the posture of grooming glove 525, so as to promote to write text by gesture.Similarly, as it was previously stated,
Gloves 525 can be used for reading text by carrying out touch feedback to user.Gloves 525 can include operation algorithm processor and
Other circuits mobile for hand of the detection from sensor 532 and finger and that touch feedback is activated to user.
Figure 21 B schematically show of the gloves 525 with touch feedback according to some embodiments of the present invention
Two embodiments 520.The gloves 525 of user's wearing can only include being used for the sensor 532 for detecting finger movement.But user wears
Wrist strap 540 can only include being used to provide a user the tactile actuator 527 of touch feedback.Wrist strap 540 can communicate with gloves 525
And receive the information moved on the finger of user.Wrist strap 540 may be in response to finger movement and provide a user touch feedback.
For herein with reference to any flow chart should be appreciated that only for convenient and clear and have selected shown method and divide
Into the separate operations of the frame expression by flow chart.It is possible that shown method, which is substituted, and is divided into separate operations, has identical
As a result.This replacement division of shown method should be understood that the other embodiment of method shown in expression.
Similarly, it will be appreciated that unless otherwise stated, have selected the ginseng herein shown only for convenient and clear
The operation execution sequence that the frame for any flow chart examined represents.The operation of shown method can be performed with alternate orders, Huo Zhetong
Shi Zhihang, there is identical result.This rearrangement of the operation of shown method should be understood to mean its of shown method
His embodiment.
Disclosed herein is different embodiments.The feature of some embodiments can be with the combinations of features of other embodiment；Therefore
Some embodiments can be the combination of the feature of multiple embodiments.For the purpose of illustration and description, the present invention is had been presented for
Embodiment it is described above.It is not exhaustion or limits the invention to disclosed precise forms.People in the art
Member many modifications, change, is replaced, changed and equivalent is possible it should be appreciated that in view of above-mentioned teaching.Therefore, should manage
Solution, appended claims are intended to all such modifications and variations that covering is fallen within the true spirit of the invention.
Although some features of the present invention have been illustrated and described, those of ordinary skill in the art now will
It will recognize that many modifications, replacement, change and equivalent.It will thus be appreciated that appended claims, which are intended to covering, falls into the present invention
True spirit in all such modifications and variations.
Claims (20)
1. a kind of ability of posture control interface arrangement, including：
One or more can be worn on the biopotential sensor on the body of user, for detecting the body from the user
One or more bioelectrical signals of body, wherein, one or more of biopotential sensors include being used to detect at least one
At least one surface nerve conduction SNC sensors of individual surface neuro-transmission signal；
Processor, at least one surface neuro-transmission signal for being configured as detecting is with corresponding to multiple known poses
The data of multiple reference signals be compared, each reference signal and one of the known poses are substantially related
Connection, to identify the known poses corresponding with least one surface neuro-transmission signal from the multiple known poses,
And the known poses identified are communicated to computerized equipment.
2. device according to claim 1, wherein, described device is configured as being assembled in the wrist of the user, and
And wherein, at least one SNC sensors are configured as detecting the electric signal of the nerve tract in the wrist.
3. device according to claim 1, further comprises：It is configured to detect at least one of the movement of the body
Motion sensor, and wherein, the processor is configured with detected movement to identify the known poses.
4. device according to claim 1, further comprises：Tactile actuator, it is configured as the known appearance identified
When gesture is registered in the computerized equipment, touch feedback is activated on the body of the user.
5. device according to claim 1, wherein, the processor is configured as by using one or more of lifes
Thing electric signal trains the data of the body for the user, by least one surface neuro-transmission signal with it is described
The each of multiple known poses is associated.
6. device according to claim 1, wherein, one or more of biopotential sensors are selected from by the following
The group of composition：Surface electromyography (sEMG) sensor, electric capacity electromyogram (cEMG) sensor, and skin conductance sensors.
7. device according to claim 1, wherein, the processor is configured as by that will have surface nerve conduction
(SNC) wavelet transform (DWT) of morther wavelet is applied to the one or more bioelectrical signals detected, from one detected
Individual or multiple bioelectrical signals filter out electromyogram (EMG) noise signal.
8. device according to claim 1, wherein, the known poses identified include at least two fingers being pressed in one
Rise, and wherein, the processor is configured as by assessing at least one surface neuro-transmission signal bag detected
The amplitude and frequency of the proportional pressure between at least two finger are included and be applied to, identifies at least two fingers pressure
Together.
9. device according to claim 8, wherein, the processor be configured as estimation at least two finger it
Between the pressure that applies.
10. a kind of method for the communication being used between ability of posture control interface arrangement and computerized equipment, methods described include：
One or more bioelectrical signals from the one or more biopotential sensors being placed on the body of user are detected,
Wherein, one or more of biopotential sensors include being used to detect at least the one of at least one surface neuro-transmission signal
Individual surface nerve conduction SNC sensors；
Using processor, by least one surface neuro-transmission signal detected with corresponding to the more of multiple known poses
The data of individual reference signal are compared, and each reference signal is obvious associated with one of the known poses；
The known poses corresponding with least one surface neuro-transmission signal are identified from the multiple known poses；And
The known poses identified are communicated to computerized equipment.
11. according to the method for claim 10, wherein, at least one surface neuro-transmission signal includes coming from wrist
In nerve tract electric signal.
12. according to the method for claim 10, wherein, identify the known poses including the use of from least one motion
Movement detected by sensor.
13. according to the method for claim 10, further comprise that working as identified known poses is registered in the computer
When changing in equipment, touch feedback is activated on the body of the user.
14. according to the method for claim 10, further comprise coming by using one or more of bioelectrical signals
Training is directed to the data of the body of the user, by least one surface neuro-transmission signal and the multiple known appearance
The each of gesture is associated.
15. according to the method for claim 10, further comprise：By that will have surface nerve conduction (SNC) morther wavelet
Wavelet transform (DWT) be applied to the one or more bioelectrical signals detected, it is raw from the one or more detected
Thing electric signal filters out electromyogram (EMG) noise signal.
16. according to the method for claim 10, wherein, identify that the known poses include：It is at least one to what is detected
Surface nerve conduction (SNC) signal carries out denoising, detects the event at least one surface neuro-transmission signal, using point
Section extracts the statistical nature in one or more of frames, and be based on to determine one or more frames of detected event
Sorting algorithm is applied to extracted statistical nature by the data, to determine the known poses.
17. according to the method for claim 10, wherein, identify that the known poses include：One or more is detected
Bioelectrical signals be applied to gesture recognition pipeline, the gesture recognition pipeline includes convolutional neural networks (CNN) and shot and long term
Remember (LSTM) neutral net.
18. according to the method for claim 10, wherein, the known poses include forcing together at least two fingers,
And wherein, identify at least two finger force together including：Assess at least one surface nerve conduction letter detected
Number include and the amplitude and frequency of the proportional pressure being applied between at least two finger.
19. according to the method for claim 18, further comprise：Pass through the bioelectrical signals for detecting one or more
It is applied to the proportional control pipeline including convolutional neural networks (CNN) and shot and long term memory (LSTM) neutral net, estimation
The pressure being applied between at least two finger.
20. according to the method for claim 19, further comprise：The shot and long term memory is trained by using auxiliary signal
Neutral net.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311690905.1A CN117687477A (en) | 2016-07-06 | 2017-07-05 | Method and apparatus for a gesture control interface of a wearable device |
| CN202311036961.3A CN117032398A (en) | 2016-07-06 | 2017-07-05 | Method and apparatus for a gesture control interface of a wearable device |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US15/202,750 US9720515B2 (en) | 2015-01-02 | 2016-07-06 | Method and apparatus for a gesture controlled interface for wearable devices |
| US15/202,750 | 2016-07-06 |
Related Child Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311690905.1A Division CN117687477A (en) | 2016-07-06 | 2017-07-05 | Method and apparatus for a gesture control interface of a wearable device |
| CN202311036961.3A Division CN117032398A (en) | 2016-07-06 | 2017-07-05 | Method and apparatus for a gesture control interface of a wearable device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107589782A true CN107589782A (en) | 2018-01-16 |
| CN107589782B CN107589782B (en) | 2024-05-14 |
Family
ID=61041788
Family Applications (3)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710543994.5A Active CN107589782B (en) | 2016-07-06 | 2017-07-05 | Method and apparatus for a gesture control interface of a wearable device |
| CN202311690905.1A Pending CN117687477A (en) | 2016-07-06 | 2017-07-05 | Method and apparatus for a gesture control interface of a wearable device |
| CN202311036961.3A Pending CN117032398A (en) | 2016-07-06 | 2017-07-05 | Method and apparatus for a gesture control interface of a wearable device |
Family Applications After (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311690905.1A Pending CN117687477A (en) | 2016-07-06 | 2017-07-05 | Method and apparatus for a gesture control interface of a wearable device |
| CN202311036961.3A Pending CN117032398A (en) | 2016-07-06 | 2017-07-05 | Method and apparatus for a gesture control interface of a wearable device |
Country Status (1)
| Country | Link |
|---|---|
| CN (3) | CN107589782B (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108345873A (en) * | 2018-03-22 | 2018-07-31 | 哈尔滨工业大学 | A kind of multiple degrees of freedom body motion information analytic method based on multilayer convolutional neural networks |
| CN108453736A (en) * | 2018-03-22 | 2018-08-28 | 哈尔滨工业大学 | A kind of multiple degrees of freedom synchronization myoelectricity control method based on deep learning |
| CN109782999A (en) * | 2019-01-30 | 2019-05-21 | 上海摩软通讯技术有限公司 | A kind of input method, input equipment and a kind of computer-readable medium |
| CN110209270A (en) * | 2019-05-14 | 2019-09-06 | 李子琪 | A kind of data glove, data glove system, bearing calibration and storage medium |
| CN110896425A (en) * | 2018-09-12 | 2020-03-20 | 意法半导体股份有限公司 | System and method for recognizing a gesture for bringing a mobile electronic device to a user's ear |
| CN111062021A (en) * | 2019-12-19 | 2020-04-24 | 清华大学 | Method and device for identity authentication based on wearable equipment |
| JP2020078077A (en) * | 2018-11-07 | 2020-05-21 | チソット・エス アー | Method for broadcasting acoustic signal |
| CN112566553A (en) * | 2018-08-13 | 2021-03-26 | 脸谱科技有限责任公司 | Real-time spike detection and identification |
| CN113423341A (en) * | 2018-11-27 | 2021-09-21 | 脸谱科技有限责任公司 | Method and apparatus for automatic calibration of wearable electrode sensor system |
| CN113467647A (en) * | 2020-03-31 | 2021-10-01 | 苹果公司 | Skin-to-skin contact detection |
| CN113811920A (en) * | 2019-05-16 | 2021-12-17 | 高通股份有限公司 | Distributed pose estimation |
| CN114442798A (en) * | 2020-11-06 | 2022-05-06 | 复旦大学附属妇产科医院 | Portable control system and control method |
| CN114494424A (en) * | 2022-04-14 | 2022-05-13 | 常州市新创智能科技有限公司 | Welding guiding method and device based on vision |
| CN115381469A (en) * | 2022-08-12 | 2022-11-25 | 歌尔股份有限公司 | Electromyographic signal acquisition device, control method and electronic equipment |
| US11941175B2 (en) | 2020-03-31 | 2024-03-26 | Apple Inc. | Skin-to-skin contact detection |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101689245A (en) * | 2007-06-27 | 2010-03-31 | 微软公司 | recognizing input gestures |
| US20100106044A1 (en) * | 2008-10-27 | 2010-04-29 | Michael Linderman | EMG measured during controlled hand movement for biometric analysis, medical diagnosis and related analysis |
| CN102132227A (en) * | 2008-03-26 | 2011-07-20 | 艾登特技术股份公司 | System and method for the multidimensional evaluation of gestures |
| CN102349037A (en) * | 2009-03-13 | 2012-02-08 | 微软公司 | Wearable electromyography-based controllers for human-computer interface |
| CN102782612A (en) * | 2010-02-24 | 2012-11-14 | 诺基亚公司 | Gesture control |
| US20140240223A1 (en) * | 2013-02-22 | 2014-08-28 | Thalmic Labs Inc. | Method and apparatus for analyzing capacitive emg and imu sensor signals for gesture control |
-
2017
- 2017-07-05 CN CN201710543994.5A patent/CN107589782B/en active Active
- 2017-07-05 CN CN202311690905.1A patent/CN117687477A/en active Pending
- 2017-07-05 CN CN202311036961.3A patent/CN117032398A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101689245A (en) * | 2007-06-27 | 2010-03-31 | 微软公司 | recognizing input gestures |
| CN102132227A (en) * | 2008-03-26 | 2011-07-20 | 艾登特技术股份公司 | System and method for the multidimensional evaluation of gestures |
| US20100106044A1 (en) * | 2008-10-27 | 2010-04-29 | Michael Linderman | EMG measured during controlled hand movement for biometric analysis, medical diagnosis and related analysis |
| CN102349037A (en) * | 2009-03-13 | 2012-02-08 | 微软公司 | Wearable electromyography-based controllers for human-computer interface |
| CN102782612A (en) * | 2010-02-24 | 2012-11-14 | 诺基亚公司 | Gesture control |
| US20140240223A1 (en) * | 2013-02-22 | 2014-08-28 | Thalmic Labs Inc. | Method and apparatus for analyzing capacitive emg and imu sensor signals for gesture control |
| US20140240103A1 (en) * | 2013-02-22 | 2014-08-28 | Thalmic Labs Inc. | Methods and devices for combining muscle activity sensor signals and inertial sensor signals for gesture-based control |
| CN105190578A (en) * | 2013-02-22 | 2015-12-23 | 赛尔米克实验室公司 | Methods and devices that combine muscle activity sensor signals and inertial sensor signals for gesture-based control |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108345873A (en) * | 2018-03-22 | 2018-07-31 | 哈尔滨工业大学 | A kind of multiple degrees of freedom body motion information analytic method based on multilayer convolutional neural networks |
| CN108453736A (en) * | 2018-03-22 | 2018-08-28 | 哈尔滨工业大学 | A kind of multiple degrees of freedom synchronization myoelectricity control method based on deep learning |
| CN112566553A (en) * | 2018-08-13 | 2021-03-26 | 脸谱科技有限责任公司 | Real-time spike detection and identification |
| CN110896425A (en) * | 2018-09-12 | 2020-03-20 | 意法半导体股份有限公司 | System and method for recognizing a gesture for bringing a mobile electronic device to a user's ear |
| US10942578B2 (en) | 2018-09-12 | 2021-03-09 | Stmicroelectronics S.R.L. | System and method for recognition of the gesture of bringing a mobile electronic device to a user ear |
| JP7289255B2 (en) | 2018-11-07 | 2023-06-09 | チソット・エス アー | How to broadcast an acoustic signal |
| JP2020078077A (en) * | 2018-11-07 | 2020-05-21 | チソット・エス アー | Method for broadcasting acoustic signal |
| CN113423341A (en) * | 2018-11-27 | 2021-09-21 | 脸谱科技有限责任公司 | Method and apparatus for automatic calibration of wearable electrode sensor system |
| CN109782999A (en) * | 2019-01-30 | 2019-05-21 | 上海摩软通讯技术有限公司 | A kind of input method, input equipment and a kind of computer-readable medium |
| CN110209270A (en) * | 2019-05-14 | 2019-09-06 | 李子琪 | A kind of data glove, data glove system, bearing calibration and storage medium |
| CN113811920A (en) * | 2019-05-16 | 2021-12-17 | 高通股份有限公司 | Distributed pose estimation |
| CN111062021A (en) * | 2019-12-19 | 2020-04-24 | 清华大学 | Method and device for identity authentication based on wearable equipment |
| CN113467647A (en) * | 2020-03-31 | 2021-10-01 | 苹果公司 | Skin-to-skin contact detection |
| US11941175B2 (en) | 2020-03-31 | 2024-03-26 | Apple Inc. | Skin-to-skin contact detection |
| CN114442798A (en) * | 2020-11-06 | 2022-05-06 | 复旦大学附属妇产科医院 | Portable control system and control method |
| CN114442798B (en) * | 2020-11-06 | 2024-05-07 | 复旦大学附属妇产科医院 | Portable control system and control method |
| CN114494424A (en) * | 2022-04-14 | 2022-05-13 | 常州市新创智能科技有限公司 | Welding guiding method and device based on vision |
| CN115381469A (en) * | 2022-08-12 | 2022-11-25 | 歌尔股份有限公司 | Electromyographic signal acquisition device, control method and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117687477A (en) | 2024-03-12 |
| CN117032398A (en) | 2023-11-10 |
| CN107589782B (en) | 2024-05-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107589782A (en) | Method and apparatus for the ability of posture control interface of wearable device | |
| US9720515B2 (en) | Method and apparatus for a gesture controlled interface for wearable devices | |
| US9612661B2 (en) | Closed loop feedback interface for wearable devices | |
| US10970936B2 (en) | Use of neuromuscular signals to provide enhanced interactions with physical objects in an augmented reality environment | |
| US20200097081A1 (en) | Neuromuscular control of an augmented reality system | |
| CN109804331B (en) | Detecting and using body tissue electrical signals | |
| Caramiaux et al. | Understanding gesture expressivity through muscle sensing | |
| Esposito et al. | A piezoresistive array armband with reduced number of sensors for hand gesture recognition | |
| US20170259167A1 (en) | Brainwave virtual reality apparatus and method | |
| US20220269346A1 (en) | Methods and apparatuses for low latency body state prediction based on neuromuscular data | |
| Wheeler et al. | Gesture-based control and EMG decomposition | |
| US11179066B2 (en) | Real-time spike detection and identification | |
| CN112805765B (en) | Information processing apparatus | |
| Aswath et al. | Human gesture recognition for real-time control of humanoid robot | |
| Chapman et al. | A wearable, open-source, lightweight forcemyography armband: On intuitive, robust muscle-machine interfaces | |
| Kumar et al. | Human-computer interface technologies for the motor impaired | |
| Karrenbach et al. | Deep learning and session-specific rapid recalibration for dynamic hand gesture recognition from EMG | |
| Naik et al. | Real-time hand gesture identification for human computer interaction based on ICA of surface electromyogram | |
| EP3860527B1 (en) | Use of neuromuscular signals to provide enhanced interactions with physical objects in an augmented reality environment | |
| Akumalla | Evaluating appropriateness of EMG and flex sensors for classifying hand gestures | |
| US20240004469A1 (en) | Intelligent human-machine interface and method which can be carried out using same | |
| CN107765837A (en) | Wearable electronic | |
| Saponas | Supporting Everyday Activities through Always-Available Mobile Computing | |
| Sivakumar et al. | Data Glove-Based Hand Gesture Recognition for Improved Human-Computer Interaction | |
| Schuff | Leveraging electromyography to enhance musician-instrument interaction using domain-specific motions |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |