WO2021219039A1 - 手势识别方法、人机交互方法、装置、设备及存储介质 - Google Patents

手势识别方法、人机交互方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2021219039A1
WO2021219039A1 PCT/CN2021/090715 CN2021090715W WO2021219039A1 WO 2021219039 A1 WO2021219039 A1 WO 2021219039A1 CN 2021090715 W CN2021090715 W CN 2021090715W WO 2021219039 A1 WO2021219039 A1 WO 2021219039A1
Authority
WO
WIPO (PCT)
Prior art keywords
signal
surface emg
emg signal
gesture
gesture recognition
Prior art date
Application number
PCT/CN2021/090715
Other languages
English (en)
French (fr)
Inventor
田彦秀
韩久琦
姚秀军
桂晨光
Original Assignee
京东数科海益信息科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 京东数科海益信息科技有限公司 filed Critical 京东数科海益信息科技有限公司
Publication of WO2021219039A1 publication Critical patent/WO2021219039A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/02—Preprocessing
    • G06F2218/04—Denoising
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/08—Feature extraction

Definitions

  • the present disclosure generally relates to the field of artificial intelligence technology, and more specifically to gesture recognition methods, human-computer interaction methods, devices, equipment, and storage media.
  • Gesture action recognition is essentially a pattern recognition problem. It needs to learn effective features from input information and use the extracted features to recognize gesture tags.
  • Gesture action recognition technology based on surface EMG signals can be divided into two categories according to whether human factors are added: one is to extract the time domain, frequency domain and time-frequency after preprocessing the collected surface EMG signals of different gesture actions Then, after performing feature reduction or feature selection on the extracted features, they are input into the classifier for model training, and the trained classifier is used for real-time prediction of gesture actions; the other type is directly preprocessing The latter surface EMG signal is used as the input, and no human factors such as feature extraction are added.
  • the deep learning framework is used to actively learn the differences of surface EMG signals of different gestures, and the trained network model structure is used in actual testing. .
  • the first type of EMG gesture recognition methods usually use signal analysis technology to manually extract a variety of signal features from surface EMG signals, and then input the extracted signal features into linear discriminant analysis, support vector machines, hidden Markov models, etc. Gesture recognition is performed in the classifier.
  • the deep neural network model is the convolutional neural network, which is widely used.
  • the second type of EMG gesture recognition technology does not rely on manual feature extraction and the complicated and tedious feature selection process, it needs to automatically learn representative depth features from a large number of input samples, and the amount of calculation is very large.
  • these two gesture recognition technologies require users to collect and record a segment of surface EMG signal data of different gestures with markers in advance. After the model training is performed with this marker data, the model can be predicted and used in practice. The implementation process is not convenient, and The user experience is poor.
  • the present disclosure provides a gesture recognition method, a human-computer interaction method, a device, a device, and a storage medium to simplify the gesture recognition process and improve the efficiency of gesture recognition.
  • the present disclosure relates to a gesture recognition method, which includes:
  • the surface EMG signal is generated by the movement of forearm muscles, and different gestures correspond to different surface EMG signals.
  • the method before obtaining the effective surface EMG signal from the surface EMG signal, the method further includes:
  • extracting the signal characteristics of the effective surface EMG signal includes:
  • the surface EMG signal is intercepted by using M data windows to obtain M segments of the surface EMG signal.
  • the interval between two adjacent segments of the surface EMG signal is a preset step, and one of the data windows includes n Data points, the preset step includes m data points;
  • the sum result of the obtained M segments of the surface EMG signal is summed, and the obtained result is taken as the signal characteristic of the effective surface EMG signal.
  • obtaining the gesture corresponding to the surface EMG signal according to the signal feature includes:
  • the gesture corresponding to the surface EMG signal is obtained.
  • obtaining an effective surface EMG signal from the surface EMG signal includes:
  • the surface EMG signal corresponding to the envelope signal whose amplitude is greater than the preset threshold is used as the effective surface EMG signal.
  • the present disclosure relates to a human-computer interaction method, which includes the gesture recognition method described in the present disclosure.
  • the present disclosure relates to a gesture recognition device, which includes:
  • the first acquisition module is used to acquire the surface EMG signal
  • the second acquisition module is configured to acquire an effective surface EMG signal from the surface EMG signal, wherein the effective surface EMG signal is the surface EMG signal in the muscle activity time period;
  • An extraction module for extracting the signal characteristics of the effective surface EMG signal
  • the processing module is configured to obtain a gesture corresponding to the surface EMG signal according to the signal characteristic.
  • the present disclosure relates to an electronic device, which includes a processor and a memory, and the processor is configured to execute a program stored in the memory to implement the gesture recognition method.
  • the electronic device is an armband device, and the armband device is used to be worn on the forearm of the upper extremity near the elbow joint.
  • the present disclosure relates to a computer-readable storage medium that stores a computer program that, when executed by a processor, implements the gesture recognition method of the present disclosure.
  • the gesture recognition method of the present disclosure obtains a surface EMG signal, and obtains an effective surface EMG signal from the surface EMG signal, thereby avoiding the interference of the invalid surface EMG signal. In some embodiments, the gesture recognition method of the present disclosure extracts the signal feature of the effective surface EMG signal, and directly obtains the gesture corresponding to the surface EMG signal according to the signal feature.
  • the gesture recognition method of the present disclosure does not require pre-acquisition of a large number of gesture samples for training, and then obtains the gesture corresponding to the surface EMG signal according to the training result, and can directly extract the signal characteristics of the effective surface EMG signal, according to the signal Features: Quickly and effectively obtain the gestures corresponding to the surface EMG signal, the operation process is simple and efficient.
  • FIG. 1 shows a schematic flowchart of a gesture recognition method provided by an embodiment of the present disclosure
  • FIG. 2 shows a schematic flowchart of an effective surface EMG signal extraction method provided by an embodiment of the present disclosure
  • FIG. 3 shows a schematic flowchart of a method for extracting surface EMG signal features according to an embodiment of the present disclosure
  • FIG. 4 shows a schematic flowchart of a specific implementation process of a gesture recognition method provided by an embodiment of the present disclosure
  • FIG. 5 shows a schematic structural diagram of a gesture recognition device provided by an embodiment of the present disclosure.
  • Fig. 6 shows a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.
  • the embodiments of the present disclosure provide a gesture recognition method, which can be applied to a single electronic device that collects a surface EMG signal and recognizes the gesture corresponding to the surface EMG signal.
  • the method can also be applied to smart terminal equipment.
  • the surface EMG signal is collected by the collection device, and then the collection device transmits the collected surface EMG signal to the smart terminal device, and the smart terminal device recognizes and obtains the surface EMG signal.
  • the corresponding gesture can also be applied to a server.
  • the surface EMG signal is collected by a collection device, and then the collection device transmits the collected surface EMG signal to the server, and the server recognizes and obtains the gesture corresponding to the surface EMG signal.
  • the method is shown in Figure 1 and includes:
  • the surface EMG signal is a bioelectric signal collected by electrodes placed on the surface of the skin. It is a non-stationary, non-linear, weak electrical signal, which has randomness and is highly susceptible to external interference. , The signal-to-noise ratio is low.
  • the surface EMG signal can reflect the extension and flexion of the human joints and the shape and position of the limbs.
  • the electrodes used to collect surface EMG signals are worn on the forearms of the upper limbs of the human body, so as to be able to monitor the movement of the forearm muscles driven by gestures.
  • the surface EMG signal monitored by the electrode worn on the forearm is generated by the movement of the forearm muscles.
  • Different gestures drive different forearm muscles. In other words, each gesture is generated by the entire forearm muscles. A part of the forearm muscle group plays a leading role, and different forearm muscle group movements will produce different surface EMG signals.
  • the gesture of OK is produced by the movement of forearm muscle A
  • the gesture of making a fist is produced by the movement of forearm muscle B
  • the gesture of scissors hand is produced by the movement of forearm muscle C
  • the gesture of metal ceremony is produced by forearm muscle D and forearm muscle E.
  • the surface EMG signal A is monitored
  • the measured object makes the fist gesture the surface EMG signal B
  • the measured object makes the scissors hand gesture the monitoring To the surface EMG signal C
  • the measured object makes a metal gesture the surface EMG signal DE is monitored.
  • the surface EMG signal after the surface EMG signal is obtained, before the effective surface EMG signal is obtained from the surface EMG signal, the surface EMG signal needs to be preprocessed to filter out the surface EMG signal. Interference, improve the accuracy of follow-up gestures.
  • noise of a predetermined frequency in the surface EMG signal is filtered out during the preprocessing process.
  • a notch filter is used to remove the noise with a frequency of 50 Hz in the surface EMG signal. Then, input the surface EMG signal obtained by the notch filter into the 6th-order Butterworth filter to obtain the surface EMG signal with a frequency of 20-200 Hz filtered by the filter.
  • the surface EMG signal with a frequency of 20-200 Hz is a useful surface EMG signal in this gesture recognition method. By preprocessing the surface EMG signal, the noise content of the signal is reduced, and the surface EMG signal that is more in line with the gesture recognition method is obtained.
  • the EMG signal is the temporal and spatial superposition of the action potentials of the motor units in many muscle fibers
  • the surface EMG signal is the combined effect of the electrical activity of the superficial muscles and nerve trunks on the surface of the skin.
  • There are two types of surface EMG signals one is the surface EMG signal corresponding to the resting potential, and the other is the surface EMG signal corresponding to the action potential.
  • the effective surface EMG signal is the surface EMG signal during the muscle activity period, that is, the effective surface EMG signal is the surface EMG signal corresponding to the action potential.
  • the surface EMG signal corresponding to the resting potential is useless for this method of gesture recognition.
  • the resting potential means that the muscle is in a relaxed state and no action is performed. And, for this method, it belongs to the useless signal of noise. Therefore, it is necessary to obtain the active segment of the surface EMG signal, that is, the starting position and the ending position of the action potential of the surface EMG signal need to be detected to obtain the effective surface EMG signal.
  • detecting the starting position and ending position of the action potential of the surface EMG signal to obtain the effective surface EMG signal includes:
  • the correction processing on the surface EMG signal is the correction processing on the preprocessed surface EMG signal.
  • a conversion operation on the preprocessed surface EMG signal to obtain a converted signal.
  • effective correction can be carried out for different measured objects, and the influence of impedance and muscle tension on the baseline threshold is reduced.
  • the converted signal is corrected based on the baseline threshold to obtain a corrected signal.
  • thr is the baseline threshold
  • MAV i is the maximum value of the sliding window in the resting state data of the surface EMG signal
  • i is the subscript of the maximum value of the sliding window in the resting state data of the surface EMG signal
  • the value is taken
  • the range is a positive integer between 1 and k
  • m is the number of sliding windows
  • A is a constant.
  • the surface EMG signal is corrected according to the baseline threshold, which effectively reduces the influence of individual differences on the surface EMG signal.
  • the obtained correction signals are imported into the kernel function one by one, and the kernel function is updated after each correction signal is imported.
  • the operations include:
  • the envelope signal ⁇ y 1 ,y 2 ,y 3 ,...y i ,... ⁇ is calculated from the correction signal ⁇ s 1 ,s 2 ,s 3 ,...s i ,... ⁇ . ⁇ .
  • the unit equidistant integral of the kernel function is calculated based on the trapezoidal method, and the envelope signal corresponding to the correction signal is obtained.
  • the envelope signal increases the action potential segment and resting in the surface EMG signal The slight difference between the potential segments reduces the fluctuation of the surface EMG signal caused by muscle tension and reduces the misjudgment of the active segment caused by muscle tension.
  • the envelope is a curve of the amplitude of a random process over time.
  • the envelope signal is a high-frequency amplitude-modulated signal, and its amplitude changes in accordance with the low-frequency modulation signal. If the peak points of the high-frequency amplitude modulation signal are connected, a curve corresponding to the low-frequency modulation signal can be obtained.
  • the envelope signal is also a new pulse signal (with a larger period), this pulse signal will also have a certain width when observed in time (there will be a period of time in each period is 0), which is the width in time is the pulse Envelope width.
  • the bandwidth of the pulse is inversely proportional to the pulse width, that is, the narrower the width of the pulse time, the larger the bandwidth on the spectrum.
  • the active segment includes a starting position and an ending position. If the amplitude of the previous one or more envelope signals is not greater than the preset threshold, when the amplitude of the envelope signal is greater than the preset threshold, it is determined that the amplitude of the envelope signal starts to be greater than the initial position of the preset threshold. Position, as the starting position of the active segment; if the amplitude of the previous one or more envelope signals is greater than the preset threshold, when the amplitude of the envelope signal is not greater than the preset threshold, it is determined that the amplitude of the envelope signal starts to be less than the preset threshold The initial position of the corresponding surface EMG signal position, as the end position of the active segment. The surface EMG signal from the start position to the end position is the effective surface EMG signal.
  • the preset threshold is set to zero. If the amplitude of the previous one or more envelope signals is not greater than zero, when the amplitude of the envelope signal is greater than zero, the position of the surface EMG signal corresponding to the initial position where the envelope signal amplitude starts to be greater than zero is determined as the active segment Start position; if the amplitude of the previous one or more envelope signals is greater than zero, when the amplitude of the envelope signal is not greater than zero, determine the position of the surface EMG signal corresponding to the initial position where the envelope signal amplitude starts to be less than zero, As the end position of the active segment.
  • the surface EMG signal from the start position to the end position is the effective surface EMG signal.
  • M data windows are used to intercept the surface EMG signal to obtain M segments of surface EMG signal.
  • the interval between two adjacent segments of surface EMG signal is a preset step.
  • the electrical signal performs a feature extraction operation, wherein one data window includes n data points, and the preset step includes m data points.
  • Figure 3 taking the feature extraction of the p-th segment of surface EMG signal as an example, the description is as follows:
  • the absolute value of all the calculated differences is summed, and the sum result is:
  • the sum result obtained by 302 is used as the feature extraction result of the p-th segment surface EMG signal.
  • the obtained feature extraction results of the M-segment surface EMG signal (ie M summation results) are summed, and the obtained result is regarded as the entire effective surface muscle Signal characteristics of electrical signals.
  • the respective feature extraction results of the M-segment surface EMG signals are summed, and the obtained result can be expressed as:
  • the signal characteristics of the surface EMG signal are extracted, which reflects the complexity of the EMG signal waveform.
  • the final result is the amplitude, frequency and duration of the EMG signal. The effect of synergy.
  • the forearm muscle group corresponding to each gesture is different, and the forearm muscle group corresponding to the signal characteristic is determined according to the signal characteristic. Further, according to the determined forearm muscle group and the pre-configured correspondence between the forearm muscle group and the gesture, the gesture corresponding to the surface EMG signal is obtained.
  • the forearm muscle group corresponding to the signal feature is determined to be forearm muscle B; according to the pre-configured forearm muscle group and gesture corresponding relationship, the forearm muscle B corresponds to the gesture of making a fist, and the gesture corresponding to the surface EMG signal is obtained For making a fist.
  • the corresponding relationship is saved as a matching template, and the matching template is used as a standard template for different gesture recognition.
  • the surface EMG signal can be input into the matching template, and the gesture corresponding to the surface EMG signal can be directly obtained.
  • the matching template can perform gesture recognition regardless of whether it is in a network environment or a non-network environment, and is not restricted by the network.
  • the matching template is stored in any smart device.
  • the obtained surface EMG signal can be input to the smart device in the form of a data cable or Bluetooth.
  • the smart device performs matching in the matching template according to the input surface EMG signal, and then the gesture corresponding to the surface EMG signal can be obtained. Further, the problem that gesture recognition cannot be performed in a non-network environment or a weak network environment is solved.
  • the gesture recognition process will be described in detail with reference to Figure 4.
  • the process includes:
  • the method provided by the embodiment of the present disclosure obtains surface EMG signals, and obtains effective surface EMG signals from the surface EMG signals, thereby avoiding the interference of invalid surface EMG signals. Further, the signal feature of the effective surface EMG signal is extracted, and the gesture corresponding to the surface EMG signal is directly obtained according to the signal feature.
  • This method does not need to collect a large number of gesture samples for training in advance, and then obtain the gesture corresponding to the surface EMG signal according to the training result, can directly extract the signal characteristics of the effective surface EMG signal, and quickly and effectively obtain the surface EMG signal corresponding to the signal characteristics. Gestures, the operation process is simple and efficient.
  • the embodiment of the present disclosure also provides a gesture recognition device.
  • the device mainly includes:
  • the first acquisition module 501 is used to acquire the surface EMG signal
  • the second acquisition module 502 is configured to acquire the effective surface EMG signal from the surface EMG signal, where the effective surface EMG signal is the surface EMG signal in the time period of muscle activity;
  • the extraction module 503 is used to extract the signal characteristics of the effective surface EMG signal.
  • the processing module 504 is configured to obtain a gesture corresponding to the surface EMG signal according to the signal characteristics.
  • the extraction module 503 is configured to use M data windows to intercept surface EMG signals to obtain M segments of surface EMG signals, and the interval between two adjacent segments of surface EMG signals is a preset step, where , One of the data windows includes n data points, and the preset step includes m data points;
  • the sum results of the obtained M-segment surface EMG signals are summed, and the obtained results are regarded as the signal characteristics of the effective surface EMG signals.
  • the processing module 504 is used to determine the forearm muscle group corresponding to the signal feature; according to the determined forearm muscle group, and the pre-configured correspondence between the forearm muscle group and the gesture, obtain all the forearm muscle groups corresponding to the surface EMG signal. The gestures.
  • the second acquisition module 502 is used to perform correction processing on the surface EMG signal to obtain a correction signal; perform an integral operation on the obtained correction signal to obtain an envelope signal;
  • the surface EMG signal corresponding to the network signal is used as the effective surface EMG signal.
  • the envelope signal increases the action potential segment and static state in the initial surface EMG signal. The slight difference between the interest potential segments reduces the fluctuation of the surface EMG signal caused by muscle tension and reduces the misjudgment of the active segment caused by muscle tension.
  • the device provided by the embodiment of the present disclosure obtains the surface EMG signal through the first obtaining module 501; and uses the second obtaining module 502 to obtain the effective surface EMG signal from the surface EMG signal, thereby avoiding the invalid surface EMG signal Interference. Then, the extraction module 503 is used to extract the signal characteristics of the effective surface EMG signal. Finally, the processing module 504 is used to directly obtain the gesture corresponding to the surface EMG signal according to the signal feature. The device does not need to collect a large number of gesture samples for training in advance, and then obtain the gesture corresponding to the surface EMG signal according to the training result, can directly extract the signal characteristics of the effective surface EMG signal, and quickly and effectively obtain the surface EMG signal corresponding to the signal characteristics. Gestures, the operation process is simple and efficient.
  • the electronic device mainly includes a processor 601 and a memory 602.
  • the electronic device further includes: a communication component 603 and a communication bus 604, wherein the processor 601, the communication component 603, and the memory 602 communicate with each other through the communication bus 604.
  • the memory 602 stores a program that can be executed by the processor 601, and the processor 601 executes the program stored in the memory 502 to achieve: obtain the surface EMG signal; obtain the effective surface EMG signal from the surface EMG signal, where , The effective surface EMG signal is the surface EMG signal in the time period of muscle activity; extracting the signal characteristics of the effective surface EMG signal; and obtaining the gesture corresponding to the surface EMG signal according to the signal characteristics.
  • the communication bus 604 mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus or the like.
  • PCI Peripheral Component Interconnect
  • EISA Extended Industry Standard Architecture
  • the communication bus 604 can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is used in FIG. 6, but it does not mean that there is only one bus or one type of bus.
  • the communication component 603 is used for communication between the above-mentioned electronic device and other devices.
  • the memory 602 may include a random access memory (Random Access Memory, RAM for short), and may also include a non-volatile memory (non-volatile memory), for example, at least one disk memory. In some embodiments, the memory may also be at least one storage device located far away from the aforementioned processor 601.
  • RAM Random Access Memory
  • non-volatile memory non-volatile memory
  • the memory may also be at least one storage device located far away from the aforementioned processor 601.
  • the above-mentioned processor 601 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc., or a digital signal processor (Digital Signal Processing, DSP for short). ), Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
  • CPU Central Processing Unit
  • NP Network Processor
  • DSP Digital Signal Processing
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • the electronic device may be an armband device, which is worn on the forearm of the upper limb near one third of the elbow joint.
  • a switch can be set on the armband device. Before the measured object makes a gesture, the switch is turned on, and after the measured object makes a gesture, the gesture is recognized.
  • the embodiment of the present disclosure also provides a human-computer interaction method, which includes the implementation method described in the above embodiment.
  • a human-computer interaction method which includes the implementation method described in the above embodiment.
  • the armband device can be used in a bionic prosthesis to serve some special groups of people, allowing this group of people to control the movement of the bionic prosthesis through their own will and muscles.
  • a bionic prosthesis Take the installation of a bionic prosthesis on the left leg of a tested object as an example:
  • the relationship between different gestures and leg movements is pre-stored in the armband device.
  • the gesture of making a fist corresponds to a step forward with the left leg.
  • the left leg needs to also step forward.
  • the measured object only needs to make a fist.
  • Surface EMG signal is obtained.
  • the measured object's gesture is obtained as a fist, and finally, according to the correspondence between the fist gesture and the left leg step forward, an instruction for the left leg to step forward is obtained.
  • the armband device sends the instruction to the bionic prosthesis after obtaining the instruction of one step forward.
  • the bionic prosthesis receives the instructions issued by the armband device and completes the walking of the measured object. Among them, the armband device and the bionic prosthesis have established a communication connection.
  • the armband device can also be used in presentations (PPT), and the relationship between different gestures and PPT execution actions can be preset.
  • PPT presentation
  • the gesture of OK corresponds to the action of the next page
  • the gesture of the scissors hand corresponds to the action of auto-play
  • the gesture of making a fist corresponds to the action of the previous page.
  • the specific implementation is also to establish a communication connection with the computer through the armband device, and the computer completes the playback of the PPT according to the specific operating instructions sent by the armband device.
  • the device provided by the embodiment of the present disclosure obtains the surface EMG signal through the processor 601; obtains the effective surface EMG signal from the surface EMG signal; extracts the signal characteristics of the effective surface EMG signal; and directly according to the signal characteristics Obtain the gesture corresponding to the surface EMG signal.
  • the effective surface EMG signal is obtained from the surface EMG signal to avoid the interference of the invalid surface EMG signal.
  • the device does not need to collect a large number of gesture samples for training in advance, and then obtain the gestures corresponding to the surface EMG signal according to the training result, can directly extract the signal characteristics of the effective surface EMG signal, and quickly and effectively obtain the surface EMG signal corresponding to the signal characteristics. Gestures, the operation process is simple and efficient.
  • a computer-readable storage medium stores a computer program.
  • the computer program runs on a computer, the computer executes the description in the above-mentioned embodiment. Gesture recognition method.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • the computer instructions may be transmitted from a website, computer, server, or data center through a cable (such as Coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, microwave, etc.) transmission to another website site, computer, server or data center.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center integrated with one or more available media.
  • the usable medium may be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape, etc.), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state hard disk).

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Human Computer Interaction (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Dermatology (AREA)
  • Neurology (AREA)
  • Neurosurgery (AREA)
  • User Interface Of Digital Computer (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)

Abstract

一种手势识别方法、人机交互方法、装置、设备及存储介质，该方法包括：获取表面肌电信号（101）；从所述表面肌电信号中获取有效表面肌电信号（102），其中，所述有效表面肌电信号为肌肉活动时间段内的表面肌电信号；提取所述有效表面肌电信号的信号特征（103）；以及根据所述信号特征，获得所述表面肌电信号对应的手势（104）。

Description

手势识别方法、人机交互方法、装置、设备及存储介质
相关申请的引用
本公开要求于2020年4月30日向中华人民共和国国家知识产权局提交的申请号为202010367114.5、发明名称为“手势识别方法、人机交互方法、装置、设备及存储介质”的发明专利申请的全部权益，并通过引用的方式将其全部内容并入本文。
领域
本公开大体上涉及人工智能技术领域，更具体地涉及手势识别方法、人机交互方法、装置、设备及存储介质。
背景
手势动作识别本质上是一个模式识别问题，需要从输入信息中学习有效的特征，并利用提取的特征识别手势标签。基于表面肌电信号的手势动作识别技术，根据是否加入人为因素可分为两类：一类是将采集到的不同手势动作的表面肌电信号预处理后，提取时域、频域以及时频域各种特征量，然后对提取的特征量进行特征降维或者特征选择后，输入到分类器中进行模型训练，将训练好的分类器用于手势动作实时预测；另一类是直接将预处理后的表面肌电信号作为输入量，不加入特征提取等人为因素，利用深度学习框架主动抓取学习不同手势动作表面肌电信号的差异性，并将训练好的网络模型结构用于实际测试中。
其中，第一类肌电手势识别方法通常运用信号分析技术从表面肌电信号中手工提取多种信号特征，然后将提取的信号特征输入到线性判别分析、支持向量机、隐马尔科夫模型等分类器中进行手势识别。第二类识别技术中深度神经网络模型为卷积神经网络的应用较为广泛。
然而，第一类肌电手势识别技术选用的信号好坏与否会对手势识别性能造成较大的影响。第二类肌电手势识别技术虽然不依 赖手工提取特征以及复杂繁琐的特征优选过程，但需要从大量输入样本中自动学习具有代表性的深度特征，计算量非常大。且这两种手势动作识别技术，均需要用户提前采集并记录一段带标记的不同手势动作表面肌电信号数据，用此标记数据进行模型训练后才能在实际中预测使用，实现过程不便捷，且用户体验较差。
概述
本公开提供了手势识别方法、人机交互方法、装置、设备及存储介质，用以简化手势识别过程，提高手势识别效率。
一方面，本公开涉及手势识别方法，其包括：
获取表面肌电信号；
从所述表面肌电信号中获取有效表面肌电信号，其中，所述有效表面肌电信号为肌肉活动时间段内的表面肌电信号；
提取所述有效表面肌电信号的信号特征；以及
根据所述信号特征，获得所述表面肌电信号对应的手势。
在某些实施方案中，所述表面肌电信号是由前臂肌群运动产生的，且不同的手势对应的表面肌电信号不同。
在某些实施方案中，从所述表面肌电信号中获取有效表面肌电信号之前，还包括：
滤除所述表面肌电信号中预设频率的噪声。
在某些实施方案中，提取所述有效表面肌电信号的信号特征，包括：
采用M个数据窗对所述表面肌电信号进行截取，获得M段表面肌电信号，相邻两段所述表面肌电信号的间隔为预设步进，其中，一个所述数据窗包括n个数据点，所述预设步进包括m个数据点；
分别对每段所述表面肌电信号进行以下特征提取操作：计算第p段所述表面肌电信号中每对相邻的数据点的差值的绝对值，对计算的所有的所述差值的绝对值进行求和处理，获得求和结果，所述p大于或等于1，且小于或等于M；以及
对获得的M段所述表面肌电信号的所述求和结果进行求和，并将获得结果作为所述有效表面肌电信号的信号特征。
在某些实施方案中，根据所述信号特征，获得所述表面肌电信号对应的手势，包括：
确定所述信号特征对应的前臂肌群；以及
根据确定的所述前臂肌群，和，预先配置的前臂肌群与手势的对应关系，获得所述表面肌电信号对应的所述手势。
在某些实施方案中，从所述表面肌电信号中获取有效表面肌电信号，包括：
对所述表面肌电信号进行校正处理，获得校正信号；
对获得的所述校正信号进行积分运算，获得包络信号；以及
将幅度大于预设阈值的包络信号对应的表面肌电信号，作为所述有效表面肌电信号。
另一方面，本公开涉及人机交互方法，其包括：本公开所述的手势识别方法。
再一方面，本公开涉及手势识别装置，其包括：
第一获取模块，用于获取表面肌电信号；
第二获取模块，用于从所述表面肌电信号中获取有效表面肌电信号，其中，所述有效表面肌电信号为肌肉活动时间段内的表面肌电信号；
提取模块，用于提取所述有效表面肌电信号的信号特征；以及
处理模块，用于根据所述信号特征，获得所述表面肌电信号对应的手势。
又一方面，本公开涉及电子设备，其包括：处理器和存储器，所述处理器，用于执行所述存储器中所存储的程序，以实现所述的手势识别方法。
在某些实施方案中，所述电子设备为臂环设备，所述臂环设备用于佩戴在上肢前臂靠近肘关节三分之一处。
另一方面，本公开涉及计算机可读存储介质，存储有计算机 程序，所述计算机程序被处理器执行时实现本公开的手势识别方法。
在某些实施方案中，本公开的手势识别方法通过获取表面肌电信号，从该表面肌电信号中获取有效表面肌电信号，避免了无效表面肌电信号的干扰。在某些实施方案中，本公开的手势识别方法提取有效表面肌电信号的信号特征，根据该信号特征，直接获取表面肌电信号对应的手势。在某些实施方案中，本公开的手势识别方法无需预先采集大量的手势样本进行训练，再根据训练结果获取表面肌电信号对应的手势，可直接提取有效表面肌电信号的信号特征，根据信号特征快速有效的获取表面肌电信号对应的手势，操作过程简便，效率高。
附图的简要说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理。
为了更清楚地说明本公开实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，对于本领域普通技术人员而言，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1示出了本公开一实施例提供的手势识别方法的流程示意图；
图2示出了本公开一实施例提供的有效表面肌电信号提取方法的流程示意图；
图3示出了本公开一实施例提供的表面肌电信号特征提取方法的流程示意图；
图4示出了本公开一实施例提供的手势识别方法的具体实施过程的流程示意图；
图5示出了本公开一实施例提供的手势识别装置的结构示意图；并且
图6示出了本公开一实施例提供的电子设备的结构示意图。
详述
为使本公开实施例的目的、技术方案和优点更加清楚，下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本公开的一部分实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都属于本公开保护的范围。
本公开实施例提供了手势识别方法，该方法可以应用于单独的电子设备，该电子设备采集表面肌电信号，并识别出该表面肌电信号所对应的手势。另外，该方法还可以应用于智能终端设备，通过采集设备采集表面肌电信号，然后该采集设备将采集的表面肌电信号传输给智能终端设备，由智能终端设备识别获得该表面肌电信号所对应的手势。当然，该方法也可以应用于服务器，通过采集设备采集表面肌电信号，然后该采集设备将采集的表面肌电信号传输给服务器，由服务器识别获得该表面肌电信号所对应的手势。
在某些实施方案中，该方法如图1所示，包括：
101，获取表面肌电信号；
102，从表面肌电信号中获取有效表面肌电信号，其中，有效表面肌电信号为肌肉活动时间段内的表面肌电信号；
103，提取有效表面肌电信号的信号特征；以及
104，根据信号特征，获得表面肌电信号对应的手势。
在某些实施方案中，表面肌电信号是放置于皮肤表面的电极采集到的一种生物电信号，是一种非平稳、非线性的微弱电信号，其具有随机性，极易受到外界干扰，信噪比较低。另外，表面肌电信号可以反映人体关节的伸屈状况以及肢体的形状和位置。
在某些实施方案中，用于采集表面肌电信号的电极佩戴在人体上肢的前臂，以能够监测手势动作所带动的前臂肌肉的运动情况。
佩戴在前臂上的电极所监测到的表面肌电信号，是由前臂肌群运动产生，不同的手势带动不同的前臂肌群运动，也就是说，每个手势的产生是由整个前臂肌群中的一部分前臂肌群起到主导作用，而不同的前臂肌群运动会产生不同的表面肌电信号。
例如，假设OK的手势是由前臂肌肉A运动产生，握拳的手势是由前臂肌肉B运动产生，剪刀手的手势是由前臂肌肉C运动产生，金属礼的手势是由前臂肌肉D和前臂肌肉E共同运动产生的。所以，当被测对象做OK的手势时，监测到表面肌电信号A；当被测对象做握拳的手势时，监测到表面肌电信号B；当被测对象做剪刀手的手势时，监测到表面肌电信号C；并且当被测对象做金属礼的手势时，监测到表面肌电信号DE。
在某些实施方案中，在获取表面肌电信号之后，从表面肌电信号中获取有效表面肌电信号之前，需要对该表面肌电信号进行预处理操作，以滤除该表面肌电信号中的干扰，提高后续手势手别的准确性。
在某些实施方案中，预处理过程中滤除表面肌电信号中预设频率的噪声。
例如，由于表面肌电信号容易受50Hz工频的干扰，因此，利用陷波器去除表面肌电信号中频率为50赫兹(Hz)的噪声。然后，将陷波器过滤后得到的表面肌电信号，输入到6阶巴特沃斯滤波器，获得该滤波器滤除后的频率20～200Hz的表面肌电信号。其中，频率为20～200Hz的表面肌电信号在该手势识别方法中为有用的表面肌电信号。通过对表面肌电信号进行预处理，降低了信号的噪声含量，且获得了更符合该手势识别方法的表面肌电信号。
在某些实施方案中，肌电信号是众多肌纤维中运动单元动作电位在时间和空间上的叠加，表面肌电信号是浅层肌肉和神经干上电活动在皮肤表面的综合效应。表面肌电信号分为两种，一种是静息电位对应的表面肌电信号，另一种是动作电位对应的表面肌电信号。其中，有效表面肌电信号为肌肉活动时间段内的表面肌电信号，即有效表面肌电信号为动作电位对应的表面肌电信号。
另外，静息电位对应的表面肌电信号对该方法的手势识别是没有用处的，静息电位是指肌肉处于放松状态，没有执行动作执行。且，对于本方法来说属于噪声类的无用信号。因此，需要获取表面肌电信号的活动段，即，需要检测表面肌电信号的动作电位的起始位置和结束位置，来获得有效表面肌电信号。
在某些实施方案中，如图2所示，检测表面肌电信号的动作电位的起始位置和结束位置，获得有效表面肌电信号包括：
201，对表面肌电信号进行校正处理，获得校正信号；
202，对获得的校正信号进行积分运算，获得包络信号；以及
203，将幅度大于预设阈值的包络信号对应的表面肌电信号，作为有效表面肌电信号。
在某些关于201的实施方案中，对表面肌电信号进行校正处理，是对经过预处理的表面肌电信号进行校正处理。
首先，对预处理的表面肌电信号执行转换操作，获得转换信号。通过对表面肌信号进行转换操作，来针对不同的被测对象进行有效校正，降低了阻抗以及肌肉紧张对基线阈值的影响。
然后，基于基线阈值校正该转换信号，获得校正信号。例如，确定基线阈值为：thr＝mean{MAV 1,MAV 2,MAV 3,...,MAV m}+A。
其中，thr为基线阈值，MAV i为表面肌电信号的静息态数据中滑动窗口的最大值，i为表面肌电信号的静息态数据中滑动窗口的最大值的下标，且取值范围为1到k之间的正整数，m为滑动窗口个数，A为常数。根据基线阈值对表面肌电信号进行校正处理，有效的降低了个体差异对表面肌电信号的影响。
在某些关于202的实施方案中，将获得的校正信号逐个导入核函数中，并在每导入一个校正信号后更新核函数，其操作包括：
对核函数进行初始化。
初始化的核函数表示为：kernel(j k)＝0，j 1,j 2,j 3,...j n。
将校正信号s i导入核函数中，核函数更新为kernel＝{j 2,...j n,s i},j 2,...j n＝0，基于梯形法计算该核函数单位等距积分，得到包络信号y i；
将校正信号s i+1导入核函数中，核函数更新为kernel＝{j 3,...j n,s i,s i+1},j 3,j 4...j n＝0，基于梯形法计算该核函数单位等距积分，得到包络信号y i+1；并且
以此类推，由校正信号{s 1,s 2,s 3,...s i,...}计算得到包络信号{y 1,y 2,y 3,...y i,...}。
基于梯形法计算该核函数单位等距积分，得到与校正信号对应的包络信号。
通过利用基线阈值对经过表面肌电信号转换得到的转换信号进行校正，然后利用核函数对校正信号进行处理得到包络信号，该包络信号增大了表面肌电信号中动作电位段与静息电位段之间的微小差异，减弱了由于肌肉紧张造成的表面肌电信号的波动，降低了由肌肉紧张造成的活动段的误判。
在某些关于203的实施方案中，包络是随机过程的振幅随着时间变化的曲线。包络信号是一个高频调幅信号，它幅度是按低频调制信号变化的。如果把高频调幅信号的峰点连接起来，就可以得到一个与低频调制信号相对应的曲线。另外，包络信号也是一个新的脉冲信号(周期更大)，这个脉冲信号在时间上观察也会有一定的宽度(每个周期内会有一段时间为0)，这是时间上宽度就是脉冲包络宽度。脉冲的带宽和脉冲宽度成反比，即脉冲时间上的宽度越窄，频谱上的带宽越大。
在某些实施方案中，活动段包括起始位置和结束位置。如果前一个或多个包络信号幅度不大于预设阈值，当包络信号的幅度大于预设阈值时，则确定该包络信号幅度开始大于预设阈值的初始位置对应的表面肌电信号的位置，作为活动段的起始位置；如果前一个或多个包络信号幅度大于预设阈值，当包络信号的幅度不大于预设阈值时，则确定该包络信号幅度开始小于预设阈值的初始位置对应的表面肌电信号的位置，作为活动段的结束位置。该起始位置到结束位置中间的表面肌电信号为有效表面肌电信号。
例如，设定的该预设阈值为零。如果前一个或多个包络信号幅度不大于零，当包络信号的幅度大于零时，则确定该包络信号 幅度开始大于零的初始位置对应的表面肌电信号的位置，作为活动段的起始位置；如果前一个或多个包络信号幅度大于零，当包络信号的幅度不大于零时，则确定该包络信号幅度开始小于零的初始位置对应的表面肌电信号的位置，作为活动段的结束位置。该起始位置到结束位置中间的表面肌电信号为有效表面肌电信号。
在某些实施方案中，采用M个数据窗对表面肌电信号进行截取，获得M段表面肌电信号，相邻两段表面肌电信号的间隔为预设步进，分别对每段表面肌电信号进行特征提取操作，其中，一个所述数据窗包括n个数据点，所述预设步进包括m个数据点。如图3所示，以对第p段表面肌电信号进行特征提取为例进行说明如下：
301，计算第p段表面肌电信号中每对相邻的数据点的差值的绝对值，其中，p大于或等于1，且小于或等于M；以及
302，对计算的所有的差值的绝对值进行求和处理，获得求和结果。
在某些关于301的实施方案中，一个数据窗的信号波长包括n个数据点，用公式表示为：DV＝{S 1，S 2，S 3，...，S i，...，S n-1，S n}，其中，S i为数据点。
分别计算每对相邻的数据点的差值的绝对值，即|S 2-S 1|，|S 3-S 2|，...，|S i-S i-1|，...，|S n-S n-1|。
在某些关于302的实施方案中，对计算的所有的差值的绝对值进行求和处理，获得求和结果为：
Figure PCTCN2021090715-appb-000001
将302所获得的求和结果作为第p段表面肌电信号的特征提取结果。
在获得M段表面肌电信号各自的特征提取结果后，对获得的M段表面肌电信号的特征提取结果(即M个求和结果)进行求和，并将获得的结果作为整个有效表面肌电信号的信号特征。
在某些实施方案中，对M段表面肌电信号各自的特征提取结 果进行求和，获得的结果可以表示为：
Figure PCTCN2021090715-appb-000002
通过对M段表面肌电信号的信号波长的简单叠加，提取表面肌电信号的信号特征，体现了肌电信号波形的复杂度，最后的获得结果是肌电信号幅值、频率以及持续时间等共同作用的效果。
在某些实施方案中，每种手势对应的前臂肌群有所不同，根据信号特征，确定该信号特征对应的前臂肌群。进一步的，根据确定的前臂肌群和预先配置的前臂肌群与手势的对应关系，获得表面肌电信号对应的手势。
例如，通过信号特征，确定该信号特征对应的前臂肌群为前臂肌肉B；根据该预先配置的前臂肌群与手势的对应关系前臂肌肉B对应握拳的手势，获得该表面肌电信号对应的手势为握拳。
在某些实施方案中，在根据信号特征获得表面肌电信号对应的手势之后，将该对应关系保存为匹配模板，并将该匹配模板作为不同手势识别的标准模板。当获得一个新的表面肌电信号后，可以将该表面肌电信号输入到匹配模板中，直接获取该表面肌电信号对应的手势。其中，该匹配模板无论是在网络环境下还是非网络环境下都可以进行手势识别，不受网络的限制。
例如，该匹配模板存储在任一智能设备中，在没有网络或者弱网环境下，可以通过数据线或者蓝牙的形式将获得的表面肌电信号输入到智能设备中。智能设备根据输入的表面肌电信号在匹配模板中进行匹配，即可获得该表面肌电信号对应的手势。进一步的，解决了在无网环境或弱网环境下无法进行手势识别的问题。
结合图4对该手势识别过程进行详细说明，该过程包括：
401，获得表面肌电信号；
402，对表面肌电信号进行预处理操作，获得处理后的表面肌电信号；
403，对处理后的表面肌电信号进行活动段检测；
404，利用滑窗算法，提取表面肌电信号的波形长度特征，获得提取特征结果；
405，根据获取的提取特征结果和匹配模板进行特征匹配；以及
406，获得表面肌电信号对应的手势。
本公开实施例提供的该方法，获取表面肌电信号，从该表面肌电信号中获取有效表面肌电信号，避免了无效表面肌电信号的干扰。进一步的，提取有效表面肌电信号的信号特征，根据该信号特征，直接获取表面肌电信号对应的手势。该方法无需预先采集大量的手势样本进行训练，再根据训练结果获取表面肌电信号对应的手势，可直接提取有效表面肌电信号的信号特征，根据信号特征快速有效的获取表面肌电信号对应的手势，操作过程简便，效率高。
本公开实施例还提供了手势识别装置，该装置的具体实施可参见方法实施例部分的描述，重复之处不再赘述，如图5所示，该装置主要包括：
第一获取模块501，用于获取表面肌电信号；
第二获取模块502，用于从表面肌电信号中获取有效表面肌电信号，其中，有效表面肌电信号为肌肉活动时间段内的表面肌电信号；
提取模块503，用于提取有效表面肌电信号的信号特征；以及
处理模块504，用于根据信号特征，获得表面肌电信号对应的手势。
在某些实施方案中，提取模块503用于采用M个数据窗对表面肌电信号进行截取，获得M段表面肌电信号，相邻两段表面肌电信号的间隔为预设步进，其中，一个所述数据窗包括n个数据点，所述预设步进包括m个数据点；
分别对每段表面肌电信号进行以下特征提取操作：计算第p段表面肌电信号中每对相邻的数据点的差值的绝对值，对计算的所有的差值的绝对值进行求和处理，获得求和结果，所述p大于或等于1，且小于或等于M；以及
对获得的M段表面肌电信号的求和结果进行求和，并将获得 结果作为有效表面肌电信号的信号特征。
在某些实施方案中，处理模块504用于确定信号特征对应的前臂肌群；根据确定的前臂肌群，和，预先配置的前臂肌群与手势的对应关系，获得表面肌电信号对应的所述手势。
在某些实施方案中，第二获取模块502用于对表面肌电信号进行校正处理，获得校正信号；对获得的校正信号进行积分运算，获得包络信号；以及将幅度大于预设阈值的包络信号对应的表面肌电信号，作为有效表面肌电信号。通过利用基线阈值对经过表面肌电信号转换得到的转换信号进行校正，然后利用核函数对校正信号进行处理得到包络信号，该包络信号增大了初始表面肌电信号中动作电位段与静息电位段之间的微小差异，减弱由于肌肉紧张造成的表面肌电信号的波动，降低了由肌肉紧张造成的活动段的误判。
本公开实施例提供的该装置，通过第一获取模块501，获取表面肌电信号；并利用第二获取模块502，从表面肌电信号中获取有效表面肌电信号，避免了无效表面肌电信号的干扰。然后，利用提取模块503，提取有效表面肌电信号的信号特征。最后，利用处理模块504，根据该信号特征，直接获取表面肌电信号对应的手势。该装置无需预先采集大量的手势样本进行训练，再根据训练结果获取表面肌电信号对应的手势，可直接提取有效表面肌电信号的信号特征，根据信号特征快速有效的获取表面肌电信号对应的手势，操作过程简便，效率高。
基于同一构思，本公开实施例中还提供了电子设备，如图6所示，该电子设备主要包括：处理器601和存储器602。在某些实施方案中，该电子设备还包括：通信组件603和通信总线604，其中，处理器601、通信组件603和存储器602通过通信总线604完成相互间的通信。其中，存储器602中存储有可被处理器601执行的程序，处理器601执行存储器502中存储的程序，以实现：获取表面肌电信号；从表面肌电信号中获取有效表面肌电信号，其中，有效表面肌电信号为肌肉活动时间段内的表面肌电信号；提取有效表面肌电信号的信号特征；以 及根据信号特征，获得表面肌电信号对应的手势。
上述电子设备中提到的通信总线604可以是外设部件互连标准(Peripheral Component Interconnect，简称PCI)总线或扩展工业标准结构(Extended Industry Standard Architecture，简称EISA)总线等。该通信总线604可以分为地址总线、数据总线、控制总线等。为便于表示，图6中仅用一条粗线表示，但并不表示仅有一根总线或一种类型的总线。
通信组件603用于上述电子设备与其他设备之间的通信。
存储器602可以包括随机存取存储器(Random Access Memory，简称RAM)，也可以包括非易失性存储器(non-volatile memory)，例如至少一个磁盘存储器。在某些实施方案中，存储器还可以是至少一个位于远离前述处理器601的存储装置。
上述的处理器601可以是通用处理器，包括中央处理器(Central Processing Unit，简称CPU)、网络处理器(Network Processor，简称NP)等，还可以是数字信号处理器(Digital Signal Processing，简称DSP)、专用集成电路(Application Specific Integrated Circuit，简称ASIC)、现场可编程门阵列(Field-Programmable Gate Array，简称FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。
在某些实施方案中，该电子设备可以为臂环设备，该臂环设备佩戴在上肢前臂靠近肘关节三分之一处。该臂环设备上可以设置一开关，被测对象在做手势之前，打开开关，在被测对象做出手势后，识别该手势。
本公开实施例还提供了人机交互方法，该方法包括以上实施例所描述的实施方法，具体实施可参见手势识别方法实施例部分的描述，重复之处不再赘述。
在某些实施方案中，该臂环设备可以应用在仿生假肢中，为一些特殊人群服务，可以让该部分人群通过自己的意志及肌肉控制仿生假肢的运动。以某被测对象的左腿安装仿生假肢为例进行说明：
首先，在该臂环设备中预先保存不同的手势和腿部运动之间的关系，例如，握拳的手势对应的是左腿向前一步。当被测对象在行走时，当右腿向前一步后，需要左腿也向前一步，此时，被测对象只需握拳即可，臂环设备采集获得被测对象的前臂肌群产生的表面肌电信号。然后，根据采集的表面肌电信号经过提取等操作获得该被测对象的手势为握拳，最后，根据握拳的手势与左腿向前一步的对应关系，获得左腿向前一步的指令。
该臂环设备在获得向前一步的指令之后，向仿生假肢发送该指令。仿生假肢接收臂环设备发出的指令，并完成被测对象的行走。其中，该臂环设备与仿生假肢已建立通信连接。
另外，该臂环设备还可以应用在演示文稿(PPT)中，可以预先设置不同的手势和PPT的执行动作之间的关系。例如，OK的手势对应是执行下一页的动作，剪刀手的手势对应的是执行自动播放的动作，握拳的手势对应的是执行上一页的动作。具体实现，也是通过臂环设备与电脑建立通信连接，电脑根据臂环设备发送的具体操作指令，完成PPT的播放。
本公开实施例提供的该设备，通过处理器601，获取表面肌电信号；从表面肌电信号中获取有效表面肌电信号；提取有效表面肌电信号的信号特征；并且根据该信号特征，直接获取表面肌电信号对应的手势。在处理过程中，从表面肌电信号中获取有效表面肌电信号，避免了无效表面肌电信号的干扰。该设备无需预先采集大量的手势样本进行训练，再根据训练结果获取表面肌电信号对应的手势，可直接提取有效表面肌电信号的信号特征，根据信号特征快速有效的获取表面肌电信号对应的手势，操作过程简便，效率高。
在本公开的又一实施例中，还提供了计算机可读存储介质，该计算机可读存储介质中存储有计算机程序，当该计算机程序在计算机上运行时，使得计算机执行上述实施例中所描述的手势识别方法。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者 其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。该计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行该计算机指令时，全部或部分地产生按照本公开实施例所述的流程或功能。该计算机可以时通用计算机、专用计算机、计算机网络或者其他可编程装置。该计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，计算机指令从一个网站站点、计算机、服务器或者数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、微波等)方式向另外一个网站站点、计算机、服务器或数据中心进行传输。该计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。该可用介质可以是磁性介质(例如软盘、硬盘、磁带等)、光介质(例如DVD)或者半导体介质(例如固态硬盘)等。
需要说明的是，在本文中，诸如“第一”和“第二”等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
以上所述仅是本公开的具体实施方式，使本领域技术人员能够理解或实现本公开。对这些实施例的多种修改对本领域的技术人员来说将是显而易见的，本文中所定义的一般原理可以在不脱离本公开的精神或范围的情况下，在其它实施例中实现。因此，本公开将不会被限制于本文所示的这些实施例，而是要符合与本文所公开的原理和新颖特点相一致的最宽的范围。

Claims (11)

  1. 手势识别方法，其包括：
    获取表面肌电信号；
    从所述表面肌电信号中获取有效表面肌电信号，其中，所述有效表面肌电信号为肌肉活动时间段内的表面肌电信号；
    提取所述有效表面肌电信号的信号特征；以及
    根据所述信号特征，获得所述表面肌电信号对应的手势。
  2. 如权利要求1所述的手势识别方法，其中，所述表面肌电信号是由前臂肌群运动产生的，且不同的手势对应的表面肌电信号不同。
  3. 如权利要求2所述的手势识别方法，其中，从所述表面肌电信号中获取有效表面肌电信号之前，还包括：
    滤除所述表面肌电信号中预设频率的噪声。
  4. 如权利要求3所述的手势识别方法，其中，提取所述有效表面肌电信号的信号特征，包括：
    采用M个数据窗对所述表面肌电信号进行截取，获得M段表面肌电信号，相邻两段所述表面肌电信号的间隔为预设步进，其中，一个所述数据窗包括n个数据点，所述预设步进包括m个数据点；
    分别对每段所述表面肌电信号进行以下特征提取操作：计算第p段所述表面肌电信号中每对相邻的数据点的差值的绝对值，对计算的所有的所述差值的绝对值进行求和处理，获得求和结果，所述p大于或等于1，且小于或等于M；以及
    对获得的M段所述表面肌电信号的所述求和结果进行求和，并将获得结果作为所述有效表面肌电信号的信号特征。
  5. 如权利要求4所述的手势识别方法，其中，根据所述信号 特征，获得所述表面肌电信号对应的手势，包括：
    确定所述信号特征对应的前臂肌群；以及
    根据确定的所述前臂肌群，和，预先配置的前臂肌群与手势的对应关系，获得所述表面肌电信号对应的所述手势。
  6. 如权利要求3至5中任一权利要求所述的手势识别方法，其中，从所述表面肌电信号中获取有效表面肌电信号，包括：
    对所述表面肌电信号进行校正处理，获得校正信号；
    对获得的所述校正信号进行积分运算，获得包络信号；以及
    将幅度大于预设阈值的包络信号对应的表面肌电信号，作为所述有效表面肌电信号。
  7. 人机交互方法，其包括：权利要求1至6中任一权利要求所述的手势识别方法。
  8. 手势识别装置，其包括：
    第一获取模块，用于获取表面肌电信号；
    第二获取模块，用于从所述表面肌电信号中获取有效表面肌电信号，其中，所述有效表面肌电信号为肌肉活动时间段内的表面肌电信号；
    提取模块，用于提取所述有效表面肌电信号的信号特征；以及
    处理模块，用于根据所述信号特征，获得所述表面肌电信号对应的手势。
  9. 电子设备，其包括：处理器和存储器，所述处理器，用于执行所述存储器中所存储的程序，以实现权利要求1至6中任一权利要求所述的手势识别方法。
  10. 如权利要求9所述的电子设备，其中，所述电子设备为臂环设备，所述臂环设备用于佩戴在上肢前臂靠近肘关节三分之一 处。
  11. 计算机可读存储介质，存储有计算机程序，其中，所述计算机程序被处理器执行时实现权利要求1至6中任一权利要求所述的手势识别方法。
PCT/CN2021/090715 2020-04-30 2021-04-28 手势识别方法、人机交互方法、装置、设备及存储介质 WO2021219039A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010367114.5A CN113589920B (zh) 2020-04-30 2020-04-30 手势识别方法、人机交互方法、装置、设备及存储介质
CN202010367114.5 2020-04-30

Publications (1)

Publication Number Publication Date
WO2021219039A1 true WO2021219039A1 (zh) 2021-11-04

Family

ID=78237601

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/090715 WO2021219039A1 (zh) 2020-04-30 2021-04-28 手势识别方法、人机交互方法、装置、设备及存储介质

Country Status (2)

Country Link
CN (1) CN113589920B (zh)
WO (1) WO2021219039A1 (zh)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113970968A (zh) * 2021-12-22 2022-01-25 深圳市心流科技有限公司 一种智能仿生手的动作预判方法
CN113986017A (zh) * 2021-12-27 2022-01-28 深圳市心流科技有限公司 一种肌电手势模板的生成方法、装置及存储介质
CN114167996A (zh) * 2022-02-14 2022-03-11 浙江强脑科技有限公司 一种基于传感器的动作预判方法、装置及存储介质
CN115114962A (zh) * 2022-07-19 2022-09-27 歌尔股份有限公司 基于表面肌电信号的控制方法、装置及可穿戴设备
CN115131878A (zh) * 2022-08-30 2022-09-30 深圳市心流科技有限公司 一种智能假肢的动作确定方法、装置、终端及存储介质
WO2023079014A1 (en) 2021-11-05 2023-05-11 Elliptic Laboratories Asa Remote presence detection system

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114690905A (zh) * 2022-03-30 2022-07-01 歌尔股份有限公司 一种手势控制方法、系统、装置、可穿戴产品及介质
CN114625257B (zh) * 2022-05-16 2022-08-16 浙江强脑科技有限公司 一种基于肌电信号的动作识别方法和识别装置
CN116400812B (zh) * 2023-06-05 2023-09-12 中国科学院自动化研究所 基于表面肌电信号的应急救援手势识别方法及装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160224236A1 (en) * 1998-05-15 2016-08-04 Advanced Touchscreen And Gestures Technologies, Llc Touch Screen Apparatus for Recognizing a Touch Gesture
CN110420025A (zh) * 2019-09-03 2019-11-08 北京海益同展信息科技有限公司 表面肌电信号处理方法、装置和可穿戴设备
CN110826625A (zh) * 2019-11-06 2020-02-21 南昌大学 一种基于表面肌电信号的手指手势分类方法
CN111026268A (zh) * 2019-12-02 2020-04-17 清华大学 一种手势识别装置和方法

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU2683859C1 (ru) * 2017-12-27 2019-04-02 ОБЩЕСТВО С ОГРАНИЧЕННОЙ ОТВЕТСТВЕННОСТЬЮ "Би-оН ЭМГ" Способ и система управления электронными устройствами с помощью электромиографического устройства считывания
CN110618754B (zh) * 2019-08-30 2021-09-14 电子科技大学 基于表面肌电信号的手势识别方法和手势识别臂带

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160224236A1 (en) * 1998-05-15 2016-08-04 Advanced Touchscreen And Gestures Technologies, Llc Touch Screen Apparatus for Recognizing a Touch Gesture
CN110420025A (zh) * 2019-09-03 2019-11-08 北京海益同展信息科技有限公司 表面肌电信号处理方法、装置和可穿戴设备
CN110826625A (zh) * 2019-11-06 2020-02-21 南昌大学 一种基于表面肌电信号的手指手势分类方法
CN111026268A (zh) * 2019-12-02 2020-04-17 清华大学 一种手势识别装置和方法

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023079014A1 (en) 2021-11-05 2023-05-11 Elliptic Laboratories Asa Remote presence detection system
CN113970968A (zh) * 2021-12-22 2022-01-25 深圳市心流科技有限公司 一种智能仿生手的动作预判方法
CN113986017A (zh) * 2021-12-27 2022-01-28 深圳市心流科技有限公司 一种肌电手势模板的生成方法、装置及存储介质
CN113986017B (zh) * 2021-12-27 2022-05-17 深圳市心流科技有限公司 一种肌电手势模板的生成方法、装置及存储介质
CN114167996A (zh) * 2022-02-14 2022-03-11 浙江强脑科技有限公司 一种基于传感器的动作预判方法、装置及存储介质
CN115114962A (zh) * 2022-07-19 2022-09-27 歌尔股份有限公司 基于表面肌电信号的控制方法、装置及可穿戴设备
CN115131878A (zh) * 2022-08-30 2022-09-30 深圳市心流科技有限公司 一种智能假肢的动作确定方法、装置、终端及存储介质
CN115131878B (zh) * 2022-08-30 2022-11-11 深圳市心流科技有限公司 一种智能假肢的动作确定方法、装置、终端及存储介质

Also Published As

Publication number Publication date
CN113589920B (zh) 2024-06-18
CN113589920A (zh) 2021-11-02

Similar Documents

Publication Publication Date Title
WO2021219039A1 (zh) 手势识别方法、人机交互方法、装置、设备及存储介质
US20220147151A1 (en) Gesture information processing method and apparatus, electronic device, and storage medium
CN104107134B (zh) 基于肌电反馈的上肢训练方法及系统
CN108904980B (zh) 基于脑电和功能性电刺激的上肢主动康复方法及装置
Huang et al. An EMG-based handwriting recognition through dynamic time warping
WO2016023349A1 (zh) 基于表面肌电信号的动作识别方法和设备
WO2020118797A1 (zh) 假肢控制方法、装置、系统、设备和存储介质
CN110418604A (zh) 用于检测电生理诱发电位变化的医疗系统和方法
CN104571504A (zh) 一种基于想象动作的在线脑-机接口方法
Li et al. Wireless sEMG-based identification in a virtual reality environment
Aceves-Fernandez et al. Methodology proposal of EMG hand movement classification based on cross recurrence plots
CN106845348B (zh) 一种基于手臂表面肌电信号的手势识别方法
CN114298089A (zh) 一种多模态力量训练辅助方法和系统
Yu et al. A novel hand gesture recognition method based on 2-channel sEMG
CN114661161A (zh) 手势交互方法、装置、系统、可穿戴设备及存储介质
WO2018214522A1 (zh) 肌电信号采集方法及装置
Veer Experimental study and characterization of SEMG signals for upper limbs
Xu et al. Convolutional Neural Network-Based Identity Recognition Using ECG at Different Water Temperatures During Bathing.
CN110547806B (zh) 一种基于表面肌电信号的手势动作在线识别方法及系统
Valbuena et al. sBCI: fast detection of steady-state visual evoked potentials
CN105686827B (zh) 一种基于微控制器的肌电信号处理及特征提取方法
CN116831874A (zh) 一种基于肌电信号的下肢康复器控制方法
Kim et al. User Recognition System Based on Spectrogram Image Conversion Using EMG Signals.
CN115299950A (zh) 肌电信号采集电路、可穿戴设备及控制方法
Pirasteh et al. EEG-based brain-computer interface methods with the aim of rehabilitating advanced stage ALS patients

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21796731

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21796731

Country of ref document: EP

Kind code of ref document: A1