CN113589920A - Gesture recognition method, man-machine interaction method, device, equipment and storage medium - Google Patents

Gesture recognition method, man-machine interaction method, device, equipment and storage medium Download PDF

Info

Publication number
CN113589920A
CN113589920A CN202010367114.5A CN202010367114A CN113589920A CN 113589920 A CN113589920 A CN 113589920A CN 202010367114 A CN202010367114 A CN 202010367114A CN 113589920 A CN113589920 A CN 113589920A
Authority
CN
China
Prior art keywords
signal
surface electromyographic
signals
gesture
gesture recognition
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010367114.5A
Other languages
Chinese (zh)
Inventor
田彦秀
韩久琦
姚秀军
桂晨光
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Haiyi Tongzhan Information Technology Co Ltd
Original Assignee
Beijing Haiyi Tongzhan Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Haiyi Tongzhan Information Technology Co Ltd filed Critical Beijing Haiyi Tongzhan Information Technology Co Ltd
Priority to CN202010367114.5A priority Critical patent/CN113589920A/en
Priority to PCT/CN2021/090715 priority patent/WO2021219039A1/en
Publication of CN113589920A publication Critical patent/CN113589920A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017Gesture based interaction, e.g. based on a set of recognized hand gestures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/02Preprocessing
    • G06F2218/04Denoising
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/08Feature extraction

Abstract

The application relates to a gesture recognition method, a man-machine interaction device, equipment and a storage medium, wherein the method comprises the following steps: acquiring a surface electromyographic signal; acquiring an effective surface electromyographic signal from the surface electromyographic signal, wherein the effective surface electromyographic signal is the surface electromyographic signal in a muscle activity time period; extracting the signal characteristics of the effective surface electromyographic signals; and acquiring a gesture corresponding to the surface electromyographic signal according to the signal characteristics. The method and the device are used for simplifying the gesture recognition process and improving the gesture recognition efficiency.

Description

Gesture recognition method, man-machine interaction method, device, equipment and storage medium
Technical Field
The present application relates to the field of artificial intelligence technologies, and in particular, to a gesture recognition method, a human-computer interaction method, an apparatus, a device, and a storage medium.
Background
Gesture recognition is essentially a pattern recognition problem, and requires learning valid features from input information and using the extracted features to identify gesture tags. The gesture action recognition technology based on the surface electromyogram signal can be divided into two types according to whether human factors are added or not: the method comprises the steps of firstly, preprocessing collected surface electromyographic signals of different gesture actions, extracting various characteristic quantities of a time domain, a frequency domain and the time-frequency domain, then performing characteristic dimension reduction or characteristic selection on the extracted characteristic quantities, inputting the characteristic quantities into a classifier for model training, and using the trained classifier for gesture action real-time prediction; the other type is that preprocessed surface electromyographic signals are directly used as input quantity, artificial factors such as feature extraction and the like are not added, a deep learning framework is used for actively capturing and learning differences of the surface electromyographic signals of different gesture actions, and a trained network model structure is used for actual testing.
The first electromyographic gesture recognition method generally uses a signal analysis technology to manually extract various signal characteristics from a surface electromyographic signal, and then inputs the extracted signal characteristics into classifiers such as a linear discriminant analysis classifier, a support vector machine classifier and a hidden Markov model for gesture recognition. The deep neural network model in the second type of recognition technology is widely applied to convolutional neural networks.
However, the signal quality selected by the first myoelectric gesture recognition technology will have a great influence on the gesture recognition performance. Although the second type of electromyographic gesture recognition technology does not depend on manual feature extraction and complex and tedious feature optimization processes, representative depth features need to be automatically learned from a large number of input samples, and the calculation amount is very large. In addition, the two gesture motion recognition technologies both require a user to acquire and record myoelectric signal data of different marked gesture motion surfaces in advance, and the myoelectric signal data can be predicted and used in practice after model training is carried out by using the marked data, so that the realization process is inconvenient and the user experience is poor.
Disclosure of Invention
The application provides a gesture recognition method, a man-machine interaction device, equipment and a storage medium, which are used for simplifying a gesture recognition process and improving gesture recognition efficiency.
In a first aspect, the present application provides a gesture recognition method, including:
acquiring a surface electromyographic signal;
acquiring an effective surface electromyographic signal from the surface electromyographic signal, wherein the effective surface electromyographic signal is the surface electromyographic signal in a muscle activity time period;
extracting the signal characteristics of the effective surface electromyographic signals;
and acquiring a gesture corresponding to the surface electromyographic signal according to the signal characteristics.
Optionally, the surface electromyography signals are generated by forearm muscle group movements, and the surface electromyography signals corresponding to different gestures are different.
Optionally, before obtaining the effective surface electromyography signal from the surface electromyography signal, the method further includes:
and filtering noise with preset frequency in the surface electromyographic signal.
Optionally, extracting a signal feature of the effective surface electromyography signal includes:
intercepting the surface electromyographic signals by adopting M data windows to obtain M sections of surface electromyographic signals, wherein the interval between two adjacent sections of surface electromyographic signals is a preset step, one data window comprises n data points, and the preset step comprises M data points;
respectively carrying out the following characteristic extraction operations on each section of the surface electromyographic signals: calculating the absolute value of the difference value of each pair of adjacent data points in the p-th section of the surface electromyogram signal, and summing all the calculated absolute values of the difference values to obtain a summation result, wherein p is greater than or equal to 1 and less than or equal to M;
and summing the summation results of the obtained M sections of the surface electromyographic signals, and taking the obtained result as the signal characteristic of the effective surface electromyographic signal.
Optionally, obtaining a gesture corresponding to the surface electromyography signal according to the signal feature includes:
determining a forearm muscle group corresponding to the signal characteristic;
and obtaining the gesture corresponding to the surface electromyographic signal according to the determined forearm muscle group and the corresponding relationship between the preset forearm muscle group and the gesture.
Optionally, obtaining an effective surface electromyography signal from the surface electromyography signal includes:
correcting the surface electromyographic signal to obtain a correction signal;
performing integral operation on the obtained correction signal to obtain an envelope signal;
and taking the surface electromyographic signal corresponding to the envelope signal with the amplitude larger than the preset threshold value as the effective surface electromyographic signal.
In a second aspect, the present application provides a human-computer interaction method, including: the gesture recognition method of the first aspect.
In a third aspect, the present application provides a gesture recognition apparatus, including:
the first acquisition module is used for acquiring a surface electromyographic signal;
the second acquisition module is used for acquiring an effective surface electromyographic signal from the surface electromyographic signal, wherein the effective surface electromyographic signal is a surface electromyographic signal within a muscle activity time period;
the extraction module is used for extracting the signal characteristics of the effective surface electromyographic signals;
and the processing module is used for obtaining the gesture corresponding to the surface electromyogram signal according to the signal characteristics.
In a fourth aspect, the present application provides an electronic device, comprising: the processor is used for executing the program stored in the memory so as to realize the gesture recognition method.
Optionally, the electronic device is an arm ring device for wearing on the forearm of the upper limb proximate to one third of the elbow joint.
In a fifth aspect, the present application provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the gesture recognition method.
Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages: according to the method provided by the embodiment of the application, the surface electromyographic signals are obtained, the effective surface electromyographic signals are obtained from the surface electromyographic signals, and interference of ineffective surface electromyographic signals is avoided. Further, the signal characteristics of the effective surface electromyogram signals are extracted, and the gestures corresponding to the surface electromyogram signals are directly acquired according to the signal characteristics. According to the method, a large number of gesture samples do not need to be collected in advance for training, the gesture corresponding to the surface electromyographic signal is obtained according to the training result, the signal characteristic of the effective surface electromyographic signal can be directly extracted, the gesture corresponding to the surface electromyographic signal is rapidly and effectively obtained according to the signal characteristic, the operation process is simple and convenient, and the efficiency is high.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
Fig. 1 is a schematic flowchart of a gesture recognition method according to an embodiment of the present disclosure;
fig. 2 is a schematic flow chart of a method for extracting an effective surface electromyogram signal according to an embodiment of the present application;
fig. 3 is a schematic flow chart of a surface electromyogram signal feature extraction method provided in the embodiment of the present application;
fig. 4 is a schematic flowchart illustrating a specific implementation process of a gesture recognition method according to an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of a gesture recognition apparatus according to an embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The embodiment of the application provides a gesture recognition method which can be applied to separate electronic equipment, wherein the electronic equipment collects surface electromyographic signals and recognizes gestures corresponding to the surface electromyographic signals. In addition, the method can also be applied to intelligent terminal equipment, the surface electromyogram signals are collected through the collection equipment, then the collected surface electromyogram signals are transmitted to the intelligent terminal equipment through the collection equipment, and gestures corresponding to the surface electromyogram signals are obtained through recognition of the intelligent terminal equipment. Of course, the method may also be applied to a server, the surface electromyogram signal is collected by a collection device, then the collected surface electromyogram signal is transmitted to the server by the collection device, and the server identifies and obtains the gesture corresponding to the surface electromyogram signal. The specific implementation process of the method is shown in fig. 1:
step 101, acquiring a surface electromyographic signal.
Specifically, the surface electromyographic signal is a bioelectricity signal acquired by an electrode placed on the surface of the skin, is a non-stable and non-linear weak electric signal, has randomness, is very easy to be interfered by the outside, and has low signal-to-noise ratio. In addition, the surface electromyographic signals can reflect the extension and flexion conditions of human joints and the shapes and positions of limbs.
In the embodiment of the application, the electrode for collecting the surface electromyographic signals is worn on the forearm of the upper limb of the human body so as to monitor the movement condition of the forearm muscle driven by the gesture action.
The surface electromyographic signals monitored by the electrodes worn on the front arms are generated by the movement of the forearm muscle groups, different gestures drive different forearm muscle groups to move, namely, each gesture is generated by a part of the forearm muscle groups in the whole forearm muscle groups to play a leading role, and different forearm muscle group movements can generate different surface electromyographic signals.
For example, assume that the OK gesture is generated by movement of forearm muscle a, the fist-making gesture is generated by movement of forearm muscle B, the scissor gesture is generated by movement of forearm muscle C, and the salute gesture is generated by movement of forearm muscle D and forearm muscle E together. Therefore, when the tested object makes OK gesture, the surface electromyogram signal A is monitored; when a measured object makes a fist-making gesture, monitoring a surface electromyographic signal B; when the tested object does the gesture of a scissor hand, monitoring a surface electromyographic signal C; when the tested object makes a metal gift gesture, the surface electromyographic signal DE is monitored.
102, obtaining an effective surface electromyographic signal from the surface electromyographic signal, wherein the effective surface electromyographic signal is a surface electromyographic signal within a muscle activity time period.
In a specific embodiment, after the surface electromyography signal is acquired and before the effective surface electromyography signal is acquired from the surface electromyography signal, the surface electromyography signal needs to be preprocessed to filter interference in the surface electromyography signal, so that accuracy of subsequent hand gestures is improved.
Specifically, noise with preset frequency in the surface electromyogram signal is filtered in the preprocessing process.
For example, because the surface electromyogram signal is easily interfered by 50Hz power frequency, a wave trap is used to remove the noise with 50Hz frequency in the surface electromyogram signal. And then, inputting the surface electromyographic signals obtained after filtering by the wave trap into a 6-order Butterworth filter to obtain the surface electromyographic signals with the frequency of 20-200 Hz, which are filtered by the filter. The surface electromyographic signals with the frequency of 20-200 Hz are useful surface electromyographic signals in the gesture recognition method. By preprocessing the surface electromyographic signals, the noise content of the signals is reduced, and the surface electromyographic signals more conforming to the gesture recognition method are obtained.
In one embodiment, the electromyographic signals are the superposition of action potentials of motor units in a plurality of muscle fibers on time and space, and the surface electromyographic signals are the combined effect of electric activities on superficial muscles and nerve trunks on the surface of skin. The surface electromyogram signals are divided into two types, one is the surface electromyogram signal corresponding to the resting potential, and the other is the surface electromyogram signal corresponding to the action potential. The effective surface electromyographic signal is a surface electromyographic signal in a muscle activity time period, namely the effective surface electromyographic signal is a surface electromyographic signal corresponding to an action potential.
In addition, the surface electromyographic signals corresponding to the resting potential are useless for gesture recognition of the method, and the resting potential means that muscles are in a relaxed state and no action is executed. And, for the present method, unwanted signals that belong to the noise class. Therefore, an active segment for acquiring the surface electromyogram signal, that is, a start position and an end position of an action potential of the surface electromyogram signal need to be detected to obtain an effective surface electromyogram signal.
Specifically, as shown in fig. 2, the starting position and the ending position of the action potential of the surface electromyography signal are detected, and the specific process of obtaining the effective surface electromyography signal is as follows:
step 201, performing correction processing on the surface muscle electrical signal to obtain a correction signal.
In a specific embodiment, the surface electromyography signal is corrected by performing a correction process on the preprocessed surface electromyography signal.
First, a conversion operation is performed on the preprocessed surface electromyogram signal to obtain a converted signal. By carrying out conversion operation on the surface muscle signals, effective correction is carried out on different tested objects, and the influence of impedance and muscle tension on a baseline threshold is reduced.
Then, the converted signal is corrected based on the baseline threshold, and a correction signal is obtained. Utensil for cleaning buttockIn vivo, the baseline threshold is determined as: thr mean MAV1,MAV2,MAV3,...,MAVm}+A
Where thr is the baseline threshold, MAViThe value of the index is the maximum value of the sliding window in the resting state data of the surface electromyogram signal, i is the subscript of the maximum value of the sliding window in the resting state data of the surface electromyogram signal, the value range is a positive integer between 1 and k, m is the number of the sliding windows, and A is a constant. And the surface myoelectric signals are corrected according to the baseline threshold, so that the influence of individual difference on the surface myoelectric signals is effectively reduced.
Step 202, performing an integration operation on the obtained correction signal to obtain an envelope signal.
In a specific embodiment, the obtained correction signals are introduced into the kernel functions one by one, and the kernel functions are updated after each introduction of a correction signal. The specific operation is as follows:
the kernel function is initialized.
The initialized kernel function is represented as: kernel (j)k)=0,j1,j2,j3,...jn
Will correct the signal siIn the import kernel function, the kernel function is updated to kernel ═ j2,...jn,si},j2,...jnCalculating the equidistant integral of the kernel function unit based on a trapezoidal method to obtain an envelope signal yi
Will correct the signal si+1In the import kernel function, the kernel function is updated to kernel ═ j3,...jn,si,si+1},j3,j4...jnCalculating the equidistant integral of the kernel function unit based on a trapezoidal method to obtain an envelope signal yi+1
By analogy, from the correction signal s1,s2,s3,...si,. DEG calculating to obtain an envelope signal y1,y2,y3,...yi,...}。
And calculating the unit equidistant integral of the kernel function based on a trapezoidal method to obtain an envelope signal corresponding to the correction signal.
The converted signal obtained by converting the surface electromyogram signal is corrected by using the baseline threshold, and then the corrected signal is processed by using the kernel function to obtain the envelope signal, wherein the envelope signal increases the tiny difference between an action potential section and a rest potential section in the surface electromyogram signal, weakens the fluctuation of the surface electromyogram signal caused by muscle tension, and reduces the misjudgment of an active section caused by the muscle tension.
And 203, taking the surface electromyographic signal corresponding to the envelope signal with the amplitude larger than the preset threshold value as an effective surface electromyographic signal.
In particular, the envelope is a curve of the amplitude of the random process over time. The envelope signal is a high frequency amplitude modulated signal whose amplitude is varied in accordance with the low frequency modulated signal. If the peaks of the high frequency amplitude modulated signal are connected, a curve corresponding to the low frequency modulated signal is obtained. In addition, the envelope signal is also a new pulse signal (with a larger period), and the pulse signal has a certain width (a period of time is 0 in each period) observed in time, which is the width of the pulse envelope in time. The bandwidth of a pulse is inversely proportional to the pulse width, i.e. the narrower the width of the pulse in time, the larger the bandwidth in the spectrum.
Specifically, the active segment includes a start position and an end position. If the amplitude of the previous one or more envelope signals is not greater than a preset threshold, when the amplitude of the envelope signals is greater than the preset threshold, determining the position of the surface electromyographic signal corresponding to the initial position where the amplitude of the envelope signals starts to be greater than the preset threshold as the initial position of the active segment; if the amplitude of the previous one or more envelope signals is larger than a preset threshold, when the amplitude of the envelope signals is not larger than the preset threshold, determining the position of the surface electromyographic signal corresponding to the initial position where the amplitude of the envelope signals starts to be smaller than the preset threshold as the end position of the active segment. The surface electromyogram signal in the middle of the starting position to the ending position is an effective surface electromyogram signal.
For example, the preset threshold is set to zero. If the amplitude of the previous envelope signal or envelope signals is not larger than zero, when the amplitude of the envelope signal is larger than zero, determining the position of the surface electromyographic signal corresponding to the initial position where the amplitude of the envelope signal is larger than zero as the initial position of the active segment; and if the amplitude of the previous one or more envelope signals is greater than zero, when the amplitude of the envelope signals is not greater than zero, determining the position of the surface electromyographic signal corresponding to the initial position where the amplitude of the envelope signals starts to be less than zero as the end position of the active segment. The surface electromyogram signal in the middle of the starting position to the ending position is an effective surface electromyogram signal.
And 103, extracting the signal characteristics of the effective surface electromyographic signals.
In a specific embodiment, M data windows are adopted to intercept the surface electromyographic signals to obtain M sections of surface electromyographic signals, the interval between two adjacent sections of surface electromyographic signals is a preset step, and feature extraction operation is performed on each section of surface electromyographic signals respectively, wherein one data window comprises n data points, and the preset step comprises M data points. As shown in fig. 3, the following description will be given taking the feature extraction of the p-th stage surface myoelectric signal as an example:
step 301, calculating an absolute value of a difference value of each pair of adjacent data points in the p-th segment of the surface electromyographic signal, wherein p is greater than or equal to 1 and less than or equal to M.
Specifically, the signal wavelength of a data window includes n data points, which are formulated as: DV ═ S1,S2,S3,...,Si,...,Sn-1,SnIn which S isiAre data points.
The absolute value of the difference, i.e. | S, for each pair of adjacent data points is calculated separately2-S1|,|S3-S2|,...,|Si-Si-1|,...,|Sn-Sn-1|。
Step 302, summing all the calculated absolute values of the difference values to obtain a summation result.
Specifically, the absolute values of all the calculated differences are summed, and the sum result is obtained as follows:
Figure BDA0002476871820000091
and taking the summation result obtained in the step 302 as a characteristic extraction result of the p-th segment surface electromyographic signal.
After the respective feature extraction results of the M segments of the surface electromyographic signals are obtained, the obtained feature extraction results (i.e., M summation results) of the M segments of the surface electromyographic signals are summed, and the obtained result is taken as the signal feature of the entire effective surface electromyographic signal.
Specifically, the feature extraction results of the M segments of surface myoelectric signals are summed, and the obtained result can be expressed as:
Figure BDA0002476871820000101
the signal characteristics of the surface electromyographic signals are extracted by simply superposing the signal wavelengths of the M sections of surface electromyographic signals, the complexity of the waveforms of the electromyographic signals is reflected, and the finally obtained result is the effect of the combined action of the amplitude, the frequency, the duration time and the like of the electromyographic signals.
And 104, acquiring a gesture corresponding to the surface electromyogram signal according to the signal characteristics.
In one embodiment, the forearm muscle group corresponding to each gesture is different, and the forearm muscle group corresponding to the signal characteristic is determined according to the signal characteristic. Further, according to the determined corresponding relation between the forearm muscle group and the gesture and the preconfigured forearm muscle group, the gesture corresponding to the surface electromyographic signal is obtained.
For example, through the signal characteristics, determining the forearm muscle group corresponding to the signal characteristics as the forearm muscle B; and obtaining the gesture corresponding to the surface electromyographic signal as a fist making gesture according to the fist making gesture corresponding to the forearm muscle B in the corresponding relationship between the preconfigured forearm muscle group and the gesture.
Specifically, after the gesture corresponding to the surface electromyogram signal is obtained according to the signal characteristics, the corresponding relation is stored as a matching template, and the matching template is used as a standard template for different gesture recognition. After a new surface electromyographic signal is obtained, the surface electromyographic signal can be input into a matching template, and a gesture corresponding to the surface electromyographic signal is directly obtained. The matching template can be used for gesture recognition in both a network environment and a non-network environment, and is not limited by a network.
For example, the matching template is stored in any intelligent device, and the obtained surface electromyographic signals can be input into the intelligent device in the form of data lines or Bluetooth under the environment without a network or a weak network. And the intelligent equipment matches the input surface electromyographic signals in the matching template, so that the gestures corresponding to the surface electromyographic signals can be obtained. Furthermore, the problem that gesture recognition cannot be carried out in a no-network environment or a weak-network environment is solved.
Specifically, the gesture recognition process is described in detail with reference to fig. 4:
step 401, obtaining a surface electromyography signal.
Step 402, performing preprocessing operation on the surface electromyographic signals to obtain processed surface electromyographic signals.
And step 403, performing active segment detection on the processed surface electromyogram signal.
And step 404, extracting the waveform length characteristic of the surface electromyogram signal by using a sliding window algorithm to obtain an extracted characteristic result.
And 405, performing feature matching according to the obtained feature extraction result and the matching template.
And 406, obtaining a gesture corresponding to the surface electromyogram signal.
According to the method provided by the embodiment of the application, the surface electromyographic signals are obtained, the effective surface electromyographic signals are obtained from the surface electromyographic signals, and interference of ineffective surface electromyographic signals is avoided. Further, the signal characteristics of the effective surface electromyogram signals are extracted, and the gestures corresponding to the surface electromyogram signals are directly acquired according to the signal characteristics. According to the method, a large number of gesture samples do not need to be collected in advance for training, the gesture corresponding to the surface electromyographic signal is obtained according to the training result, the signal characteristic of the effective surface electromyographic signal can be directly extracted, the gesture corresponding to the surface electromyographic signal is rapidly and effectively obtained according to the signal characteristic, the operation process is simple and convenient, and the efficiency is high.
The embodiment of the present application further provides a gesture recognition apparatus, and specific implementation of the apparatus may refer to the description of the method embodiment section, and repeated details are not repeated, as shown in fig. 5, the apparatus mainly includes:
the first obtaining module 501 is configured to obtain a surface electromyogram signal.
A second obtaining module 502, configured to obtain an effective surface electromyography signal from the surface electromyography signal, where the effective surface electromyography signal is a surface electromyography signal in a muscle activity time period.
An extracting module 503, configured to extract a signal feature of the effective surface electromyogram signal.
And the processing module 504 is configured to obtain a gesture corresponding to the surface electromyogram signal according to the signal characteristics.
In a specific embodiment, the extracting module 503 is specifically configured to intercept the surface electromyography signals by using M data windows to obtain M sections of surface electromyography signals, where an interval between two adjacent sections of surface electromyography signals is a preset step, where one data window includes n data points, and the preset step includes M data points;
respectively carrying out the following characteristic extraction operations on each section of surface myoelectric signals: calculating the absolute value of the difference value of each pair of adjacent data points in the p-th section of surface electromyographic signals, and summing the absolute values of all the calculated difference values to obtain a summation result, wherein p is greater than or equal to 1 and less than or equal to M;
and summing the obtained summation results of the M sections of surface electromyographic signals, and taking the obtained result as the signal characteristic of the effective surface electromyographic signal.
In an embodiment, the processing module 504 is specifically configured to determine a forearm muscle group corresponding to the signal feature; and obtaining the gesture corresponding to the surface electromyographic signal according to the determined forearm muscle group and the corresponding relationship between the preset forearm muscle group and the gesture.
In a specific embodiment, the second obtaining module 502 is specifically configured to perform correction processing on the surface muscle electrical signal to obtain a correction signal; performing integral operation on the obtained correction signal to obtain an envelope signal; and taking the surface electromyographic signal corresponding to the envelope signal with the amplitude larger than the preset threshold value as an effective surface electromyographic signal. The converted signal obtained by converting the surface electromyogram signal is corrected by using a baseline threshold, and then the corrected signal is processed by using a kernel function to obtain an envelope signal, wherein the envelope signal increases the tiny difference between an action potential section and a rest potential section in the initial surface electromyogram signal, weakens the fluctuation of the surface electromyogram signal caused by muscle tension, and reduces the misjudgment of an active section caused by the muscle tension.
According to the device provided by the embodiment of the application, a first acquisition module 501 is used for acquiring a surface electromyographic signal; and the second obtaining module 502 is utilized to obtain the effective surface electromyogram signal from the surface electromyogram signal, so that the interference of the ineffective surface electromyogram signal is avoided. Further, by means of the extraction module 503, the signal characteristics of the effective surface electromyogram signal are extracted. Finally, by using the processing module 504, the gesture corresponding to the surface electromyogram signal is directly acquired according to the signal characteristics. The device does not need to collect a large number of gesture samples in advance for training, obtains gestures corresponding to the surface electromyographic signals according to training results, can directly extract signal characteristics of the effective surface electromyographic signals, and quickly and effectively obtains the gestures corresponding to the surface electromyographic signals according to the signal characteristics, and is simple and convenient in operation process and high in efficiency.
Based on the same concept, an embodiment of the present application further provides an electronic device, as shown in fig. 6, the electronic device mainly includes: a processor 601 and a memory 602. Specifically, the electronic device further includes: a communication component 603 and a communication bus 604, wherein the processor 601, the communication component 603 and the memory 602 communicate with each other via the communication bus 604. The memory 602 stores a program executable by the processor 601, and the processor 601 executes the program stored in the memory 502 to implement the following steps: acquiring a surface electromyographic signal; acquiring an effective surface electromyographic signal from the surface electromyographic signal, wherein the effective surface electromyographic signal is the surface electromyographic signal in a muscle activity time period; extracting signal characteristics of the effective surface electromyographic signals; and acquiring a gesture corresponding to the surface electromyogram signal according to the signal characteristics.
The communication bus 604 mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus 604 may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 6, but this is not intended to represent only one bus or type of bus.
The communication component 603 is used for communication between the above-mentioned electronic device and other devices.
The Memory 602 may include a Random Access Memory (RAM) or a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. Alternatively, the memory may be at least one storage device located remotely from the processor 601.
The Processor 601 may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like, and may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic devices, discrete gates or transistor logic devices, and discrete hardware components.
In one embodiment, the electronic device may be an arm-ring device worn on the forearm of the upper limb proximate one third of the elbow joint. The arm ring device can be provided with a switch, the switch is turned on before the tested object makes a gesture, and the gesture is recognized after the tested object makes the gesture.
The embodiment of the application also provides a human-computer interaction method, which includes the implementation methods described in the above embodiments, and for specific implementation, reference may be made to the description of the embodiment of the gesture recognition method, and repeated details are not repeated.
Specifically, the arm ring equipment can be applied to the bionic artificial limb to serve some special crowds, and the movement of the bionic artificial limb can be controlled by the partial crowds through own will and muscles. The bionic artificial limb is installed on the left leg of a certain tested object as an example for explanation:
first, the relationship between different gestures and leg movements is previously saved in the armring device, for example, a fist-making gesture corresponds to a left leg forward step. When the tested object walks, the right leg needs to move forward one step, and the left leg needs to move forward one step, at the moment, the tested object only needs to hold a fist, and the surface electromyographic signals generated by the forearm muscle group of the tested object are acquired and obtained by the arm ring equipment. And finally, obtaining a command of the left leg for one step ahead according to the corresponding relation between the fist making gesture and the left leg for one step ahead.
The arm ring device sends the instruction to the biomimetic prosthesis after obtaining the instruction of the previous step. The bionic artificial limb receives the instruction sent by the arm ring equipment and finishes the walking of the tested object. Wherein the arm ring equipment is in communication connection with the bionic artificial limb.
In addition, the arm ring device can also be applied to a presentation (PPT), and the relationship between different gestures and execution actions of the PPT can be preset. For example, the gesture of OK corresponds to the action of executing the next page, the gesture of scissors corresponds to the action of executing the auto-play, and the gesture of making a fist corresponds to the action of executing the previous page. The specific implementation is that communication connection is established between the arm ring equipment and the computer, and the computer completes PPT playing according to specific operation instructions sent by the arm ring equipment.
According to the device provided by the embodiment of the application, the surface electromyogram signal is acquired through the processor 601; obtaining an effective surface electromyographic signal from the surface electromyographic signal; extracting signal characteristics of the effective surface electromyographic signals; and directly acquiring the gesture corresponding to the surface electromyogram signal according to the signal characteristics. In the processing process, the effective surface electromyographic signals are obtained from the surface electromyographic signals, and the interference of ineffective surface electromyographic signals is avoided. The device does not need to collect a large number of gesture samples in advance for training, then obtains gestures corresponding to the surface electromyographic signals according to training results, can directly extract signal characteristics of the effective surface electromyographic signals, and quickly and effectively obtains the gestures corresponding to the surface electromyographic signals according to the signal characteristics, and is simple and convenient in operation process and high in efficiency.
In yet another embodiment of the present application, there is also provided a computer-readable storage medium having stored therein a computer program which, when run on a computer, causes the computer to execute the gesture recognition method described in the above embodiment.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wirelessly (e.g., infrared, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that includes one or more of the available media. The available media may be magnetic media (e.g., floppy disks, hard disks, tapes, etc.), optical media (e.g., DVDs), or semiconductor media (e.g., solid state drives), among others.
It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The foregoing are merely exemplary embodiments of the present invention, which enable those skilled in the art to understand or practice the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (11)

1. A gesture recognition method, comprising:
acquiring a surface electromyographic signal;
acquiring an effective surface electromyographic signal from the surface electromyographic signal, wherein the effective surface electromyographic signal is the surface electromyographic signal in a muscle activity time period;
extracting the signal characteristics of the effective surface electromyographic signals;
and acquiring a gesture corresponding to the surface electromyographic signal according to the signal characteristics.
2. The gesture recognition method according to claim 1, wherein the surface electromyography signals are generated by forearm muscle group movements, and the surface electromyography signals are different for different gestures.
3. The gesture recognition method according to claim 2, wherein before obtaining the effective surface electromyography signal from the surface electromyography signal, further comprising:
and filtering noise with preset frequency in the surface electromyographic signal.
4. The gesture recognition method according to claim 3, wherein extracting the signal feature of the effective surface electromyography signal includes:
intercepting the surface electromyographic signals by adopting M data windows to obtain M sections of surface electromyographic signals, wherein the interval between two adjacent sections of surface electromyographic signals is a preset step, one data window comprises n data points, and the preset step comprises M data points;
respectively carrying out the following characteristic extraction operations on each section of the surface electromyographic signals: calculating the absolute value of the difference value of each pair of adjacent data points in the p-th section of the surface electromyogram signal, and summing all the calculated absolute values of the difference values to obtain a summation result, wherein p is greater than or equal to 1 and less than or equal to M;
and summing the summation results of the obtained M sections of the surface electromyographic signals, and taking the obtained result as the signal characteristic of the effective surface electromyographic signal.
5. The gesture recognition method according to claim 4, wherein obtaining the gesture corresponding to the surface electromyography signal according to the signal feature comprises:
determining a forearm muscle group corresponding to the signal characteristic;
and obtaining the gesture corresponding to the surface electromyographic signal according to the determined forearm muscle group and the corresponding relationship between the preset forearm muscle group and the gesture.
6. The gesture recognition method according to claim 3, wherein obtaining an effective surface electromyography signal from the surface electromyography signal comprises:
correcting the surface electromyographic signal to obtain a correction signal;
performing integral operation on the obtained correction signal to obtain an envelope signal;
and taking the surface electromyographic signal corresponding to the envelope signal with the amplitude larger than the preset threshold value as the effective surface electromyographic signal.
7. A human-computer interaction method, comprising: the gesture recognition method of any one of claims 1-6.
8. A gesture recognition apparatus, comprising:
the first acquisition module is used for acquiring a surface electromyographic signal;
the second acquisition module is used for acquiring an effective surface electromyographic signal from the surface electromyographic signal, wherein the effective surface electromyographic signal is a surface electromyographic signal within a muscle activity time period;
the extraction module is used for extracting the signal characteristics of the effective surface electromyographic signals;
and the processing module is used for obtaining the gesture corresponding to the surface electromyogram signal according to the signal characteristics.
9. An electronic device, comprising: a processor and a memory, the processor to execute a program stored in the memory to implement the gesture recognition method of any of claims 1-6.
10. The electronic device of claim 9, wherein the electronic device is an arm ring device configured to be worn on the forearm of the upper limb proximate one third of the elbow joint.
11. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the gesture recognition method according to any one of claims 1 to 6.
CN202010367114.5A 2020-04-30 2020-04-30 Gesture recognition method, man-machine interaction method, device, equipment and storage medium Pending CN113589920A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202010367114.5A CN113589920A (en) 2020-04-30 2020-04-30 Gesture recognition method, man-machine interaction method, device, equipment and storage medium
PCT/CN2021/090715 WO2021219039A1 (en) 2020-04-30 2021-04-28 Gesture recognition method and apparatus, man-machine interaction method and apparatus, and device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010367114.5A CN113589920A (en) 2020-04-30 2020-04-30 Gesture recognition method, man-machine interaction method, device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN113589920A true CN113589920A (en) 2021-11-02

Family

ID=78237601

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010367114.5A Pending CN113589920A (en) 2020-04-30 2020-04-30 Gesture recognition method, man-machine interaction method, device, equipment and storage medium

Country Status (2)

Country Link
CN (1) CN113589920A (en)
WO (1) WO2021219039A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114625257A (en) * 2022-05-16 2022-06-14 浙江强脑科技有限公司 Action recognition method and device based on electromyographic signals
CN114690905A (en) * 2022-03-30 2022-07-01 歌尔股份有限公司 Gesture control method, system and device, wearable product and medium
CN116400812A (en) * 2023-06-05 2023-07-07 中国科学院自动化研究所 Emergency rescue gesture recognition method and device based on surface electromyographic signals

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
NO20211334A1 (en) 2021-11-05 2023-05-08 Elliptic Laboratories Asa Remote presence detection system
CN113970968B (en) * 2021-12-22 2022-05-17 深圳市心流科技有限公司 Intelligent bionic hand action pre-judging method
CN113986017B (en) * 2021-12-27 2022-05-17 深圳市心流科技有限公司 Myoelectric gesture template generation method and device and storage medium
CN114167996B (en) * 2022-02-14 2022-05-17 浙江强脑科技有限公司 Sensor-based action pre-judging method and device and storage medium
CN115131878B (en) * 2022-08-30 2022-11-11 深圳市心流科技有限公司 Method, device, terminal and storage medium for determining motion of intelligent artificial limb

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110420025A (en) * 2019-09-03 2019-11-08 北京海益同展信息科技有限公司 Surface electromyogram signal processing method, device and wearable device
CN110618754A (en) * 2019-08-30 2019-12-27 电子科技大学 Surface electromyogram signal-based gesture recognition method and gesture recognition armband
CN110826625A (en) * 2019-11-06 2020-02-21 南昌大学 Finger gesture classification method based on surface electromyographic signals
CN111026268A (en) * 2019-12-02 2020-04-17 清华大学 Gesture recognition device and method

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6610917B2 (en) * 1998-05-15 2003-08-26 Lester F. Ludwig Activity indication, external source, and processing loop provisions for driven vibrating-element environments

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110618754A (en) * 2019-08-30 2019-12-27 电子科技大学 Surface electromyogram signal-based gesture recognition method and gesture recognition armband
CN110420025A (en) * 2019-09-03 2019-11-08 北京海益同展信息科技有限公司 Surface electromyogram signal processing method, device and wearable device
CN110826625A (en) * 2019-11-06 2020-02-21 南昌大学 Finger gesture classification method based on surface electromyographic signals
CN111026268A (en) * 2019-12-02 2020-04-17 清华大学 Gesture recognition device and method

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114690905A (en) * 2022-03-30 2022-07-01 歌尔股份有限公司 Gesture control method, system and device, wearable product and medium
CN114625257A (en) * 2022-05-16 2022-06-14 浙江强脑科技有限公司 Action recognition method and device based on electromyographic signals
CN114625257B (en) * 2022-05-16 2022-08-16 浙江强脑科技有限公司 Action recognition method and device based on electromyographic signals
CN116400812A (en) * 2023-06-05 2023-07-07 中国科学院自动化研究所 Emergency rescue gesture recognition method and device based on surface electromyographic signals
CN116400812B (en) * 2023-06-05 2023-09-12 中国科学院自动化研究所 Emergency rescue gesture recognition method and device based on surface electromyographic signals

Also Published As

Publication number Publication date
WO2021219039A1 (en) 2021-11-04

Similar Documents

Publication Publication Date Title
CN113589920A (en) Gesture recognition method, man-machine interaction method, device, equipment and storage medium
Fajardo et al. EMG hand gesture classification using handcrafted and deep features
Powar et al. A novel pre-processing procedure for enhanced feature extraction and characterization of electromyogram signals
Phinyomark et al. Wavelet-based denoising algorithm for robust EMG pattern recognition
Oweis et al. ANN-based EMG classification for myoelectric control
Peng et al. Feasibility of NeuCube spiking neural network architecture for EMG pattern recognition
Fatima et al. Towards a low cost Brain-computer Interface for real time control of a 2 DOF robotic arm
Li et al. Wireless sEMG-based identification in a virtual reality environment
Hamedi et al. Facial gesture recognition using two-channel bio-sensors configuration and fuzzy classifier: A pilot study
CN106845348B (en) Gesture recognition method based on arm surface electromyographic signals
Dzitac et al. Identification of ERD using fuzzy inference systems for brain-computer interface
Too et al. Exploring the relation between EMG pattern recognition and sampling rate using spectrogram
Ison et al. Beyond user-specificity for emg decoding using multiresolution muscle synergy analysis
CN111110268B (en) Human body muscle sound signal prediction method based on random vector function connection network technology
Ke et al. Intersected EMG heatmaps and deep learning based gesture recognition
McCool et al. Improved pattern recognition classification accuracy for surface myoelectric signals using spectral enhancement
CN111839847A (en) Myoelectric prosthesis control method, myoelectric prosthesis control device, electronic equipment and computer readable medium
CN111923048A (en) Electromyographic signal classification and exoskeleton robot control method and device
Qiu et al. Elimination of power line interference from ECG signals using recurrent neural networks
Veer A flexible approach for segregating physiological signals
Guerrero et al. Novel electromyography signal envelopes based on binary segmentation
CN110604578A (en) Human hand and hand motion recognition method based on SEMG
Islam et al. Myoelectric pattern recognition performance enhancement using nonlinear features
Silva-Acosta et al. Automatic estimation of continuous elbow flexion–extension movement based on electromyographic and electroencephalographic signals
Li et al. sEMG signal filtering study using synchrosqueezing wavelet transform with differential evolution optimized threshold

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information

Address after: 601, 6 / F, building 2, No. 18, Kechuang 11th Street, Daxing District, Beijing, 100176

Applicant after: Jingdong Technology Information Technology Co.,Ltd.

Address before: 601, 6 / F, building 2, No. 18, Kechuang 11th Street, Daxing District, Beijing, 100176

Applicant before: Jingdong Shuke Haiyi Information Technology Co.,Ltd.

Address after: 601, 6 / F, building 2, No. 18, Kechuang 11th Street, Daxing District, Beijing, 100176

Applicant after: Jingdong Shuke Haiyi Information Technology Co.,Ltd.

Address before: 601, 6 / F, building 2, No. 18, Kechuang 11th Street, Beijing Economic and Technological Development Zone, Beijing 100176

Applicant before: BEIJING HAIYI TONGZHAN INFORMATION TECHNOLOGY Co.,Ltd.

CB02 Change of applicant information