CN114569142A - Gesture recognition method and system based on brain-like calculation and gesture recognition device - Google Patents
Gesture recognition method and system based on brain-like calculation and gesture recognition device Download PDFInfo
- Publication number
- CN114569142A CN114569142A CN202210189979.6A CN202210189979A CN114569142A CN 114569142 A CN114569142 A CN 114569142A CN 202210189979 A CN202210189979 A CN 202210189979A CN 114569142 A CN114569142 A CN 114569142A
- Authority
- CN
- China
- Prior art keywords
- gesture
- data set
- super
- sample
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/389—Electromyography [EMG]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1116—Determining posture transitions
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/389—Electromyography [EMG]
- A61B5/397—Analysis of electromyograms
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/725—Details of waveform analysis using specific filters therefor, e.g. Kalman or adaptive filters
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
- A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Veterinary Medicine (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Public Health (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Artificial Intelligence (AREA)
- Physiology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Psychiatry (AREA)
- Signal Processing (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Dentistry (AREA)
- Evolutionary Computation (AREA)
- Fuzzy Systems (AREA)
- Mathematical Physics (AREA)
- User Interface Of Digital Computer (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a gesture recognition method, a system and a gesture recognition device based on brain-like computing, wherein the method comprises the following steps: acquiring multi-channel electromyographic signals of a user at an arm, and filtering to obtain a first data set; performing segmentation processing to extract effective gesture active segments and acquiring gesture features from the extracted gesture active segments; inputting the test data set into a recognition model; in the recognition model, a first memory is used for mapping the test data set into a super-dimensional vector in a coding mode, similarity calculation is carried out on the super-dimensional vector and a sample super-dimensional vector with the highest similarity, a sample gesture corresponding to the sample super-dimensional vector is used as a prediction gesture, and the prediction gesture is output. The invention can achieve ideal recognition accuracy rate by performing gesture recognition on the myoelectric signals on the surface of the human arm only by learning on a small amount of data, is convenient to calculate, does not need iteration for multiple times, has small training model, and is suitable for directly learning and predicting in hardware local.
Description
Technical Field
The invention relates to the field of gesture recognition, in particular to a gesture recognition method and system based on brain-like calculation and a gesture recognition device.
Background
In the concept of metastic space, gesture recognition is an important medium for human-computer interaction. The gesture recognition technology can provide technical support for man-machine interaction in the meta universe, and the actions and the tracks of human hands in reality are displayed in a virtual space in real time to interact with the virtual world.
In the prior art, most gesture recognition technologies are based on a computer vision algorithm, and the gesture recognition effect becomes poor or even can not be recognized directly under the condition that light is dim or hands are shielded by the method. The gesture recognition method is often used with other peripheral devices, such as a handle and other media devices, to obtain feedback information in the virtual world.
In addition, the gesture recognition technology based on deep learning or machine learning algorithm usually needs a large amount of data to perform learning training, iteration is needed for multiple times, and the trained algorithm model needs a large storage space and can only be stored in a large-scale PC (personal computer) terminal, so that learning and prediction on a single chip microcomputer or FPGA (field programmable gate array) hardware are difficult to perform locally, and the application of the gesture recognition technology in the aspect of a microprocessor is greatly limited.
Therefore, there is a need for a gesture recognition scheme that can learn and predict locally on the hardware to solve the above problems.
Disclosure of Invention
In view of this, the invention provides a gesture recognition method, system and device based on brain-like computing, and the specific scheme is as follows:
a gesture recognition method based on brain-like calculation comprises the following steps:
acquiring multi-channel electromyographic signals of a user at an arm to obtain a multi-channel electromyographic data set related to gestures;
filtering the multi-channel electromyographic data set to obtain a first data set;
the first data set is processed in a segmented mode to extract effective gesture active sections, and gesture features are obtained from the extracted gesture active sections to obtain a test data set;
acquiring a trained recognition model for the user, wherein the recognition model is an algorithm model constructed based on super-dimensional calculation, the test data set is input into the recognition model for gesture prediction, a first memory and a second memory are preset in the recognition model, and the second memory comprises a sample super-dimensional vector carrying a sample gesture;
in the recognition model, the first memory is used for encoding and mapping the test data set into a super-dimensional vector, similarity calculation is carried out on the super-dimensional vector and the sample super-dimensional vector, the sample super-dimensional vector with the highest similarity is found, and a sample gesture corresponding to the sample super-dimensional vector is used as a prediction gesture and is output.
In a specific embodiment, the segmentation process specifically includes:
before collecting a multi-channel electromyographic data set of a user gesture, collecting a multi-channel electromyographic signal of a user in a resting state in advance to obtain a resting state threshold value;
and extracting effective gesture active segments based on the resting state threshold.
In a specific embodiment, the training process of the recognition model includes:
acquiring a multichannel electromyographic signal of a certain user at an arm to acquire a sample data set of a plurality of sample gestures of the user;
filtering the sample data set to obtain a first training data set;
processing the first training data set in a segmentation mode to extract effective gesture active segments, extracting gesture features from the extracted gesture active segments, and marking corresponding gesture categories to obtain training set data;
constructing an initial algorithm model based on super-dimensional calculation, and setting the data dimension of the model as D;
respectively initializing a first memory and a second memory in the initial algorithm model;
inputting the training data set into the initial algorithm model for training to obtain a sample super-dimensional vector, and storing the sample super-dimensional vector into the second memory;
and after the training is finished, outputting an algorithm model to obtain an identification model for the user.
In a specific embodiment, the electromyographic signals of the user are collected through a multi-channel bracelet arranged in an electrode array.
In a specific embodiment, the electrode array arrangement in the multi-channel bracelet comprises a 4 × 8 array or a 4 × 16 array.
A brain-like computation based gesture recognition system comprising the following:
the acquisition unit is used for acquiring the electromyographic signals of the user at the arm to obtain a multi-channel electromyographic data set related to the gesture;
the filtering unit is used for carrying out filtering processing on the multi-channel electromyographic data set to obtain a first data set;
the preprocessing unit is used for processing the first data set in a segmentation mode to extract an effective gesture active segment and acquiring gesture features from the extracted gesture active segment to obtain a test data set;
the model acquisition unit is used for acquiring a trained recognition model for the user, the recognition model is an algorithm model constructed based on super-dimensional calculation, the test data set is input into the recognition model for gesture prediction, a first memory and a second memory are preset in the recognition model, and the second memory comprises a sample super-dimensional vector carrying a sample gesture;
and the prediction unit is used for mapping the test data set into a super-dimensional vector in the identification model by the first memory, carrying out similarity calculation with the sample super-dimensional vector, searching the sample super-dimensional vector with the highest similarity, and outputting a sample gesture corresponding to the sample super-dimensional vector as a prediction gesture.
In one embodiment, the pre-processing unit comprises:
before collecting a multi-channel electromyographic data set of a user gesture, collecting a multi-channel electromyographic signal of a user in a resting state in advance to obtain a resting state threshold value;
and extracting effective gesture active segments based on the resting state threshold.
In a specific embodiment, the system further comprises a model training unit;
the model training unit specifically comprises:
acquiring a multi-channel electromyographic signal of a certain user at an arm to acquire a sample data set of a plurality of sample gestures of the user;
filtering the sample data set to obtain a first training data set;
processing the first training data set in a segmented mode to extract an effective gesture active segment, extracting gesture features from the gesture active segment, and marking corresponding gesture categories to obtain training set data;
constructing an initial algorithm model based on super-dimensional calculation, and setting the data dimension of the model as a super-dimensional vector;
respectively initializing a first memory and a second memory in the initial algorithm model;
inputting the training data set into the initial algorithm model for training to obtain a sample super-dimensional vector, and storing the sample super-dimensional vector into the second memory;
and after the training is finished, outputting an algorithm model to obtain an identification model for the user.
In a specific embodiment, the collecting unit collects the electromyographic signals of the user through a multi-channel bracelet arranged in an electrode array;
the electrode array arrangement form in the multi-channel bracelet comprises a 4 x 8 array or a 4 x 16 array.
A gesture recognition apparatus comprising:
the acquisition module is used for acquiring multi-channel electromyographic signals at the arm of the user;
the main control module is integrated on the FPGA, is electrically connected with the acquisition module, is used for controlling the acquisition module and executes the gesture recognition method based on the brain-like calculation.
Has the advantages that: the invention provides a gesture recognition method, a system and a gesture recognition device based on brain-like calculation, which can avoid the condition of poor recognition effect caused by light and shading problems by performing gesture recognition on the myoelectric signals on the surfaces of human arms, and have very high gesture recognition accuracy based on the myoelectric signals. The scheme can achieve ideal recognition accuracy rate only by learning on a small amount of data; and the method is convenient to calculate, does not need to iterate for multiple times, has a small training model, and is suitable for directly learning and predicting in hardware local. The gesture recognition device integrates the gesture recognition method into hardware local areas such as an FPGA (field programmable gate array), and is convenient for learning and predicting directly on the hardware local areas.
Drawings
FIG. 1 is a schematic flow chart illustrating a gesture recognition method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating a gesture recognition method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an electromyographic signal waveform according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a waveform of a segmented electromyographic signal according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a gesture recognition system according to an embodiment of the present invention.
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Reference numerals are as follows: 1-a collecting unit; 2-a filtering unit; 3-a pretreatment unit; 4-a model acquisition unit; 5-a prediction unit; 6-model training unit.
Detailed Description
Hereinafter, various embodiments of the present disclosure will be described more fully. The present disclosure is capable of various embodiments and of modifications and variations therein. However, it should be understood that: there is no intention to limit the various embodiments of the present disclosure to the specific embodiments disclosed herein, but rather, the disclosure is to cover all modifications, equivalents, and/or alternatives falling within the spirit and scope of the various embodiments of the present disclosure.
The invention provides a gesture recognition method, a system and a gesture recognition device based on brain-like calculation, which can avoid the situation of poor recognition effect caused by light and shielding problems by performing gesture recognition on the myoelectric signals on the surfaces of human arms, have very high gesture recognition accuracy based on the myoelectric signals, are suitable for providing a reliable and convenient gesture recognition technology for the Yuanzhou space, and achieve a complete closed-loop interaction effect.
The scheme of the invention can achieve ideal recognition accuracy rate only by learning on a small amount of data; and the calculation is convenient, iteration is not needed for multiple times, the training model is small, and the method is suitable for directly learning and predicting in hardware local.
The terminology used in the various embodiments of the disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the various embodiments of the disclosure. As used herein, the singular forms are intended to include the plural forms as well, unless the context clearly indicates otherwise. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of the disclosure belong. The terms (such as those defined in commonly used dictionaries) should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein in various embodiments.
Example 1
The embodiment 1 of the invention discloses a gesture recognition method based on brain-like calculation, which carries out gesture recognition prediction based on human arm muscle electrical signals. The flow block diagram of the gesture recognition method is shown in the attached figure 1, and the specific scheme is as follows:
a gesture recognition method based on brain-like calculation comprises the following steps:
101. acquiring multi-channel electromyographic signals of a user at an arm to obtain a multi-channel electromyographic data set related to gestures;
102. filtering the multi-channel electromyography data set to obtain a first data set;
103. the first data set is processed in a segmented mode to extract effective gesture active sections, and gesture features are obtained from the extracted gesture active sections to obtain a test data set;
104. acquiring a trained recognition model for the user, inputting a test data set into the recognition model for gesture prediction, wherein a first memory and a second memory are preset in the recognition model, and the second memory comprises a sample super-dimensional vector carrying a sample gesture;
105. in the recognition model, a first memory is used for mapping the test data set into a super-dimensional vector in a coding mode, similarity calculation is carried out on the super-dimensional vector and a sample super-dimensional vector with the highest similarity, a sample gesture corresponding to the sample super-dimensional vector is used as a prediction gesture, and the prediction gesture is output.
The principle schematic diagram of the gesture recognition method is shown in the specification and the attached figure 2. The scheme of the embodiment is that the gesture recognition is carried out based on the electric signals of the muscle of the human arm. Compared with the traditional gesture recognition scheme based on the visual algorithm, the scheme is not affected by the problems of light, shielding and the like, and the recognition effect is stable. Each gesture can generate a corresponding electromyographic signal, so that the gesture recognition accuracy based on the electromyographic signals is high.
In addition, because different people make the same gesture, and myoelectric data are different, the gesture recognition method of this embodiment needs to perform corresponding training for each user when in use, so as to obtain a recognition model for each user. Different users can recognize different data of the model, and training and learning are performed in a targeted manner, so that prediction is more accurate.
In this embodiment, the myoelectric signal is collected using a flexible multi-channel bracelet. Place flexible multichannel bracelet in user's arm position, gather the N passageway flesh electricity data set of relevant gesture, to the person of wearing, the collection of flexible multichannel bracelet all is very convenient with wearing. Wherein, the electrodes in the multi-channel bracelet are arranged according to a specific array. Preferably, the electrode array arrangement in the multi-channel bracelet includes a 4 × 8 array, a 4 × 16 array or other array forms. The flexible electrodes need more electrode arrangement to acquire more electromyographic data, and the more the electrodes, the more the data density information. Too few electrodes obtain too little data to accurately recognize gestures. Too many electrodes not only increase the acquisition cost, but also increase the data volume for subsequent data processing. The 4 × 8 array and the 4 × 16 array selected in this embodiment can be adapted to most of the acquisition situations.
Specifically, the collected multi-channel electromyography data set is subjected to filtering processing, including high-low pass filtering and 50Hz power frequency notch filtering, so as to remove other noises and obtain a first data set. When the algorithm is trained, the filtering processing is also carried out by adopting the scheme. An image of the electromyographic signals acquired by the flexible multi-channel bracelet after filtering is shown in figure 3 in the specification.
The segmentation process specifically includes: acquiring myoelectric signals of a user in a resting state in advance before acquiring a multi-channel myoelectric data set of gestures of the user to obtain a resting state threshold; and dividing the gesture active segment according to the resting state threshold value. An exemplary graph of the waveform of fig. 3 after being subjected to the segmentation processing is shown in fig. 4 of the specification. The electromyographic signals of a person in a resting state are obviously different from those in a non-resting state, and the embodiment excludes data of the user in the resting state by acquiring a resting state threshold value and only retains the electromyographic signals in the non-resting state.
The acquisition process of the gesture features comprises the following steps: obtaining an effective gesture activity section to obtain a first gesture activity section; and then calculating the gesture characteristics of each channel in the first gesture activity section. The gesture features can be embodied by the average value of each channel data and the like.
And carrying out segmentation processing on the first data set, and then extracting gesture features. In one embodiment, the first data set may also be labeled with a corresponding gesture tag and compared to the predicted gesture to verify whether the prediction is correct. The sequence steps of extracting the gesture features and marking the gesture labels can be set randomly.
In this embodiment, a pre-trained recognition model is used for gesture prediction. Particularly, the recognition mode is an algorithm model constructed based on super dimensional Computing (HDC for short), and the recognition model of the embodiment is obviously different from the traditional AI algorithm based on the neural network. By adopting the HDC algorithm, an ideal recognition result can be achieved by using less data, and the model is light in weight and is suitable for learning and predicting models locally in hardware such as an FPGA (field programmable gate array) and a singlechip.
In the prediction process, the recognition model encodes the test data into a super-dimensional vector (Hyper-vector), and then performs similarity calculation on the super-dimensional vector and the super-dimensional vector in the associative memory to find out similar gesture categories and obtain a final gesture result.
Since the electromyogram signal of each user is different, a corresponding recognition model needs to be customized according to the user. And identifying preset itememory and associative memory in the model. The elementary memory is used as the first memory, and the associative memory is used as the second memory. The recognition model has memory and response, and can realize real perception capability instead of depending on picture learning to improve recognition accuracy. Related sample super-dimensional vectors are stored in the associated memory, each sample super-dimensional vector corresponds to a corresponding sample gesture, and the corresponding gesture can be predicted as long as the matched sample super-dimensional vector is found. The stored sample super-dimensional vectors are equivalent to memory, and corresponding gestures are predicted to be equivalent to perception according to the matched sample super-dimensional vectors.
When the gesture is predicted by adopting the trained recognition model, the item memory acquires data of a corresponding channel in the data set and gesture data to perform coding operation, wherein the coding operation comprises addition operation and matrix multiplication operation, and a super-dimensional vector is obtained. Then, similarity calculation is carried out on the super-dimensional vector and the sample super-dimensional vector of the corresponding gesture in the associated memory, and the sample super-dimensional vector with the highest similarity with the super-dimensional vector is found.
And if the similarity between the super-dimensional vector of the user and a certain sample super-dimensional vector is the highest, taking the sample gesture corresponding to the sample super-dimensional vector as the predicted gesture of the user.
Wherein, the training process of the recognition model comprises the following steps:
201. the method comprises the steps of collecting a multi-channel electromyographic signal of a certain user at an arm, and obtaining a sample data set of a plurality of sample gestures of the user. A flexible multi-channel bracelet is placed at the arm position of a user, and an N-channel myoelectric data set of a preset sample gesture is collected.
202. And filtering the sample data set to obtain a first training data set. The filtering process comprises high-low pass filtering and 50Hz power frequency notch filtering.
203. And processing the first training data set in a segmented mode to extract an effective gesture active segment, extracting gesture features from the gesture active segment, and marking corresponding gesture categories to obtain training set data. The segmentation process, gesture feature extraction and labeling gesture categories are the same as in the previous section.
204. And constructing an initial algorithm model based on super-dimensional calculation, and setting the data dimension of the model as D. And (3) building an identification model, wherein the dimension of a Hyper-vector (Hyper-vector) of the identification model is set to be D, and D is a set parameter and is a vector with a large dimension, so that the model is called a Hyper-dimensional vector. The data type of the recognition model is a super-dimensional vector, and the storage comprises an itememory and an associative memory.
205. And respectively initializing the first memory and the second memory in the initial algorithm model. After the model is built, initializing the itememory and the associative memory of the recognition model respectively. After initializing the associative memory, an initial super-dimensional vector is obtained. During training, both the itemememory and the associationmemory encode data.
206. And inputting the training data set into an initial algorithm model for training to obtain a sample super-dimensional vector carrying the sample gesture, and storing the sample super-dimensional vector into a second memory.
Specifically, the first memory encodes the gesture features and gesture categories of each channel in the training data set to obtain a super-dimensional vector; the second memory presets initial super-dimensional vectors of all gesture categories after initialization; and carrying out coding operation on the super-dimensional vector and the initial super-dimensional vector corresponding to the gesture category to obtain a sample super-dimensional vector carrying the sample gesture. In this embodiment, the encoding operation includes an addition operation, a matrix multiplication operation, and other operations commonly used in the encoding process.
207. After the training is completed, the second memory is processed by e.g. a polarization process to simplify the computational consumption.
208. And outputting the processed algorithm model to obtain an identification model for the user.
The learning method based on the neural network cannot be memorized, so that the model is huge and the data processing is slow. The so-called memory ability of the current artificial intelligence model is processed by using past information, but the memory ability is not real memory ability, for example, when a baseball player prepares to receive a baseball flying from a distance, the baseball player does not receive the baseball through accurate calculation, but utilizes the muscle reaction after years of training, and the AI does not have the natural response ability similar to the human muscle memory.
The gesture recognition method of the embodiment is based on the recognition model, can generate memory, can greatly reduce the calculation requirement, enables the task to be completed more quickly and effectively, and can achieve an ideal result only by needing less data volume. More importantly, the HDC algorithm is light in weight, is suitable for direct learning and prediction on hardware local, and can save a lot of cost in the aspects of data transmission and model storage.
The embodiment provides a gesture recognition method based on brain-like calculation, which can avoid the situation of poor recognition effect caused by light and shading problems by performing gesture recognition on the myoelectric signals on the surfaces of human arms, and has very high gesture recognition accuracy based on the myoelectric signals. The scheme can achieve ideal recognition accuracy rate only by learning on a small amount of data; and the calculation is convenient, iteration is not needed for multiple times, the training model is small, and the method is suitable for directly learning and predicting in hardware local.
Example 2
The embodiment 2 of the invention discloses a gesture recognition system based on brain-like computing, the gesture recognition method of the embodiment 1 is systematized, the specific structure of the system is shown as the attached figure 5 in the specification, and the specific scheme is as follows:
a brain-like computation based gesture recognition system comprising the following:
the system comprises an acquisition unit 1, a gesture recognition unit and a gesture recognition unit, wherein the acquisition unit is used for acquiring multi-channel electromyographic signals of a user at an arm to obtain a multi-channel electromyographic data set related to the gesture; the myoelectric signals are collected using a flexible multi-channel bracelet. Place flexible multichannel bracelet at user's arm position, gather the N passageway flesh electricity data set of relevant gesture, to the person of wearing, the collection of flexible multichannel bracelet all very convenient with wearing. Wherein, the electrodes in the multi-channel bracelet are arranged according to a specific array. Preferably, the electrode array arrangement in the multi-channel bracelet includes a 4 × 8 array, a 4 × 16 array or other array forms.
The filtering unit 2 is used for carrying out filtering processing on the multi-channel electromyographic data set to obtain a first data set;
the preprocessing unit 3 is used for processing the first data set in a segmentation mode to extract an effective gesture active segment, and acquiring gesture features from the extracted gesture active segment to obtain a test data set; the segmentation processing specifically includes: acquiring myoelectric signals of a user in a resting state in advance before acquiring a multi-channel myoelectric data set of gestures of the user to obtain a resting state threshold; and dividing the gesture active segment according to the resting state threshold value. An exemplary graph of the waveform of fig. 3 after being subjected to the segmentation processing is shown in fig. 4 of the specification. The electromyographic signals of a person in a resting state are obviously different from those in a non-resting state, and the embodiment excludes data of the user in the resting state by acquiring a resting state threshold value and only retains the electromyographic signals in the non-resting state.
The model obtaining unit 4 is configured to obtain a trained recognition model for the user, and input the test data set into the recognition model for gesture prediction, where the recognition model is preset with a first memory and a second memory, and the second memory includes a sample super-dimensional vector carrying a sample gesture;
and the prediction unit 5 is used for mapping the test data set into a super-dimensional vector in the identification model by the first memory, carrying out similarity calculation with the sample super-dimensional vector, searching for the sample super-dimensional vector with the highest similarity, and outputting a sample gesture corresponding to the sample super-dimensional vector as a prediction gesture.
Still include model training unit 6, specifically include: acquiring a multi-channel electromyographic signal of a certain user at an arm to acquire a sample data set of a plurality of sample gestures of the user; filtering the sample data set to obtain a first training data set; the first training data set is processed in a segmented mode to extract an effective gesture active segment, gesture features are extracted from the gesture active segment, corresponding gesture categories are marked, and training set data are obtained; constructing an initial algorithm model based on super-dimensional calculation, and setting the data dimension of the model as a super-dimensional vector; respectively initializing a first memory and a second memory in the initial algorithm model; inputting a training data set into an initial algorithm model for training, and performing coding mapping on the training data set by using a first memory and a second memory to obtain a sample super-dimensional vector carrying a sample gesture, and storing the sample super-dimensional vector into the second memory; and after the training is finished, outputting an algorithm model to obtain an identification model for the user.
In this embodiment, a pre-trained recognition model is used for gesture prediction. Particularly, the recognition mode is an algorithm model constructed based on super dimensional Computing (HDC for short), and the recognition model of the embodiment is obviously different from the traditional AI algorithm based on the neural network. By adopting the HDC algorithm, an ideal recognition result can be achieved by using less data, and the model is light in weight and is suitable for learning and predicting models locally in hardware such as an FPGA (field programmable gate array) and a singlechip.
The embodiment discloses a gesture recognition system based on brain-like computing, which systematizes the gesture recognition method of the embodiment 1 to make the gesture recognition system more practical.
Example 3
The embodiment discloses a gesture recognition device, which combines the gesture recognition method based on brain-like computing of the embodiment 1 with an FPGA. The specific scheme is as follows:
a gesture recognition device comprises an acquisition module and a main control module.
And the acquisition module is used for acquiring the multichannel electromyographic signals at the arms of the user. The collection module uses flexible multichannel bracelet to gather the flesh electricity signal. Place flexible multichannel bracelet in user's arm position, gather the N passageway flesh electricity data set of relevant gesture, to the person of wearing, the collection of flexible multichannel bracelet all is very convenient with wearing. Wherein, the electrodes in the multi-channel bracelet are arranged according to a specific array. Preferably, the electrode array arrangement in the multi-channel bracelet comprises a 4 × 8 array or a 4 × 16 array or other array forms.
The main control module is integrated on the FPGA, is electrically connected with the acquisition module, and is used for controlling the acquisition module to acquire signals and executing the gesture recognition method based on the brain-like calculation in the embodiment 1.
A gesture recognition method based on brain-like computation comprises the following steps:
101. acquiring electromyographic signals of a user at an arm to obtain a multi-channel electromyographic data set related to gestures;
102. filtering the multi-channel electromyography data set to obtain a first data set;
103. processing the first data set in a segmentation mode to extract effective gesture active segments, and acquiring gesture features from the extracted gesture active segments to obtain a test data set;
104. acquiring a trained recognition model for the user, inputting a test data set into the recognition model for gesture prediction, wherein a first memory and a second memory are preset in the recognition model, and the second memory comprises a sample super-dimensional vector carrying a sample gesture;
105. in the recognition model, a first memory is used for mapping the test data set into a super-dimensional vector in a coding mode, similarity calculation is carried out on the super-dimensional vector and a sample super-dimensional vector with the highest similarity, a sample gesture corresponding to the sample super-dimensional vector is used as a prediction gesture, and the prediction gesture is output.
The embodiment provides a gesture recognition device, and combines the gesture recognition method based on brain-like computation of the embodiment 1 with an FPGA, so that the gesture recognition method can be directly learned and predicted on hardware local.
The invention provides a gesture recognition method, a system and a gesture recognition device based on brain-like calculation, which can avoid the condition of poor recognition effect caused by light and shading problems by performing gesture recognition on the myoelectric signals on the surfaces of human arms, and have very high gesture recognition accuracy based on the myoelectric signals. The scheme can achieve ideal recognition accuracy rate only by learning on a small amount of data; and the calculation is convenient, iteration is not needed for multiple times, the training model is small, and the method is suitable for directly learning and predicting in hardware local. The gesture recognition method is integrated on hardware local areas such as an FPGA (field programmable gate array), so that learning and prediction can be directly performed on the hardware local areas conveniently.
Those skilled in the art will appreciate that the figures are merely schematic representations of one preferred implementation scenario and that the blocks or flow diagrams in the figures are not necessarily required to practice the present invention. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario may be distributed in the devices in the implementation scenario according to the description of the implementation scenario, or may be located in one or more devices different from the present implementation scenario with corresponding changes. The modules of the implementation scenario may be combined into one module, or may be further split into multiple sub-modules. The above-mentioned invention numbers are merely for description and do not represent the merits of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of the present invention, however, the present invention is not limited thereto, and any variations that can be made by those skilled in the art are intended to fall within the scope of the present invention.
Claims (10)
1. A gesture recognition method based on brain-like calculation is characterized by comprising the following steps:
acquiring multi-channel electromyographic signals of a user at an arm to obtain a multi-channel electromyographic data set related to gestures;
filtering the multi-channel electromyographic data set to obtain a first data set;
the first data set is processed in a segmented mode to extract effective gesture active sections, and gesture features are obtained from the extracted gesture active sections to obtain a test data set;
acquiring a trained recognition model for the user, wherein the recognition model is an algorithm model constructed based on super-dimensional calculation, the test data set is input into the recognition model for gesture prediction, a first memory and a second memory are preset in the recognition model, and the second memory comprises a sample super-dimensional vector carrying a sample gesture;
in the recognition model, the first memory is used for encoding and mapping the test data set into a super-dimensional vector, similarity calculation is carried out on the super-dimensional vector and the sample super-dimensional vector, the sample super-dimensional vector with the highest similarity is found, and a sample gesture corresponding to the sample super-dimensional vector is used as a prediction gesture and is output.
2. The gesture recognition method according to claim 1, wherein the segmentation process specifically comprises:
before collecting a multi-channel electromyographic data set of a user gesture, collecting a multi-channel electromyographic signal of a user in a resting state in advance to obtain a resting state threshold value;
and extracting effective gesture active segments based on the resting state threshold.
3. The gesture recognition method according to claim 1, wherein the training process of the recognition model comprises:
acquiring a multi-channel electromyographic signal of a certain user at an arm to acquire a sample data set of a plurality of sample gestures of the user;
filtering the sample data set to obtain a first training data set;
processing the first training data set in a segmented mode to extract effective gesture active sections, extracting gesture features from the extracted gesture active sections, and marking corresponding gesture categories to obtain training set data;
constructing an initial algorithm model based on super-dimensional calculation, and setting the data dimension of the model as D;
respectively initializing a first memory and a second memory in the initial algorithm model;
inputting the training data set into the initial algorithm model for training to obtain a sample super-dimensional vector, and storing the sample super-dimensional vector into the second memory;
and after the training is finished, outputting an algorithm model to obtain a recognition model for the user.
4. The gesture recognition method according to claim 1, wherein the electromyographic signals of the user are collected through a multi-channel bracelet arranged in an electrode array.
5. The gesture recognition method according to claim 1, wherein the arrangement form of the electrode arrays in the multi-channel bracelet comprises a 4 x 8 array or a 4 x 16 array.
6. A gesture recognition system based on brain-like computation is characterized by comprising the following components:
the acquisition unit is used for acquiring the electromyographic signals of the user at the arm to obtain a multi-channel electromyographic data set related to the gesture;
the filtering unit is used for carrying out filtering processing on the multi-channel electromyographic data set to obtain a first data set;
the preprocessing unit is used for processing the first data set in a segmentation mode to extract an effective gesture active segment and acquiring gesture features from the extracted gesture active segment to obtain a test data set;
the model acquisition unit is used for acquiring a trained recognition model for the user, the recognition model is an algorithm model constructed based on super-dimensional calculation, the test data set is input into the recognition model for gesture prediction, a first memory and a second memory are preset in the recognition model, and the second memory comprises a sample super-dimensional vector carrying a sample gesture;
and the prediction unit is used for mapping the test data set into a super-dimensional vector in the identification model by the first memory, carrying out similarity calculation with the sample super-dimensional vector, searching the sample super-dimensional vector with the highest similarity, and outputting a sample gesture corresponding to the sample super-dimensional vector as a prediction gesture.
7. The gesture recognition system of claim 6, wherein the preprocessing unit comprises:
the method comprises the steps that before a multichannel electromyography data set of user gestures is collected, multichannel electromyography signals of a user in a resting state are collected in advance to obtain a resting state threshold value;
and extracting effective gesture active segments based on the resting state threshold.
8. The gesture recognition system of claim 6, further comprising a model training unit;
the model training unit specifically comprises:
acquiring a multi-channel electromyographic signal of a certain user at an arm to acquire a sample data set of a plurality of sample gestures of the user;
filtering the sample data set to obtain a first training data set;
processing the first training data set in a segmented mode to extract an effective gesture active segment, extracting gesture features from the gesture active segment, and marking corresponding gesture categories to obtain training set data;
constructing an initial algorithm model based on super-dimensional calculation, and setting the data dimension of the model as a super-dimensional vector;
respectively initializing a first memory and a second memory in the initial algorithm model;
inputting the training data set into the initial algorithm model for training to obtain a sample super-dimensional vector, and storing the sample super-dimensional vector into the second memory;
and after the training is finished, outputting an algorithm model to obtain an identification model for the user.
9. The gesture recognition system of claim 6, wherein the collection unit collects electromyographic signals of the user through a multi-channel bracelet arranged in an electrode array;
the electrode array arrangement form in the multi-channel bracelet comprises a 4 x 8 array or a 4 x 16 array.
10. A gesture recognition apparatus, comprising:
the acquisition module is used for acquiring multi-channel electromyographic signals at the arm of the user;
the main control module is integrated on the FPGA, is electrically connected with the acquisition module, and is used for controlling the acquisition module and executing the gesture recognition method based on brain-like calculation according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210189979.6A CN114569142A (en) | 2022-02-28 | 2022-02-28 | Gesture recognition method and system based on brain-like calculation and gesture recognition device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210189979.6A CN114569142A (en) | 2022-02-28 | 2022-02-28 | Gesture recognition method and system based on brain-like calculation and gesture recognition device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114569142A true CN114569142A (en) | 2022-06-03 |
Family
ID=81772277
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210189979.6A Pending CN114569142A (en) | 2022-02-28 | 2022-02-28 | Gesture recognition method and system based on brain-like calculation and gesture recognition device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114569142A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090327171A1 (en) * | 2008-06-26 | 2009-12-31 | Microsoft Corporation | Recognizing gestures from forearm emg signals |
| CN106980367A (en) * | 2017-02-27 | 2017-07-25 | 浙江工业大学 | A kind of gesture identification method based on myoelectricity topographic map |
| CN112732090A (en) * | 2021-01-20 | 2021-04-30 | 福州大学 | Muscle cooperation-based user-independent real-time gesture recognition method |
| WO2022027822A1 (en) * | 2020-08-03 | 2022-02-10 | 南京邮电大学 | Electromyographic signal-based intelligent gesture action generation method |
| CN114081439A (en) * | 2021-10-11 | 2022-02-25 | 浙江柔灵科技有限公司 | Brain-like algorithm for sleep staging by applying prefrontal single-channel electroencephalogram signals |
-
2022
- 2022-02-28 CN CN202210189979.6A patent/CN114569142A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090327171A1 (en) * | 2008-06-26 | 2009-12-31 | Microsoft Corporation | Recognizing gestures from forearm emg signals |
| CN106980367A (en) * | 2017-02-27 | 2017-07-25 | 浙江工业大学 | A kind of gesture identification method based on myoelectricity topographic map |
| WO2022027822A1 (en) * | 2020-08-03 | 2022-02-10 | 南京邮电大学 | Electromyographic signal-based intelligent gesture action generation method |
| CN112732090A (en) * | 2021-01-20 | 2021-04-30 | 福州大学 | Muscle cooperation-based user-independent real-time gesture recognition method |
| CN114081439A (en) * | 2021-10-11 | 2022-02-25 | 浙江柔灵科技有限公司 | Brain-like algorithm for sleep staging by applying prefrontal single-channel electroencephalogram signals |
Non-Patent Citations (1)
| Title |
|---|
| ABBAS RAHIMI等: "Hyperdimensional biosignal processing: A case study for EMG-based hand gesture recognition", 2016 IEEE INTERNATIONAL CONFERENCE ON REBOOTING COMPUTING (ICRC), pages 1 - 8 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Tang et al. | Triple cross-domain attention on human activity recognition using wearable sensors | |
| Kasabov et al. | Evolving spatio-temporal data machines based on the NeuCube neuromorphic framework: Design methodology and selected applications | |
| CN112861604B (en) | Myoelectric action recognition and control method irrelevant to user | |
| CN111126263B (en) | Electroencephalogram emotion recognition method and device based on double-hemisphere difference model | |
| CN113693613B (en) | Electroencephalogram signal classification method, electroencephalogram signal classification device, computer equipment and storage medium | |
| CN111317468A (en) | Electroencephalogram signal classification method and device, computer equipment and storage medium | |
| CN110333783B (en) | Irrelevant gesture processing method and system for robust electromyography control | |
| Shin et al. | Korean sign language recognition using EMG and IMU sensors based on group-dependent NN models | |
| CN111714118A (en) | Brain cognition model fusion method based on ensemble learning | |
| CN111870242A (en) | Intelligent gesture action generation method based on electromyographic signals | |
| US20220012489A1 (en) | Apparatus and method for motor imagery classification using eeg | |
| CN114998983A (en) | Limb rehabilitation method based on augmented reality technology and posture recognition technology | |
| KR20200018868A (en) | Method for Adaptive EEG signal processing using reinforcement learning and System Using the same | |
| CN111753683A (en) | Human body posture identification method based on multi-expert convolutional neural network | |
| Xu et al. | Real-time finger force prediction via parallel convolutional neural networks: a preliminary study | |
| CN113143295A (en) | Equipment control method and terminal based on motor imagery electroencephalogram signals | |
| Sykacek et al. | Probabilistic methods in BCI research | |
| Zea et al. | Real-time hand gesture recognition: a long short-term memory approach with electromyography | |
| CN112884062B (en) | Motor imagery classification method and system based on CNN classification model and generated countermeasure network | |
| CN107766773A (en) | Various dimensions control method and control device based on gesture | |
| US20230101539A1 (en) | Physiological electric signal classification processing method and apparatus, computer device and storage medium | |
| CN114569142A (en) | Gesture recognition method and system based on brain-like calculation and gesture recognition device | |
| CN116524380A (en) | Target detection method based on brain-computer signal fusion | |
| CN114795246A (en) | Brain region local-global attention-based electroencephalogram emotion classification method and system | |
| CN114997229A (en) | Electroencephalogram signal-based motor imagery classification method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |