CN116662773A - Model acquisition system, gesture recognition method, gesture recognition device, apparatus and storage medium - Google Patents

Model acquisition system, gesture recognition method, gesture recognition device, apparatus and storage medium Download PDF

Info

Publication number
CN116662773A
CN116662773A CN202310515750.1A CN202310515750A CN116662773A CN 116662773 A CN116662773 A CN 116662773A CN 202310515750 A CN202310515750 A CN 202310515750A CN 116662773 A CN116662773 A CN 116662773A
Authority
CN
China
Prior art keywords
gesture
current
gesture recognition
recognition model
meta
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310515750.1A
Other languages
Chinese (zh)
Inventor
王友好
王译
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Yinghe Brain Science Co ltd
Original Assignee
Shenzhen Yinghe Brain Science Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Yinghe Brain Science Co ltd filed Critical Shenzhen Yinghe Brain Science Co ltd
Priority to CN202310515750.1A priority Critical patent/CN116662773A/en
Publication of CN116662773A publication Critical patent/CN116662773A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01—Indexing scheme relating to G06F3/01
    • G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Medical Informatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Dermatology (AREA)
  • General Health & Medical Sciences (AREA)
  • Neurology (AREA)
  • Neurosurgery (AREA)
  • User Interface Of Digital Computer (AREA)
  • Image Analysis (AREA)

Abstract

The application relates to the field of electrophysiological signal identification, and discloses a model acquisition system, a gesture identification method, a device, equipment and a storage medium, wherein the model acquisition system comprises: the input unit is used for acquiring current myoelectricity data and gesture labels of a current individual; and a processing unit: under the condition that a personal gesture recognition model does not exist in a current individual, acquiring an initial gesture recognition model, wherein the initial gesture recognition model is generated according to a plurality of element learning training generated based on historical myoelectricity data and gesture labels of a plurality of individuals, a support set of each element learning task is generated based on the historical myoelectricity data and gesture labels of the individuals at earlier time, and a query set is generated based on the historical myoelectricity data and gesture labels of the individuals at later time; and fine-tuning the initial gesture recognition model according to the current myoelectricity data and the gesture label of the current individual to obtain a current gesture recognition model. And constructing a meta-learning task by utilizing myoelectricity data and gesture labels based on the time sequence, and enhancing the robustness of the current gesture recognition model.

Description

Model acquisition system, gesture recognition method, gesture recognition device, apparatus and storage medium
Technical Field
The embodiment of the application relates to the field of electrophysiological signal identification, in particular to a model acquisition system, a gesture identification method, a gesture identification device, gesture identification equipment and a storage medium.
Background
Electromyography (EMG) is an electrophysiological signal obtained by recording the firing of motor neurons of the muscle via electromyographic electrodes. It contains rich neural information that can be decoded into a number of limb-related activity signals. The acquisition of the EMG has the characteristics of no harm to human bodies, easy acquisition, easy operation and the like, and has good application prospect in the field of gesture recognition and classification, in particular to medical treatment, entertainment and other industries related to machine control.
The traditional myoelectric gesture recognition uses a large amount of single individual pre-collected data to train a corresponding pattern recognition classifier, and a common classifier model comprises a support vector machine, a random forest and other models of traditional machine learning, and a model constructed based on a convolutional neural network, a cyclic neural network and other methods in deep learning. At present, myoelectric signals have a plurality of inherent problems in gesture recognition application, and as the muscle nerve distribution of an individual has specificity and is influenced by factors such as skin impedance, muscle structure and the like which are highly dependent on individual characteristics, the myoelectric signals have obvious individual differences, and the classification accuracy depends on a large amount of data training of the individual. In addition, due to the influence of factors such as electrode displacement and environment, the change of the electromyographic signal characteristics is very rapid, so that even a model trained on a large amount of individual data still has the accuracy gradually reduced along with the increase of the service time. Based on the reasons, the model trained based on a large amount of individual data often causes the problems of over-fitting, unstable performance of the model and the like due to factors such as limited data collection amount, waste of priori knowledge and the like.
In order to solve the problem of priori knowledge waste, the method of transfer learning provides a scheme, but the transfer learning faces the problem of catastrophic forgetting, and in order to ensure model accuracy, a large amount of individual data is still required for training, model training and model application scene switching are still complex, and training and adjustment efficiency is low.
Disclosure of Invention
The embodiment of the application aims to provide a model acquisition system, a gesture recognition method, a gesture recognition device, equipment and a storage medium, and aims to accurately acquire a current gesture recognition model adapting to the characteristics of a current individual through a small amount of individual data by using a meta-learning mode, so that the training and adjustment efficiency of the gesture recognition model is improved, and further, the accurate recognition of the current individual gesture is completed.
To solve one or more of the above problems and to achieve the above objects, an embodiment of the present application provides a model acquisition system including: the input unit is used for acquiring current myoelectricity data of a current individual and gesture labels, wherein the gesture labels comprise gesture information; a processing unit configured to: under the condition that a personal gesture recognition model does not exist in a current individual, acquiring an initial gesture recognition model, wherein the initial gesture recognition model is generated by performing meta-learning training according to a plurality of meta-learning tasks generated based on historical myoelectricity data and gesture labels of a plurality of individuals, and each meta-learning task is generated based on the historical myoelectricity data and gesture labels of the same individual; and fine-tuning the initial gesture recognition model according to the current myoelectricity data and the gesture label of the current individual to obtain the current gesture recognition model of the current individual.
To solve one or more of the above problems and achieve the above objects, an embodiment of the present application provides a model acquisition method, including: acquiring current myoelectricity data and a gesture label of a current individual, wherein the gesture label comprises gesture information; under the condition that a personal gesture recognition model does not exist in the current individual, an initial gesture recognition model is obtained, wherein the initial gesture recognition model is generated by meta-learning training according to a plurality of meta-learning tasks generated based on historical myoelectricity data and gesture labels of a plurality of individuals, and each meta-learning task is generated based on the historical myoelectricity data and gesture labels of the same individual; and fine-tuning the initial gesture recognition model according to the current myoelectricity data and the gesture label of the current individual to obtain the current gesture recognition model of the current individual.
To solve one or more of the above problems and achieve the above objects, an embodiment of the present application provides a gesture recognition method, including: acquiring a current gesture recognition model of a current individual through the model acquisition system; acquiring real-time myoelectricity data of the current individual; and acquiring the gesture of the current individual according to the real-time myoelectricity data through the current gesture recognition model.
To solve one or more of the above problems and achieve the above objects, an embodiment of the present application further provides a gesture recognition apparatus, including: the first acquisition module is used for acquiring a current gesture recognition model of a current individual through the model acquisition system; the second acquisition module is used for acquiring the real-time myoelectricity data of the current individual; and the recognition module is used for acquiring the gesture of the current individual according to the real-time myoelectricity data through the current gesture recognition model.
To solve one or more of the above problems and achieve the above objects, an embodiment of the present application further provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the gesture recognition method or the model acquisition method.
To solve one or more of the above problems and achieve the above objects, embodiments of the present application further provide a computer-readable storage medium storing a computer program that when executed by a processor, implements the above gesture recognition method or performs the above model acquisition method.
According to the model acquisition system provided by the application, under the condition that a personal gesture recognition model does not exist in a current individual, current myoelectricity data and gesture information of the current individual are acquired through an input unit, an initial gesture recognition model is subjected to fine adjustment according to the current myoelectricity data and gesture labels of the current individual, so that the current gesture recognition model suitable for the current individual is obtained, the initial gesture recognition model is generated through meta-learning training according to a plurality of meta-learning tasks generated based on historical myoelectricity data and gesture labels of a plurality of individuals, and each meta-learning task is generated based on the historical myoelectricity data and gesture labels of the same individual. According to the method, a plurality of meta-learning tasks respectively based on the historical myoelectricity data and the gesture labels of the same individual are formed according to the historical myoelectricity data and the gesture labels of the plurality of individuals, an initial gesture recognition model which is adaptive to the population and has good generalization capability is obtained in a meta-learning training mode, when the method is applied to a specific current individual, the initial gesture recognition model is finely adjusted according to the current myoelectricity data and the gesture labels of the current individual, so that the initial gesture recognition model is accurately and efficiently finely adjusted under the support of a small amount of individual data, the current gesture recognition model adaptive to the current individual is obtained, and the accuracy of gesture recognition is further ensured.
Drawings
One or more embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements, and in which the figures of the drawings are not to be taken in a limiting sense, unless otherwise indicated.
FIG. 1 is a schematic diagram of a model acquisition system in an embodiment of the application;
FIG. 2 is a schematic diagram of a meta learning task structure in an embodiment of the present application;
FIG. 3 is a schematic diagram of a meta learning training process in an embodiment of the present application;
FIG. 4 is a flow chart of a model acquisition method in another embodiment of the application;
FIG. 5 is a flow chart of a gesture recognition method in another embodiment of the present application;
FIG. 6 is a schematic diagram of a gesture recognition apparatus according to another embodiment of the present application;
fig. 7 is a schematic structural diagram of an electronic device according to another embodiment of the present application.
Detailed Description
As known from the background art, the traditional model training mode has the problems of over-fitting and unstable model performance, and the model obtained through transfer learning has low training and adjusting efficiency, and needs to rely on a large amount of individual data to ensure the recognition accuracy. Therefore, how to provide a model acquisition method with high training and adjustment efficiency to ensure accurate and efficient recognition of gestures, which can meet the requirements of efficient switching of different application scenarios, is an urgent problem to be solved.
In order to solve the above-described problems, an embodiment of the present application provides a model acquisition system including: the input unit is used for acquiring current myoelectricity data of a current individual and gesture labels, wherein the gesture labels comprise gesture information; a processing unit configured to: under the condition that a personal gesture recognition model does not exist in a current individual, acquiring an initial gesture recognition model, wherein the initial gesture recognition model is generated by performing meta-learning training according to a plurality of meta-learning tasks generated based on historical myoelectricity data and gesture labels of a plurality of individuals, and each meta-learning task is generated based on the historical myoelectricity data and gesture labels of the same individual; and fine tuning the initial gesture recognition model according to the current myoelectricity data and the gesture label of the current individual to obtain the current gesture recognition model of the current individual.
Before a current gesture recognition model of a current individual is obtained, the model obtaining system obtains historical myoelectricity data and gesture labels according to a plurality of individuals, forms a meta-learning task based on the historical myoelectricity data and the gesture labels of each individual, and generates a relatively universal initial gesture recognition model in a meta-learning training mode. When the current gesture recognition model of the current individual is acquired under the condition that the personal gesture recognition model does not exist in the current individual, acquiring current myoelectricity data and gesture labels according to the current individual through an input unit, and fine-tuning an initial gesture recognition model according to the acquired current myoelectricity data and gesture labels to acquire the current gesture recognition model applicable to the current individual. The method comprises the steps of obtaining a plurality of meta-learning tasks according to historical myoelectricity data and gesture labels of a plurality of individuals, generating each meta-learning task based on the historical myoelectricity data and gesture labels of the same individual, combining an initial gesture recognition model which is obtained in a meta-learning training mode, is adaptive to people and has good generalization capability, and fine-tuning the initial gesture recognition model according to current myoelectricity data and gesture labels of the current individuals when the initial gesture recognition model is applied to the specific current individuals. Therefore, under the support of a small amount of individual data, the initial gesture recognition model is accurately and efficiently subjected to fine adjustment, the current gesture recognition model adapting to the current individual is obtained, and further the accuracy of gesture recognition is ensured.
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the embodiments of the present application will be described in detail below with reference to the accompanying drawings. However, it will be understood by those of ordinary skill in the art that in various embodiments of the present application, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, the claimed technical solution of the present application can be realized without these technical details and various changes and modifications based on the following embodiments. The following embodiments are divided for convenience of description, and should not be construed as limiting the specific implementation of the present application, and the embodiments can be mutually combined and referred to without contradiction.
Implementation details of the model acquisition system described in the present application will be specifically described below with reference to specific embodiments, and the following details are provided only for facilitating understanding, and are not essential to the present solution.
A first aspect of an embodiment of the present application provides a model acquisition system, as shown in fig. 1, including an input unit 101 and a processing unit 102:
the input unit 101 is configured to obtain current myoelectricity data of a current individual and a gesture tag, where the gesture tag includes gesture information.
Specifically, when a current gesture recognition model with a better recognition effect for a current individual is acquired, firstly, myoelectricity data of the current individual in a current state and gesture labels corresponding to gestures of the individual during myoelectricity data acquisition are acquired and recorded through the input unit 101, and the current myoelectricity data and the gesture labels of the current individual are acquired. The gesture labels are data labels created when the gestures are stored, and different gesture labels correspond to different gestures, for example, gesture label 1 corresponds to an individual vertical index finger, gesture label 2 corresponds to an individual vertical middle finger, gesture label 3 corresponds to an individual fist making and the like. In specific applications, the meaning of each label can be set according to needs, and the specific setting and meaning of the label are not limited in this embodiment.
For example, when current myoelectricity data of a current individual is collected, the input unit 101 automatically prompts the current individual in a voice or text mode, gestures corresponding to a plurality of preset gesture labels are made according to a certain sequence in a specified time, myoelectric signals generated by a muscle motor neuron under different gestures made by the current individual are recorded through myoelectricity electrodes, and the myoelectric signals and the gesture labels are coded and stored according to time intervals corresponding to the gestures. The method has the advantages that the current myoelectricity data and the gesture labels of the current individual in the current state are obtained, so that the initial gesture recognition model can be finely adjusted conveniently, and the current gesture recognition model aiming at the current state of the current individual can be obtained efficiently and accurately.
A processing unit 102 configured to: under the condition that a personal gesture recognition model does not exist in a current individual, acquiring an initial gesture recognition model, wherein the initial gesture recognition model is generated by performing meta-learning training according to a plurality of meta-learning tasks generated based on historical myoelectricity data and gesture labels of a plurality of individuals, and each meta-learning task is generated based on the historical myoelectricity data and gesture labels of the same individual; and fine tuning the initial gesture recognition model according to the current myoelectricity data and the gesture label of the current individual to obtain the current gesture recognition model of the current individual.
Specifically, before or after the processing unit 102 obtains the current myoelectricity data and the gesture label of the current individual through the input unit 101, according to a preconfigured program, whether the personal gesture recognition model of the current individual is stored in advance is detected according to the identity of the current individual, and if it is detected that the personal gesture recognition model does not exist in the current individual, the processing unit 102 obtains the initial gesture recognition model. The initial gesture recognition model is generated by performing meta-learning training according to a plurality of meta-learning tasks generated based on the historical myoelectricity data and gesture labels of a plurality of individuals, and each meta-learning task is generated based on the historical myoelectricity data and gesture labels of the same individual. Upon acquiring the current gesture recognition model for the current individual, the processing unit 102 may read the pre-trained initial gesture recognition model from the memory address where the initial gesture recognition model is stored by way of communication. The processing unit 102 may also read the historical myoelectricity data and the gesture labels of a plurality of individuals from the designated storage address, generate a plurality of meta-learning tasks based on the historical myoelectricity data and the gesture labels of each individual, and then perform meta-learning training on an untrained meta-learner to generate an initial gesture recognition model. The processing unit 102 may also directly read a plurality of meta-learning tasks pre-generated based on the historical myoelectricity data of different individuals and the gesture labels to train the untrained meta-learner to generate an initial gesture recognition model. The specific acquisition mode of the initial gesture recognition model is not limited in this embodiment.
After acquiring the current myoelectricity data and the gesture label of the current individual and the initial gesture recognition model, the processing unit 102 performs fine adjustment on the initial gesture recognition model for the current individual according to the current myoelectricity data and the gesture label of the current individual, for example, by taking the current myoelectricity data as input information and the gesture corresponding to the gesture label as supervision information, performs supervised learning on the initial gesture recognition model, so that the initial gesture recognition model has better pertinence and adaptability to the current individual, and takes the initial gesture recognition model after fine adjustment as the current gesture recognition model of the current individual. By fine tuning the initial gesture recognition model for the current individual according to the current myoelectricity data and the gesture label of the current individual, the pertinence of the obtained current gesture recognition model is improved, and the accuracy of gesture recognition for the current individual is ensured.
Further, the initial gesture recognition model may be generated by training in the following manner: training a network model with gradient reverse transfer based on historical myoelectricity data and gesture labels of a plurality of individuals to obtain an original gesture recognition model; generating a plurality of meta-learning tasks according to the historical myoelectricity data and gesture labels of a plurality of individuals, wherein each meta-learning task is generated based on the historical myoelectricity data and gesture labels of the same individual; and performing meta learning training on a meta learner based on the original gesture recognition model by utilizing each meta learning task to acquire the initial gesture recognition model.
Specifically, when the processing unit 102 or other model training device trains an initial gesture recognition model according to the historical myoelectricity data and gesture labels of a plurality of individuals, the processing unit may train the network model with gradient reverse transfer based on the historical myoelectricity data and gesture labels of the plurality of individuals, the training mode includes supervised training, unsupervised training, and the like, to obtain an original gesture recognition model, and then form a plurality of meta-learning tasks from the myoelectricity data set formed by the historical myoelectricity data and gesture labels of the plurality of individuals according to preset constraint conditions, where the data included in each meta-learning task is from the same individual. For example, a myoelectricity data set composed of a plurality of individual historical myoelectricity data and gesture labels is randomly sampled to extract the plurality of individual historical myoelectricity data and gesture labels. One or more meta-learning tasks are then generated based on the historical myoelectric data and gesture labels for each individual acquired by random sampling. As such, each meta-learning task of the number of tasks contains data from the same individual. And then, according to the generated multiple meta learning tasks, performing meta learning training on a meta learner based on the original gesture recognition model by utilizing each meta learning task to optimize parameters of the meta learner, so as to obtain the initial gesture recognition model. Here, the obtaining of the original gesture recognition model and the forming of the meta learning task may be performed simultaneously, or the meta learning task may be formed first, and then the original gesture recognition model may be obtained. The myoelectricity data and gesture labels of a plurality of individuals are randomly screened out according to preset constraint conditions by acquiring the myoelectricity data and gesture labels of the plurality of individuals, one or more meta-learning tasks are generated based on the myoelectricity data and gesture labels of each individual, and meta-learning training is carried out on a meta-learner based on an original gesture recognition model by utilizing the generated meta-learning tasks, so that the trained initial gesture recognition model has good generalization capability for different individuals and newly added individuals.
It is worth mentioning that the network model with gradient reverse transfer is any one of a convolutional neural network CNN model, a long-short-term neural network LSTM model and a cyclic neural network RNN model.
For example, a neural network model optimized based on a random gradient descent optimization algorithm may be used as a base model, e.g., a 3-layer convolutional neural network model of an Adam optimizer may be used as a base model, and model training may be performed in conjunction with a preset loss function, e.g., a cross entropy loss function. And taking the historical myoelectricity data in the myoelectricity data set as input, taking gestures of the historical myoelectricity data as supervision signals, calculating a loss value by using a preset loss function, optimizing model parameters based on gradient updating of the loss value, and completing training of the model to obtain an original gesture recognition model.
In addition, after current myoelectricity data and gesture labels of a current individual are collected each time, a myoelectricity data set containing historical myoelectricity data and gesture labels of a plurality of individuals can be subjected to data expansion, and then an original gesture model is retrained based on the expanded myoelectricity data set, so that the generalization effect of the original gesture recognition model is improved. The retraining may be performed periodically according to a preset period, or may be performed after the myoelectric dataset is expanded to a certain extent, which is not limited in this embodiment.
In another example, the meta-learning tasks include a Support Set (Support Set) and a Query Set (Query Set), and the meta-learning training of the meta-learner based on the original gesture recognition model with each meta-learning task includes: for each meta learning task, firstly assigning parameters of a meta learner to a base learner, and then training the base learner according to myoelectricity data in a support set and gesture labels corresponding to the myoelectricity data so as to optimize the parameters of the base learner; acquiring a prediction error of a gesture label prediction result of the base learner after parameter optimization according to myoelectricity data in the query set and gesture labels corresponding to the myoelectricity data; and according to the prediction error, carrying out gradient update on the element learner to optimize parameters of the element learner.
Specifically, the meta-learning task includes a support set and a query set, where the support set has an important set of N-way K-shot, i.e., N types of samples in the support set, each type of sample has K marked data, and the query set may include N types of samples, each type of sample has K marked data, where N, K, N and K are positive integers. Taking the example that 6 gesture tags are included in the support set, 3 myoelectricity data are correspondingly collected by each gesture tag, two gesture tags are correspondingly collected by each gesture tag, in this embodiment, reference may be made to fig. 2 for the metadata learning task generated according to the historical myoelectricity data and gesture tags of each individual, the obtained myoelectricity data are represented by the marked data, and the samples are gesture tags of each type.
In the process of performing meta learning training on a meta learner based on an original gesture recognition model according to each meta learning task, for each meta learning task, copying parameters in the meta learner through the base learner, then reading gesture labels corresponding to myoelectricity data and myoelectricity data in a supporting set, predicting the gesture labels by using the base learner by taking myoelectricity data as input, and performing gradient updating according to a loss value between a prediction result and the gesture labels corresponding to the myoelectricity data so as to optimize the parameters in the base learner. And then reading myoelectricity data in the query set and gesture labels corresponding to the myoelectricity data, taking the myoelectricity data corresponding to each gesture label in the query set as input, predicting the gesture labels by using the parameter-optimized base learner, and acquiring a prediction error of a gesture label prediction result output by the parameter-optimized base learner according to the gesture labels corresponding to the myoelectricity data in the query set. And then, according to the obtained prediction error, carrying out gradient update on the element learner to obtain the new optimized parameter, and taking the new parameter as the parameter of the element learner. And in the next training condition, the parameters optimized by the meta learner are copied into the base learner again. And performing multiple rounds of training until the prediction error of the gesture label is not reduced, stopping training, and taking the meta-learner after parameter optimization as an initial gesture recognition model. The required initial gesture recognition model is accurately and efficiently obtained based on the original gesture recognition model by carrying out double-layer cyclic training and parameter optimization according to the support set and the query set in the meta-learning task. And the obtained initial gesture recognition model can adapt to a specific task only by carrying out fine adjustment on a small number of gradient update steps and a meta-learning task related to the specific task.
In addition, the preset constraint condition can also comprise that in the process of generating the meta-learning task, the myoelectricity data and the gesture label adopted in the process of generating the support set can be obtained from the data with relatively front time in the myoelectricity data of the individual, and in the process of generating the query set, the myoelectricity data and the gesture label adopted in the process of obtaining the data with relatively rear time in the myoelectricity data of the individual. The support set is generated through the earlier myoelectricity data of the individual, the query set is generated through the later myoelectricity data, so that the trained meta-learner is a model for predicting the myoelectricity data with later events based on the myoelectricity data with earlier time, the coincidence degree of the trained model and the actual gesture recognition process is further improved, and the gesture prediction accuracy of the model is further improved.
In another example, training the base learner to optimize parameters of the base learner based on myoelectric data in the support set and gesture labels corresponding to the myoelectric data includes: taking myoelectricity data in the support set as input, taking a gesture label corresponding to the myoelectricity data as a supervision signal, and obtaining a loss value of a gesture label prediction result of the base learner; and carrying out gradient update on the base learner according to the loss value to obtain new parameters of the base learner.
Specifically, when training the base learner, myoelectricity data is used as an input signal, gesture labels corresponding to myoelectricity data recorded in a supporting set are used as monitoring signals in the training process, the base learner predicts the gesture labels according to the input myoelectricity data, a loss value of the gesture label prediction result is calculated through a preset loss function, such as a cross entropy loss function, a value comparison loss function or an index loss function, the gesture label prediction result and the monitoring signals, and then gradient updating is performed on the base learner according to the loss value to obtain parameters optimized by the base learner. Through the mode of supervision training, parameter optimization of the basic learner is accurately completed, and the effect of meta learning training is ensured.
In another example, obtaining the prediction error of the parameter-optimized base learner gesture label prediction result includes: acquiring a gesture label prediction result of myoelectricity data according to the parameter-optimized base learner and the myoelectricity data in the query set; and acquiring a loss value corresponding to the gesture label prediction result by presetting a loss function, gesture label prediction results and gesture labels corresponding to myoelectricity data in the query set, and acquiring a prediction error according to the loss values corresponding to all myoelectricity data in the query set.
Specifically, myoelectricity data in the query set is used as input, gesture label prediction is performed by using a base learner after parameter optimization, and gesture labels corresponding to the myoelectricity data in the query set are used as supervision signals. And combining a preset loss function, such as a cross entropy loss function, a pair value loss function or an exponential loss function, a gesture label predicted by the base learner after parameter optimization and a gesture label corresponding to myoelectricity data in the query set, acquiring a loss value of a gesture label prediction result of the base learner after parameter optimization, and acquiring a prediction error based on the acquired loss value corresponding to all myoelectricity data in the query set, so that gradient update of the element learner according to the prediction error is facilitated, and further parameter optimization of the element learner is completed. And the prediction error of the prediction result is accurately obtained through a preset loss function, so that the parameter adjustment of the meta learner is conveniently and accurately carried out, and the model optimization is completed.
Further, according to the loss values corresponding to all myoelectricity data in the query set, a prediction error is obtained, including: and obtaining the average value of the loss values corresponding to all myoelectricity data in the query set, and taking the average value as a prediction error. The average here may be any one of an arithmetic average, a weighted average, a geometric average, a root mean square average, and a harmonic average. Preferably, the average value is an arithmetic average value. Specifically, in the process of obtaining the prediction error of the base learner after parameter optimization, the loss values corresponding to the myoelectric data in the query set can be obtained one by one, then the loss values corresponding to the myoelectric data are arithmetically averaged to obtain the arithmetical average value of the loss values corresponding to all the myoelectric data, and the obtained arithmetical average value is used as the prediction error. The arithmetic average is used as the prediction error by carrying out arithmetic average on the loss values corresponding to all myoelectric data in the query set, so that the prediction error is obtained as accurately as possible, and the influence of accidental factors of single prediction on the prediction result is avoided.
Furthermore, in the meta-learning training process of the meta-learner based on the original gesture recognition model by utilizing each meta-learning task, the learning rate of the meta-learner during gradient updating according to the myoelectricity data and the gesture labels corresponding to the myoelectricity data in the support set is smaller than the learning rate of the meta-learner during gradient updating according to the myoelectricity data and the gesture labels corresponding to the myoelectricity data in the query set. Through limiting the learning rate of the inner circulation training according to the support set and the outer circulation training according to the query set, the element learner can converge as soon as possible by combining the inner circulation with the high learning rate with the outer circulation with the low learning rate, and the training efficiency of the element learner is improved.
In summary, in the meta learner training based on the original gesture recognition model, a process of performing a round of meta learning training based on a meta learning task may refer to fig. 3, which includes:
step 301, obtaining a meta learning task adopted by current round training.
Step 302, assigning parameters of the meta learner to the base learner.
And 303, training the base learner according to the support set data of the meta learning task adopted by the current round training, and obtaining the optimized parameters of the base learner after gradient updating.
Step 304, based on the parameter optimized basic learner, calculating a loss value of the meta learning task as a prediction error according to the query set data of the meta learning task adopted by the current round training, and calculating a corresponding gradient.
And 305, updating parameters of the meta learner according to the calculated gradient to complete meta learning training of the current round.
In another alternative example, the initial gesture recognition model may also be generated by training in the following manner: generating a plurality of meta-learning tasks according to the historical myoelectricity data and gesture labels of a plurality of individuals, wherein each meta-learning task is generated based on the historical myoelectricity data and gesture labels of the same individual; and performing meta learning training on a meta learner based on the network model with gradient reverse transfer by utilizing each meta learning task to acquire an initial gesture recognition model.
Specifically, when the initial gesture recognition model is obtained, historical myoelectricity data and gesture labels of a plurality of individuals can be randomly obtained, one or more meta-learning tasks are generated based on the historical myoelectricity data and the gesture labels of each individual, a plurality of meta-learning tasks are finally obtained, and each meta-learning task is generated based on the historical myoelectricity data and the gesture labels of the same individual. And then taking a grid model with gradient reverse transfer as a meta learner, and performing meta learning training on the network model by utilizing each meta learning task to obtain an initial gesture recognition model. The parameters of the gradient reverse transfer grid model at the initial stage of training are obtained randomly. The meta-task training mode in meta-learning drives the model to optimize parameters in a direction with stronger generalization, and the accuracy of the model is improved. The initial gesture recognition model is obtained by directly obtaining a grid model with random parameters and gradient reverse transfer and through meta-learning training, the obtaining process of the initial gesture recognition model is simplified, the obtaining efficiency of the model is improved, and only a small number of gradient updating steps and a meta-learning task related to a specific task are required to be finely adjusted so as to adapt to the specific task.
In another example, fine tuning the initial gesture recognition model based on current myoelectricity data and gesture labels of the current individual includes: performing supervision training on the initial gesture recognition model according to current myoelectricity data and gesture labels of the current individuals; and according to the preset gradient updating step number, carrying out gradient updating on the initial gesture recognition model to obtain new parameters so as to form a current gesture recognition model of the current individual.
Specifically, when the initial gesture recognition model is finely adjusted according to the current myoelectricity data and the gesture label of the current individual, the fine adjustment can be achieved in a supervision training mode, namely, the current myoelectricity data of the current individual is used as input data, the gesture label corresponding to the current myoelectricity data is used as a supervision signal, the supervision training is carried out, in the gradient updating process, the initial gesture recognition model is subjected to gradient updating according to the preset gradient updating step number, so that optimized parameters are obtained, and the current gesture recognition model of the current individual is formed. And the fine adjustment of the initial gesture recognition model is accurately realized through supervision training and preset gradient updating steps, so that the recognition accuracy of the current gesture recognition model is ensured.
In another example, the processing unit 102 is further configured to: generating a current meta-learning task of the current individual according to the current myoelectricity data and the gesture label of the current individual; and according to the current meta learning task, performing meta learning training on a meta learner based on the initial gesture recognition model to generate a personal gesture recognition model of the current individual.
Specifically, after acquiring the current myoelectricity data and the gesture label of the current individual, the processing unit 102 further generates a current meta-learning task of the current individual according to the current myoelectricity data and the gesture label of the current individual. And then according to the current meta learning task of the current individual, performing meta learning training on the meta learner based on the initial gesture recognition model for the current individual, so that the meta learner after parameter optimization has better pertinence and adaptability to the current individual, and taking the meta learner after the meta learning training as the personal gesture recognition model of the current individual. For example, myoelectricity data when gestures corresponding to different gesture labels are made in 2 minutes under the current state of an individual are collected, according to the collected current myoelectricity data and gesture labels, the initial gesture recognition model is finely adjusted, meanwhile, according to preset constraint conditions, myoelectricity data with previous time and corresponding gesture labels are taken as a support set from the 2 minutes data, current meta-learning training tasks of the current individual are generated by taking the myoelectricity data with the later time and the corresponding gesture labels as a query set, meta-learning training is conducted on a meta-learner based on the initial gesture recognition model according to the current meta-learning tasks, and the personal gesture recognition model of the current individual is obtained.
In addition, when generating the current meta-learning task according to the current myoelectricity data and the gesture labels of the current individual, myoelectricity data when gestures corresponding to different gesture labels are made in 2 minutes in the current state of the individual can be collected, meta-learning training tasks are generated according to the myoelectricity data and the gesture labels corresponding to the myoelectricity data in the previous minute, and meta-learning verification tasks are generated according to the myoelectricity data and the gesture labels corresponding to the myoelectricity data in the next minute. After cloud learning training is carried out on the initial gesture recognition model according to the meta learning training task, gesture prediction accuracy of the trained initial gesture recognition model is acquired and detected according to the meta learning verification task, and under the condition that the gesture prediction accuracy reaches a preset threshold and the loss value of a gesture label prediction result is not reduced, the meta learning training is judged to be completed, adjustment is not needed, and the obtained initial gesture recognition model is used as a personal gesture recognition model of a current individual. For example, the threshold of the prediction accuracy is set to 0.9, that is, if the initial gesture recognition model after meta-learning training has ninety percent or more probability of correctly predicting the gesture of the current individual, and the loss value of the gesture label prediction result is no longer reduced, the training is judged to be completed. Under the condition that the gesture prediction precision does not reach a preset threshold and/or the loss value of the gesture label prediction result is still reduced, judging that training is not completed, continuously performing repeated training and parameter fine adjustment on the initial gesture recognition model according to a meta-learning training task, or collecting myoelectricity data of a current individual for a longer time and gesture labels corresponding to the myoelectricity data, generating a new meta-learning task to perform retraining and parameter fine adjustment on the initial gesture recognition model until the gesture prediction precision reaches the preset threshold and the loss value of the gesture label prediction result is not reduced. And the accuracy of the obtained personal gesture recognition model is ensured by detecting the prediction accuracy through the verification task.
It should be noted that, when the meta learning task is generated, the training task may be generated according to the data collected earlier, the verification task may be generated according to the data collected later, the training task may be generated according to the data collected later, the verification task may be generated according to the data collected earlier, all the collected data may be used for the meta learning task generation, or only part of the data may be selected for the meta learning task generation, which is not limited in this embodiment.
Further, the model acquisition system further includes: a storage unit; a storage unit for storing a personal gesture recognition model; a processing unit further configured to: under the condition that a personal gesture recognition model exists in a current individual, fine tuning is carried out on the personal gesture recognition model according to current myoelectricity data and gesture labels of the current individual to obtain the current gesture recognition model of the current individual; generating a current meta-learning task of the current individual according to myoelectricity data and gesture labels of the current individual, and continuing meta-learning training on the personal gesture recognition model according to the current meta-learning task to generate a parameter-optimized personal gesture recognition model.
Specifically, the processing unit 102 detects whether the personal gesture recognition model of the current individual is stored in advance according to the identity of the current individual, reads the personal gesture recognition model of the current individual under the condition that the personal gesture recognition model of the current individual is stored in the storage unit, and then performs fine adjustment on the personal gesture recognition model of the current individual according to the current myoelectricity data and the gesture label of the current individual to obtain the current gesture recognition model of the current individual. That is, in the case that the current individual is detected not to be a new user, the pre-stored personal gesture recognition model of the current individual is read and used as a basis for acquiring the current gesture recognition model. By storing and reusing the individual gesture recognition model, the prior experience waste caused by the generation of the individual gesture recognition model in the current gesture recognition model generation process is avoided.
In addition, the processing unit 102 is further configured to generate a current meta-learning task of the current individual according to the current myoelectric data and the gesture label of the current individual, and then perform meta-learning training on the current state of the current individual according to the current meta-learning task of the current individual, so that the personal gesture recognition model has better pertinence and adaptability to the current individual, perform parameter optimization on the personal gesture recognition model in a meta-learning training manner, and store the personal gesture recognition model after parameter optimization through the storage unit.
The present embodiment is not particularly limited in the kind of the processing unit. The processing unit may be hardware for performing logic operations, such as a single-chip microcomputer, a microprocessor, a programmable logic controller (PLC, programmable Logic Controller) or a Field programmable gate array (FPGA, field-Programmable Gate Array), or a software program, a function module, a function, a target library (objects) or a Dynamic-Link library (Dynamic-Link Libraries) for implementing the above functions on a hardware basis. Or a combination of the two.
Another aspect of the embodiment of the application provides a gesture model generating method. The flow of the gesture model generating method may refer to fig. 4, which includes the following steps:
step 401, current myoelectricity data and a gesture label of a current individual are acquired, wherein the gesture label contains gesture information.
Step 402, under the condition that a personal gesture recognition model does not exist in the current individual, acquiring an initial gesture recognition model, wherein the initial gesture recognition model is generated by performing meta-learning training according to a plurality of meta-learning tasks generated based on historical myoelectricity data and gesture labels of a plurality of individuals, and each meta-learning task is generated based on the historical myoelectricity data and gesture labels of the same individual.
Step 403, fine tuning the initial gesture recognition model according to the current myoelectricity data and the gesture label of the current individual to obtain the current gesture recognition model of the current individual.
It is to be noted that this embodiment is a method embodiment corresponding to the system embodiment, and this embodiment may be implemented in cooperation with the system embodiment. The related technical details mentioned in the system embodiment are still valid in this embodiment, and in order to reduce repetition, they are not described here again. Accordingly, the related technical details mentioned in the present embodiment can also be applied to the system embodiment.
In another aspect of the embodiments of the present application, a gesture recognition method may refer to fig. 5, where the gesture recognition method includes the following steps:
step 501, a current gesture recognition model of a current individual is obtained. Specifically, the terminal device for recognizing the gesture of the current individual acquires the current gesture recognition model which is acquired by fine tuning the initial gesture recognition model for the current individual through the model acquisition system.
Step 502, acquiring real-time myoelectricity data of a current individual. Specifically, when gesture recognition is performed, myoelectricity data of a current individual is acquired in real time through myoelectricity electrodes, and the acquired real-time myoelectricity data is input into a current gesture recognition model.
Step 503, acquiring the gesture of the current individual according to the real-time myoelectricity data through the current gesture recognition model. Specifically, after the real-time myoelectricity data of the current individual is obtained, the current gesture recognition model predicts the gesture of the current individual according to the input real-time myoelectricity data, and outputs the predicted gesture of the current individual.
In one example, after the current gesture recognition model of the current individual is acquired by the model acquisition system, the method further includes: acquiring the prediction precision of a current gesture recognition model; under the condition that the prediction accuracy does not meet the preset threshold value, re-acquiring current myoelectricity data and gesture labels of the current individuals in a new preset duration; and re-trimming the current gesture recognition model according to the re-acquired current myoelectricity data and the gesture label.
Specifically, due to the characteristics of myoelectricity data, the prediction accuracy of a current gesture recognition model of a current individual is gradually reduced along with the increase of the use time, so that after the current gesture recognition model of the current individual is obtained, the prediction accuracy of the current gesture recognition model is monitored, under the condition that the prediction accuracy of the current gesture recognition model is insufficient, the myoelectricity data corresponding to different gesture labels and when different gestures are made in a preset time length under the current state of the current individual are re-obtained, and the current gesture recognition model is re-finely adjusted according to the re-obtained current myoelectricity data and gesture labels, so that the current gesture recognition model can be more adaptive to the current state of the current individual, and the gesture recognition accuracy is ensured. And in consideration of the characteristic that the electromyographic signals are easy to change, the gesture recognition model is updated and iterated periodically, so that the accuracy of a gesture recognition result obtained by using the gesture recognition model is ensured.
Another aspect of the embodiments of the present application further provides a gesture recognition apparatus, referring to fig. 6, including:
the first obtaining module 601 is configured to obtain, by using the model obtaining system described above, a current gesture recognition model of a current individual.
A second obtaining module 602, configured to obtain real-time myoelectricity data of a current individual;
the recognition module 603 is configured to obtain, according to the real-time myoelectricity data, a gesture of a current individual through the current gesture recognition model.
In one example, the gesture recognition apparatus further includes, a third acquisition model; the third acquisition module is used for acquiring the prediction precision of the current gesture recognition model; under the condition that the prediction accuracy does not meet the preset threshold value, re-acquiring current myoelectricity data and gesture labels of the current individuals in a new preset duration; and re-trimming the current gesture recognition model according to the re-acquired current myoelectricity data and the gesture label.
It is to be noted that this embodiment is an apparatus embodiment corresponding to the method embodiment, and this embodiment may be implemented in cooperation with the method embodiment. The related technical details mentioned in the method embodiment are still valid in this embodiment, and in order to reduce repetition, they are not described here again. Accordingly, the related technical details mentioned in the present embodiment may also be applied in the method embodiment.
Another aspect of an embodiment of the present application further provides an electronic device, referring to fig. 7, including: comprising at least one processor 701; and a memory 702 communicatively coupled to the at least one processor 701; the memory 702 stores instructions executable by the at least one processor 701, the instructions being executable by the at least one processor 701 to enable the at least one processor 701 to perform a gesture recognition method as described above, or a model acquisition method as described above.
Where memory 702 and processor 701 are connected by a bus, the bus may comprise any number of interconnected buses and bridges, the buses connecting the various circuits of the one or more processors 701 and memory 702 together. The bus may also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art, and therefore, will not be described any further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver may be one element or may be a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. The data processed by the processor 701 is transmitted over a wireless medium via an antenna, which further receives the data and transmits the data to the processor 701.
The processor 701 is responsible for managing the bus and general processing and may provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. And memory 702 may be used to store data used by processor 701 in performing operations.
Another aspect of the embodiments of the present application also provides a computer-readable storage medium storing a computer program. The computer program, when executed by a processor, implements a gesture recognition method as described above, or a model acquisition method as described above.
That is, it will be understood by those skilled in the art that all or part of the steps in implementing the methods of the embodiments described above may be implemented by a program stored in a storage medium, where the program includes several instructions for causing a device (which may be a single-chip microcomputer, a chip or the like) or a processor (processor) to perform all or part of the steps in the methods of the embodiments of the application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
It will be understood by those of ordinary skill in the art that the foregoing embodiments are specific examples of carrying out the invention and that various changes in form and details may be made therein without departing from the spirit and scope of the invention.

Claims (20)

1. A model acquisition system, comprising:
the input unit is used for acquiring current myoelectricity data of a current individual and gesture labels, wherein the gesture labels comprise gesture information;
a processing unit configured to:
under the condition that a personal gesture recognition model does not exist in the current individual, an initial gesture recognition model is obtained, wherein the initial gesture recognition model is generated by meta-learning training according to a plurality of meta-learning tasks generated based on historical myoelectricity data and gesture labels of a plurality of individuals, each meta-learning task comprises a support set and a query set, the support set is generated based on the historical myoelectricity data and gesture labels of the individuals at a first time, the query set is generated based on the historical myoelectricity data and gesture labels of the individuals at a second time, and the first time is earlier than the second time;
and fine-tuning the initial gesture recognition model according to the current myoelectricity data and the gesture label of the current individual to obtain the current gesture recognition model of the current individual.
2. The model acquisition system of claim 1, wherein the initial gesture recognition model is trainable to be generated by:
training a network model with gradient reverse transfer based on historical myoelectricity data and gesture labels of a plurality of individuals to obtain an original gesture recognition model;
generating a plurality of meta-learning tasks according to the historical myoelectricity data and gesture labels of a plurality of individuals, wherein each meta-learning task is generated based on the historical myoelectricity data and gesture labels of the same individual;
and performing meta learning training on a meta learner based on the original gesture recognition model by utilizing each meta learning task to acquire the initial gesture recognition model.
3. The model acquisition system according to claim 2, wherein the network model with gradient reverse transfer is any one of a convolutional neural network CNN model, a long-short-term neural network LSTM model, and a cyclic neural network RNN model.
4. The model acquisition system according to claim 2, wherein the performing meta-learning training on the meta-learner based on the original gesture recognition model using each of the meta-learning tasks includes:
For each meta learning task, firstly assigning parameters of the meta learner to a base learner, and then training the base learner according to myoelectricity data in the support set and gesture labels corresponding to the myoelectricity data so as to optimize the parameters of the base learner;
acquiring a prediction error of a gesture label prediction result of the base learner after parameter optimization according to myoelectricity data in the query set and gesture labels corresponding to the myoelectricity data;
and according to the prediction error, carrying out gradient update on the meta learner so as to optimize parameters of the meta learner.
5. The model acquisition system of claim 4, wherein training the base learner to optimize parameters of the base learner based on myoelectric data in the support set and gesture labels corresponding to myoelectric data comprises:
taking myoelectricity data in the support set as input, taking a gesture label corresponding to the myoelectricity data as a supervision signal, and obtaining a loss value of a gesture label prediction result of the base learner;
and according to the loss value, carrying out gradient update on the base learner so as to obtain new parameters of the base learner.
6. The model acquisition system according to claim 4, wherein the acquiring the prediction error of the gesture label prediction result of the base learner after parameter optimization includes:
acquiring the gesture label prediction result of myoelectricity data according to the base learner after parameter optimization and myoelectricity data in the query set;
and acquiring a loss value corresponding to the gesture label prediction result through presetting a loss function, the gesture label prediction result and gesture labels corresponding to the myoelectricity data in the query set, and acquiring the prediction error according to the loss values corresponding to all myoelectricity data in the query set.
7. The model acquisition system according to claim 6, wherein the acquiring the prediction error according to the loss values corresponding to all myoelectric data in the query set includes:
and obtaining an average value of loss values corresponding to all myoelectricity data in the query set, and taking the average value as the prediction error.
8. The model acquisition system according to claim 4, wherein in the meta-learning training of the meta-learner based on the original gesture recognition model by using each meta-learning task, a learning rate at the time of gradient update of the base learner according to myoelectric data and gesture labels corresponding to myoelectric data in the support set is smaller than a learning rate at the time of gradient update of the meta-learner according to myoelectric data and gesture labels corresponding to myoelectric data in the query set.
9. The model acquisition system of claim 1, wherein the initial gesture recognition model is further trainable to be generated by:
generating a plurality of meta-learning tasks according to the historical myoelectricity data and gesture labels of a plurality of individuals, wherein each meta-learning task is generated based on the historical myoelectricity data and gesture labels of the same individual;
and performing meta learning training on a meta learner based on the network model with gradient reverse transfer by utilizing each meta learning task to acquire the initial gesture recognition model.
10. The model acquisition system of claim 1, wherein fine tuning the initial gesture recognition model based on current myoelectricity data and gesture labels of the current individual comprises:
performing supervision training on the initial gesture recognition model according to the current myoelectricity data and the gesture label of the current individual;
and according to the preset gradient updating step number, carrying out gradient updating on the initial gesture recognition model to obtain new parameters so as to form the current gesture recognition model of the current individual.
11. The model acquisition system according to any one of claims 1 to 10, wherein the processing unit is further configured to:
Generating a current meta-learning task of the current individual according to the current myoelectricity data and the gesture label of the current individual;
and performing meta learning training on a meta learner based on the initial gesture recognition model according to the current meta learning task to generate the personal gesture recognition model of the current individual.
12. The model acquisition system of claim 11, wherein the model acquisition system further comprises: a storage unit;
the storage unit is used for storing the personal gesture recognition model;
the processing unit is further configured to:
when the current individual has the personal gesture recognition model, fine tuning the personal gesture recognition model according to the current myoelectricity data and gesture labels of the current individual to obtain the current gesture recognition model of the current individual;
generating the current meta-learning task of the current individual according to the myoelectricity data and the gesture label of the current individual, and continuing meta-learning training on the personal gesture recognition model according to the current meta-learning task to generate the personal gesture recognition model with optimized parameters.
13. A model acquisition system, comprising:
the input unit is used for acquiring current myoelectricity data of a current individual and gesture labels, wherein the gesture labels comprise gesture information;
a processing unit configured to:
under the condition that a personal gesture recognition model does not exist in the current individual, acquiring an initial gesture recognition model, wherein the initial gesture recognition model is generated by meta-learning training according to a plurality of meta-learning tasks generated based on historical myoelectricity data and gesture labels of a plurality of individuals; performing fine adjustment on the initial gesture recognition model according to the current myoelectricity data and the gesture label of the current individual to obtain a current gesture recognition model of the current individual;
generating a current meta-learning task of the current individual according to the myoelectricity data and the gesture label of the current individual, continuing meta-learning training on the initial gesture recognition model according to the current meta-learning task, and generating the personal gesture recognition model, wherein the personal gesture recognition model is used for fine tuning according to the subsequently acquired current myoelectricity data and gesture label so as to obtain the updated current gesture recognition model of the current individual;
The current meta-learning task comprises a support set and a query set, wherein the support set is generated based on historical myoelectricity data and gesture labels of a current individual at a third time, the query set is generated based on historical myoelectricity data and gesture labels of the current individual at a fourth time, and the third time is earlier than the fourth time.
14. A model acquisition method, characterized by comprising:
acquiring current myoelectricity data and a gesture label of a current individual, wherein the gesture label comprises gesture information;
under the condition that a personal gesture recognition model does not exist in the current individual, an initial gesture recognition model is obtained, wherein the initial gesture recognition model is generated by meta-learning training according to a plurality of meta-learning tasks generated based on historical myoelectricity data and gesture labels of a plurality of individuals, each meta-learning task comprises a support set and a query set, the support set is generated based on the historical myoelectricity data and gesture labels of the individuals at a first time, the query set is generated based on the historical myoelectricity data and gesture labels of the individuals at a second time, and the first time is earlier than the second time;
and fine-tuning the initial gesture recognition model according to the current myoelectricity data and the gesture label of the current individual to obtain the current gesture recognition model of the current individual.
15. A method of gesture recognition, comprising: acquiring a current gesture recognition model of a current individual by the model acquisition system of any one of claims 1 to 13;
acquiring real-time myoelectricity data of the current individual;
and acquiring the gesture of the current individual according to the real-time myoelectricity data through the current gesture recognition model.
16. The gesture recognition method according to claim 15, further comprising, after the current gesture recognition model is acquired by the model acquisition system according to any one of claims 1 to 13:
acquiring the prediction precision of the current gesture recognition model;
under the condition that the prediction precision does not meet a preset threshold value, re-acquiring current myoelectricity data and gesture labels of the current individual in a new preset time period;
and re-trimming the current gesture recognition model according to the re-acquired current myoelectricity data and gesture labels.
17. A gesture recognition apparatus, comprising:
a first acquisition module for acquiring a current gesture recognition model of a current individual by the model acquisition system of any one of claims 1 to 13;
The second acquisition module is used for acquiring the real-time myoelectricity data of the current individual;
and the recognition module is used for acquiring the gesture of the current individual according to the real-time myoelectricity data through the current gesture recognition model.
18. The gesture recognition apparatus of claim 17, further comprising a third acquisition model;
the third acquisition module is used for acquiring the prediction precision of the current gesture recognition model;
under the condition that the prediction precision does not meet a preset threshold value, re-acquiring current myoelectricity data and gesture labels of the current individual in a new preset time period;
and re-trimming the current gesture recognition model according to the re-acquired current myoelectricity data and gesture labels.
19. An electronic device, comprising:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the model acquisition method of claim 14 or the gesture recognition method of claim 15 or 16.
20. A computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the model acquisition method according to claim 14 or the gesture recognition method according to claim 15 or 16.
CN202310515750.1A 2022-03-29 2022-03-29 Model acquisition system, gesture recognition method, gesture recognition device, apparatus and storage medium Pending CN116662773A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310515750.1A CN116662773A (en) 2022-03-29 2022-03-29 Model acquisition system, gesture recognition method, gesture recognition device, apparatus and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310515750.1A CN116662773A (en) 2022-03-29 2022-03-29 Model acquisition system, gesture recognition method, gesture recognition device, apparatus and storage medium
CN202210341755.2A CN114781439B (en) 2022-03-29 2022-03-29 Model acquisition system, gesture recognition method, gesture recognition device, apparatus and storage medium

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CN202210341755.2A Division CN114781439B (en) 2022-03-29 2022-03-29 Model acquisition system, gesture recognition method, gesture recognition device, apparatus and storage medium

Publications (1)

Publication Number Publication Date
CN116662773A true CN116662773A (en) 2023-08-29

Family

ID=82426454

Family Applications (2)

Application Number Title Priority Date Filing Date
CN202210341755.2A Active CN114781439B (en) 2022-03-29 2022-03-29 Model acquisition system, gesture recognition method, gesture recognition device, apparatus and storage medium
CN202310515750.1A Pending CN116662773A (en) 2022-03-29 2022-03-29 Model acquisition system, gesture recognition method, gesture recognition device, apparatus and storage medium

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN202210341755.2A Active CN114781439B (en) 2022-03-29 2022-03-29 Model acquisition system, gesture recognition method, gesture recognition device, apparatus and storage medium

Country Status (2)

Country Link
CN (2) CN114781439B (en)
WO (1) WO2023185887A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114781439B (en) * 2022-03-29 2023-05-30 深圳市应和脑科学有限公司 Model acquisition system, gesture recognition method, gesture recognition device, apparatus and storage medium
CN116595443B (en) * 2023-07-17 2023-10-03 山东科技大学 Wireless signal book gesture recognition method based on meta learning
CN117292404B (en) * 2023-10-13 2024-04-19 哈尔滨工业大学 High-precision gesture data identification method, electronic equipment and storage medium

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8447704B2 (en) * 2008-06-26 2013-05-21 Microsoft Corporation Recognizing gestures from forearm EMG signals
US20200046284A1 (en) * 2017-07-28 2020-02-13 Meltin Mmi Co., Ltd. System, method, and program for recognizing myoelectric signal-originating motion
CN110598676B (en) * 2019-09-25 2022-08-02 南京邮电大学 Deep learning gesture electromyographic signal identification method based on confidence score model
CN110796207B (en) * 2019-11-08 2023-05-30 中南大学 Fatigue driving detection method and system
CN110852447B (en) * 2019-11-15 2023-11-07 腾讯云计算（北京）有限责任公司 Meta learning method and apparatus, initializing method, computing device, and storage medium
GB2588951A (en) * 2019-11-15 2021-05-19 Prevayl Ltd Method and electronics arrangement for a wearable article
CN111103976B (en) * 2019-12-05 2023-05-02 深圳职业技术学院 Gesture recognition method and device and electronic equipment
CN111339837B (en) * 2020-02-08 2022-05-03 河北工业大学 Continuous sign language recognition method
CN111870242A (en) * 2020-08-03 2020-11-03 南京邮电大学 Intelligent gesture action generation method based on electromyographic signals
CN112818768B (en) * 2021-01-19 2022-08-26 南京邮电大学 Transformer substation reconstruction and extension violation behavior intelligent identification method based on meta-learning
CN113971437B (en) * 2021-09-24 2024-01-19 西北大学 Cross-domain gesture recognition method based on commercial Wi-Fi equipment
CN114781439B (en) * 2022-03-29 2023-05-30 深圳市应和脑科学有限公司 Model acquisition system, gesture recognition method, gesture recognition device, apparatus and storage medium

Also Published As

Publication number Publication date
CN114781439B (en) 2023-05-30
CN114781439A (en) 2022-07-22
WO2023185887A1 (en) 2023-10-05

Similar Documents

Publication Publication Date Title
CN114781439B (en) Model acquisition system, gesture recognition method, gesture recognition device, apparatus and storage medium
US11734319B2 (en) Question answering method and apparatus
US11551103B2 (en) Data-driven activity prediction
CN110619423B (en) Multitask prediction method and device, electronic equipment and storage medium
CN111382906B (en) Power load prediction method, system, equipment and computer readable storage medium
EP3726435A1 (en) Deep neural network training method and apparatus, and computer device
Lison Model-based bayesian reinforcement learning for dialogue management
CN115034430A (en) Carbon emission prediction method, device, terminal and storage medium
WO2023010861A1 (en) Wake-up method, apparatus, device, and computer storage medium
WO2019052430A1 (en) Method and apparatus for self-service of mobile terminal
US10909322B1 (en) Unusual score generators for a neuro-linguistic behavioral recognition system
CN116569194A (en) Joint learning
US11914956B1 (en) Unusual score generators for a neuro-linguistic behavioral recognition system
CN112101417A (en) Continuous learning method and device based on condition batch normalization
CN112115994A (en) Training method and device of image recognition model, server and storage medium
Feng et al. DME: An Adaptive and Just-in-Time Weighted Ensemble Learning Method for Classifying Block-Based Concept Drift Steam
CN114638379A (en) Edge side multi-agent OPC UA information analysis and decision method
CN109325402B (en) Signal processing method, system and computer storage medium
CN110889396A (en) Energy internet disturbance classification method and device, electronic equipment and storage medium
CN117493956A (en) Method, device, equipment and storage medium for identifying brain electricity emotion crossing database
Piazentin et al. A simulator for Freeman K-sets in Java
US20240172984A1 (en) Electroencephalogram (eeg) emotion recognition method based on spiking convolutional neural network
CN110134250B (en) Human-computer interaction signal processing method, device and computer readable storage medium
US20240178854A1 (en) Signal processor
CN109754091B (en) Self-adaptive learning engine training system based on brain wave technology and application thereof

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination