CN113297935A - Feature adaptive motion recognition system - Google Patents
Feature adaptive motion recognition system Download PDFInfo
- Publication number
- CN113297935A CN113297935A CN202110514238.6A CN202110514238A CN113297935A CN 113297935 A CN113297935 A CN 113297935A CN 202110514238 A CN202110514238 A CN 202110514238A CN 113297935 A CN113297935 A CN 113297935A
- Authority
- CN
- China
- Prior art keywords
- electrode
- missing
- electrodes
- existing
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003044 adaptive effect Effects 0.000 title claims description 14
- 230000009471 action Effects 0.000 claims abstract description 57
- 230000011218 segmentation Effects 0.000 claims description 42
- 238000009826 distribution Methods 0.000 claims description 17
- 238000007637 random forest analysis Methods 0.000 claims description 15
- 230000003183 myoelectrical effect Effects 0.000 claims description 12
- 230000004044 response Effects 0.000 claims description 12
- 230000001502 supplementing effect Effects 0.000 claims description 12
- 238000011156 evaluation Methods 0.000 claims description 8
- 239000012535 impurity Substances 0.000 claims description 6
- 238000010276 construction Methods 0.000 claims description 5
- 238000000638 solvent extraction Methods 0.000 claims description 5
- 238000000034 method Methods 0.000 abstract description 49
- 230000008569 process Effects 0.000 abstract description 21
- 230000008859 change Effects 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 9
- 239000011159 matrix material Substances 0.000 description 9
- 210000003811 finger Anatomy 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000003066 decision tree Methods 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000013589 supplement Substances 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 208000018737 Parkinson disease Diseases 0.000 description 1
- 239000002390 adhesive tape Substances 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000002567 electromyography Methods 0.000 description 1
- 238000007636 ensemble learning method Methods 0.000 description 1
- 210000004932 little finger Anatomy 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 210000004243 sweat Anatomy 0.000 description 1
- 210000003813 thumb Anatomy 0.000 description 1
- 210000000707 wrist Anatomy 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/12—Classification; Matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/08—Feature extraction
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
The invention provides a characteristic self-adaptive action recognition system, which comprises: the sensing terminal comprises a plurality of electromyographic electrodes and is used for acquiring corresponding electromyographic signals generated by user actions and sending the corresponding electromyographic signals to the client; the client is used for receiving the electromyographic signals sent by the sensing terminal and uploading the electromyographic signals to a server; a server for extracting one or more feature values from each of the electromyographic signals and recognizing the user's motion based on the feature values of the electromyographic signals. The feature self-adaptive motion recognition system provided by the invention supports the dynamic change of a feature space in the motion recognition process.
Description
Technical Field
The invention relates to the technical field of human-computer interaction and machine learning, in particular to a feature adaptive action recognition system.
Background
Conventional machine learning techniques typically assume that the data stream (including offline training and online prediction data) has a fixed feature space, however this assumption does not hold in many real-world scenarios. In fact, the increase and decrease of the feature space of the data stream may be caused by many reasons, such as the change of the feature space caused by the variability of the electrodes and the diversity of the recognition requirements in the physiological calculation, so that the recognition model constructed on the basis of the machine learning technology on the initial feature space is no longer suitable for the changed feature space.
Taking the motion recognition (more specifically, gesture recognition) shown in fig. 1 as an example, many factors (e.g., motion artifacts, hair and sweat on the skin surface, electrical noise from power lines and external power sources, etc.) may affect the quality of physiological signals (e.g., electromyographic signals) collected by the electrodes, and as time progresses, a case may occur in which physiological signals from some existing electrodes are absent (as indicated by the absence of some existing electrodes); in addition, as the recognition needs or rehabilitation goals change, additional electrodes may need to be added to support these changes (see fig. 2). The absence and the addition of the electrodes can cause the absence and the addition of features extracted from physiological signals acquired by the electrodes, namely, the feature space changes, so that the existing recognition model constructed based on the initial feature space cannot adapt to the motion recognition after the change of the feature space.
To cope with dynamic changes in feature space, one solution that is currently common is to reconstruct the recognition model using the feature space changed data set. However, this solution has three limitations: firstly, the learned useful knowledge in the existing recognition model is ignored; secondly, a large amount of data is needed for reconstructing the identification model, and the requirement of the reconstructed identification model on the sample size is possibly difficult to meet by the incremental data set; more importantly, this solution is difficult to adapt to the dynamic changes of the feature space.
Disclosure of Invention
To overcome the problems in the prior art, according to an embodiment of the present invention, there is provided a feature adaptive motion recognition system, including: the sensing terminal comprises a plurality of electromyographic electrodes and is used for acquiring corresponding electromyographic signals generated by user actions and sending the corresponding electromyographic signals to the client; the client is used for receiving the electromyographic signals sent by the sensing terminal and uploading the electromyographic signals to a server; a server for extracting one or more feature values from each of the electromyographic signals and recognizing an action of the user based on the feature values of the electromyographic signals, the server configured to: learning and constructing a recognition model based on an initial data set, wherein the initial data set comprises characteristic values of physiological signals of all collected myoelectric electrodes corresponding to user actions and labels corresponding to the user actions, and the labels are used for indicating the categories of the actions; acquiring a plurality of characteristic values of electromyographic signals of a plurality of electromyographic electrodes of user actions to be identified; determining whether a characteristic value from one or more of the all electrodes is missing from the acquired characteristic values; in response to determining that a characteristic value from one or more of the all electrodes is missing from the acquired characteristic values, treating the one or more electrodes as one or more missing electrodes, supplementing the acquired characteristic values with characteristic values from the one or more missing electrodes; and taking the acquired characteristic value as the input of the recognition model to obtain an action recognition result.
In the above system, the supplementing the feature values from the one or more missing electrodes in the acquired feature values includes: for each missing electrode of the one or more missing electrodes, selecting an un-missing electrode of the existing electrode set that is most relevant to the missing electrode, and supplementing the feature value from the missing electrode according to the feature value from the selected un-missing electrode in the obtained feature values; wherein the non-missing electrode represents an electrode in the existing electrode set other than the one or more missing electrodes.
In the above system, the server is further providedIs set to execute: for each electrode in the existing electrode set, constructing an electrode sketch for that electrode based on the initial data set; after the action recognition result is obtained, updating the electrode sketch of each electrode in the existing electrode set; wherein supplementing feature values from the missing electrode according to feature values from the selected non-missing electrode in the acquired feature values comprises: the eigenvalues from the missing electrode are calculated based on the following equation:wherein i indicates that the missing electrode is the ith electrode in the existing electrode set, j indicates that the selected non-missing electrode is the jth electrode in the existing electrode set, t indicates the round corresponding to the acquired characteristic value, (o) indicates the existing electrode,representing the characteristic value, S, from the missing electrodeiElectrode sketch showing the missing electrode, SjAn electrode sketch representing selected non-missing electrodes,representing a characteristic value from the selected non-missing electrodes among the acquired characteristic values; and supplementing the characteristic value from the missing electrode in the acquired characteristic values.
In the above system, the server is further configured to perform: for each missing electrode of the one or more missing electrodes, calculating a degree of correlation between the missing electrode and each non-missing electrode of the existing electrode set; wherein the degree of correlation between two electrodes includes the pearson correlation coefficient between the electrode sketches of the two electrodes and the spatial distance between the two electrodes.
In the above system, the server is further configured to perform: determining whether a characteristic value from a newly added electrode exists in the acquired characteristic values; and in response to determining that a feature value from a newly added electrode exists in the obtained feature values, constructing an incremental dataset based on the obtained feature values, updating the identification model based on the incremental dataset, and adding the newly added electrode to the existing electrode set.
In the above system, the constructing the recognition model based on the initial data set includes: constructing a plurality of individual classifiers based on the initial dataset to form a random forest, the random forest being used as the recognition model; and updating the recognition model based on the incremental dataset comprises: calculating an evaluation value for each individual classifier in the recognition model based on:wherein, S (h)m) Representing the m-th individual classifier h in the recognition modelmThe evaluation value of (1); acc (h)m) Represents hmThe classification accuracy of (2); # featuremRepresentation construction hmThe number of all features used;representing the number of eigenvalues from all electrodes in the existing electrode set, t indicating the round to which the obtained eigenvalues correspond; selecting individual classifiers with a preset proportion from all the individual classifiers in the identification model from small to large according to the evaluation values as the individual classifiers to be updated; and updating the individual classifier to be updated based on the incremental dataset.
In the above system, the updating the individual classifier to be updated based on the incremental data set includes: calculating the impurity degree of leaf nodes which can be reached by the incremental dataset in the individual classifier to be updated; if the leaf node's impurity level reaches a predetermined threshold, then a splitting operation is performed on the leaf node.
In the above system, the updating the individual classifier to be updated based on the incremental data set includes: for each internal node in the individual classifier to be updated, a candidate score for that internal node is calculated based on the following equationDivergence indexes corresponding to each candidate segmentation threshold in the segmentation threshold set are as follows:wherein v represents an internal node v in the individual classifier to be updated; dvRepresenting a subset of the incremental dataset within which an internal node v can be reached; phi (v) represents the partitioning property of the internal node v; τ (v) represents the current segmentation threshold for internal node v; qLAnd QRRespectively representing D when the current segmentation threshold is tau (v)vThe label distribution in (1); q'LAnd Q'RRespectively, D when using the candidate segmentation thresholdvThe label distribution in (1); i DvI denotes DvThe size of (d); i DL,τL represents the subset size of the left child node which can reach the internal node v in the incremental data set when the current segmentation threshold is tau (v); i DR,τL represents the subset size of the right child node which can reach the internal node v in the incremental data set when the current segmentation threshold is tau (v); 2JSD (P)1,P2)=KL(P1P Pavg)+KL(P2P Pavg) And P isavg=(P1+P2) (ii)/2, wherein KL (. cndot.) represents the Kullback-Leibler divergence; and selecting the candidate segmentation threshold corresponding to the maximum divergence index from the candidate segmentation threshold set to update the current segmentation threshold of the internal node.
In the above system, the server is further configured to: and constructing an electrode sketch for the newly added electrode.
In the above system, the constructing the recognition model based on the initial data set includes: constructing a plurality of individual classifiers based on the initial dataset to form a random forest, the random forest being used as the recognition model; and the server is further configured to: after obtaining an action recognition result, selecting an individual classifier to be updated from the recognition model; and for the individual classifier to be updated, carrying out clipping operation on the internal node of the individual classifier.
The embodiment of the invention can adapt to the dynamic change of the characteristic space. In a real action recognition scene, the characteristic loss is generally unplanned and unexpected, so for the lost characteristic, the invention selects the non-lost electrode with the highest correlation degree with the lost electrode, and solves the characteristic value from the lost electrode according to the characteristic value from the non-lost electrode, so that the action recognition can be carried out by utilizing the existing recognition model under the condition of the characteristic loss. In addition, the characteristic increase is usually planned and expected, so the invention updates the recognition model on the basis of the new characteristics, so that the recognition model is more consistent with the actual motion recognition scene, thereby improving the motion recognition accuracy. The identification model is updated by adjusting the structure and/or the segmentation threshold, so that new knowledge is learned while useful knowledge learned by the identification model is not lost, and the requirement of the updated identification model on the sample size is met by the incremental data set. Further, the recognition model can be updated even in the case of no new added features, for example, useless internal nodes are reduced, and thus the accuracy of motion recognition is further improved.
Drawings
Example embodiments will be described in detail with reference to the accompanying drawings, which are intended to depict example embodiments and should not be construed as limiting the intended scope of the claims. The drawings are not to be considered as drawn to scale unless explicitly indicated.
FIG. 1 is a schematic diagram of incremental changes in electrodes over time in a gesture recognition application;
FIG. 2 is a schematic diagram of the incremental change in feature space over time according to one embodiment of the present invention;
FIG. 3 is a schematic diagram of a feature adaptive motion recognition system according to one embodiment of the present invention;
FIG. 4 is a schematic diagram of a plurality of clients in a feature adaptive action recognition system according to one embodiment of the present invention;
FIG. 5 is a flow diagram of a server performing a feature adaptive action recognition method according to one embodiment of the present invention;
FIG. 6 is a flow diagram of a server performing a feature adaptive action recognition method according to another embodiment of the present invention;
FIG. 7 is a schematic diagram of an individual classifier according to one embodiment of the invention;
FIG. 8 is a schematic diagram showing the dynamic changes of the electrodes of the myoelectric arm ring in the experimental part of the present invention;
FIG. 9 is a schematic diagram of two gestures during the experimental verification process of the present invention with a smart phone as the client.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail by embodiments with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Before describing embodiments of the present invention, the symbols used in the embodiments and the corresponding definitions are first introduced, as shown in table 1:
TABLE 1
According to one embodiment of the invention, a feature adaptive motion recognition system is provided, which is adapted to implement motion recognition in the absence of features.
Referring to fig. 3, the feature-adaptive motion recognition system 300 includes three parts, namely a sensing terminal 301, a client 302 and a remote cloud server 303, wherein:
the sensing terminal 301 senses the intention of the user by using an electromyographic device, is composed of a plurality of electromyographic electrodes, and can send the collected corresponding electromyographic signals generated by the user actions to the client in real time. In one embodiment, the electromyographic closed loop can be used as a sensing terminal to collect the electromyographic signals of eight channels, wherein the sampling frequency is 200hz, and the collected electromyographic signals are transmitted to the client in real time through the bluetooth device.
The client 302 can receive the electromyographic signals from the sensing terminal and upload the collected signals to a remote server. The client is composed of a control module, a signal display module, a signal uploading module and the like (not shown). The control module is mainly used for controlling the data acquisition process and recording the acquired surface electromyographic signals; the signal display module can visually display the acquired signals in real time through a line graph; the signal uploading module can automatically upload the acquired data to a remote server. In one embodiment, the recognition System can support multiple Clients, including a smartphone client and a personal computer client (as shown in fig. 4), and is therefore also called a Multi Clients/Server interaction System (McS).
The remote cloud server 303 is primarily used to extract feature values from raw surface electromyography signals, build initial recognition models, update existing models using streaming data, and recognize user actions (e.g., gestures), and is configured to perform feature-adaptive action recognition (as described below). The remote cloud server is capable of providing high performance computing services and returning computing results to the client device. The electromyographic data streams, the calculation result streams and the control command streams of the sensing terminal, the multiple clients and the remote cloud can realize dynamic cooperation through wireless technologies such as Bluetooth and WiFi.
According to one embodiment of the invention, the server is used for realizing action recognition in the case of feature loss.
Referring to fig. 5, in summary, the server for implementing action recognition in the case of feature loss includes: step S11, learning and constructing a recognition model based on an initial data set (also called a sample set), wherein the initial data set comprises initial characteristics of all electrodes in an existing electrode set corresponding to multiple physiological signal acquisitions and labels corresponding to the multiple physiological signal acquisitions, and the labels are used for indicating the categories of actions. For one of ordinary skill in the art, this step may be done prior to performing motion recognition; s12, acquiring a plurality of characteristic values of electromyographic signals of a plurality of electromyographic electrodes of user actions; s13, determining whether the acquired characteristic values lack the characteristic values from one or more electrodes in the existing electrode set or not, obtaining the missing electrodes in the existing electrode set in response to determining that the characteristic values from one or more electrodes in the existing electrode set are lacked, and supplementing the characteristic values from the missing electrodes in the acquired characteristic values; s14, taking the obtained characteristic value as the input of an identification model to obtain an action identification result; and S15, if the action recognition is not finished, returning to the step S12, otherwise, finishing the method.
Feature adaptive motion recognition is described below in conjunction with fig. 5. In particular, construction of the recognition model and continuous motion recognition based on the recognition model are described in rounds (also referred to as acquisition rounds) over time. Wherein the data collected in round 1 is used for the construction of the recognition model; in the 2 nd, 3 rd, … th, N rounds, data are collected from the objects to be recognized, respectively, and motion recognition is performed by the constructed recognition model based on the collected data.
And S11, constructing an identification model based on the initial data set. Step S11 includes the following sub-steps:
and S111, constructing an initial data set. The initial data set comprises characteristic values of physiological signals of all collected myoelectric electrodes corresponding to the action of a user and a label corresponding to the action of the user, and the label is used for indicating the category of the action.
The electrode from which the physiological signal was obtained in round 1 was taken as an existing electrode, and the array of a plurality of existing electrodes constituted an existing electrode set (in round 1, there were no missing electrodes and additional electrodes). As shown in fig. 1, an array of existing electrodes may be arranged on the arm for acquiring physiological signals (e.g. myoelectric signals).
Specifically, constructing the initial data set includes:
n in round 1 using each existing electrode in the existing electrode set1The next physiological signal acquisition to obtain n1A sample of a physiological signal.
For each existing electrodeN collected in round 11Extracting d characteristics from each physiological signal sample in the physiological signal samples to obtain initial characteristics from the existing electrode in the 1 st roundIs n1X d matrix, i.e.Wherein (o) represents an existing electrode, 1 represents the 1 st round, i represents the ith existing electrode in the existing electrode set,representing the number of existing electrodes in the existing electrode set in the 1 st round;
from the initial features of all existing electrodes from the set of existing electrodes in round 1 and n acquired in round 11Constructing an initial data set D by using labels corresponding to physiological signal samples1,Wherein,initial features from all existing electrodes in the existing electrode set in round 1 are shown,is composed ofOf a matrix, i.e.WhereinRepresenting the number of existing electrodes in the existing electrode set in the 1 st round; y is1Denotes n collected from round 11Labels corresponding to individual physiological signal samples (labels indicating the category of action, e.g. gesture), Y1Is n1X c matrix, i.e.Where c represents the total number of categories.
In the above, for each physiological signal sample acquired in the 1 st round for each existing electrode, d feature values are extracted therefrom. For example, 9(d ═ 9) feature values as shown in table 2 may be extracted:
TABLE 2
And S112, constructing a random forest based on the initial data set as a recognition model H for motion recognition. Wherein the random forest comprises M decision trees (i.e., M individual classifiers), and each individual classifier is trained from an initial data set; each trained individual taxonomy classifier includes a root node, an internal node, and a leaf node, each internal node having a partitioning attribute (corresponding to a feature in the initial dataset) and a partitioning threshold for classifying datasets arriving at the internal node.
The result of motion recognition on a given sample x for a recognition model with M individual classifiers can be expressed as follows:
where H (x) represents the output of the recognition model for a given sample x,represents the m-th individual classifier hmFor the output of a given sample x on the c 'th class, M represents the total number of individual classifiers in the recognition model, 1 ≦ c' ≦ c, and c represents the total number of classes.
S113, constructing an initial electrode sketch of each existing electrode in the existing electrode set, wherein the electrode sketch is used for describing the characteristics of physiological signals acquired by the corresponding electrode.
From the initial data set constructed in substep S111, the corresponding acquired data for each existing electrode in the existing electrode set in round 1 may be obtainedWherein i represents the ith existing electrode in the existing electrode set,indicates the number of existing electrodes in the existing electrode set in run 1, and as described above,indicating the initial eigenvalues from the ith existing electrode in round 1,
for the ith existing electrode in the existing electrode set, the current can be measured from D1,iIn (1)In selection of liAn electrode sketch S for constructing the existing electrodeiSo that (S)i)TSi≈(D1,i)TD1,iAnd li=n1. Wherein S isiIs 1iX d matrix, i.e.For example, the classification matrix sketch technique proposed by Xin Mu et al may be employed to selectTo construct an electrode sketch. By constructing an electrode sketch, a lower dimensional matrix (e.g., S) may be usedi) To approximateHigher dimensional matrices (e.g. of)。
S12, acquiring a plurality of characteristic values of electromyographic signals of a plurality of electromyographic electrodes of the user action.
An existing electrode set for constructing a recognition model is arranged on a target to be recognized (for example, a human body or an animal whose action is to be recognized), and in addition, a newly added electrode can be arranged on the target to be recognized. N in the t (t ≧ 2) th round using electrodes arranged on the target to be recognizedtAcquiring n of the electrodes in the t roundtA sample of a physiological signal. It is understood that, in the t-th round, some existing electrodes in the existing electrode set for constructing the recognition model may be missing due to the cause (i.e., missing electrodes), so that the characteristic value of the electromyogram signal from the missing electrode in the t-th round cannot be acquired.
For each of the plurality of electrodes (i.e., including the non-missing electrode and the added electrode in the existing electrode set, wherein the non-missing electrode refers to the electrode in the existing electrode set other than the missing electrode), the n collected from the electrode in the t-th roundtD features are extracted from each physiological signal in the physiological signals, and a feature value from the electrode in the t round is obtained.
And constructing a characteristic value according to the characteristics of the electromyographic signals from the plurality of electrodes (namely, the electrodes which are not lost and are newly added in the existing electrode set) in the t-th round, so as to obtain the characteristic value collected from the target to be identified.
Step S13. determine whether the feature values obtained in step S12 are missing feature values from one or more existing electrodes in the existing electrode set, obtain a missing electrode in the existing electrode set in response to determining that the feature values from one or more electrodes in the existing electrode set are missing, and supplement the feature values from the missing electrode in the obtained feature values. Step S13 includes the following sub-steps:
s131, determining whether the acquired characteristic values lack the characteristic values from one or more existing electrodes in the existing electrode set according to the physiological signals acquired in the t round for constructing the acquired characteristic values.
For example, a mark may be added to the physiological signal sent by the electrode to indicate from which electrode the signal came. From this identification, it can be determined from which existing electrode(s) the characteristic value is missing from the acquired characteristic values.
S132, in response to determining that characteristic values of one or more electrodes in the existing electrode set are missing, one or more missing electrodes in the existing electrode set are obtained.
For example, if the physiological signal acquired in the t-th pass does not contain an identification of some existing electrode(s), then that electrode is considered to be the missing electrode.
And S133, for each missing electrode, selecting the non-missing electrode with the maximum correlation degree with the missing electrode from the existing electrode set.
According to an embodiment of the present invention, for each missing electrode, the following sub-steps S1331-1334 are performed:
and S1331, calculating the correlation between the physiological signals acquired by the missing electrode and each non-missing electrode. The pearson correlation coefficient between the electrode sketches of the two electrodes is taken as the correlation between the physiological signals acquired by the two electrodes, and is shown as the following formula:
in the formula (2), the first and second groups,electrode sketch S representing ith electrodeiElectrode sketch S with jth electrodejThe Pearson correlation coefficient between them, i ≠ j,has a value range of [ -1,1 [)]And is andthe larger the value of (A) indicates that the correlation between the physiological signals acquired by the ith electrode and the jth electrode is higher; cov denotes covariance; e represents expectation;electrode sketch S representing ith electrodeiThe average value of (a) of (b),electrode sketch S representing ith electrodeiStandard deviation of (2). It is to be understood that the calculationAndmanner and calculation ofAndin a similar manner, it can be calculated with reference to equations (3) and (4)And
and S1332, calculating the space distance between the missing electrode and each non-missing electrode. Wherein the spatial distance between the two electrodes can be calculated according to:
wherein,represents the spatial distance between the ith and jth electrodes, i ≠ j,has a value range of (0, 1)];pos(ei)=<haxisi,vaxisi>,pos(ej)=<haxisj,vaxisj>Respectively representing the coordinates of the ith electrode and the jth electrode on a two-dimensional space; i | pos (e)i),pos(ej) | | represents the euclidean distance between the ith electrode and the jth electrode.
In addition, for two electrodes in the same row or column of the electrode array (as shown in fig. 1), after the spatial distance between the two electrodes is calculated, a predetermined increment can be added to further improve the spatial position correlation between the two electrodes in the same row or column of the array. Preferably, the increment can be set to 1, i.e. 1Wherein 1 isThe upper limit value of (3).
And S1333, calculating distribution differences between the missing electrodes and each non-missing electrode according to the correlation between the physiological signals acquired by the electrodes and the space distance between the electrodes, wherein the distribution differences are used for measuring the correlation degree between the electrodes. Wherein the difference in distribution between the two electrodes can be calculated according to:
wherein DD (i.j) represents a difference in distribution between the ith electrode and the jth electrode, i ≠ j; alpha and beta represent parameters and are regularization coefficients used for limiting the value distribution of the DD (i.j) within a certain range; preferably, when α is 1 and β is 4, DD (i.j) can be made to have a value in the range of [0,1];Representing a correlation between physiological signals acquired by the ith electrode and the jth electrode;representing the spatial distance between the ith and jth electrodes. If the distribution difference between the two electrodes is larger, the two electrodes have more similar signal characteristics and closer spatial positions; conversely, if the distribution difference between two electrodes is small, it means that the two electrodes have signal characteristics with large difference and far spatial positions.
S1334, selecting the non-missing electrodes corresponding to the maximum distribution difference, as shown in the following formula:
j=argmaxj'(DD(i,j')),j≠i (7)
s134, for each missing electrode, searching the feature value from the selected non-missing electrode in the t-th round in the feature values obtained in step S12, and calculating the feature value from the missing electrode in the t-th round according to the searched feature value of the non-missing electrode.
Suppose thatBy pseudo-inverse calculationI.e., the characteristic value from the missing electrode in the t-th run, as shown in the following equation:
wherein i represents that the missing electrode is the first in the existing electrode setEach electrode, j indicates that the non-missing electrode selected for the missing electrode is the first electrode in the existing electrode setA plurality of electrodes, each of which is provided with a plurality of electrodes,representing the number of existing electrodes in the existing electrode set in the t-th round;representing the eigenvalues from the ith electrode in the existing set of electrodes in the t-th run,representing a characteristic value from a jth electrode in the existing electrode set in the tth round; siElectrode sketching, S, representing the ith electrode in an existing electrode setjAn electrode sketch representing the jth electrode in the existing electrode set.
S135. in the feature values acquired in step S12, the feature values from each missing electrode in the t-th round are supplemented.
And S14, taking the acquired characteristic value as the input of the recognition model to obtain an action recognition result.
And S15, if the action recognition is not finished, returning to the step S12, otherwise, finishing the method.
According to an embodiment of the present invention, in step S15, if the motion recognition is not finished, the following processing is further executed before returning to step S12:
updating the electrode sketch for each existing electrode in the existing electrode set, and making t equal to t + 1. Similar to constructing the initial electrode sketch, updating the electrode sketch for each existing electrode in the existing electrode set includes: root of herbaceous plantObtaining a characteristic value of each existing electrode from the existing electrode set in the t round according to the obtained characteristic value; from the eigenvalues of each existing electrode from the set of existing electrodes in the t-th run and n acquired in the t-th runtThe label corresponding to each physiological signal sample constructs the corresponding acquired data of each existing electrode in the existing electrode set in the t roundWherein,representing the eigenvalues from the ith existing electrode in the t-th run,indicating the number of existing electrodes in the existing electrode set in the t-th run, YtRepresenting n collected in the t-th roundtLabel corresponding to a sample of a physiological signal, YtIs ntX c matrix, i.e.c represents the total number of categories; for the ith existing electrode in the existing electrode set, the current can be measured from Dt,iIn (1)In selection of liUpdating the electrode profile S of the existing electrodeiSo that (S)i)TSi≈(Dt,i)TDt,iAnd li=nt. Wherein S isiIs 1iX d matrix, i.e.
In the above-described embodiments, an electrode sketch is constructed and updated for each existing electrode in the existing electrode set, which can be used to calculate correlations between physiological signals acquired by the electrodes, as well as to calculate eigenvalues from the missing electrodes. In other embodiments, the electrode sketch may not be constructed for the existing electrodes, for example, the spatial distance between the electrodes may be used to measure the distribution difference (i.e., the degree of correlation) between the electrodes; and for a missing electrode in the t (t ≧ 2) round, the eigenvalue from that missing electrode in the t round can be complemented with the eigenvalue from that electrode in the previous round.
Furthermore, although in the above described embodiments pearson's correlation coefficient is used to calculate the correlation between physiological signals acquired by the electrodes, in other embodiments other linear correlation methods may be used to measure the correlation between variables; in addition, the spatial distance between the electrodes may also be calculated using the euclidian distance of the electrodes in the three-dimensional space.
According to an embodiment of the invention, the server may further update the recognition model in case of feature increase to make the recognition model more fit to the actual action recognition scenario.
Referring to fig. 6, in summary, feature-adaptive action recognition by a server includes: s21, learning and constructing an identification model based on an initial data set, wherein the initial data set comprises characteristic values of physiological signals of all myoelectric electrodes corresponding to the user action and labels corresponding to the user action, and the labels are used for indicating the category of the action; s22, acquiring a plurality of characteristic values of a plurality of electromyographic signals of user actions; s23, determining whether the acquired characteristic values lack the characteristic values from one or more electrodes in the existing electrode set or not, obtaining the missing electrodes in the existing electrode set in response to determining that the characteristic values from one or more electrodes in the existing electrode set are lacked, and supplementing the characteristic values from the missing electrodes in the acquired characteristic values; s24, taking the obtained characteristic value as the input of an identification model to obtain an action identification result; s25, determining whether a characteristic value from a newly added electrode exists in the obtained characteristic values, responding to the determination that the characteristic value from the newly added electrode exists, constructing an incremental data set, updating an identification model based on the incremental data set, and adding the newly added electrode into an existing electrode set; and S26, if the action recognition is not finished, returning to the S22, otherwise, finishing the method.
The server feature adaptive action recognition will be described below in conjunction with fig. 6.
S21, constructing an identification model based on the initial data set, wherein the identification model comprises the following steps: constructing an initial dataset D for round 11,The initial data set comprises characteristic values of physiological signals of all myoelectric electrodes corresponding to the user action and labels corresponding to the user action, and the labels are used for indicating the category of the action; constructing a random forest as an identification model for action identification based on the initial data set, wherein the random forest comprises M individual classifiers, each individual classifier comprises a root node, an internal node and a leaf node, and each internal node has a segmentation attribute and a segmentation threshold; and constructing an initial electrode sketch of each existing electrode in the existing electrode set.
And S22, acquiring a plurality of characteristic values of a plurality of electromyographic signals of the user action.
The existing electrode set for constructing the recognition model is arranged on the target to be recognized, and in addition, the newly added electrodes can be arranged on the target to be recognized. N in the t (t ≧ 2) th round using electrodes arranged on the target to be recognizedtAcquiring n of the electrodes (including the non-missing electrodes and the newly added electrodes in the existing electrode set) in the t round of physiological signal acquisitiontAnd d features are extracted from each physiological signal.
Step S23. determine whether the feature values obtained in step S22 are missing feature values from one or more existing electrodes in the existing electrode set, obtain missing electrodes in the existing electrode set in response to determining that the feature values from one or more electrodes in the existing electrode set are missing, and supplement the feature values from the missing electrodes in the obtained feature values.
And S24, taking the acquired characteristic value as the input of the recognition model to obtain an action recognition result.
And S25, determining whether the characteristic value from the newly added electrode exists in the acquired characteristic values, constructing an incremental data set in response to the determination that the characteristic value from the newly added electrode exists, updating the identification model based on the incremental data set, and adding the newly added electrode into the existing electrode set. Step S25 includes the following sub-steps:
s251, determining whether there are features from the newly added electrode (i.e., one or more electrodes outside the existing electrode set) in the acquired feature values according to the physiological signals acquired in the t-th round for constructing the acquired feature values.
For example, by adding a flag to the physiological signal sent by the electrode to indicate from which electrode the signal came. From this identification, it can be determined from which newly added electrode(s) the characteristic value(s) from among the acquired characteristic values exist.
And S252, in response to determining that the characteristic value from the newly added electrode exists, determining the newly added electrode. For example, the newly added electrode may be determined from an identification contained in the physiological signal.
And S253, constructing an increment data set under the condition that a newly added electrode exists.
Specifically, the incremental data set D of the t-th round is constructed based on the acquired characteristic valuest,Wherein,representing the eigenvalues of all existing electrodes from the set of existing electrodes in the t-th run,representing the number of eigenvalues from all existing electrodes in the existing electrode set in the t-th run,representing the number of existing electrodes in the existing electrode set in the t-th round;representing the eigenvalues from all newly added electrodes in the t-th run,representing the number of eigenvalues from all newly added electrodes in the t-th round,representing the number of newly added electrodes in the t-th round; y istRepresenting n collected in the t-th roundtLabel corresponding to a sample of a physiological signal, YtIs ntX c matrix, i.e.And c represents the total number of categories.
S254, updating the identification model based on the incremental data set. The method comprises the following substeps:
s2541, selecting an individual classifier to be updated in the recognition model.
The individual classifier is evaluated by its accuracy and all the features used to construct it, as shown in the following equation:
wherein, S (h)m) Represents the m-th individual classifier hmM is more than or equal to 1 and less than or equal to M, wherein M represents the total number of the individual classifiers; acc (h)m) Represents hmThe classification accuracy of (2); # featuremRepresentation construction hmThe number of all features used;representing the number of eigenvalues from all existing electrodes in the existing electrode set at round t.
After the evaluation values of all the individual classifiers in the identification model are obtained, the individual classifiers are ranked according to the evaluation values from small to large, and the top M × δ individual classifiers are selected as the individual classifiers to be updated. Wherein δ represents a preset update ratio of the individual classifiers, and M represents the total number of the individual classifiers.
S2542, for each individual classifier to be updated, performing structure adjustment and/or segmentation threshold adjustment on the individual classifier based on the incremental data set.
A. Structural adjustment (FIDE _ s for short):
for each individual classifier to be updated, the impurity degree (e.g., measured by error rate, entropy, kini index, etc.) of a leaf node that can be reached by the incremental dataset in the individual classifier is calculated, and if the calculated impurity degree of the leaf node reaches a predetermined threshold (i.e., indicating that the leaf node is not pure), the splitting operation is performed on the leaf node.
B. Segmentation threshold adjustment (FIDE _ m for short):
and presetting a corresponding candidate segmentation threshold value set for each internal node in each individual classifier.
For each internal node in each individual classifier to be updated, calculating a divergence index corresponding to each candidate segmentation threshold value in the candidate segmentation threshold value set of the internal node according to the following formula:
2JSD(P1,P2)=KL(P1 PPavg)+KL(P2 PPavg) (11)
Pavg=(P1+P2)/2 (12)
wherein v represents an internal node v in the individual classifier to be updated; dvRepresents a subset of reachable nodes v in the incremental dataset; phi (v) represents the segmentation attribute of node v (corresponding to a feature in the feature space); τ (v) represents the current segmentation threshold for node v; qLAnd QRRespectively representing D when the current segmentation threshold is tau (v)vThe label distribution in (1); q'LAnd Q'RRespectively representing D when using the candidate segmentation thresholdvThe label distribution in (1); i DvI denotes DvThe size of (d); i DL,τI represents the subset size of the left child node which can reach the node v in the incremental data set when the current segmentation threshold is tau (v); i DR,τI represents the subset size of the right child node which can reach the node v in the incremental data set when the current segmentation threshold is tau (v); pavgRepresents the distribution P1And P2The mean value of (a); Jensen-Shannon Divergence (JSD) is a symmetric extension of Kullback-Leibler Divergence (KL), JSD (P1,P2) For measuring two distributions P1And P2The distance of (c). In formula (10), JSD (Q'L,QL) And JSD (Q'R,QR) Respectively representing the JSD distances of the left and right subtrees of node v when it uses the candidate and current partitioning thresholds τ (v). In the formula (10), the divergence index has a value range of [0,1 ]]A larger divergence index indicates a better corresponding candidate segmentation threshold.
After traversing all candidate segmentation threshold values in the candidate segmentation threshold value set, selecting the candidate segmentation threshold value corresponding to the maximum divergence index to update the current segmentation threshold value of the node v.
The update process of the recognition model can be seen in algorithm 1.
S255, adding the newly added electrode into the existing electrode set
And S26, if the action recognition is not finished, returning to the S22, otherwise, finishing the method.
According to an embodiment of the present invention, in step S26, if the motion recognition is not finished, the electrode sketch of each existing electrode in the existing electrode set is updated before returning to step S22, wherein the electrode sketch is constructed for the new feature value, and t is made t + 1.
The feature adaptive motion recognition method provided by the embodiment can realize motion recognition by using the existing recognition model under the condition of feature value missing, and update the recognition model on the basis of newly added feature values, so that the recognition model is more consistent with the actual motion recognition scene, and the precision of motion recognition is improved. The identification model is updated by adjusting the structure and/or the segmentation threshold, so that new knowledge is learned while useful knowledge learned by the identification model is not lost, and the requirement of the updated identification model on the sample size is met by the incremental data set.
In the above-described embodiment, the recognition model is updated as the feature value increases. According to another embodiment of the invention, no matter whether the newly added electrode exists or not, the recognition model can be updated by reducing useless internal nodes after the action recognition result is obtained, so that the action recognition accuracy is further improved.
Specifically, after an action recognition result is obtained, constructing an incremental data set of the t-th turn based on the obtained characteristic values, and selecting an individual classifier to be updated in the recognition model; for each individual classifier to be updated, calculating a subtree penalty (i.e., evaluating the penalty value of the subtree having the internal node as the root node) and a leaf penalty (i.e., the penalty value if the internal node is clipped to a leaf node) for its internal node; nodes with a clipping sub-tree loss greater than a leaf loss or nodes unreachable by the incremental dataset.
Referring to fig. 7, assume that the classification label of the sample on the node v is 1 (same as the leaf node class determination mechanism, determined in the training process according to the majority voting method), and two child nodes v thereoflAnd vrThe classification labels of (1) and (0) are respectively. In the updating process, assuming that 10 samples in the incremental dataset reach the node v and the real labels thereof are {1,1,1,1,1,1,1,1,0,0} respectively, all the samples are judged to belong to class 1, wherein two samples are judged to belong to class 1Break error, with a loss of 20%; the 10 samples arrive at node v along a branch of node vlAnd vrIn time, assuming that 5 samples reach two branches, the labels are {1,1,1,1,1} and {1,1,1,0,0}, respectively, then these are determined as {1,1,1,1,1} and {0,0,0,0,0}, respectively, i.e., 0 and 3 samples are determined as being wrong, and the loss is 0% and 60%, respectively. At this time, for node v, its subtree loss is vlAnd vrThe sum of the weighted losses of 5/10 × 0% +5/10 × 60% + 30%; whereas if v is considered a leaf node, the loss is only 20%. The subtree loss is now greater than the leaf loss, and therefore the internal node v will be pruned away.
Experimental part:
to verify the beneficial effects of the feature adaptive motion recognition system provided by the present invention, the above two updating methods (including FIDE _ s and FIDE _ m) performed by the server of the present invention are compared with the following five related methods, which are described in detail as follows:
extreme random trees (ET): an ensemble learning method based on decision trees in which segmentation attributes and segmentation thresholds are selected randomly.
K-grouped random forest (KSRF): a grouped random forest method is used for grouping different characteristics and respectively constructing a decision tree model.
Improved feature incremental random forest (iFIRF): the original FIRF can only adapt to the characteristic increment scene, and the iFIRF is improved by adopting a characteristic missing value processing strategy, wherein the characteristic missing value processing strategy is similar to the characteristic missing value processing strategy in the FIDE.
Common feature missing value processing Method (MCFI): applicants consider the reduced features as feature missing and fill in with the most common features in the feature values. For added features, applicants have used methods similar to those in FIDE _ s and FIDE _ m to process two variations of constructing MCFI, including MCFI _ s and MCFI _ m.
For the ET and KSRF methods, applicants train the initial recognition model using all the initial data sets and the data streams during the incremental update, so ET and KSRF are the most ideal recognition models, i.e., ideally, FIDE _ s and FIDE _ m only have an effect close to ET and KSRF, but hardly exceed them.
(1) Experimental verification under personal computer client
During the experiment, four males and two females participated in the data collection, and their average age was 23.33 years. From the demonstration video, each subject sequence performed 15 different gestures, including five finger gestures (i.e., "bend thumb", "bend index finger", "bend middle finger", "bend ring finger", and "bend little finger"), six wrist gestures (i.e., "bend", "extend", "adduction", "abduction", "ext", "extorsion", and "pronation"), and other four gestures (i.e., "open palm", "spread open", "fist making", and "finger"). In the acquisition process, each gesture is repeated eight times; meanwhile, the applicant only reserves partial electrodes of the myoelectric arm ring type, and the rest electrodes are shielded by using adhesive tapes so as to simulate the increase and decrease processes of the electrodes in a real situation. FIG. 8 illustrates the evolution of electrodes, initially, only two electrodes (electrode 1 and electrode 2) are available to build an initial gesture recognition model; during the second, third and fourth increments, applicants successively added two electrodes. At the same time, electrode 1 disappeared during the third increment and electrode 2 disappeared during the fourth increment.
Table 3 gesture recognition accuracy (%) comparison during different incremental updates using a personal computer as the client
Table 4 model update time (seconds) and gesture recognition time (milliseconds) in different incremental update processes using a personal computer as a client
Gesture recognition time (Hao second) in different incremental updating processes
The applicant compared the recognition accuracy of FIDE _ s and FIDE _ m in this scenario with the five comparison methods. Generally, the gesture recognition accuracy increases as the number of electrodes increases, from 45% of the initial model to 90% after the fourth incremental update. Although one electrode is lost in the third incremental updating process and the fourth incremental updating process, the accuracy of gesture recognition is less affected, and the result is improved compared with the result of the last incremental updating process. Comparing the FIDE _ s and FIDE _ m methods, FIDE _ m updates batches at each increment, and recognition accuracy is superior to the FIDE _ s method. In the fourth incremental update batch, FIDE _ m achieved 90.83% identification accuracy, better than the MCFI _ s, MCFI _ m and iFIRF methods, and closer to the experimental results in the ideal case (i.e., KSRF and ET).
Applicants also compared the model update and the average time consumption for recognizing a gesture, and the results are shown in table 3. Average model update times for FIDE _ s and FIDE _ m are 0.86 seconds and 5.50 seconds, respectively, which results are acceptable to the user in most cases; in terms of time consumption in recognizing one gesture, only about 1 millisecond is required to recognize one gesture using FIDE _ s and FIDE _ m, meaning that the gesture recognition system can achieve a fast response.
(2) Experimental verification under personal computer client
A total of 16 subjects, including four females and 12 males, were recruited in this experiment, with a mean age of 69.4 years. Each subject made two different gestures, including a fist and a finger, as shown in FIG. 9, were diagnostic gestures specified on the Unified Parkinson's Disease Rating Scale (UPDRS). In the experimental process, a testee is required to wear a myoelectric arm ring on a hand used by the testee; meanwhile, the myoelectric arm ring automatically sends the collected surface myoelectric signal to the smart phone client through Bluetooth; and finally, the smart phone sends the electromyographic signals to a remote server through WiFi.
In addition, the electrode increment updating process, the feature extraction and the parameter setting in the experiment are all similar to those in table 2, the electrode increment is updated 4 times, as shown in fig. 8, only two electrodes are available on the myoelectric arm ring in the initial state, two electrodes are gradually added each time in the subsequent increment updating process, and the electrode 1 and the electrode 2 are removed in the third and fourth increment processes. In terms of feature extraction, we will extract nine-dimensional features from each electrode as shown in table 2.
Table 5 gesture recognition accuracy (%) comparison in different incremental updating processes using a smartphone as a client
Table 6 model update time (sec) versus gesture recognition time (ms) during different incremental updates using a smartphone as a client
Gesture recognition time (Hao second) in different incremental updating processes
The experimental results using the smartphone as the client are shown in table 5, and when all eight electrodes are available, the final gesture recognition accuracy of the FIDE _ s and FIDE _ m methods is 92.65% and 94.14%, which is better than that of other comparison methods, but the recognition results in an ideal case are still different. Model updating and recognition efficiency are also important influence factors in the real scene gesture recognition problem, time consumption for incremental model updating and gesture recognition is shown in table 6, and the time consumption for model updating of FIDE _ s and FIDE _ m methods is 7.57 seconds respectively; the maximum time taken for the FIDE _ s and FIDE _ m methods to recognize a user gesture is 0.77 milliseconds, which means that the FIDE _ s and FIDE _ m methods require only a small time consumption when recognizing a user gesture, which is feasible in practical application scenarios.
It should be noted that some exemplary methods are depicted as flowcharts. Although a flowchart may describe the operations as being performed serially, it can be appreciated that many of the operations can be performed in parallel, concurrently, or with synchronization. In addition, the order of operations may be rearranged, for example, step S25 may be performed in synchronization with step S23 or before step S23. A process may terminate when an operation is completed, but may have additional steps not included in the figure or embodiment.
The above-described methods may be implemented by hardware, software, firmware, middleware, pseudocode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or pseudo code, the program code or code segments to perform the tasks may be stored in a computer readable medium such as a storage medium, and a processor may perform the tasks.
It should be appreciated that the software-implemented exemplary embodiment is typically encoded on some form of program storage medium or implemented over some type of transmission medium. The program storage medium may be any non-transitory storage medium such as a magnetic disk (e.g., a floppy disk or a hard drive) or an optical disk (e.g., a compact disk read only memory or "CD ROM"), and may be read only or random access. Similarly, the transmission medium may be twisted wire pairs, coaxial cable, optical fiber, or some other suitable transmission medium known to the art.
Although the present invention has been described by way of preferred embodiments, the present invention is not limited to the embodiments described herein, and various changes and modifications may be made without departing from the scope of the present invention.
Claims (10)
1. A feature adaptive motion recognition system, the system comprising:
the sensing terminal comprises a plurality of electromyographic electrodes and is used for acquiring corresponding electromyographic signals generated by user actions and sending the corresponding electromyographic signals to the client;
the client is used for receiving the electromyographic signals sent by the sensing terminal and uploading the electromyographic signals to a server;
a server for extracting one or more feature values from each electromyographic signal and recognizing an action of the user based on the feature values of the electromyographic signal, the server configured to:
learning and constructing a recognition model based on an initial data set, wherein the initial data set comprises characteristic values of physiological signals of all collected myoelectric electrodes corresponding to user actions and labels corresponding to the user actions, and the labels are used for indicating the categories of the actions;
acquiring a plurality of characteristic values of electromyographic signals of a plurality of electromyographic electrodes of user actions to be identified;
determining whether a characteristic value from one or more of the all electrodes is missing from the acquired characteristic values;
in response to determining that a characteristic value from one or more of the all electrodes is missing from the acquired characteristic values, treating the one or more electrodes as one or more missing electrodes, supplementing the acquired characteristic values with characteristic values from the one or more missing electrodes; and
and taking the acquired characteristic value as the input of the recognition model to obtain an action recognition result.
2. The system of claim 1, wherein supplementing the acquired feature values with feature values from the one or more missing electrodes comprises:
for each missing electrode of the one or more missing electrodes, selecting an un-missing electrode of the existing electrode set that is most relevant to the missing electrode, and supplementing the feature value from the missing electrode according to the feature value from the selected un-missing electrode in the obtained feature values;
wherein the non-missing electrode represents an electrode in the existing electrode set other than the one or more missing electrodes.
3. The system of claim 2, wherein the server is further configured for:
for each electrode in the existing electrode set, constructing an electrode sketch for that electrode based on the initial data set; and
after obtaining the action recognition result, updating the electrode sketch of each electrode in the existing electrode set;
wherein supplementing feature values from the missing electrode according to feature values from the selected non-missing electrode in the acquired feature values comprises:
the eigenvalues from the missing electrode are calculated based on the following equation:
wherein i indicates that the missing electrode is the ith electrode in the existing electrode set, j indicates that the selected non-missing electrode is the jth electrode in the existing electrode set, t indicates the round corresponding to the acquired characteristic value, (o) indicates the existing electrode,representing the characteristic value, S, from the missing electrodeiElectrode sketch showing the missing electrode, SjAn electrode sketch representing selected non-missing electrodes,representing a characteristic value from the selected non-missing electrodes among the acquired characteristic values; and
supplementing the feature value from the missing electrode in the acquired feature value.
4. The system of claim 3, wherein the server is further configured for:
for each missing electrode of the one or more missing electrodes, calculating a degree of correlation between the missing electrode and each non-missing electrode of the existing electrode set;
wherein the degree of correlation between two electrodes includes the pearson correlation coefficient between the electrode sketches of the two electrodes and the spatial distance between the two electrodes.
5. The system according to any of claims 1-4, wherein the server is further configured for:
determining whether a characteristic value from a newly added electrode exists in the acquired characteristic values; and
in response to determining that a feature value from a newly added electrode exists in the obtained feature values, constructing an incremental dataset based on the obtained feature values, updating the identification model based on the incremental dataset, and adding the newly added electrode to the existing electrode set.
6. The system of claim 5, wherein the building a recognition model based on the initial dataset comprises:
constructing a plurality of individual classifiers based on the initial dataset to form a random forest, the random forest being used as the recognition model; and
updating the recognition model based on the incremental dataset comprises:
calculating an evaluation value for each individual classifier in the recognition model based on:
wherein, S (h)m) Representing the m-th individual classifier h in the recognition modelmThe evaluation value of (1); acc (h)m) Represents hmThe classification accuracy of (2); # featuremRepresentation construction hmThe number of all features used;representing the number of eigenvalues from all electrodes in the existing electrode set, t indicating the round to which the obtained eigenvalues correspond;
selecting individual classifiers with a preset proportion from all the individual classifiers in the identification model from small to large according to the evaluation values as the individual classifiers to be updated; and
updating the individual classifier to be updated based on the incremental dataset.
7. The system of claim 6, wherein the updating the individual classifiers to be updated based on the incremental dataset comprises:
calculating the impurity degree of leaf nodes which can be reached by the incremental dataset in the individual classifier to be updated;
if the leaf node's impurity level reaches a predetermined threshold, then a splitting operation is performed on the leaf node.
8. The system of claim 6 or 7, wherein the updating the individual classifier to be updated based on the incremental dataset comprises:
for each internal node in the individual classifier to be updated, calculating a divergence index corresponding to each candidate segmentation threshold in the candidate segmentation threshold set of the internal node based on the following formula:
wherein v represents an interior in the individual classifier to be updatedA node v; dvRepresenting a subset of the incremental dataset within which an internal node v can be reached; phi (v) represents the partitioning property of the internal node v; τ (v) represents the current segmentation threshold for internal node v; qLAnd QRRespectively representing D when the current segmentation threshold is tau (v)vThe label distribution in (1); q'LAnd Q'RRespectively, D when using the candidate segmentation thresholdvThe label distribution in (1); i DvI denotes DvThe size of (d); i DL,τL represents the subset size of the left child node which can reach the internal node v in the incremental data set when the current segmentation threshold is tau (v); i DR,τL represents the subset size of the right child node which can reach the internal node v in the incremental data set when the current segmentation threshold is tau (v); 2JSD (P)1,P2)=KL(P1PPavg)+KL(P2 PPavg) And P isavg=(P1+P2) (ii)/2, wherein KL (. cndot.) represents the Kullback-Leibler divergence;
and selecting the candidate segmentation threshold corresponding to the maximum divergence index from the candidate segmentation threshold set to update the current segmentation threshold of the internal node.
9. The system of claim 5, wherein the server is further configured to:
and constructing an electrode sketch for the newly added electrode.
10. The system according to any of claims 1-4, wherein said building a recognition model based on the initial dataset comprises:
constructing a plurality of individual classifiers based on the initial dataset to form a random forest, the random forest being used as the recognition model; and
the server is further configured to:
after obtaining an action recognition result, selecting an individual classifier to be updated from the recognition model; and
and for the individual classifier to be updated, carrying out clipping operation on the internal node of the individual classifier.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110514238.6A CN113297935A (en) | 2021-05-12 | 2021-05-12 | Feature adaptive motion recognition system |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110514238.6A CN113297935A (en) | 2021-05-12 | 2021-05-12 | Feature adaptive motion recognition system |
Publications (1)
Publication Number | Publication Date |
---|---|
CN113297935A true CN113297935A (en) | 2021-08-24 |
Family
ID=77321450
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202110514238.6A Pending CN113297935A (en) | 2021-05-12 | 2021-05-12 | Feature adaptive motion recognition system |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN113297935A (en) |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140204013A1 (en) * | 2013-01-18 | 2014-07-24 | Microsoft Corporation | Part and state detection for gesture recognition |
WO2021052045A1 (en) * | 2019-09-17 | 2021-03-25 | 北京海益同展信息科技有限公司 | Body movement recognition method and apparatus, computer device and storage medium |
-
2021
- 2021-05-12 CN CN202110514238.6A patent/CN113297935A/en active Pending
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140204013A1 (en) * | 2013-01-18 | 2014-07-24 | Microsoft Corporation | Part and state detection for gesture recognition |
WO2021052045A1 (en) * | 2019-09-17 | 2021-03-25 | 北京海益同展信息科技有限公司 | Body movement recognition method and apparatus, computer device and storage medium |
Non-Patent Citations (1)
Title |
---|
YINGWEI ZHANG 等: "A Feature Adaptive Learning Method for High-Density sEMG-Based Gesture Recognition", 《PROC. ACM INTERACT. MOB. WEARABLE UBIQUITOUS TECHNOL.》 * |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP7422946B2 (en) | Automatic construction of neural network architecture using Bayesian graph search | |
CN112800998B (en) | Multi-mode emotion recognition method and system integrating attention mechanism and DMCCA | |
US20200334451A1 (en) | Motion behavior pattern classification method, system and device | |
JP6831453B2 (en) | Signal search device, method, and program | |
CN107609588B (en) | Parkinson patient UPDRS score prediction method based on voice signals | |
CN108703824B (en) | Bionic hand control system and control method based on myoelectricity bracelet | |
CN112256123A (en) | Brain load-based control work efficiency analysis method, equipment and system | |
CN111797674B (en) | MI electroencephalogram signal identification method based on feature fusion and particle swarm optimization algorithm | |
Kini et al. | Large margin mixture of AR models for time series classification | |
CN113158861B (en) | Motion analysis method based on prototype comparison learning | |
WO2018036547A1 (en) | Data processing method and device thereof | |
KR20210095671A (en) | Image processing method and related device | |
CN112256122B (en) | Control work efficiency analysis method, device and system based on mental fatigue | |
CN117407772B (en) | Method and system for classifying training multi-element time sequence data by supervising and comparing learning network model | |
CN112256124A (en) | Emotion-based control work efficiency analysis method, equipment and system | |
CN113807176A (en) | Small sample video behavior identification method based on multi-knowledge fusion | |
US12114984B2 (en) | System and apparatus for generating imaging information based on at least a signal | |
CN112200025A (en) | Operation and control work efficiency analysis method, device and system | |
CN112465054B (en) | FCN-based multivariate time series data classification method | |
CN114757310B (en) | Emotion recognition model and training method, device, equipment and readable storage medium thereof | |
CN113297935A (en) | Feature adaptive motion recognition system | |
CN114548262B (en) | Feature level fusion method for multi-mode physiological signals in emotion calculation | |
CN114398991B (en) | Electroencephalogram signal emotion recognition method based on Transformer structure search | |
CN112861796B (en) | Feature adaptive motion recognition method | |
Suresh et al. | Uncertain data analysis with regularized XGBoost |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
WD01 | Invention patent application deemed withdrawn after publication | ||
WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20210824 |