CN113598759B - Myoelectricity feature optimization-based lower limb action recognition method and system - Google Patents

Myoelectricity feature optimization-based lower limb action recognition method and system Download PDF

Info

Publication number
CN113598759B
CN113598759B CN202111066419.3A CN202111066419A CN113598759B CN 113598759 B CN113598759 B CN 113598759B CN 202111066419 A CN202111066419 A CN 202111066419A CN 113598759 B CN113598759 B CN 113598759B
Authority
CN
China
Prior art keywords
lower limb
feature
optimization
signal
signals
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111066419.3A
Other languages
Chinese (zh)
Other versions
CN113598759A (en
Inventor
曹佃国
王加帅
武玉强
张中才
王金强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qufu Normal University
Original Assignee
Qufu Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qufu Normal University filed Critical Qufu Normal University
Priority to CN202111066419.3A priority Critical patent/CN113598759B/en
Publication of CN113598759A publication Critical patent/CN113598759A/en
Application granted granted Critical
Publication of CN113598759B publication Critical patent/CN113598759B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/103Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1123Discriminating type of movement, e.g. walking or running
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/103Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1126Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/24Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316Modalities, i.e. specific diagnostic methods
    • A61B5/389Electromyography [EMG]
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/24Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316Modalities, i.e. specific diagnostic methods
    • A61B5/389Electromyography [EMG]
    • A61B5/397Analysis of electromyograms
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7203Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235Details of waveform analysis
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235Details of waveform analysis
    • A61B5/725Details of waveform analysis using specific filters therefor, e.g. Kalman or adaptive filters
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235Details of waveform analysis
    • A61B5/7253Details of waveform analysis characterised by using transforms
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235Details of waveform analysis
    • A61B5/7264Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Veterinary Medicine (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Public Health (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Physiology (AREA)
  • Artificial Intelligence (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Psychiatry (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Dentistry (AREA)
  • Evolutionary Computation (AREA)
  • Fuzzy Systems (AREA)
  • Mathematical Physics (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)

Abstract

The invention provides a lower limb action recognition method and system based on myoelectricity feature optimization, wherein the method comprises the following steps: acquiring a plurality of electromyographic signals corresponding to the lower limb actions in real time; extracting main characteristic section signals from the electromyographic signals respectively; respectively extracting the characteristics of each main characteristic segment signal, and carrying out characteristic optimization according to the correlation between the electromyographic signal energy and the action; fusing the optimized multiple features to obtain a feature vector; and based on the feature vector, performing lower limb action recognition by adopting a pre-trained lower limb action recognition model. According to the invention, by introducing weighted feature optimization based on the correlation degree of muscles and motions, the feature redundancy is reduced, and meanwhile, as much useful information as possible is reserved, so that the recognition efficiency and accuracy are improved.

Description

Myoelectricity feature optimization-based lower limb action recognition method and system
Technical Field
The invention belongs to the technical field of action recognition, and particularly relates to a lower limb action recognition method and system based on myoelectricity feature optimization.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
The lower limb action recognition technology based on the surface electromyographic signals has the characteristics of safety, real time and convenience, and is widely applied to various fields such as sports medicine, biomedicine, rehabilitation engineering, intelligent artificial limbs and the like.
Lower limb motion recognition techniques typically include multiple stages of feature extraction, processing, and classification based on features. For example, chinese patent CN110238863B proposes a lower limb rehabilitation robot control method and system based on electroencephalogram-myoelectric signals, which uses electroencephalogram-myoelectric coherence analysis to perform 3 simple lower limb motion recognition. Chinese patent CN107092861B proposes a lower limb motion recognition method based on pressure and acceleration sensors, after extracting the required features, dimension reduction processing is adopted to reduce the 48-dimension features to 5 dimensions, and 6 simple lower limb motions are recognized by a simple SVM classifier. Chongqing university automation institute Dan Xin and the like propose a lower limb movement characteristic extraction and classification method based on surface electromyographic signals, and 5 simple lower limb movements are identified offline by utilizing sEMG through simple characteristic fusion.
In the prior art including the implementation method, during the characteristic processing process of the electromyographic signals corresponding to the lower limb actions, a part of researchers perform dimension reduction processing on the characteristics, so that part of effective characteristics can be lost while the redundancy of the characteristics is reduced, the data volume can be reduced, the recognition speed can be improved, but the distinguishing degree of the actions is reduced, and the recognition rate is reduced; some researchers do not process the extracted features, input the features into a classifier for recognition after simple feature fusion, and the recognition speed is low, the delay is high and the timeliness is poor although all effective signals are reserved, so that the method has no practical popularization and application value. In the classifier training model process, part of personnel uses the most original support vector machine to carry out simple classification, the process of parameter optimization is not provided, the recognition rate cannot be guaranteed, better parameters can be obtained by carrying out parameter optimization through a genetic algorithm, the recognition rate is improved to a certain extent, but the traditional genetic algorithm selection operator is easy to fall into a local optimal solution, has poor adaptability and is not suitable for actual action recognition.
Disclosure of Invention
In order to overcome the defects of low recognition rate, poor adaptability and low practical application value of the classification method, the invention provides a lower limb action recognition method and system based on electromyographic signal feature optimization, a weighted feature optimization algorithm based on muscle and motion correlation is introduced, the feature redundancy is reduced, simultaneously, as much useful information as possible is reserved, and the recognition rate is improved while the recognition speed is improved.
To achieve the above object, one or more embodiments of the present invention provide the following technical solutions:
a lower limb action recognition method based on myoelectricity feature optimization comprises the following steps:
acquiring a plurality of electromyographic signals corresponding to the lower limb actions in real time;
extracting main characteristic section signals from the electromyographic signals respectively;
respectively extracting the characteristics of each main characteristic segment signal, and carrying out characteristic optimization according to the correlation between the electromyographic signal energy and the action;
and fusing the optimized multiple features to obtain feature vectors, and performing lower limb action recognition by adopting a pre-trained lower limb action recognition model based on the feature vectors.
Further, after a plurality of electromyographic signals of the lower limb are obtained, denoising pretreatment is performed on the plurality of electromyographic signals: the wavelet transformation threshold denoising method and the digital filtering threshold denoising method remove baseline drift, high-frequency noise and low-frequency noise, and the 50Hz power frequency interference is removed by using a wave trap.
Further, the main characteristic segment signal extraction adopts a framing energy method:
framing the electromyographic signals, and calculating the total energy of signals in each frame;
if the total energy in a certain frame is larger than the set threshold value and the total energy in the set number of frames is larger than the set threshold value, the frame is used as the initial frame of an action segment;
the electromyographic signals of the set time from the initial frame are extracted to obtain the main characteristic section signals.
Further, the feature optimization according to the correlation between the electromyographic signal energy and the action comprises:
extracting the characteristics of the main characteristic section signals;
acquiring signal energy values of corresponding channels of each muscle in the main characteristic section signals;
calculating the weight of the channel in the main characteristic section signal according to the proportion of the signal energy value of each channel accounting for the summation of the signal energy of all channels;
and carrying out weighted optimization on the characteristics of each channel signal of the main characteristic segment signal according to the weights to obtain optimized characteristics.
Further, the lower limb action recognition model is obtained by training based on a support vector machine classifier.
Further, when training is performed based on the support vector machine classifier, a genetic algorithm based on combination of tournament and sorting is adopted for parameter optimization.
Further, the process of parameter optimization includes:
(1) Determining an initial population, and calculating the fitness of individuals in the population based on feature vectors obtained through feature extraction, optimization and fusion according to training data;
(2) Sequencing individuals in the population from small to large in sequence according to the fitness;
(3) Dividing the ordered individuals into a plurality of grades according to the sequence;
(4) According to the principle that superior persons are selected more and inferior persons are selected less, the diversity of the population is considered as much as possible under the condition that all excellent individuals exist, the multiple grades of individuals are selected according to the set probability respectively, and then individual interpolation and cross mutation are carried out to obtain a new population;
(5) And (3) calculating the fitness of the population, if the fitness does not reach the target value, repeating the steps (2) - (5) until the fitness reaches the target value, wherein the penalty factors and the numerical values of the kernel parameters are applied to the support vector machine.
One or more embodiments provide a lower limb motion recognition system optimized based on electromyographic signal characteristics, comprising:
the signal acquisition module is used for acquiring a plurality of electromyographic signals corresponding to the lower limb actions in real time;
the signal preprocessing module is used for extracting main characteristic section signals of the electromyographic signals respectively;
the feature extraction module is used for extracting features of each main feature segment signal respectively;
the feature optimization module is used for performing feature optimization on the extracted features according to the correlation between the electromyographic signal energy and the action;
the feature fusion module is used for fusing the optimized multiple features to obtain feature vectors;
and the action recognition module is used for performing lower limb action recognition by adopting a pre-trained lower limb action recognition model based on the feature vector.
One or more embodiments provide an electronic device including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the electromyographic signal feature optimization based lower limb motion recognition method when executing the program.
One or more embodiments provide a computer readable storage medium storing a computer program which, when executed by a processor, implements the electromyographic signal feature optimization based lower limb motion recognition method.
The one or more of the above technical solutions have the following beneficial effects:
the method for optimizing the weighted characteristics based on the correlation degree of the muscles and the motions is provided, and each channel is weighted according to the specific gravity of energy values of different channels in each electromyographic signal, so that the electromyographic signals of the same muscle have stronger differentiation degree for different actions; the redundancy of the same feature under different actions can be effectively reduced, the distinguishing degree of the feature is improved, the optimal feature vector containing all effective information is obtained, the characteristic of improving the recognition rate of complex actions is achieved, and the complex 6 lower limb actions can be recognized by utilizing the surface electromyographic signals of a single signal type.
The genetic algorithm is improved by combining the tournament and the sequencing, the problem that a selection operator is easy to fall into a local optimal solution is solved, a global optimal solution with strong adaptability is obtained, useful information is more comprehensively reserved, and therefore the method has better adaptability and is convenient for practical application. By improving two steps in the recognition process, timeliness and accuracy of lower limb action recognition are improved, adaptability of a classification model is enhanced, and the method is verified online through online action recognition, so that a high online action recognition rate is obtained, and the method has a high practical application value.
The main characteristic segment signals are extracted by a framing energy method, so that the data volume is reduced, the characteristic extraction time is shortened, the characteristic precision is improved, and the real-time performance of the system is ensured.
The signal is preprocessed and noise reduced by using a composite wavelet denoising method and a wave trap, and high-frequency noise, baseline drift and 50Hz power frequency interference of the electromyographic signals can be effectively removed.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention.
FIG. 1 is a flow chart of a lower limb action recognition method based on electromyographic signal feature optimization in an embodiment of the invention;
FIG. 2 is a schematic diagram of noise reduction of surface electromyographic signals in an embodiment of the invention;
FIG. 3 is a schematic diagram of detecting an active segment of a myoelectric signal on a right leg lifting action surface according to an embodiment of the present invention;
fig. 4 is a schematic diagram of main feature segment extraction for a right leg lifting action surface electromyographic signal in an embodiment of the invention;
FIG. 5 is a graph showing the comparison of wavelet packet coefficients and energy characteristics of 8 groups of muscles for 6 actions according to an embodiment of the present invention;
FIG. 6 is a graph showing a convergence curve of population fitness after genetic algorithm improvement in an embodiment of the present invention.
Detailed Description
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the invention. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the present invention. As used herein, the singular is also intended to include the plural unless the context clearly indicates otherwise, and furthermore, it is to be understood that the terms "comprises" and/or "comprising" when used in this specification are taken to specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof.
Example 1
The embodiment discloses a lower limb action recognition method based on electromyographic signal feature optimization, which can accurately recognize 6 lower limb actions of lifting/putting right leg, lifting/putting left leg, sitting to standing and standing to sitting by using 8-channel sEMG signals. The method comprises an offline motion recognition classification model training stage and an online motion recognition stage, as shown in fig. 1, wherein the offline motion recognition classification model training stage mainly comprises the steps of signal acquisition, preprocessing, feature extraction, weighted feature optimization based on the correlation degree between muscles and motions, feature fusion and classification model training; the online action recognition stage comprises signal acquisition, preprocessing, feature extraction, weighted feature optimization based on the correlation degree of muscles and motions, feature fusion and the like, and the fused features are input into an offline action recognition classification model to recognize the actions of the lower limbs in real time.
Offline classification model training phase
Step 1: and acquiring electromyographic signals of a plurality of muscles corresponding to the lower limb movements.
In this embodiment, surface electromyographic signals of 8 muscles of 4 subjects with 6 lower limb actions are collected: left/right rectus crudely, left/right biceps crudely, left/right tibialis anterior, left/right gastrocnemius.
Step 2: preprocessing the electromyographic signals and extracting main characteristic segment signals.
The surface electromyographic signal sEMG preprocessing comprises two major parts of noise reduction of signals and main characteristic segment signal extraction. The sEMG signal is a non-stable and nonlinear weak electric signal and is often accompanied by noise and interference, so that the removal of the noise and the interference can greatly improve the quality of data and obtain a higher action recognition rate. The sEMG after noise and interference removal is purer and smoother, but a plurality of useless rest signals exist in the sEMG, so that the difficulty of signal processing is increased, and the timeliness is reduced. Therefore, extracting the main characteristic segment signal, which has a small data amount but contains most of the effective signals of one action, from the lengthy sEMG signal increases the timeliness of the action recognition. In the embodiment, a wavelet transformation threshold denoising method and a digital filtering threshold denoising method are adopted to remove baseline drift, high-frequency noise and low-frequency noise of a surface electromyographic signal, a trap is used to remove 50Hz power frequency interference, and a framing energy method is used to extract a main characteristic section signal.
The step 2 specifically includes:
step 2.1: the baseline drift, high-frequency noise and low-frequency noise are removed by adopting a wavelet transformation threshold denoising method and a digital filtering threshold denoising method, 50Hz power frequency interference is removed by utilizing a wave trap, and a smooth and pure signal with low signal-to-noise ratio is obtained through two steps, wherein the noise reduction effect is shown in figure 2.
Step 2.2: extracting a main characteristic segment signal based on a framing energy method from the noise-reduced signal:
the main characteristic section signals extracted in this embodiment are: data within 0.5s to 2.5s after the start of the motion activity segment, all sEMG signal lengths used for subsequent studies were 2s signals. When the main characteristic segment signals are extracted, the starting points of actions are detected, and the starting points of the signals can be effectively detected by a signal framing energy method.
The main characteristic segment signal extraction method based on the framing energy method comprises the following specific steps:
(1) Selecting proper frame length and frame shift, and framing the signal:
(M-1)×I+L=N (1)
where M is the total frame number of the signal, I is the incremental frame step, L is the frame length, i.e., the signal length of each frame, and N is the total length of the signal.
(2) Calculate the total energy En (i) per frame signal:
(3) Based on the signal energy at stationary stand-up, an adaptive threshold th is calculated:
(4) If En is greater than th in a certain frame and greater than th in all 3 frames thereafter, the frame is the start frame of the signal action segment.
(5) 2s data after the starting point of the signal action segment is 0.5s is intercepted, and a main characteristic segment signal is obtained:
wherein SN is the detected starting point, FS is the number of starting point frames, F s For sampling frequency, MSN is the sampling point 0.5s after the starting point, MEN is the sampling point 2s after the MSN sampling point, MSN is the sampling pointThe signal of MEN is the main characteristic segment signal.
The main characteristic section signals are extracted by adopting a signal framing energy method, the signal length to be processed is shortened, the data volume is reduced, the timeliness of motion recognition is improved, the monitoring of the starting point is shown in figure 3, the self-adaptive threshold value is obtained by calculating the energy sum of the rest state, and the starting point and the ending point of the right leg lifting motion are judged by the threshold value. As shown in fig. 4, the surface electromyographic signal is divided into a start segment signal, a main feature segment signal, and an end segment signal, and the main feature segment signal is extracted for feature extraction and optimization.
Step 3: and carrying out feature extraction and feature optimization on the main feature segment signals:
in the embodiment, 4 features such as time domain features and time-frequency domain features are extracted, and the extracted features have small differences under different actions in consideration of the complexity and stability of electromyographic signals, so that the weighted optimization based on the correlation degree of muscles and motions is considered to improve the distinguishing degree of the same features under different actions, reduce the data redundancy, increase the feature distinguishing degree and facilitate the improvement of the action recognition rate.
The step 3 specifically includes:
step 3.1: the extracted features of this example are shown in table 1.
TABLE 1sEMG action recognition related physiological response characteristics
The 4 features in table 1 are calculated as follows:
wherein x is i Is the amplitude of the ith sampling point of the surface electromyographic signal, M is the total number of windows after signal framing, n is the window length, f (x) =1 when x is not less than th, otherwise f (x) = 0,S j The wavelet packet coefficients for the j-th window.
Step 3.2: calculating signal energy values of channels corresponding to each muscle when the lower limb performs different actions based on weighted feature optimization of the muscle and the motion correlation, and establishing an energy table to obtain contribution degrees; calculating the correlation degree of the muscles and the motions according to the contribution degree in the energy table, and establishing a correlation degree table of the muscles and the motions to obtain a correlation coefficient; and according to the correlation coefficient, giving different weights to each channel, and completing the optimization of each feature. The specific implementation steps are as follows:
(1) Marking the extracted main characteristic segment signal as X, and taking absolute value to obtain absX;
(2) Calculating the signal energy value, i.e. the sum of the amplitudes, of each channel to obtain submasX i Where i=1, 2..n, is the number of channels, this example uses 8 channels, so n=8, and an energy table is built, see table 2;
(3) Summing the energy of all channels to obtain submasX, wherein the weight calculation mode of each channel is as follows:
wherein C is i For the optimization coefficient of the ith channel, i is the serial number of the channel, n is the total number of the channels, and a characteristic optimization coefficient table is established, see table 3;
(4) The proposed features and feature optimization coefficients C i Multiplying to obtain the optimized characteristics.
Wherein feature is i Respectively represent the extracted sEMG MAV 、sEMG RMS 、sEMG WA And sEMG EC1 Features, features i Is the feature after optimization. After the characteristics are weighted and optimized, the redundancy of the characteristics is reduced among different channels under the same action; the degree of differentiation of features between identical channels under different actions is enhanced. Wavelet packet coefficient energy feature sEMG for right leg lifting action EC1 The comparison of the graphs before and after the optimization is shown in FIG. 5, from which it can be seen that the original wavelet packet coefficient energy characteristics sEMG EC1 The redundancy before optimization is high, the division is difficult, and after the weighted characteristics of the muscle and the motion correlation are optimized, the WFO-EC1 characteristics with low redundancy and high division are obtained.
Table 2 energy meter
TABLE 3 correlation of muscle to motion
Step 4: the genetic algorithm is improved, so that the optimal SVM parameters are found:
in the embodiment, the penalty factor c and the kernel parameter g are optimized by utilizing the capability of the genetic algorithm for obtaining the optimal solution, the genetic algorithm is improved by utilizing the method of combining the tournament and the sorting, a grade elimination system is established, individuals in the population are sorted according to the size of the fitness, the individuals in the population are divided into four grades of poor, medium, good and excellent, and when the next generation selection is carried out, the populations in the four grades are selected according to a certain proportion, so that the proportion of excellent individuals in the population is ensured to be large, and the diversity of the population is maintained.
The specific implementation process of the step 4 is as follows:
(1) Determining initial population number, and fusing characteristic vector X p As training data of genetic algorithm, calculating fitness of individuals in the population, and recording a matrix recording each parameter and fitness of the population as oldpop;
(2) Sequencing individuals in the population from small to large in sequence according to the fitness value, and recording the individuals as a matrix sortpop;
(3) The ordered individuals are sequentially classified into four grades of poor, medium, good and excellent and are sequentially denoted as C bad ,C mid ,C well ,C good And c=c bad +C mid +C well +C good C is the total population.
(4) According to the principle of selecting more superior individuals and less inferior individuals, the diversity of the population is considered as much as possible under the condition of ensuring that all excellent individuals exist, and the poor, medium, good and excellent individuals with four grades are selected according to the probabilities of P, P+delta, P+2delta and P+3delta (0 < P <1,0< delta < 1).
(5) Recombining the individuals selected according to the proportion to obtain a new population newpop, wherein the individuals contained in the population are marked as follows:
C new =[C bad ×P]+[C mid ×(P+δ)]+[C well ×(P+2δ)]+[C good ×(P+3δ)] (11)
(6) Step (4) discards a portion of the population, resulting in incomplete population matrices, and therefore requires interpolation at newpop. The interpolation principle adopts the superior and inferior elimination principle, if C-C is less than or equal to C good Then from C good Randomly selecting C-C individuals and adding the C-C individuals into a new population newpop, if C-C is more than or equal to C good Then C is as follows good All individuals in (a) are interpolated into a new population, and C is selected well C-C-C in (3) good The random individuals are interpolated into newpop, so that the population number of newpop is C, and cross mutation is carried out in newpop, thus obtaining more excellent offspring.
(7) And (3) calculating the fitness of the population by adopting a 5-fold cross validation method, if the fitness does not reach the target value, repeating the steps (2) - (6) until the fitness reaches the target value, and recording the values of the penalty factor c and the kernel parameter g at the moment to obtain bestc and bestg.
(8) And applying bestc and bestg to a support vector machine to obtain a training network of an improved genetic algorithm support vector machine (IGA-SVM), training an optimal classification model through a training set, and finally inputting a test set to the classification model to obtain the optimal action recognition rate.
In the embodiment, by improving the genetic algorithm, the average convergence rate and average fitness of the population are obviously higher than those of the genetic algorithm before optimization, as can be seen from fig. 6, the improved genetic algorithm has high convergence rate and high fitness, and the support vector machine is optimized by using the improved genetic algorithm to obtain the IGA-SVM.
Step 5: the surface electromyographic signals obtained in the step 1 are preprocessed in the step 2 to extract main characteristic segment signals, the characteristics of the step 3 are extracted and optimized, the genetic algorithm is improved and the classifier is optimized in the step 4, the off-line classification test of 6 lower limb actions is completed, the high recognition rate recognition of the lower limb actions is realized, and the classification result is shown in the table 4.
TABLE 4 test classification results
From the table, the average recognition rate of 6 lower limb actions reaches 94.75% by applying the feature optimization and improving the genetic algorithm-support vector machine classifier.
In the embodiment, 8 paths of sEMG signals are acquired by using 8 channels of sensors, 6 types of lower limb actions in daily life are successfully identified with a high identification rate of 94.75%, and the identification rate of the lower limb actions is greatly improved.
(II) Online action recognition phase
The action recognition method can be applied to a rehabilitation platform of the lower limb exoskeleton robot, and the patient is driven to do rehabilitation exercises by recognizing the lower limb actions of the demonstrator.
The recognition model of 6 actions of the lower limb is trained through an offline action recognition stage, and the model is applied to an online action recognition stage.
Specifically, the recognition model of 6 actions of the lower limb in the offline action recognition stage is packaged into a dynamic link library through Matlab, new data acquired in real time is input into the model, and a specific action instruction is returned.
The method for acquiring and processing the new data in real time comprises the following specific steps:
step 1: firstly, the Cshharp is used for compiling upper computer software, and a server of Delsys Trigno is connected, so that a numerical signal of the myoelectric sensor is obtained.
Step 2: 8 channels corresponding to off-line action recognition are selected and placed at 8 muscles of the lower limb: left/right rectus leg, left/right biceps leg, left/right tibialis anterior, left/right gastrocnemius, and surface electromyographic signals are acquired.
Step 3: preprocessing the electromyographic signals, denoising the signals by using a method of combining wavelet transformation threshold denoising, digital filtering threshold denoising and 50Hz power frequency interference removal by using a wave trap, and extracting main characteristic section signals by using a framing energy method.
Step 4: and carrying out feature extraction and weighted feature optimization based on the correlation degree of muscles and motions on the extracted main feature segment signals, extracting 4 features such as time domain features and time-frequency domain features as well as the feature extraction in the off-line action recognition stage, and carrying out weighted optimization on the features to obtain all effective features with low data redundancy and high feature distinction degree.
Step 5: packaging the recognition models of 6 actions of the lower limb in the offline action recognition stage into a dynamic link library through Matlab, and implanting the dynamic link library into upper computer software written by Caharp; and 4 features mentioned in the step 4 are fused into feature vectors and input into the motion recognition model to finish online motion recognition, wherein the average recognition rate of the online motion recognition reaches 90.6% through verification, and specific classification results are shown in Table 5, wherein the motions 1-6 respectively represent lifting right legs, placing right legs, lifting left legs, placing left legs, sitting and standing.
TABLE 5 Online behavior recognition test classification results
The actions identifiable in the online action recognition stage are limited to the 6 types of lower limb actions in the offline action recognition stage, and the more the number of actions is, the more the number of online action recognition is, and the feasibility is provided.
Example two
The embodiment aims to provide a lower limb action recognition system based on electromyographic signal characteristic optimization.
A lower limb motion recognition system based on electromyographic signal feature optimization, comprising:
the signal acquisition module is used for acquiring a plurality of electromyographic signals corresponding to the lower limb actions in real time;
the signal preprocessing module is used for extracting main characteristic section signals of the electromyographic signals respectively;
the feature extraction module is used for extracting features of each main feature segment signal respectively;
the feature optimization module is used for performing feature optimization on the extracted features according to the correlation between the electromyographic signal energy and the action;
the feature fusion module is used for fusing the optimized multiple features to obtain feature vectors;
and the action recognition module is used for performing lower limb action recognition by adopting a pre-trained lower limb action recognition model based on the feature vector.
Example III
An object of the present embodiment is to provide an electronic apparatus.
An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the program:
acquiring a plurality of electromyographic signals corresponding to the lower limb actions in real time;
extracting main characteristic section signals from the electromyographic signals respectively;
respectively carrying out feature extraction and feature optimization on each main feature segment signal, and fusing a plurality of optimized features to obtain feature vectors;
and based on the feature vector, performing lower limb action recognition by adopting a pre-trained lower limb action recognition model.
Example IV
An object of the present embodiment is to provide a computer-readable storage medium.
A computer-readable storage medium having stored thereon a computer program, upper computer software, etc., the program being executed by a processor to:
acquiring a plurality of electromyographic signals corresponding to the lower limb actions in real time;
extracting main characteristic section signals from the electromyographic signals respectively;
respectively carrying out feature extraction and feature optimization on each main feature segment signal, and fusing a plurality of optimized features to obtain feature vectors;
and based on the feature vector, performing lower limb action recognition by adopting a pre-trained lower limb action recognition model.
The steps of the second to fourth embodiments correspond to the first embodiment, and the detailed description of the second embodiment refers to the relevant description of the first embodiment.
The technical scheme provides a lower limb action recognition method and system based on electromyographic signal feature optimization, and provides a weighted feature optimization algorithm based on muscle and motion relativity, so that feature redundancy is reduced, simultaneously, as much useful information as possible is reserved, recognition rate is improved while recognition speed is improved, and popularization and application of online recognition are facilitated; the genetic algorithm is improved by combining the tournament and the sequencing, the problem that a selection operator is easy to fall into a local optimal solution is solved, a global optimal solution with strong adaptability is obtained, useful information is more comprehensively reserved, and therefore the method has better adaptability and is convenient for practical application. By improving two steps in the recognition process, timeliness and accuracy of lower limb action recognition are improved, adaptability of a classification model is enhanced, and the method is verified online through online action recognition, so that a high online action recognition rate is obtained, and the method has a high practical application value.
It will be appreciated by those skilled in the art that the modules or steps of the invention described above may be implemented by general-purpose computer means, alternatively they may be implemented by program code executable by computing means, whereby they may be stored in storage means for execution by computing means, or they may be made into individual integrated circuit modules separately, or a plurality of modules or steps in them may be made into a single integrated circuit module. The present invention is not limited to any specific combination of hardware and software.
While the foregoing description of the embodiments of the present invention has been presented in conjunction with the drawings, it is not intended to limit the scope of the invention, and it will be apparent to those skilled in the art that various modifications, improvements or variations can be made without the need for inventive faculty, while remaining within the scope of the invention.

Claims (6)

1. The lower limb action recognition method based on myoelectricity feature optimization is characterized by comprising the following steps of:
acquiring a plurality of electromyographic signals of a plurality of muscles corresponding to the lower limb actions in real time;
extracting main characteristic section signals from the electromyographic signals respectively;
respectively extracting the characteristics of each main characteristic segment signal, and carrying out characteristic optimization according to the correlation between the electromyographic signal energy and the action;
fusing the optimized multiple features to obtain feature vectors, and performing lower limb action recognition by adopting a pre-trained lower limb action recognition model based on the feature vectors;
the feature optimization according to the correlation between the electromyographic signal energy and the action comprises the following steps:
extracting the characteristics of the main characteristic section signals;
acquiring signal energy values of corresponding channels of each muscle in the main characteristic section signals, namely amplitude sums of absolute values of the main characteristic section signals;
calculating the weight of the channel in the main characteristic section signal according to the proportion of the signal energy value of each channel accounting for the summation of the signal energy of all channels;
weighting and optimizing the characteristics of each channel signal of the main characteristic segment signal according to the weight to obtain optimized characteristics;
the lower limb action recognition model is obtained by training based on a support vector machine classifier; when training is performed based on the support vector machine classifier, a genetic algorithm based on combination of tournament and sequencing is adopted for parameter optimization;
the parameter optimization process comprises the following steps:
(1) Determining an initial population, and calculating the fitness of individuals in the population based on feature vectors obtained through feature extraction, optimization and fusion according to training data;
(2) Sequencing individuals in the population from small to large in sequence according to the fitness;
(3) Dividing the ordered individuals into a plurality of grades according to the sequence;
(4) According to the principle that superior persons are selected more and inferior persons are selected less, the diversity of the population is considered as much as possible under the condition that all excellent individuals exist, the multiple grades of individuals are selected according to the set probability respectively, and then individual interpolation and cross mutation are carried out to obtain a new population;
(5) And (3) calculating the fitness of the population, if the fitness does not reach the target value, repeating the steps (2) - (5) until the fitness reaches the target value, wherein the penalty factors and the numerical values of the kernel parameters are applied to the support vector machine.
2. The method for identifying the lower limb actions based on myoelectricity feature optimization according to claim 1, wherein after a plurality of myoelectricity signals of the lower limb are acquired, denoising pretreatment is further performed on the plurality of myoelectricity signals: the baseline drift, high-frequency noise and low-frequency noise are removed by adopting a wavelet transformation threshold denoising method and a digital filtering threshold denoising method, and 50Hz power frequency interference is removed by utilizing a wave trap.
3. The method for identifying the lower limb actions based on myoelectricity feature optimization according to claim 1, wherein the main feature segment signal extraction adopts a framing energy method:
framing the electromyographic signals, and calculating the total energy of signals in each frame;
if the total energy in a certain frame is larger than the set threshold value and the total energy in the set number of frames is larger than the set threshold value, the frame is used as the initial frame of an action segment;
the electromyographic signals of the set time from the initial frame are extracted to obtain the main characteristic section signals.
4. Lower limb motion recognition system based on electromyographic signal characteristic optimization, which is characterized by comprising:
the signal acquisition module is used for acquiring a plurality of electromyographic signals of a plurality of muscles corresponding to the lower limb actions in real time;
the signal preprocessing module is used for extracting main characteristic section signals of the electromyographic signals respectively;
the feature extraction module is used for extracting features of each main feature segment signal respectively;
the feature optimization module is used for performing feature optimization on the extracted features according to the correlation between the electromyographic signal energy and the action;
the feature fusion module is used for fusing the optimized multiple features to obtain feature vectors;
the action recognition module is used for performing lower limb action recognition by adopting a pre-trained lower limb action recognition model based on the feature vector;
the feature optimization according to the correlation between the electromyographic signal energy and the action comprises the following steps:
extracting the characteristics of the main characteristic section signals;
acquiring signal energy values of corresponding channels of each muscle in the main characteristic section signals, namely amplitude sums of absolute values of the main characteristic section signals;
calculating the weight of the channel in the main characteristic section signal according to the proportion of the signal energy value of each channel accounting for the summation of the signal energy of all channels;
weighting and optimizing the characteristics of each channel signal of the main characteristic segment signal according to the weight to obtain optimized characteristics;
the lower limb action recognition model is obtained by training based on a support vector machine classifier; when training is performed based on the support vector machine classifier, a genetic algorithm based on combination of tournament and sequencing is adopted for parameter optimization;
the parameter optimization process comprises the following steps:
(1) Determining an initial population, and calculating the fitness of individuals in the population based on feature vectors obtained through feature extraction, optimization and fusion according to training data;
(2) Sequencing individuals in the population from small to large in sequence according to the fitness;
(3) Dividing the ordered individuals into a plurality of grades according to the sequence;
(4) According to the principle that superior persons are selected more and inferior persons are selected less, the diversity of the population is considered as much as possible under the condition that all excellent individuals exist, the multiple grades of individuals are selected according to the set probability respectively, and then individual interpolation and cross mutation are carried out to obtain a new population;
(5) And (3) calculating the fitness of the population, if the fitness does not reach the target value, repeating the steps (2) - (5) until the fitness reaches the target value, wherein the penalty factors and the numerical values of the kernel parameters are applied to the support vector machine.
5. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements a myoelectric feature optimization-based lower limb motion recognition method as claimed in any one of claims 1-3 when the program is executed by the processor.
6. A computer readable storage medium storing a computer program, wherein the program when executed by a processor implements a method for identifying a lower limb motion based on myoelectric feature optimization as claimed in any one of claims 1 to 3.
CN202111066419.3A 2021-09-13 2021-09-13 Myoelectricity feature optimization-based lower limb action recognition method and system Active CN113598759B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111066419.3A CN113598759B (en) 2021-09-13 2021-09-13 Myoelectricity feature optimization-based lower limb action recognition method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111066419.3A CN113598759B (en) 2021-09-13 2021-09-13 Myoelectricity feature optimization-based lower limb action recognition method and system

Publications (2)

Publication Number Publication Date
CN113598759A CN113598759A (en) 2021-11-05
CN113598759B true CN113598759B (en) 2023-09-22

Family

ID=78310376

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111066419.3A Active CN113598759B (en) 2021-09-13 2021-09-13 Myoelectricity feature optimization-based lower limb action recognition method and system

Country Status (1)

Country Link
CN (1) CN113598759B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115034273B (en) * 2021-12-27 2023-09-01 驻马店市中心医院 Myoelectricity biofeedback equipment and system based on pattern recognition
CN116019467A (en) * 2023-03-28 2023-04-28 潍坊医学院附属医院 Feature extraction method of facial nerve electromyographic signals
CN117331047A (en) * 2023-12-01 2024-01-02 德心智能科技(常州)有限公司 Human behavior data analysis method and system based on millimeter wave radar

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102764167A (en) * 2012-06-12 2012-11-07 天津大学 Myoelectric prosthesis control source lead optimization method based on correlation coefficients
CN103577639A (en) * 2013-10-31 2014-02-12 浙江大学 Multi-objective optimization method for synthesis of rotary detection sound field
CN104537382A (en) * 2015-01-12 2015-04-22 杭州电子科技大学 Electromyographic signal gait recognition method for optimizing support vector machine based on genetic algorithm
CN105389442A (en) * 2015-12-09 2016-03-09 天津大学 Reverse design method for coupling genetic algorithm, neural network and numerical simulation
CN105550437A (en) * 2015-12-09 2016-05-04 天津大学 Reverse design method for indoor environment based on genetic algorithm
CN105595990A (en) * 2016-01-27 2016-05-25 浙江铭众科技有限公司 Intelligent terminal device for evaluating and distinguishing quality of electrocardiosignal
KR101711302B1 (en) * 2015-10-26 2017-03-02 한양대학교 산학협력단 Discriminative Weight Training for Dual-Microphone based Voice Activity Detection and Method thereof
CN107255785A (en) * 2017-04-28 2017-10-17 南京邮电大学 Based on the analog-circuit fault diagnosis method for improving mRMR
CN109948465A (en) * 2019-02-28 2019-06-28 浙江工业大学 A kind of surface electromyogram signal classification method based on multiple target
CN110974212A (en) * 2019-11-07 2020-04-10 曲阜师范大学 Electrocardio and myoelectric characteristic fused rehabilitation training motion state monitoring method and system
CN112036357A (en) * 2020-09-09 2020-12-04 曲阜师范大学 Upper limb action recognition method and system based on surface electromyogram signal
CN112971800A (en) * 2021-01-27 2021-06-18 东南大学 Deep learning-based electrocardio premature beat signal processing method, module and device
CN113229831A (en) * 2021-05-10 2021-08-10 燕山大学 Movement function monitoring and management method based on myoelectricity and myooxygen signals
WO2021161901A1 (en) * 2020-02-13 2021-08-19 富士フイルム株式会社 Feature value selection method, feature value selection program, multiclass classification method, multiclass classification program, feature value selection device, multiclass classification device, and feature value set

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102764167A (en) * 2012-06-12 2012-11-07 天津大学 Myoelectric prosthesis control source lead optimization method based on correlation coefficients
CN103577639A (en) * 2013-10-31 2014-02-12 浙江大学 Multi-objective optimization method for synthesis of rotary detection sound field
CN104537382A (en) * 2015-01-12 2015-04-22 杭州电子科技大学 Electromyographic signal gait recognition method for optimizing support vector machine based on genetic algorithm
KR101711302B1 (en) * 2015-10-26 2017-03-02 한양대학교 산학협력단 Discriminative Weight Training for Dual-Microphone based Voice Activity Detection and Method thereof
CN105389442A (en) * 2015-12-09 2016-03-09 天津大学 Reverse design method for coupling genetic algorithm, neural network and numerical simulation
CN105550437A (en) * 2015-12-09 2016-05-04 天津大学 Reverse design method for indoor environment based on genetic algorithm
CN105595990A (en) * 2016-01-27 2016-05-25 浙江铭众科技有限公司 Intelligent terminal device for evaluating and distinguishing quality of electrocardiosignal
CN107255785A (en) * 2017-04-28 2017-10-17 南京邮电大学 Based on the analog-circuit fault diagnosis method for improving mRMR
CN109948465A (en) * 2019-02-28 2019-06-28 浙江工业大学 A kind of surface electromyogram signal classification method based on multiple target
CN110974212A (en) * 2019-11-07 2020-04-10 曲阜师范大学 Electrocardio and myoelectric characteristic fused rehabilitation training motion state monitoring method and system
WO2021161901A1 (en) * 2020-02-13 2021-08-19 富士フイルム株式会社 Feature value selection method, feature value selection program, multiclass classification method, multiclass classification program, feature value selection device, multiclass classification device, and feature value set
CN112036357A (en) * 2020-09-09 2020-12-04 曲阜师范大学 Upper limb action recognition method and system based on surface electromyogram signal
CN112971800A (en) * 2021-01-27 2021-06-18 东南大学 Deep learning-based electrocardio premature beat signal processing method, module and device
CN113229831A (en) * 2021-05-10 2021-08-10 燕山大学 Movement function monitoring and management method based on myoelectricity and myooxygen signals

Non-Patent Citations (7)

* Cited by examiner, † Cited by third party
Title
Bimodal Anxiety State Assessment Based on Electromyography and Electroencephalogram;Diancong Zhang;2020 International Conference on Sensing, Measurement & Data Analytics in the era of Artificial Intelligence (ICSMD);全文 *
Classification of Electroencephalogram Data from Hand Grasp and Release Movements for BCI Controlled Prosthesis;Lange, G;3RD INTERNATIONAL CONFERENCE ON SYSTEM-INTEGRATED INTELLIGENCE: NEW CHALLENGES FOR PRODUCT AND PRODUCTION ENGINEERING;全文 *
Supervised and unsupervised learning in processing myographic patterns;Shamsin, M;BIGDATA CONFERENCE (FORMERLY INTERNATIONAL CONFERENCE ON BIG DATA AND ITS APPLICATIONS);第1117卷;全文 *
基于sEMG信号的上肢自主康复训练动作模式识别;陈彦钊;中国博士学位论文全文数据库 (医药卫生科技辑)(第1期);全文 *
基于表面肌电信号的手部动作识别与交互;李奂良;中国优秀硕士学位论文全文数据库 (基础科学辑)(第8期);全文 *
高清.基于肌电信号的手势动作识别.中国优秀硕士学位论文全文数据库.2020,(第06期),E080-6. *
黄虎,谢洪波.基于蚁群算法的表面肌电信号特征选择.北京生物医学工程.2012,第31卷(第02期),164-169. *

Also Published As

Publication number Publication date
CN113598759A (en) 2021-11-05

Similar Documents

Publication Publication Date Title
CN113598759B (en) Myoelectricity feature optimization-based lower limb action recognition method and system
CN110974212A (en) Electrocardio and myoelectric characteristic fused rehabilitation training motion state monitoring method and system
CN113065526B (en) Electroencephalogram signal classification method based on improved depth residual error grouping convolution network
CN102073881A (en) Denoising, feature extraction and pattern recognition method for human body surface electromyography signals
CN108958474A (en) A kind of action recognition multi-sensor data fusion method based on Error weight
CN109979161A (en) A kind of tumble detection method for human body based on convolution loop neural network
CN112396001B (en) Rope skipping number statistical method based on human body posture estimation and TPA (tissue placement model) attention mechanism
CN110503082A (en) A kind of model training method and relevant apparatus based on deep learning
CN110032987B (en) Surface electromyographic signal classification method based on cerebellar neural network model
CN107280659A (en) The processing method and system of a kind of electrocardiosignal
CN112036357B (en) Upper limb action recognition method and system based on surface electromyographic signals
CN111950460B (en) Muscle strength self-adaptive stroke patient hand rehabilitation training action recognition method
CN114159080B (en) Training and recognition method and device for upper limb rehabilitation robot movement intention recognition model
CN109814716A (en) A kind of motion intention coding/decoding method based on dynamic surface electromyography signal
Guan et al. Research on human behavior recognition based on deep neural network
Rana et al. A lightweight dnn for ecg image classification
CN114569143A (en) Myoelectric gesture recognition method based on attention mechanism and multi-feature fusion
CN109886109B (en) Behavior identification method based on deep learning
Wang et al. Diversity amplification and data generation of Chinese Sign Language based on Generative Adversarial Network
CN116738295B (en) sEMG signal classification method, system, electronic device and storage medium
CN116898459A (en) Surface electromyographic signal action recognition method based on transfer learning and support vector machine
CN115568856B (en) Multi-finger force continuous estimation method based on collaborative mapping reconstruction
CN112668424B (en) RBSAGAN-based data augmentation method
Souza et al. Evaluation of Pattern Recognition in Myoelectric Signal Using Netlab GLM
CN111832694A (en) Data preprocessing method and device, training system, wearable device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant