CN109784412A - The multiple sensor signals fusion method based on deep learning for gait classification - Google Patents
The multiple sensor signals fusion method based on deep learning for gait classification Download PDFInfo
- Publication number
- CN109784412A CN109784412A CN201910063463.5A CN201910063463A CN109784412A CN 109784412 A CN109784412 A CN 109784412A CN 201910063463 A CN201910063463 A CN 201910063463A CN 109784412 A CN109784412 A CN 109784412A
- Authority
- CN
- China
- Prior art keywords
- layer
- gait
- data
- imu
- sensor signals
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005021 gait Effects 0.000 title claims abstract description 58
- 238000007500 overflow downdraw method Methods 0.000 title claims abstract description 16
- 238000013135 deep learning Methods 0.000 title claims abstract description 4
- 206010017577 Gait disturbance Diseases 0.000 claims abstract description 32
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 29
- 101000739754 Homo sapiens Semenogelin-1 Proteins 0.000 claims abstract description 26
- 102100037550 Semenogelin-1 Human genes 0.000 claims abstract description 26
- 230000004927 fusion Effects 0.000 claims abstract description 18
- 238000013528 artificial neural network Methods 0.000 claims abstract description 9
- 230000003183 myoelectrical effect Effects 0.000 claims abstract description 9
- 230000006870 function Effects 0.000 claims description 25
- 238000012549 training Methods 0.000 claims description 18
- 230000001133 acceleration Effects 0.000 claims description 13
- 210000003205 muscle Anatomy 0.000 claims description 11
- 238000012360 testing method Methods 0.000 claims description 11
- 208000011644 Neurologic Gait disease Diseases 0.000 claims description 10
- 206010056242 Parkinsonian gait Diseases 0.000 claims description 9
- 238000005070 sampling Methods 0.000 claims description 8
- 230000005284 excitation Effects 0.000 claims description 7
- 241000272525 Anas platyrhynchos Species 0.000 claims description 6
- 206010013033 Diplegia Diseases 0.000 claims description 6
- 206010017585 Gait spastic Diseases 0.000 claims description 6
- 210000001015 abdomen Anatomy 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 5
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 4
- 239000013598 vector Substances 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 2
- 238000004088 simulation Methods 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 abstract description 4
- 230000000694 effects Effects 0.000 abstract description 2
- 238000000034 method Methods 0.000 description 7
- 210000003141 lower extremity Anatomy 0.000 description 6
- 230000008859 change Effects 0.000 description 2
- 210000001097 facial muscle Anatomy 0.000 description 2
- 210000001699 lower leg Anatomy 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 208000020084 Bone disease Diseases 0.000 description 1
- 208000021642 Muscular disease Diseases 0.000 description 1
- 201000009623 Myopathy Diseases 0.000 description 1
- 230000006931 brain damage Effects 0.000 description 1
- 231100000874 brain damage Toxicity 0.000 description 1
- 208000029028 brain injury Diseases 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 230000006806 disease prevention Effects 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 210000002082 fibula Anatomy 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000003862 health status Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
Abstract
The invention belongs to biometrics identification technology fields, and in particular to a kind of multiple sensor signals fusion method based on deep learning for gait classification.The present invention classifies to abnormal gait by constructing deep neural network, and is merged using convolutional neural networks to the multi-source heterogeneous information source data from IMU inertia sensing unit and SEMG surface myoelectric；Fusion content includes data Layer (CNN input layer), characteristic layer (pond CNN layer 1 to convolutional layer 2) and decision-making level's (CNN output layer) fusion, multi-source heterogeneous sensor information is extracted to complete, improve classifier nicety of grading, data prediction workload is reduced simultaneously, improves classification accuracy and identification effect.It is verified, present invention classifying quality in a variety of abnormal gait classification tasks is obviously improved compared with single mode sensor, in embodiment in lifted six classification task of abnormal gait, classification accuracy reaches 99.15%, and more single IMU information source CNN network promotes about three percentage points.
Description
Technical field
The invention belongs to bio signal field of sensing technologies, and in particular to a kind of multiple sensor signals for gait classification
Fusion method.
Background technique
Gait refers to the posture showed when people walks, is one of human body important biomolecule feature.Abnormal gait mostly with disease
Change position is related, as the important feature of reflection human health status and capacity, obtains accurate believable gait in time and believes
Breath, training abnormal gait classifier carries out timely early warning to abnormal gait, and it is monitored and is assessed for a long time, examines in medical treatment
There is important directive significance in disconnected, disease prevention.
Fusion MSDF (Multi-sensor Data Fusion) technology is extensive first in military affairs
It uses, recently as the increasingly mature perfect of biomedical information acquisition technique, especially wearable device and body Sensor Network
The rise of (Body Sensor Network), causes concern to the Multi-sensor Fusion of medical signals.
The gait recognition method of mainstream mainly divides computer vision scheme based on video and image procossing and is based at present
The sensor plan of pavement and wearable sensor such as IMU.Had based on sensor based on lower extremity movement information, based on lower limb surface
Myoelectricity is based on numerous single sensor solutions such as plantar nervous arch, but single-sensor data compared with multisensor are thin, no
It can obtain that object is complete, comprehensive information.
Summary of the invention
It is an object of the invention to propose a kind of nicety of grading is high, computation complexity is low more sensings for gait classification
Device signal fused method.
Multiple sensor signals fusion method proposed by the present invention for gait classification is based on deep neural network technology
, classified by constructing deep neural network to abnormal gait, and using neural network to the number of multiple information sources data
It is merged according to layer, characteristic layer, decision-making level, so that complete extract multi-source heterogeneous sensor information, improves classifier nicety of grading.
Specific step is as follows:
Step 1 acquires normal gait information.Utilize IMU (inertia sensing unit) module and SEMG for being worn on human body
(surface myoelectric) module acquires signal when human normal walking, obtains normal gait 3-axis acceleration information and muscle groups of lower extremitates table
Facial muscle power information；
Step 2, acquisition abnormity gait information.Human body simulation typical case abnormal gait row is acquired using the IMU for being worn on human body
Signal when walking obtains abnormal gait 3-axis acceleration information and muscle groups of lower extremitates surface myoelectric information；
Step 3, data prediction are merged with data Layer.By original each channel data from two kinds of sensors according to when
Between stamp do alignment of data, will be normalized after IMU data and EMG data filtering；Walking cadence according to target representative row will
Initial data respectively does windowing cutting pretreatment, and each data queue is stamped respective labels according to gait classification, is distinguished
For two groups of data labels pair of IMU, SEMG data；Wherein, timestamp is aligned in the group of tri- number of axle evidence of IMU, SEMG multichannel number
According to group in timestamp alignment and IMU, SEMG data group between timestamp be aligned；
Step 4 builds Fusion Features depth convolutional neural networks (CNN), which is six layer structure, convolutional layer one
A is connect entirely with one A of pond layer, and one B of convolutional layer is connect entirely with one B of pond layer, and one A of pond layer and one B of pond layer are fused to convolution
Layer two, two connection pool layer two of convolutional layer, pond layer two connect full connection hidden layer, full connection hidden layer be connected to connect entirely it is defeated
Layer out；Two convolutional layers one (one A of convolutional layer, one B of convolutional layer) arranged side by side, pond layers one are defined, convolutional layer two, a pond are defined
Change layer two, defines full articulamentum and each activation primitive；
Step 5, by data label obtained in step 3 to being divided into training set and test set, after timestamp is aligned
(IMU enters one A of convolutional layer to the convolutional layer one that IMU and SEMG data are respectively fed in groups in step 4, and SEMG enters convolutional layer one
B it) is trained；Data complete feature-based fusion in convolutional layer two；
Step 6 repeats step 3 to step 5 three times, obtains three models；
Step 7 utilizes test set assessment models after the completion of training, the prediction result of three models is done majority vote choosing
It lifts, completes the fusion of decision data layer.
The present invention Step 1: in two, the IMU acquisition unit can dispose but be not limited to shank, in shoes, loins, but require
Normal gait is consistent with the mounting means of abnormal gait.
The present invention is Step 1: the SEMG acquisition unit can dispose but be not limited to gastrocnemius belly of muscle, tibialis anterior flesh in two
Abdomen, but require normal gait consistent with the mounting means of abnormal gait.
In step 2 of the present invention, it is described typical case abnormal gait include but is not limited to hemiplegic gait, parkinsonian gait, duck step by step
State, steppage gait, diplegia gait.
Described that initial data is respectively done to windowing cutting in step 3 of the present invention, data window length representative value is but not
It is limited to two gait cycles, piecewise step value is desirable between window but is not limited to a quarter of windowing length, and the label form is
But it is not limited to one-hot coding.
In step 4 of the present invention, the input matrix Shape of one A of convolutional layer and one B of convolutional layercov1AWith Shapecov1B
It is defined as every segment data window in step 3 (data source port number, the sampling from respective signal data under current sampling frequency
Points) namely:
Shapecov1A: (NIMU, Twindows*fc)
Shapecov1B: (NSEMG, Twindows*fc)
Wherein, NIMUFor IMU data channel number, NSEMGFor SEMG data channel number, TwindowsFor data window time domain length, fc
For current sampling frequency.
The output layer nodal point number is defined as gait category number namely a kind of normal gait and all exceptions in "current" model
The sum of gait.
The optimization algorithm of the CNN network can be but be not limited to Adam optimizer.
Each layer excitation function can be but be not limited to ReLU function, Sigmod function, tanh function, and output layer motivates letter
Number is Softmax function.
The loss function of the CNN network is multiclass logarithm loss function.
In step 7 of the present invention, the implementation of Decision-level fusion is three model majority voting systems.
The invention has the advantages that completing lower extremity movement acceleration information and lower limb table using CNN deep neural network
The data Layer and Feature-level fusion of facial muscle power information complete the Decision-level fusion of data using more CNN model vote in majority, thus
The data of IMU unit Yu the multi-source heterogeneous sensor of SEMG unit are made full use of in the building of abnormal gait classifier, are improved and are divided
Class device accuracy.Verified, this model classifying quality in a variety of abnormal gait classification tasks has significantly compared with single mode sensor
It is promoted.As in embodiments of the present invention in six classification task of abnormal gait of lifting, classification accuracy can reach 99.15%, more singly
IMU information source CNN network promotes about three percentage points.
Detailed description of the invention
Fig. 1 is the multiple sensor signals fusion method knot based on depth convolutional neural networks that the present invention is used for gait classification
Composition.
Fig. 2 is the structure chart for the depth convolutional neural networks that the present invention is built.
Fig. 3 is that pretreated acceleration transducer contained by a segment signal window of neural network is sent into the embodiment of the present invention
Data.
Fig. 4 is that pretreated surface myoelectric number contained by a segment signal window of neural network is sent into the embodiment of the present invention
According to.
Fig. 5 is acceleration signal example (interception 10s) under hemiplegic gait in the embodiment of the present invention (drawing circle gait).
Fig. 6 is acceleration signal example under parkinsonian gait (festinating gait) in the embodiment of the present invention (interception 10s).
Fig. 7 is gluteus medius myopathic gait (duck step by step state) in the embodiment of the present invention.
Fig. 8 is acceleration signal example under steppage gait in the embodiment of the present invention (interception 10s).
Fig. 9 is acceleration signal example under diplegia gait (scissor-like gait) in the embodiment of the present invention (interception 10s).
Figure 10 is model after 7 wheel of training in embodiment in the ROC curve in training set.
Figure 11 is that confusion matrix of the model in test set after 7 wheels is trained in embodiment.
Specific embodiment
Develop simultaneously preferred embodiment with reference to the accompanying drawing, and the present invention will be described in detail.
The present invention provides a kind of multiple sensor signals fusions based on depth convolutional neural networks for gait classification
Method, method structural schematic diagram are as shown in Figure 1.This method is realized using following steps:
Step 1 acquires normal gait information.IMU and SEMG hardware acquisition system are fixed on left and right shank by bandage
Outside, wherein IMU Y-axis and horizontal plane, X-axis is vertical with human coronary face, and Z axis is vertical with human body sagittal plane.SEMG two is logical
Road is affixed on gastrocnemius belly of muscle and tibialis anterior belly of muscle respectively, and every 4 centimetres of channel two panels electrode spacing, reference electrode is affixed on fibula external malleolus
Locate skin.Setting systematic sampling rate is 512Hz, and setting IMU accelerometer response is ± 2g, when acquisition human normal is walked
Motor message, acquisition step number are no less than 100 steps, obtain the left and right lower limb 3-axis acceleration information and two pieces of passes under normal gait
The surface myoelectric information of key muscle.
Step 2, acquisition abnormity gait information.IMU and SEMG hardware acquisition system is placed and the same step of collecting flowchart
One, the surface myoelectric information of left and right lower limb 3-axis acceleration information and two pieces of crucial muscle under acquisition abnormity gait.At this
Choose five kinds of typical abnormal gaits in embodiment in abnormal gait because caused by from common causatives such as myopathy, osteopathy, brain damages
Simulated, respectively hemiplegic gait (draw circle gait), parkinsonian gait (festinating gait), (duck is step by step for gluteus medius myopathic gait
State), steppage gait, diplegia gait (scissor-like gait).Original signal example (interception 2s) is such as Figure 10 institute under five kinds of abnormal gaits
Show.Wherein, Fig. 5 is hemiplegic gait (drawing circle gait), Fig. 6 is parkinsonian gait (festinating gait), Fig. 7 is gluteus medius myopathic gait
(duck step by step state), Fig. 8 are steppage gait, Fig. 9 is diplegia gait (scissor-like gait).
Step 3, data prediction are merged with the data Layer of signal.IMU initial data is filtered by six rank low pass Butterworths
Wave device, SEMG initial data do smoothing processing after extracting signal envelope.Next by the Five-channel number from two kinds of sensors
Alignment of data is done according to according to timestamp, triple channel IMU data and two channel EMG data are done into data normalization processing.According to mesh
Mark representative row walks cadence and initial data is respectively done to windowing cutting pretreatment and each data queue is stamped phase according to gait classification
Label is answered, two groups of data labels pair of respectively IMU, SEMG data are obtained.Every segment signal window takes in the present embodiment
All sampled points in 2000ms, IMU data and SEMG data respectively obtain three axis 3*1024=3072 point, two at 512Hz
2*1024=2048, channel point.Number label is 0 to 5, and wherein the label of hemiplegic gait (drawing circle gait) is 0, Parkinson's step
The label of state (festinating gait) is 1, and the label of normal gait is 2, and the label of gluteus medius myopathic gait (duck step by step state) is 3, across
The label of threshold gait is 4, and the label of diplegia gait (scissor-like gait) is 5.Fig. 3 is a segment signal window institute in this preferred embodiment
Example containing signal.Wherein, Fig. 3 is pretreated acceleration transducer data, and Fig. 4 is pretreated surface myoelectric data.
Step 4, builds Fusion Features CNN depth convolutional neural networks, and each layer structure of CNN network is as shown in Figure 2.Definition
Two convolutional layers one (one A of convolutional layer, one B of convolutional layer) arranged side by side, pond layers one (one A of pond layer, one B of pond layer) define one
A convolutional layer two, pond layer two define full articulamentum and each activation primitive.Wherein, one A of convolutional layer and one B of convolutional layer
Input matrix is defined as every segment data window in step 3, and (data source port number comes from respective signal number under current sampling frequency
According to sampling number) namely:
Shapecov1A: (3,2*512)
Shapecov1B: (2,2*512)
The IMU and SEMG data characteristics are exported in one A of pond layer and one B of pond layer respectively, and are commonly connected to convolution
Layer two completes the Feature-level fusion of data in convolutional layer two.
The optimization algorithm of the CNN network has been provided in connection with the advantages of two kinds of optimization algorithms of AdaGrad and RMSProp
Adam optimizer.Its gradient updating formula are as follows:
Wherein, default learning rate α is set as 0.001, ε=10^-8, and divisor is avoided to become 0.It is public for gradient mean value
Formula are as follows:
mt=β1mt-1+(1-β1)gt
Wherein, β1Coefficient is exponential decay rate, control weight distribution (momentum and current gradient), takes 0.9 in the present embodiment,
gtFor the gradient of t time step.For gradient variance, formula are as follows:
vt=β2vt-1+(1-β2)gt 2
Wherein, β2Coefficient is exponential decay rate, the influence situation of the gradient square before controlling, and is taken in the present embodiment
0.999, gtFor the gradient of t time step.
Each layer excitation function is set as the linear correcting unit of ReLU, formula are as follows:
F (x)=max (0, x)
Output layer excitation function is Softmax function, formula are as follows:
Wherein, wj(j=0,1,2,3,4,5) for from full articulamentum to the weight vectors of full connection output layer.
The loss function of the CNN network is multiclass logarithm loss function or multiclass cross entropy cost function, this implementation
The cross entropy cost function concrete form of convolutional neural networks in example are as follows:
Wherein, yi ′For i-th of value in label, yiFor the respective components in the vector through softmax normalized output.
The output layer nodal point number be defined as in "current" model gait category number namely a kind of normal gait and all abnormal gaits it
With.The present embodiment is six classification task of abnormal gait, therefore six are defined as output layer number of nodes.
Step 5, by data label obtained in step 3 to being divided into training set and test set, after timestamp is aligned
(IMU enters one A of convolutional layer to the convolutional layer one that IMU and SEMG data are respectively fed in groups in step 4, and SEMG enters convolutional layer one
B it) is trained.Data complete feature-based fusion in convolutional layer two.CNN network be five and half full connection structures, one A of convolutional layer with
Layer one A in pond is connected entirely, and one B of convolutional layer is connect entirely with one B of pond layer, and one A of pond layer and one B of pond layer are fully connected to convolutional layer
Two, two connection pool layer two of convolutional layer, pond layer two connect full connection hidden layer, and the full hidden layer that connects is connected to full connection output
Layer.Label obtained in step 3 is switched into one-hot coding (10000,01000,001000,000100,000010,000001),
And taking 65% to do training set after corresponding on information window, label, remaining 35% does test set.Training set is sent into step 4
CNN carry out more wheel training.
In the present embodiment, epochs namely all training set exercise wheel numbers take 8, batch size namely every wheel training institute
The sequence sets number taken takes 128, and learning rate takes 0.0001 to be trained.Test set assessment models classification effect is utilized after the completion of training
Fruit.Figure 10 is that ROC curve of the model in training set, Figure 11 are obscuring in test set after 7 wheel of training in this preferred embodiment
Matrix.
Step 6 repeats step 3 and obtains three models three times to step 5, assesses mould using test set after the completion of training
The prediction result of three models is done majority vote election by type, completes the fusion of decision data layer.It is verified, in the present embodiment
Model is in the six classification accuracy (ratio that model is judged as the total real example sample number of the sample Zhan of real example) of test set
99.15%, more single IMU information source CNN network improves three percentage points.
It should be pointed out that above-described embodiment is merely to illustrate the present invention, being achieved in that for each step can be
Variation, various modifications to these embodiments are it will be apparent that therefore all for those skilled in the art
The equivalents and improvement carried out on the basis of theory of the invention general and spirit, should all protection scope of the present invention it
It is interior.
Claims (9)
1. a kind of multiple sensor signals fusion method based on deep learning for gait classification, which is characterized in that pass through structure
Deep neural network is built to classify to abnormal gait, and using neural network to the data Layer of multiple information sources data, characteristic layer,
Decision-making level is merged, so that complete extract multi-source heterogeneous sensor information, the specific steps are as follows:
Step 1 acquires normal gait information: being walked using IMU module and SEMG module the acquisition human normal for being worn on human body
When signal, obtain normal gait 3-axis acceleration information and muscle groups of lower extremitates surface myoelectric information；
Step 2, acquisition abnormity gait information: when acquiring the walking of human body simulation typical case abnormal gait using the IMU for being worn on human body
Signal, obtain abnormal gait 3-axis acceleration information and muscle groups of lower extremitates surface myoelectric information；
Step 3, data prediction are merged with data Layer: by original each channel data from two kinds of sensors according to timestamp
Alignment of data is done, will be normalized after IMU data and EMG data filtering；Walking cadence according to target representative row will be original
Data respectively do windowing cutting pretreatment, and each data queue is stamped respective labels according to gait classification, are respectively
Two groups of data labels pair of IMU, SEMG data；Wherein, timestamp is aligned in the group of tri- number of axle evidence of IMU, SEMG multi-channel data
Group in timestamp alignment and IMU, SEMG data group between timestamp be aligned；
Step 4 builds Fusion Features depth convolutional neural networks (CNN), which is six layer structure:
One A of convolutional layer is connect entirely with one A of pond layer, and one B of convolutional layer is connect entirely with one B of pond layer, one A of pond layer and pond layer one
B is fused to convolutional layer two, two connection pool layer two of convolutional layer, and pond layer two connects full connection hidden layer, and the full hidden layer that connects connects
It is connected to full connection output layer；
Define two convolutional layers one arranged side by side: one A of convolutional layer, one B of convolutional layer, define pond layer one, define a convolutional layer two,
Pond layer two defines full articulamentum and each activation primitive；
Step 5, by data label obtained in step 3 to being divided into training set and test set, the IMU after timestamp is aligned with
The convolutional layer one: IMU that SEMG data are respectively fed in groups in step 4 enters one A of convolutional layer, and SEMG enters one B of convolutional layer；Into
Row training；Data complete feature-based fusion in convolutional layer two；
Step 6 repeats step 3 to step 5 three times, obtains three models；
Step 7 utilizes test set assessment models after the completion of training, the prediction result of three models is done majority vote election, complete
It is merged at decision data layer.
2. multiple sensor signals fusion method according to claim 1, which is characterized in that Step 1: IMU described in two is adopted
Collection unit be placed in shank, in shoes, loins, normal gait is consistent with the mounting means of abnormal gait；The SEMG acquisition unit
It is placed in gastrocnemius belly of muscle, tibialis anterior belly of muscle, normal gait is consistent with the mounting means of abnormal gait.
3. multiple sensor signals fusion method according to claim 1, which is characterized in that typical case described in step 2 is abnormal
Gait includes hemiplegic gait, parkinsonian gait, duck state, steppage gait, diplegia gait step by step.
4. multiple sensor signals fusion method according to claim 1,2 or 3, which is characterized in that will described in step 3
Initial data respectively does windowing cutting, and data window length representative value is two gait cycles, and piecewise step value takes away window between window
The a quarter of length, the label form are one-hot coding.
5. multiple sensor signals fusion method according to claim 4, which is characterized in that convolutional layer one described in step 4
The input matrix of A and one B of convolutional layer are defined as every segment data window in step 3: data source port number, under current sampling frequency
Sampling number from respective signal data；
The output layer nodal point number is defined as gait category number namely a kind of normal gait and all abnormal gaits in "current" model
The sum of.
6. according to claim 1, multiple sensor signals fusion method described in 2,3 or 5, which is characterized in that the CNN network
Optimization algorithm uses Adam optimizer；
The multiple sensor signals fusion method, which is characterized in that each layer excitation function is ReLU function, Sigmod letter
Several or tanh function, output layer excitation function are Softmax function.
7. multiple sensor signals fusion method according to claim 6, which is characterized in that the loss letter of the CNN network
Number is multiclass logarithm loss function.
8. multiple sensor signals fusion method according to claim 7, which is characterized in that the Adam optimizer, ladder
Spend more new formula are as follows:
Wherein, α is default learning rate, and ε is decimal, and divisor is avoided to become 0；For gradient mean value, formula are as follows:
mt=β1mt-1+(1-β1)gt
Wherein, β1Coefficient is exponential decay rate, controls the weight distribution of momentum and current gradient, gtFor the gradient of t time step；
For gradient variance, formula are as follows:
vt=β2vt-1+(1-β2)gt 2
Wherein, β2Coefficient is exponential decay rate, the influence situation of the gradient square before controlling；
Each layer excitation function is set as the linear correcting unit of ReLU, formula are as follows:
F (x)=max (0, x)
Output layer excitation function is Softmax function, formula are as follows:
Wherein, wj(j=0,1,2,3,4,5) for from full articulamentum to the weight vectors of full connection output layer.
9. multiple sensor signals fusion method according to claim 8, which is characterized in that the loss letter of the CNN network
Number is multiclass logarithm loss function or multiclass cross entropy cost function, concrete form are as follows:
Wherein, yi' for i-th of value in label, yiFor the respective components in the vector through softmax normalized output.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910063463.5A CN109784412A (en) | 2019-01-23 | 2019-01-23 | The multiple sensor signals fusion method based on deep learning for gait classification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910063463.5A CN109784412A (en) | 2019-01-23 | 2019-01-23 | The multiple sensor signals fusion method based on deep learning for gait classification |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109784412A true CN109784412A (en) | 2019-05-21 |
Family
ID=66502101
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910063463.5A Pending CN109784412A (en) | 2019-01-23 | 2019-01-23 | The multiple sensor signals fusion method based on deep learning for gait classification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109784412A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110215216A (en) * | 2019-06-11 | 2019-09-10 | 中国科学院自动化研究所 | Based on the with different levels Activity recognition method in skeletal joint point subregion, system |
| CN110236550A (en) * | 2019-05-30 | 2019-09-17 | 清华大学 | A kind of body gait prediction meanss based on multi-modal deep learning |
| CN110507288A (en) * | 2019-08-29 | 2019-11-29 | 重庆大学 | Vision based on one-dimensional convolutional neural networks induces motion sickness detection method |
| CN110537921A (en) * | 2019-08-28 | 2019-12-06 | 华南理工大学 | Portable gait multi-sensing data acquisition system |
| CN110801226A (en) * | 2019-11-01 | 2020-02-18 | 西安交通大学 | Human knee joint moment testing system method based on surface electromyographic signals and application |
| CN111611859A (en) * | 2020-04-21 | 2020-09-01 | 河北工业大学 | Gait recognition method based on GRU |
| CN111950460A (en) * | 2020-08-13 | 2020-11-17 | 电子科技大学 | Muscle strength self-adaptive stroke patient hand rehabilitation training action recognition method |
| CN112067015A (en) * | 2020-09-03 | 2020-12-11 | 青岛歌尔智能传感器有限公司 | Step counting method and device based on convolutional neural network and readable storage medium |
| CN112818927A (en) * | 2021-02-26 | 2021-05-18 | 上海交通大学 | Real-time classification method and system for human body lower limb movement modes |
| CN113286311A (en) * | 2021-04-29 | 2021-08-20 | 沈阳工业大学 | Distributed perimeter security protection environment sensing system based on multi-sensor fusion |
| CN115019393A (en) * | 2022-06-09 | 2022-09-06 | 天津理工大学 | Exoskeleton robot gait recognition system and method based on convolutional neural network |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108345846A (en) * | 2018-01-29 | 2018-07-31 | 华东师范大学 | A kind of Human bodys' response method and identifying system based on convolutional neural networks |
| CN108734208A (en) * | 2018-05-15 | 2018-11-02 | 重庆大学 | Multi-source heterogeneous data fusion system based on multi-modal depth migration study mechanism |
| CN108958482A (en) * | 2018-06-28 | 2018-12-07 | 福州大学 | A kind of similitude action recognition device and method based on convolutional neural networks |
-
2019
- 2019-01-23 CN CN201910063463.5A patent/CN109784412A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108345846A (en) * | 2018-01-29 | 2018-07-31 | 华东师范大学 | A kind of Human bodys' response method and identifying system based on convolutional neural networks |
| CN108734208A (en) * | 2018-05-15 | 2018-11-02 | 重庆大学 | Multi-source heterogeneous data fusion system based on multi-modal depth migration study mechanism |
| CN108958482A (en) * | 2018-06-28 | 2018-12-07 | 福州大学 | A kind of similitude action recognition device and method based on convolutional neural networks |
Non-Patent Citations (1)
| Title |
|---|
| 王中伟: "基于多感知节点数据融合的体态行为分析研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110236550A (en) * | 2019-05-30 | 2019-09-17 | 清华大学 | A kind of body gait prediction meanss based on multi-modal deep learning |
| CN110215216A (en) * | 2019-06-11 | 2019-09-10 | 中国科学院自动化研究所 | Based on the with different levels Activity recognition method in skeletal joint point subregion, system |
| CN110215216B (en) * | 2019-06-11 | 2020-08-25 | 中国科学院自动化研究所 | Behavior identification method and system based on skeletal joint point regional and hierarchical level |
| CN110537921A (en) * | 2019-08-28 | 2019-12-06 | 华南理工大学 | Portable gait multi-sensing data acquisition system |
| CN110507288A (en) * | 2019-08-29 | 2019-11-29 | 重庆大学 | Vision based on one-dimensional convolutional neural networks induces motion sickness detection method |
| CN110801226A (en) * | 2019-11-01 | 2020-02-18 | 西安交通大学 | Human knee joint moment testing system method based on surface electromyographic signals and application |
| CN111611859A (en) * | 2020-04-21 | 2020-09-01 | 河北工业大学 | Gait recognition method based on GRU |
| CN111950460A (en) * | 2020-08-13 | 2020-11-17 | 电子科技大学 | Muscle strength self-adaptive stroke patient hand rehabilitation training action recognition method |
| CN112067015A (en) * | 2020-09-03 | 2020-12-11 | 青岛歌尔智能传感器有限公司 | Step counting method and device based on convolutional neural network and readable storage medium |
| CN112067015B (en) * | 2020-09-03 | 2022-11-22 | 青岛歌尔智能传感器有限公司 | Step counting method and device based on convolutional neural network and readable storage medium |
| CN112818927A (en) * | 2021-02-26 | 2021-05-18 | 上海交通大学 | Real-time classification method and system for human body lower limb movement modes |
| CN113286311A (en) * | 2021-04-29 | 2021-08-20 | 沈阳工业大学 | Distributed perimeter security protection environment sensing system based on multi-sensor fusion |
| CN113286311B (en) * | 2021-04-29 | 2024-04-12 | 沈阳工业大学 | Distributed perimeter security environment sensing system based on multi-sensor fusion |
| CN115019393A (en) * | 2022-06-09 | 2022-09-06 | 天津理工大学 | Exoskeleton robot gait recognition system and method based on convolutional neural network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109784412A (en) | The multiple sensor signals fusion method based on deep learning for gait classification | |
| WO2021143353A1 (en) | Gesture information processing method and apparatus, electronic device, and storage medium | |
| CN106503799B (en) | Deep learning model based on multiple dimensioned network and the application in brain status monitoring | |
| CN109770912A (en) | A kind of abnormal gait classification method based on depth convolutional neural networks | |
| EP3895605A1 (en) | Methods and systems for determining abnormal cardiac activity | |
| CN104771163B (en) | EEG feature extraction method based on CSP and R CSP algorithms | |
| CN108597609A (en) | A kind of doctor based on LSTM networks is foster to combine health monitor method | |
| CN109993093A (en) | Road anger monitoring method, system, equipment and medium based on face and respiratory characteristic | |
| CN108491077A (en) | A kind of surface electromyogram signal gesture identification method for convolutional neural networks of being divided and ruled based on multithread | |
| CN110013248A (en) | Brain electricity tensor mode identification technology and brain-machine interaction rehabilitation system | |
| EP3836836B1 (en) | Real-time spike detection and identification | |
| CN113111865B (en) | Fall behavior detection method and system based on deep learning | |
| CN109770913A (en) | A kind of abnormal gait recognition methods based on reverse transmittance nerve network | |
| Shao et al. | Single-channel SEMG using wavelet deep belief networks for upper limb motion recognition | |
| CN109976526A (en) | A kind of sign Language Recognition Method based on surface myoelectric sensor and nine axle sensors | |
| CN112884063B (en) | P300 signal detection and identification method based on multi-element space-time convolution neural network | |
| Xu et al. | Intelligent emotion detection method based on deep learning in medical and health data | |
| Nguyen et al. | IMU-based spectrogram approach with deep convolutional neural networks for gait classification | |
| CN106991409A (en) | A kind of Mental imagery EEG feature extraction and categorizing system and method | |
| CN113010013A (en) | Wasserstein distance-based motor imagery electroencephalogram migration learning method | |
| Jung et al. | Deep neural network-based gait classification using wearable inertial sensor data | |
| Nafea et al. | Multi-sensor human activity recognition using CNN and GRU | |
| Liu et al. | A fully connected deep learning approach to upper limb gesture recognition in a secure FES rehabilitation environment | |
| CN107045624A (en) | A kind of EEG signals pretreatment rolled into a ball based on maximum weighted and sorting technique | |
| CN110916672A (en) | Old people daily activity monitoring method based on one-dimensional convolutional neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190521 |