CN111797743A - Human behavior recognition method based on bidirectional LSTM - Google Patents

Human behavior recognition method based on bidirectional LSTM Download PDF

Info

Publication number
CN111797743A
CN111797743A CN202010596167.4A CN202010596167A CN111797743A CN 111797743 A CN111797743 A CN 111797743A CN 202010596167 A CN202010596167 A CN 202010596167A CN 111797743 A CN111797743 A CN 111797743A
Authority
CN
China
Prior art keywords
behavior
layer
behavior recognition
bidirectional lstm
lstm
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010596167.4A
Other languages
Chinese (zh)
Inventor
潘赟
肖沛文
朱怀宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN202010596167.4A priority Critical patent/CN111797743A/en
Publication of CN111797743A publication Critical patent/CN111797743A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/30—Noise filtering

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Biology (AREA)
  • Multimedia (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Human Computer Interaction (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

A human behavior recognition method based on bidirectional LSTM comprises a training stage and a recognition stage, wherein in the training stage, three-axis acceleration original data are collected through an accelerometer, then filtering denoising and data segmentation preprocessing are carried out on sensor data, finally, a data segment obtained through segmentation and a behavior label obtained during collection are utilized to train and verify a behavior recognition network model based on bidirectional LSTM, and a training model with the highest recognition rate on a verification set is selected to be applied to the recognition stage. The method has the advantages of high accuracy and few parameters. The invention provides a deep bidirectional LSTM human body behavior recognition method based on bidirectional LSTM, which can provide a recognition result with higher accuracy and rapidness.

Description

Human behavior recognition method based on bidirectional LSTM
Technical Field
The invention belongs to the field of human behavior recognition, and relates to a human behavior recognition method based on deep learning and sensor data.
Background
The human behavior recognition technology can fully reflect human behavior states and physiological information, and has wide application prospects in the fields of motion tracking, body building exercise, daily monitoring, medical rehabilitation, man-machine interaction, virtual reality, intelligent environments and the like. Based on the behavior recognition of the inertial sensor, the physical information such as acceleration, angular velocity and direction generated by human motion is collected by using sensors such as an accelerometer, a gyroscope and a direction sensor and is used for recognizing the current human behavior.
The accuracy of the traditional human behavior identification method depends on the extraction and selection of features to a great extent, and researchers need to extract the time domain, frequency domain and other features of signals by means of professional knowledge and skills and carry out feature reduction by combining feature engineering. The neural network can convert the original data into higher-order, more abstract and more complex expressions through a simple nonlinear model, and can replace an artificial feature extraction process of behavior recognition through automatic feature extraction.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a deep bidirectional LSTM human body behavior recognition method based on bidirectional LSTM, which can provide a higher-accuracy and faster recognition result.
The technical scheme adopted by the invention for solving the technical problems is as follows:
a human behavior recognition method based on bidirectional LSTM comprises the following steps:
step (1): the method comprises the steps of collecting sensor data under different human behavior categories, wherein the collected action categories are set as M ═ M1，m2，...，mJJ is a predefined number of behavior categories;
step (2): carrying out filtering and denoising processing on the sensor data by using a low-pass filtering method;
and (3): segmenting the filtered sensor data to obtain data segments, and for each data segment Pk＝[A1，A2，...，AL]∈RN×LAnd L is the length of the fragment,
Figure BDA0002557483920000011
1, 2, L, N is the sensor data dimension, k is 1, 2,.., K and K are the number of data segments, the behavior class label of the data segment is marked as j, and j corresponds to the behavior class mj，mjE.m, J1, 2, J, all data segments together forming a sample set S, S ═ P1，P2，...，PK]；
And (4): constructing a deep bidirectional LSTM behavior recognition network model based on bidirectional LSTM;
and (5): dividing a sample set S into a model training set STAnd model validation set SVUsing the model training set STTraining a deep bidirectional LSTM behavior recognition network model and performing verification on a verification set SVCarrying out model verification;
and (6): and (4) identifying the behavior by using the trained deep bidirectional LSTM network model to obtain an identification result.
Further, in the step (1), the collected sensor data includes data of three axes x, y, and z of the accelerometer, and the collected motion set M includes 6 types of walking, jogging, going upstairs, going downstairs, standing, and sitting.
Still further, in the step (3), the segmentation is performed by using a sliding window method with a window length of L and an overlap coverage of ρ%.
Further, in the step (4), the constructed deep bidirectional LSTM behavior recognition network model includes an input layer, a fully-connected layer, a hidden layer and an output layer, and for each input data segment Pinput＝ [A1，A2，...，AL]∈RN×LConverted into by using the full connection layer
Figure BDA0002557483920000021
Figure BDA0002557483920000022
i＝1，2，...，L，NfcThe number of nodes of the full connection layer; the hidden layer comprises a forward propagation layer and a backward propagation layer, the depth of the forward propagation layer and the depth of the backward propagation layer are 3, each layer adopts LSTM units, and the number of the LSTM units in each layer is 64; the outputs of the forward pass and the reverse pass are spliced through a splicing operation,and inputting the sequence of the last spliced moment into an output layer, wherein the number of nodes of the output layer is 6, the index of the maximum value output by the output layer is used as a final identification category label corresponding to the number of the behavior categories.
Further, in the step (5), a training set S is usedTAnd STTraining a deep bidirectional LSTM behavior recognition network model according to the class label corresponding to each sample segment, and performing verification on the model in a verification set SVThe verification is carried out, the accuracy is used as the evaluation standard of the model, and the verification set S is selectedVAnd obtaining model parameters under the iteration cycle with the highest accuracy to obtain a final depth bidirectional LSTM behavior recognition network model.
Further, in the step (6), a low-pass filtering method is adopted to perform filtering and denoising processing on accelerometer data acquired in real time, a sliding window method with a window length of L and an overlapping coverage rate of ρ% is adopted to perform segmentation to obtain a data segment at the current moment, and the data segment is input into a trained deep bidirectional LSTM behavior recognition network to recognize the current behavior category.
The invention has the following beneficial effects: the method can provide a higher-accuracy and faster identification result.
Drawings
FIG. 1 is a flow chart of a behavior recognition method according to the present invention;
FIG. 2 is a deep bidirectional LSTM behavior recognition network model of the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
Referring to fig. 1 and 2, a bidirectional LSTM-based human behavior recognition method includes the following steps:
step (1): the implementation method comprises the steps of collecting sensor data under different human behavior categories, using a WISDM public data set, placing a smart phone in a trouser pocket, collecting accelerometer sensor data, wherein the sampling frequency is 20Hz, the number of people participating in data collection is 36, the collected actions are 6 types, namely walking, going upstairs, going downstairs, jogging, sitting and standing, the data set comprises 1098209 pieces of sampling data, and each piece of sampling data comprises an experimenter number, behaviors and 3-axis acceleration data.
Step (2): the method comprises the steps of utilizing a low-pass filtering method to carry out filtering and denoising processing on sensor data, and adopting a moving average filter with the window length of 5 to carry out denoising processing on accelerometer sensor data.
And (3): the implementation method comprises the steps of firstly removing missing points in a data set, then carrying out data segmentation by using a sliding window method with the window length of 60 and the overlapping coverage rate of 50%, carrying out total segmentation to obtain 35998 samples, wherein each sample is 60 points long, contains a 3-axis acceleration time sequence, and marks a behavior class label of each sample.
And (4): a deep bidirectional LSTM behavior recognition network model is constructed based on bidirectional LSTM, and the bidirectional LSTM behavior recognition network model designed by the implementation method is shown in figure 2. The model comprises an input layer, a full connection layer, a hidden layer and an output layer, wherein the number of nodes of the full connection layer is 32, the hidden layer comprises a forward propagation layer and a backward propagation layer, the depth of the forward propagation layer and the depth of the backward propagation layer are 3, each layer adopts an LSTM unit, and the number of the LSTM units of each layer is 64; and the output of the forward transmission and the reverse transmission is spliced through splicing operation, the sequence of the last moment after splicing is input into an output layer, the number of nodes of the output layer is 6, the index of the maximum value output by the output layer is used as a final identification category label corresponding to the number of behavior categories.
And (5): training and verifying a deep bidirectional LSTM behavior recognition network model, dividing 70% of samples obtained in the step (3) into a training set, and dividing 30% of the samples into a verification set, namely 25198 samples and 10800 samples are respectively contained in the training set and the verification set, training the deep bidirectional LSTM behavior recognition network model by using the model training set, initially setting the learning rate of the network to be 0.0025, and reducing the learning rate along with the increase of iteration times, namely after 50 iterations are carried out, setting the learning rate to be 0.8 originally. In order to enhance the generalization capability of the network and relieve the over-fitting problem, L2 regularization is adopted to limit the weight, the regularization coefficient is 0.0015, an Adam optimizer is adopted to train the model parameters, and the loss function adopts a cross entropy loss function. Model validation and selection is performed on the validation set. The recognition accuracy of the finally selected model on the validation set was 98.5%. Table 1 shows the identification confusion matrix of the model on the test set. The vertical direction is the real category and the horizontal direction is the recognition category.
Walking machine Go upstairs Go downstairs Jogging Sitting position Station
Walking machine 4232 18 20 1 2 0
Go upstairs 14 1135 27 10 0 1
Go downstairs 13 30 882 8 0 0
Jogging 8 4 7 3361 0 0
Sitting position 0 0 0 0 559 4
Station 0 0 0 0 0 464
TABLE 1
And (6): and (4) identifying the behavior by using the trained deep bidirectional LSTM network model to obtain an identification result. The method comprises the steps of placing a smart phone or other mobile equipment in a trouser pocket to collect real-time data of an accelerometer sensor, carrying out filtering and denoising processing on the accelerometer data collected in real time by adopting a moving average filtering method with the window length of 5, segmenting by adopting a sliding window method with the window length of 60 and the overlapping coverage rate of 50% to obtain a data segment at the current moment, inputting the data segment in the step (5) into a trained bidirectional LSTM behavior recognition network, and recognizing the current behavior category.

Claims (6)

1. A human behavior recognition method based on bidirectional LSTM is characterized by comprising the following steps:
step (1): the method comprises the steps of collecting sensor data under different human behavior categories, wherein the collected action categories are set as M ═ M1，m2，...，mJJ is a predefined number of behavior categories;
step (2): carrying out filtering and denoising processing on the sensor data by using a low-pass filtering method;
and (3): segmenting the filtered sensor data to obtain data segments, and for each data segment Pk＝[A1，A2，...，AL]∈RN×LAnd L is the length of the fragment,
Figure FDA0002557483910000011
Figure FDA0002557483910000012
n is the dimension of the sensor data, K is 1, 2, and K is the number of data segments, the behavior class label of the data segment is marked as j, and j corresponds to the behavior class mj，mjE.m, J1, 2, J, all data segments together forming a sample set S, S ═ P1，P2，...，PK]；
And (4): constructing a deep bidirectional LSTM behavior recognition network model based on bidirectional LSTM;
and (5): dividing a sample set S into a model training set STAnd model validation set SVTraining by means of modelsCollection STTraining a deep bidirectional LSTM behavior recognition network model and performing verification on a verification set SVCarrying out model verification;
and (6): and (4) identifying the behavior by using the trained deep bidirectional LSTM network model to obtain an identification result.
2. The bi-directional LSTM based behavior recognition method of claim 1, wherein: in the step (1), the collected sensor data includes data of three axes x, y and z of the accelerometer, and the collected motion set M includes 6 types including walking, jogging, going upstairs, going downstairs, standing and sitting.
3. The bidirectional LSTM-based behavior recognition method according to claim 1 or 2, wherein in the step (3), the segmentation is performed by using a sliding window method with a window length L and an overlapping coverage rate p%.
4. The bi-directional LSTM based behavior recognition method of claim 1 or 2, wherein in step (4), the constructed deep bi-directional LSTM behavior recognition network model comprises an input layer, a fully connected layer, a hidden layer and an output layer, and for each input data segment Pinput＝[A1，A2，...，AL]∈RN×LConverted into by using the full connection layer
Figure FDA0002557483910000021
Figure FDA0002557483910000022
NfcThe number of nodes of the full connection layer; the hidden layer comprises a forward propagation layer and a backward propagation layer, the depth of the forward propagation layer and the depth of the backward propagation layer are 3, each layer adopts LSTM units, and the number of the LSTM units in each layer is 64; the output of the forward transmission and the reverse transmission is spliced through splicing operation, the sequence of the last moment after splicing is input into an output layer, the number of nodes of the output layer is 6, the maximum value of the output layer is located according to the number of behavior categoriesQuoted as the final identification category label.
5. The bi-directional LSTM based behavior recognition method of claim 1 or 2, wherein in step (5), a training set S is usedTAnd STTraining a deep bidirectional LSTM behavior recognition network model according to the class label corresponding to each sample segment, and performing verification on a verification set SVThe accuracy is used as the evaluation standard of the model, and the verification set S is selectedVAnd obtaining model parameters under the iteration cycle with the highest accuracy to obtain a final depth bidirectional LSTM behavior recognition network model.
6. The bidirectional LSTM-based behavior recognition method of claim 1 or 2, wherein in the step (6), the accelerometer data collected in real time is filtered and denoised by a low-pass filtering method, a data segment at the current moment is obtained by segmenting by a sliding window method with a window length L and an overlapping coverage rate p%, and the data segment is input into a trained deep bidirectional LSTM behavior recognition network to recognize the current behavior category.
CN202010596167.4A 2020-06-28 2020-06-28 Human behavior recognition method based on bidirectional LSTM Pending CN111797743A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010596167.4A CN111797743A (en) 2020-06-28 2020-06-28 Human behavior recognition method based on bidirectional LSTM

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010596167.4A CN111797743A (en) 2020-06-28 2020-06-28 Human behavior recognition method based on bidirectional LSTM

Publications (1)

Publication Number Publication Date
CN111797743A true CN111797743A (en) 2020-10-20

Family

ID=72804594

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010596167.4A Pending CN111797743A (en) 2020-06-28 2020-06-28 Human behavior recognition method based on bidirectional LSTM

Country Status (1)

Country Link
CN (1) CN111797743A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113742669A (en) * 2021-08-18 2021-12-03 浙江工业大学 User authentication method based on twin network

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113742669A (en) * 2021-08-18 2021-12-03 浙江工业大学 User authentication method based on twin network
CN113742669B (en) * 2021-08-18 2024-05-14 浙江工业大学 User authentication method based on twin network

Similar Documents

Publication Publication Date Title
CN104143079B (en) The method and system of face character identification
CN106650687B (en) Posture correction method based on depth information and skeleton information
CN107219924B (en) A kind of aerial gesture identification method based on inertial sensor
CN106096662A (en) Human motion state identification based on acceleration transducer
CN105160310A (en) 3D (three-dimensional) convolutional neural network based human body behavior recognition method
CN109101938B (en) Multi-label age estimation method based on convolutional neural network
CN108229268A (en) Expression Recognition and convolutional neural networks model training method, device and electronic equipment
CN107462258A (en) A kind of step-recording method based on mobile phone 3-axis acceleration sensor
CN108257158A (en) A kind of target prediction and tracking based on Recognition with Recurrent Neural Network
CN106228200A (en) A kind of action identification method not relying on action message collecting device
CN105205449A (en) Sign language recognition method based on deep learning
CN111476161A (en) Somatosensory dynamic gesture recognition method fusing image and physiological signal dual channels
CN105943206A (en) Prosthetic hand control method based on MYO armlet
CN103400123A (en) Gait type identification method based on three-axis acceleration sensor and neural network
CN113723312B (en) Rice disease identification method based on visual transducer
CN103886030B (en) Cost-sensitive decision-making tree based physical information fusion system data classification method
CN110197224A (en) Aerial hand-written character track restoration methods based on the confrontation study of feature space depth
CN110363233A (en) A kind of the fine granularity image-recognizing method and system of the convolutional neural networks based on block detector and Fusion Features
CN112464738A (en) Improved naive Bayes algorithm user behavior identification method based on mobile phone sensor
CN109903053A (en) A kind of anti-fraud method carrying out Activity recognition based on sensing data
CN108960171B (en) Method for converting gesture recognition into identity recognition based on feature transfer learning
CN110501741A (en) A kind of useful signal detection method and system
CN111738178A (en) Wearing mask facial expression recognition method based on deep learning
CN107506781A (en) A kind of Human bodys' response method based on BP neural network
CN113642432A (en) Method for identifying human body posture by convolutional neural network based on covariance matrix transformation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination