CN113947119A - Method for detecting human gait by using plantar pressure signals - Google Patents

Method for detecting human gait by using plantar pressure signals Download PDF

Info

Publication number
CN113947119A
CN113947119A CN202111210035.4A CN202111210035A CN113947119A CN 113947119 A CN113947119 A CN 113947119A CN 202111210035 A CN202111210035 A CN 202111210035A CN 113947119 A CN113947119 A CN 113947119A
Authority
CN
China
Prior art keywords
layer
pressure
gait
model
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
CN202111210035.4A
Other languages
Chinese (zh)
Inventor
吴化平
陈海宁
苏彬彬
由淋元
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University of Technology ZJUT
Original Assignee
Zhejiang University of Technology ZJUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University of Technology ZJUT filed Critical Zhejiang University of Technology ZJUT
Priority to CN202111210035.4A priority Critical patent/CN113947119A/en
Publication of CN113947119A publication Critical patent/CN113947119A/en
Withdrawn legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/08—Feature extraction
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/1036—Measuring load distribution, e.g. podologic studies
    • A61B5/1038—Measuring plantar pressure during gait
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/112—Gait analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/12—Classification; Matching

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Biophysics (AREA)
  • Data Mining & Analysis (AREA)
  • Biomedical Technology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Animal Behavior & Ethology (AREA)
  • Dentistry (AREA)
  • Surgery (AREA)
  • Pathology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Evolutionary Biology (AREA)
  • Signal Processing (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Physiology (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

The invention discloses a method for detecting human gait by using plantar pressure signals. The method comprises the steps that a flexible array pressure sensor is used for collecting plantar pressure information, and a movement gait data set is established; carrying out pretreatment operation; establishing an improved MobileNet-V3 algorithm model; sending the preprocessed movement gait data set into a model for training and testing; and deploying the tested model at the mobile phone end, and transmitting the left foot pressure data and the right foot pressure data acquired in real time to the mobile phone end to predict the current gait. The method of the invention adopts the improved network model, realizes better accuracy and generalization, finally achieves the purpose of improving the model performance, and realizes the technical effect of ensuring the lightweight of the algorithm model and simultaneously having higher performance.

Description

Method for detecting human gait by using plantar pressure signals
Technical Field
The invention belongs to a human body gait detection method in the technical field of human body gait recognition and deep learning, and particularly relates to a method for detecting human body gait by utilizing plantar pressure signals and an improved neural network.
Background
In recent years, with the rapid development of artificial intelligence technology, research on human gait recognition using deep learning technology has received more and more attention from relevant experts and scholars. At present, the technology of human motion gait recognition has important application in many fields, such as health detection, rehabilitation and nursing, man-machine cooperation and other fields, and the human motion gait recognition technology is introduced into virtual reality to bring better experience to users, so the research of the human motion gait recognition technology is necessary.
In the deep learning era, the research of human gait recognition algorithm is advanced to a certain extent. The method can be divided into two categories according to different feature information used by the algorithm, wherein one category is that the related information of key points of human gait is taken as features, for example, the classification recognition is finished by means of RNNs, LSTM, CNN and the like by utilizing the joint distribution histogram of the key points or the rotation and displacement features of 3D positions; one is to use plantar pressure information as a feature, such as clustering human body actions using a self-organizing map neural network (SOM) and plantar pressure sensing information.
With the increasing popularity of 5G technologies and smart phones, there is a greater demand for deep learning algorithms to be deployed on mobile terminals. On the other hand, as the number of layers of the neural network model is gradually increased to achieve higher performance, the number of layers can reach thousands, however, the oversized model is not suitable for being deployed on a mobile terminal, which promotes the generation of lightweight networks such as Squeezenet, Mobilenet and the like, and the Mobilenet network model is developed through V1, V2 and V3.
Although the MobileNet-V3 network model is improved to a certain extent on the basis of the MobileNet-V2 network model, for example, the detection precision of the MobileNet-V3 network model on a COCO data set is approximately the same as that of the MobileNet-V2 network model, the problems of further improving the inference speed and the classification accuracy still exist.
Disclosure of Invention
The invention provides a method for detecting human gait by using plantar pressure signals, aiming at the problem that an algorithm model needs to have higher classification accuracy while being light.
The technical scheme adopted by the invention is as follows:
(1) the method comprises the steps that a flexible array pressure sensor is used for collecting plantar pressure information, and a movement gait data set is established;
(2) after operations such as bilinear interpolation, normalization, cascade connection and the like are carried out on the motion gait data set obtained in the step (1), the preprocessing process of the motion gait data set is completed;
(3) establishing an improved MobileNet-V3 algorithm model;
(4) sending the movement gait data set which is preprocessed in the step (2) into the model in the step (3) for training and testing;
(5) and deploying the tested model at the mobile phone end, and transmitting the pressure data of the left foot and the right foot acquired by the flexible array pressure sensor to the mobile phone end in real time through Bluetooth to predict the current gait.
In the step (1), the flexible array pressure sensor mainly comprises a plurality of pressure sensitive sensors which are arranged in a row-column array, the pressure sensitive sensors are uniformly distributed on the left foot and the right foot of a human body respectively, the pressure sensitive sensors are arranged on the sole of the foot respectively according to the shape of the foot, each pressure sensitive sensor is of a sandwich structure and comprises two layers of electrode materials and one layer of force sensitive material, and the layer of force sensitive material is sandwiched between the two layers of electrode materials.
In the flexible array pressure sensor described above,
the pressure-sensitive sensors in all the column directions are connected to the column one-out-of-one switch and then output, one end of each pressure-sensitive sensor in the same column direction is connected to the same input end of the column one-out-of-one switch, one ends of the pressure-sensitive sensors in different column directions are respectively connected to different input ends of the column one-out-of-one switch, and the only output end of the column one-out-of-one switch is led out through a reference resistor and then connected to an external receiving circuit;
the pressure-sensitive sensors in each row direction are connected to the row one-out-of-multiple switch and then output, the other ends of the pressure-sensitive sensors in the same row direction are connected together and then connected to the same input end of the row one-out-of-multiple switch, one ends of the pressure-sensitive sensors in different row directions are respectively connected to different input ends of the row one-out-of-multiple switch, and the only output end of the row one-out-of-multiple switch is led out and connected to an external receiving circuit;
the pressure-sensitive sensors are arranged in sequence, the other end of each pressure-sensitive sensor is connected to the other end of the last pressure-sensitive sensor through a diode, and the other end of the last pressure-sensitive sensor is connected to one input end of the one-row-multiple-selection switch through a diode. Thereby reducing crosstalk between the sensors by means of the series diodes.
The data collected by the flexible array pressure sensor is in time sequence by the vertical axis, the horizontal axis represents the data collected by different sensors, and each gait has 128 columns of effective data.
In the step (1), the data collected by the flexible array pressure sensors are extracted by respectively using the flexible array pressure sensors under five human body gaits of standing, sitting, walking on flat ground, going upstairs and going downstairs, taking 1.5 seconds as the length of a time window and taking 1 second as the step length of a time sliding window, and effective sample data is obtained to form a motion gait data set.
In the step (2), the sensor data respectively collected at the left and right feet of the human body are stacked and cascaded together in the channel direction.
In the step (3), the improved MobileNet-V3 algorithm model is obtained by replacing all inverted residual error structures in the original MobileNet-V3 algorithm model with hourglass structures;
the inverse residual structure is mainly formed by sequentially connecting a first layer of point-by-point convolution layer PW, a layer of depth convolution layer DW and a second layer of point-by-point convolution layer PW, wherein the input of the first layer of point-by-point convolution layer PW is used as the input of the inverse residual structure, and the input of the first layer of point-by-point convolution layer PW and the output of the second layer of point-by-point convolution layer PW are subjected to addition operation and then are used as the output of the inverse residual structure.
The hourglass structure is mainly formed by sequentially connecting a first layer of depth convolution layer DW, a first layer of point-by-point convolution layer PW, a second layer of point-by-point convolution layer PW and a second layer of depth convolution layer DW, wherein the input of the first layer of depth convolution layer DW is used as the input of the hourglass structure, and the input of the first layer of depth convolution layer DW and the output of the second layer of depth convolution layer DW are subjected to addition operation and then are used as the output of the hourglass structure; the hourglass structure is formed by reversing the depth convolution layer and the point-by-point convolution layer in the inverted residual structure, and finally adding one depth convolution layer.
The depth convolution layer performs a convolution operation using one convolution kernel for each channel of the input, and the point-by-point convolution performs a convolution operation on the input using a convolution kernel of size 1 × 1.
Simultaneously, all h-swish activation functions and relu6 activation functions of other neural network layers except an output layer in the original MobileNet-V3 algorithm model are replaced by the hash function, when the input of the hash function is less than zero, the function value is not directly equal to zero but smoothly transits to zero, and the softmax activation function is reserved in the output layer;
meanwhile, a compression excitation module is added between a second layer of point-by-point convolution layer PW and a second layer of depth convolution layer DW of each hourglass structure in an original MobileNet-V3 algorithm model, the compression excitation module mainly comprises a layer of global average pooling layer, two continuous layers of full-connection layers and a scale operation, feature graphs output by the second layer of point-by-point convolution layer PW are respectively input into the global average pooling layer and the scale operation of the compression excitation module, the global average pooling layer output sequentially passes through the two continuous layers of full-connection layers and then is input into the scale operation, a miactivation function is arranged behind each full-connection layer, the output of the final compression excitation module is obtained through the scale operation, and then the final output is input into the second layer of depth convolution layer DW. The feature map output by the second layer point-by-point convolution layer PW is compressed into a vector of 1 × 1 × C by a compression excitation module, and then the vector and the original feature map are subjected to scale operation. The attention mechanism is arranged and applied to the hourglass structure, so that important characteristic information is emphasized, useless detail information is restrained, the network can learn more key and useful information, and the model learning efficiency is improved.
And (4) scaling operation, namely multiplying the weighted value of each channel calculated by the compressed excitation module by a two-dimensional matrix of the channel corresponding to the characteristic diagram output by the second layer point-by-point convolution layer.
According to the model, an hourglass structure is adopted to replace a reversed residual error structure in an original model, higher dimensional features and richer space information in a data set are extracted, meanwhile, a mish function is used to replace a relu function and an h-swish function, gradient transition is smooth, information flow is guaranteed, on the basis, an attention mechanism used in the original model is reserved, and the model is applied to the hourglass structure, so that the model obtains better performance.
In the step (5), the model is implanted into a gait recognition application program, the gait recognition application program is mainly divided into two parts, one part is responsible for data acquisition and display, and the other part is responsible for data reading and analysis.
When data acquisition is started, the sensor acquisition module uploads display data to a mobile phone end by using Bluetooth low frequency so as to draw a real-time plantar pressure dynamic image on a user interface, and acquires pressure data at high frequency and stores the pressure data in the storage module; when data acquisition is stopped, the sensor acquisition module actively reads data in the storage module and uploads the data to the mobile phone end, the mobile phone end calls a motion posture analysis program to input plantar pressure data into the algorithm model, and an analysis result of the model is received and displayed.
According to the invention, on the basis of the original MobileNet-V3 network model, an hourglass structure (sandglass block) is adopted to replace the original inverted residual structure, more characteristic information is kept, meanwhile, the original hswish function is replaced by the Mish function, and the smoother activation function allows better information to enter the neural network, so that better accuracy and generalization are obtained, the purpose of improving the model performance is finally achieved, and the technical effect of ensuring the lightweight of the algorithm model and still having higher performance is realized.
The inverted residual error structure in the original MobileNet-V3 network is completely replaced by the hourglass structure, the position of the bottleneck of the hourglass structure is different from the inverted residual error structure, the bottleneck of the inverted residual error structure is positioned at the front-rear joint of the two residual error blocks, the bottleneck of the hourglass structure is positioned in the middle of the hourglass structure, the hourglass structure establishes short connection between higher dimensional features, and meanwhile, deep convolution is increased, so that the network learns more features.
The invention takes the hash function as the activation function, the hash function enables the gradient descending process to be smoother, the situation that the gradient is suddenly zero can not occur, and the information flow between network layers is ensured.
The present invention retains the attention of the original network structure and applies it to an hourglass structure. Through the method, the MobileNet-V3 network model is improved, so that the classification performance of the model is improved to a certain degree.
In addition, the improved Mobile Net-V3 network model is deployed at the mobile phone end, the mobile phone end receives left and right foot pressure data through Bluetooth and inputs the data into the algorithm model, and finally, gait analysis results are received and displayed at the mobile phone end.
The invention has the beneficial effects that:
the method of the invention adopts the improved network model, realizes better accuracy and generalization, finally achieves the purpose of improving the model performance, and realizes the technical effect of ensuring the lightweight of the algorithm model and simultaneously having higher performance.
The improved Mobile Net-V3 network model is deployed at the mobile phone end, so that the purpose that the mobile phone end receives left and right foot pressure data by using Bluetooth and inputs the left and right foot pressure data into the algorithm model to detect the current human gait is achieved.
Drawings
FIG. 1 is a flow chart of the steps of the method of the present invention.
FIG. 2 is a comparison graph of the structure of the sand leakage in the method of the present invention and the structure of the inverted residual error of the original MobileNet-V3 model.
FIG. 3 is a flow chart of a compression excitation module for a sandscreen structure in the method of the present invention.
Fig. 4 is a diagram of an array layout of a flexible array sensor of the present invention.
Detailed Description
The following detailed description of embodiments of the invention refers to the accompanying drawings.
The specific implementation process and conditions of the invention are as follows:
examples
Step 1, sole pressure data are collected, and a motion gait data set is constructed.
The method comprises the steps of firstly, utilizing flexible array pressure sensors to collect plantar pressure data under different human motion gaits, respectively and uniformly distributing 64 flexible force-sensitive sensors according to the shapes of left and right feet of a human body, wherein the structure of each sensor is a sandwich structure, namely, a layer of force-sensitive material is sandwiched between two layers of electrode materials, crosstalk is reduced between the sensors in a mode of serially connecting diodes, the sampling frequency during high-frequency sampling is set to be 20Hz, the longitudinal axis of the collected data is a time sequence, the transverse axis represents the data collected by different sensors, and each gait has 128 rows of effective data.
The maximum duration of one gait cycle is about 1.5 seconds obtained by actual multiple measurements, 1.5 seconds are taken as the length of a time window of a pressure sequence, 1 second is taken as the step length of a time sliding window to extract data under different human motion gaits, 12669 groups of effective samples are extracted in total, the shape and the size of each group of pressure data are 30 multiplied by 128 and contain left and right foot synchronous pressure data, and the 12669 groups of samples can respectively correspond to one of five human gaits of standing, sitting, walking on flat ground, going up stairs and going down stairs according to the human gaits when the group of samples are collected.
Step 2, preprocessing the motion gait data set, and specifically comprises the following substeps:
(a) firstly, separating left and right foot data in an acquired motion gait data matrix with the size of 12669 multiplied by 30 multiplied by 128, obtaining the sizes of the left and right foot data matrix after separation, wherein the sizes of the left and right foot data matrix are 12669 multiplied by 30 multiplied by 64, then adjusting the shape of the last one-dimensional data to 8 multiplied by 8, and finally obtaining the sizes of the left and right foot data, wherein the sizes of the left and right foot data are 12669 multiplied by 30 multiplied by 8.
(b) The last two dimensions of the left and right foot data are resampled to 32 × 32 by bilinear interpolation, and the left and right foot data size becomes 12669 × 30 × 32 × 32. The bilinear interpolation is to use 4 adjacent points to perform linear interpolation in one direction, then perform linear interpolation in the other direction, and the value in the area surrounded by the 4 points can be estimated by the comprehensive formula obtained by the two linear interpolations.
(c) And normalizing the last two dimensions of the two data matrixes to enable all the data on each channel to be in a distribution with the mean value of 0 and the variance of 1 on each channel, and specifically, subtracting the mean value of the data on each channel from all the values on each channel and dividing the mean value by the standard deviation of the data on each channel.
(d) The left and right foot data are concatenated into a size of 12669 × 60 × 32 × 32 as a model input by the method of concat, i.e., stacking in the direction of the channel.
(e) And finally, processing the finished data set according to the following steps of 4: a ratio of 1 randomly partitions the training set and the test set.
And 3, building an algorithm model based on the improved MobileNet-V3. The MobileNet-V3 network structure is divided into a Large version and a Small version, the Small version is aimed at, and the improved algorithm model comprises the following steps:
(a) residual error model
The invention replaces the inverted residual structure in the original model with an hourglass structure.
As can be seen from fig. 2, the hourglass structure and the inverse residual structure both include depth Convolution, which performs 3 × 3 Convolution on each channel of the input, and Pointwise Convolution, which performs 1 × 1 Convolution on the input.
The hourglass structure is different from the inverted residual structure in that the inverted residual block firstly performs point-by-point convolution on input to realize the operation of increasing dimensionality, and then extracts pressure image space characteristics through depth convolution and performs point-by-point convolution to realize the operation of reducing dimensionality. The hourglass structure still follows the previous network and uses the depth separation convolution, but the sequence of the depth convolution and the point-by-point convolution is reversed, the first step is to perform the depth convolution on the input, extract the spatial characteristics, perform two-step point-by-point convolution to realize the operation of firstly reducing the dimension and then increasing the dimension, and the last step outputs the result after the depth convolution.
(b) Activating a function
Compared with the original model, the method has the difference in the activation function that the original model uses h-swish and relu6 as the activation function and is completely replaced by the mish activation function, and the output layer uses softmax as the activation function.
The application of the hash activation function is such that when the input is negative the function is not completely truncated, it still allows a relatively small negative gradient to flow in, thus ensuring the flow of information. Since the hash function ensures the smoothness of each point, the gradient descending effect is better than relu 6. The formula of the hash activation function is as follows:
Mish(x)＝x*tanh(log(1+ex))
Figure BDA0003308575550000061
where x represents an input value, tanh represents a hyperbolic tangent function, and e represents a natural constant.
(c) Attention mechanism
The improvement reserves the attention mechanism of the original MobileNet-V3 network model and adds the attention mechanism into the hourglass structure, and the attention mechanism is introduced, namely a compressed excitation module (Squeeze-and-excitation module) is added into the network structure, and the compressed excitation module is applied before the last layer of the hourglass structure, namely the second point-by-point convolution is performed, then the SE operation is performed, and then the deep convolution is performed.
As can be seen from fig. 3, the compression and excitation module mainly comprises a compression operation and an excitation operation, the compression operation is global average pooling, and assuming that the size of the original input is W × H × C, the output is 1 × 1 × C; the excitation operation consists of two fully-connected layers, the first fully-connected layer has C × SERadio neurons, SERadio is a scaling parameter, the input shape is 1 × 1 × C, the output shape is 1 × 1 × C × SERadio, the second fully-connected layer has C neurons, the input shape is 1 × 1 × C × SERadio, the output shape is 1 × 1 × C, and finally the vector with the shape of 1 × 1 × C is subjected to scale operation with the original characteristic diagram.
Finally, the improved MobileNet-V3 network model is constructed based on the original network model of MobileNet-V3 by combining the hourglass structure and the mish activation function.
The frames and parameters of the layers in the improved MobileNet-V3 network model are as follows:
Figure BDA0003308575550000071
and 4, training and testing the built model by using the preprocessed data set.
The improved MobileNet-V3 network model is realized by adopting a python programming language based on a pyroch framework, meanwhile, the Batchsize of the training process is set to be 64, the epoch is set to be 500, the learning rate is set to be 0.01, an Adam optimizer is adopted to optimize the model, and finally, a training set is thrown into the model for training.
After the model training is finished, the test set is input into the model, and the human motion gait corresponding to a certain group of data samples can be judged by the model.
And 5, deploying the tested model at the mobile phone end, transmitting the left foot pressure data and the right foot pressure data acquired by the sensor acquisition module to the mobile phone end by using Bluetooth, inputting the data into the algorithm model by the mobile phone end, and receiving and displaying a model gait analysis result.
After the model test is finished, the algorithm model is implanted into an Android gait recognition application program, the Android gait recognition application program is mainly divided into two parts, one part is responsible for data acquisition and display, and the other part is responsible for data reading and analysis.
When the mobile phone end commands the sensor acquisition module to start acquiring data, the sensor acquisition module acquires pressure data at high frequency and stores the pressure data in the storage module, and uploads display data to the mobile phone end by using Bluetooth low frequency, and the mobile phone end preprocesses the acquired data and sends the preprocessed data to a display program so as to draw a real-time plantar pressure dynamic image on a user interface. The mobile phone end starts a sub thread to start timing when receiving the display data, and the data acquisition and display stage is finished when the timing is finished.
When data acquisition is stopped, the sensor acquisition module actively reads data in the storage module and uploads the data to the mobile phone end, the mobile phone end receives the data and stores the data in the local database, then the motion posture analysis program is called to input plantar pressure data into the algorithm model, and an analysis result of the model is received and displayed.
Comparative example
Different from the embodiment, the original MobileNet-V3 network model is adopted.
On a computer with the same configuration, the CPU was used to train and test the improved MobileNet-V3 network model and the original MobileNet-V3 network model using the same data set, and the performance of the two models on the test set is shown in the following table:
Figure BDA0003308575550000081
compared with the original MobileNet-V3 network model, the improved MobileNet-V3 network model has higher accuracy and higher detection speed, and simultaneously reduces the size of the model, so that the model is more suitable for being deployed in a mobile terminal. Therefore, the special cascade mode of plantar pressure information is utilized, and the hourglass structure applying the attention mechanism and the use of the mish activation function on the MobileNet-V3 network model are combined, so that the model ensures light weight and improves the performance of the model to a certain extent.

Claims (7)

1. A method for detecting human gait by using plantar pressure signals is characterized by comprising the following steps:
(1) the method comprises the steps that a flexible array pressure sensor is used for collecting plantar pressure information, and a movement gait data set is established;
(2) after operations such as bilinear interpolation, normalization, cascade connection and the like are carried out on the motion gait data set obtained in the step (1), the preprocessing process of the motion gait data set is completed;
(3) establishing an improved MobileNet-V3 algorithm model;
(4) sending the movement gait data set which is preprocessed in the step (2) into the model in the step (3) for training and testing;
(5) and deploying the tested model at the mobile phone end, and transmitting the pressure data of the left foot and the right foot acquired by the flexible array pressure sensor to the mobile phone end in real time through Bluetooth to predict the current gait.
2. The method of claim 1, wherein the method further comprises the step of using the plantar pressure signal to detect gait of the human body, wherein the method further comprises the following steps: in the step (1), the flexible array pressure sensor mainly comprises a plurality of pressure sensitive sensors which are arranged in a row-column array, the left foot and the right foot of a human body are respectively and uniformly distributed with the plurality of pressure sensitive sensors, the structure of each pressure sensitive sensor is a sandwich structure and comprises two layers of electrode materials and one layer of force sensitive material, and the layer of force sensitive material is sandwiched between the two layers of electrode materials.
3. The method of claim 1, wherein the method further comprises the step of using the plantar pressure signal to detect gait of the human body, wherein the method further comprises the following steps: in the flexible array pressure sensor, the pressure-sensitive sensors in each column direction are connected to the column one-more-selection switch and then output, and one end of each pressure-sensitive sensor in the same column direction is connected to the same input end of the column one-more-selection switch; the pressure-sensitive sensors in each row direction are connected to the row one-more-selection switch and then output, and the other ends of the pressure-sensitive sensors in the same row direction are connected together and then connected to the same input end of the row one-more-selection switch; the other end of each pressure-sensitive sensor is connected to the other end of the last pressure-sensitive sensor through a respective diode, and the other end of the last pressure-sensitive sensor is connected to one input end of the row one-out-of-row switch through a diode.
4. The method of claim 1, wherein the method further comprises the step of using the plantar pressure signal to detect gait of the human body, wherein the method further comprises the following steps: in the step (1), the data collected by the flexible array pressure sensors are extracted by respectively using the flexible array pressure sensors under five human body gaits of standing, sitting, walking on flat ground, going upstairs and going downstairs, taking 1.5 seconds as the length of a time window and taking 1 second as the step length of a time sliding window, and effective sample data is obtained to form a motion gait data set.
5. The method of claim 2, wherein the step of detecting gait of the human body using plantar pressure signals comprises: in the step (2), the sensor data respectively collected at the left and right feet of the human body are stacked and cascaded together in the channel direction.
6. The method of claim 1, wherein the method further comprises the step of using the plantar pressure signal to detect gait of the human body, wherein the method further comprises the following steps: in the step (3), the improved MobileNet-V3 algorithm model is obtained by replacing all inverted residual error structures in the MobileNet-V3 algorithm model with hourglass structures;
the hourglass structure is mainly formed by sequentially connecting a first layer of depth convolution layer DW, a first layer of point-by-point convolution layer PW, a second layer of point-by-point convolution layer PW and a second layer of depth convolution layer DW, wherein the input of the first layer of depth convolution layer DW is used as the input of the hourglass structure, and the input of the first layer of depth convolution layer DW and the output of the second layer of depth convolution layer DW are subjected to addition operation and then are used as the output of the hourglass structure;
simultaneously, all h-swish activation functions and relu6 activation functions of other neural network layers except an output layer in the MobileNet-V3 algorithm model are replaced by a hash function, and a softmax activation function is reserved in the output layer; and simultaneously, adding a compression excitation module between a second layer point-by-point convolution layer PW and a second layer depth convolution layer DW of each hourglass structure in the MobileNet-V3 algorithm model, wherein the compression excitation module mainly comprises a layer of global average pooling layer, two continuous layers of full-connection layers and a scale operation, the feature diagram output by the second layer point-by-point convolution layer PW is respectively input into the global average pooling layer and the scale operation of the compression excitation module, the output of the global average pooling layer is sequentially input into the scale operation after passing through the two continuous layers of full-connection layers, an sh miactivation function is respectively arranged behind each layer of full-connection layers, and the output of the final compression excitation module is obtained through the scale operation and is further input into the second layer depth convolution layer DW.
7. The method of claim 1, wherein the method further comprises the step of using the plantar pressure signal to detect gait of the human body, wherein the method further comprises the following steps: in the step (5), the model is implanted into a gait recognition application program, the gait recognition application program is mainly divided into two parts, one part is responsible for data acquisition and display, and the other part is responsible for data reading and analysis.
CN202111210035.4A 2021-10-18 2021-10-18 Method for detecting human gait by using plantar pressure signals Withdrawn CN113947119A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111210035.4A CN113947119A (en) 2021-10-18 2021-10-18 Method for detecting human gait by using plantar pressure signals

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111210035.4A CN113947119A (en) 2021-10-18 2021-10-18 Method for detecting human gait by using plantar pressure signals

Publications (1)

Publication Number Publication Date
CN113947119A true CN113947119A (en) 2022-01-18

Family

ID=79331000

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111210035.4A Withdrawn CN113947119A (en) 2021-10-18 2021-10-18 Method for detecting human gait by using plantar pressure signals

Country Status (1)

Country Link
CN (1) CN113947119A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115227238A (en) * 2022-08-04 2022-10-25 河北工业大学 Gait recognition system based on wearable strain sensor and construction method thereof
CN116434348A (en) * 2023-06-14 2023-07-14 武汉纺织大学 Human body action real-time identification method and system based on flexible strain sensor

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115227238A (en) * 2022-08-04 2022-10-25 河北工业大学 Gait recognition system based on wearable strain sensor and construction method thereof
CN116434348A (en) * 2023-06-14 2023-07-14 武汉纺织大学 Human body action real-time identification method and system based on flexible strain sensor
CN116434348B (en) * 2023-06-14 2023-09-01 武汉纺织大学 Human body action real-time identification method and system based on flexible strain sensor

Similar Documents

Publication Publication Date Title
CN113947119A (en) Method for detecting human gait by using plantar pressure signals
CN109902583B (en) Skeleton gesture recognition method based on bidirectional independent circulation neural network
CN114220035A (en) Rapid pest detection method based on improved YOLO V4
CN111178288B (en) Human body posture recognition method and device based on local error layer-by-layer training
CN110610158A (en) Human body posture identification method and system based on convolution and gated cyclic neural network
CN109726662A (en) Multi-class human posture recognition method based on convolution sum circulation combination neural net
CN111753683A (en) Human body posture identification method based on multi-expert convolutional neural network
CN111104902B (en) Hemiplegia gait classification method based on graph convolution network
CN109902623A (en) A kind of gait recognition method based on perception compression
CN102651072A (en) Classification method for three-dimensional human motion data
CN115966010A (en) Expression recognition method based on attention and multi-scale feature fusion
CN113408321A (en) Real-time target detection method and device for lightweight image and video data
CN112668486A (en) Method, device and carrier for identifying facial expressions of pre-activated residual depth separable convolutional network
CN115294563A (en) 3D point cloud analysis method and device based on Transformer and capable of enhancing local semantic learning ability
CN113095251A (en) Human body posture estimation method and system
CN114550047B (en) Behavior rate guided video behavior recognition method
CN116311349A (en) Human body key point detection method based on lightweight neural network
Özbay et al. 3D Human Activity Classification with 3D Zernike Moment Based Convolutional, LSTM-Deep Neural Networks.
CN116188435A (en) Medical image depth segmentation method based on fuzzy logic
CN112465054B (en) FCN-based multivariate time series data classification method
Wang et al. Design of static human posture recognition algorithm based on CNN
CN110840459B (en) Human body balance ability obtaining method and system, computer equipment and medium
CN114612758A (en) Target detection method based on deep grouping separable convolution
CN113011506A (en) Texture image classification method based on depth re-fractal spectrum network
CN108427967B (en) Real-time image clustering method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WW01 Invention patent application withdrawn after publication

Application publication date: 20220118

WW01 Invention patent application withdrawn after publication