CN115731602A - Human body activity recognition method, device, equipment and storage medium based on topological representation - Google Patents
Human body activity recognition method, device, equipment and storage medium based on topological representation Download PDFInfo
- Publication number
- CN115731602A CN115731602A CN202110975148.7A CN202110975148A CN115731602A CN 115731602 A CN115731602 A CN 115731602A CN 202110975148 A CN202110975148 A CN 202110975148A CN 115731602 A CN115731602 A CN 115731602A
- Authority
- CN
- China
- Prior art keywords
- data
- sensor
- topological
- sensors
- activity recognition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 230000037081 physical activity Effects 0.000 title claims abstract description 19
- 230000000694 effects Effects 0.000 claims abstract description 33
- 238000007781 pre-processing Methods 0.000 claims abstract description 19
- 238000000605 extraction Methods 0.000 claims abstract description 14
- 238000012549 training Methods 0.000 claims description 26
- 238000012360 testing method Methods 0.000 claims description 22
- 238000010586 diagram Methods 0.000 claims description 20
- 238000004590 computer program Methods 0.000 claims description 17
- 238000012545 processing Methods 0.000 claims description 16
- 230000004927 fusion Effects 0.000 claims description 6
- 238000012512 characterization method Methods 0.000 claims 2
- 238000007619 statistical method Methods 0.000 abstract description 3
- 230000006870 function Effects 0.000 description 5
- 230000001133 acceleration Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000004083 survival effect Effects 0.000 description 4
- 238000001914 filtration Methods 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000002688 persistence Effects 0.000 description 2
- 238000007637 random forest analysis Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 210000003423 ankle Anatomy 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000002459 sustained effect Effects 0.000 description 1
- 210000000707 wrist Anatomy 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/10—Pre-processing; Data cleansing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Medical Informatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Mathematical Physics (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Testing Or Calibration Of Command Recording Devices (AREA)
Abstract
The invention provides a human activity recognition method, a human activity recognition device, human activity recognition equipment and a storage medium based on topological representation, wherein the human activity recognition method comprises the following steps: acquiring data of a plurality of sensors, wherein the plurality of sensors are placed at different parts of a human body; preprocessing the data of each sensor to obtain the preprocessed data of each sensor; carrying out topological feature extraction on the preprocessed data of each sensor to obtain feature data of each sensor; classifying the characteristic data of each sensor by using the trained classifier to obtain a classification result corresponding to each sensor; and fusing the classification results corresponding to the plurality of sensors to obtain the category of the human body activity. According to the invention, the classification results of a plurality of sensors are fused, so that the identification accuracy is improved, and in addition, the topological characteristics of the sensor data are used as the input data of the classifier, so that the key information lost in the conventional statistical analysis can be obtained, and the identification accuracy is further improved.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a human body activity identification method, a human body activity identification device, human body activity identification equipment and a storage medium based on topological representation.
Background
Human Activity Recognition (HAR) is to define and classify the things it is doing by collecting and analyzing the motion data of people, and has wide application in health management, medical monitoring, human-computer interaction, safety, entertainment, etc., except that the data in the form of video images is adopted to train the Recognition model, the sensor data set is also widely applied to Human Activity Recognition. The sensor data sequences are collected by a smartphone or wearable sensor, and the collected sensor sequences are then analyzed and classified as known and well-defined movements or activities, such as sitting, running, walking up and down stairs, etc. In recent years, with the development of sensor devices, various sensors capable of accurately acquiring human behavior data are widely used on wearable devices such as wristbands, watches, mobile phones and the like, and therefore, human activity recognition research based on the wearable sensors also becomes a hotspot in behavior recognition. At present, human activity recognition technology based on wearable inertial sensors has been successful, but the classification accuracy of human activities by using wearable inertial sensors in practical applications is not high enough.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention provides a human body activity identification method, a human body activity identification device, human body activity identification equipment and a storage medium based on topological representation, and the identification accuracy can be effectively improved.
The specific technical scheme provided by the invention is as follows: a human activity recognition method based on topological representation comprises the following steps:
acquiring data of a plurality of sensors, wherein the plurality of sensors are placed at different parts of a human body;
preprocessing the data of each sensor to obtain the preprocessed data of each sensor;
carrying out topological feature extraction on the preprocessed data of each sensor to obtain feature data of each sensor;
classifying the characteristic data of each sensor by using a trained classifier to obtain a classification result corresponding to each sensor;
and fusing the classification results corresponding to the plurality of sensors to obtain the category of the human body activity.
Further, preprocessing the data of each sensor to obtain preprocessed data of each sensor, including:
denoising the data of each sensor to obtain the denoising data of each sensor;
normalizing the de-noising data of each sensor to obtain normalized data of each sensor;
and performing sliding window processing on the normalized data of each sensor to obtain a plurality of window data of each sensor, and taking the plurality of window data of each sensor as the preprocessing data of the sensor.
Further, performing topological feature extraction on the preprocessed data of each sensor to obtain feature data of each sensor, including:
respectively carrying out phase space reconstruction on the plurality of window data of each sensor to obtain a plurality of point cloud data of each sensor;
respectively carrying out continuous coherence on a plurality of point cloud data of each sensor to obtain a continuous coherence map of each point cloud data;
and extracting the topological features of each point cloud data according to the continuous concoction chart of each point cloud data, obtaining a plurality of topological features corresponding to each sensor, and taking the plurality of topological features corresponding to each sensor as the characteristic data of the sensor.
Further, classifying the feature data of each sensor by using the trained classifier, wherein the classifying comprises the following steps:
dividing a plurality of topological features corresponding to each sensor into training data and testing data;
training a classifier by using the training data to obtain a trained classifier;
and classifying the test data by using the trained classifier.
Further, each topological feature includes features of a plurality of channels, and the method for dividing the plurality of topological features corresponding to each sensor into training data and test data includes:
splicing the characteristics of the plurality of channels of each topological characteristic in the plurality of topological characteristics to obtain a plurality of spliced characteristics corresponding to each sensor;
and dividing the spliced characteristics corresponding to each sensor into training data and test data.
Further, the fusion of the classification results corresponding to the plurality of sensors includes: and fusing the classification results corresponding to the plurality of sensors by using a voting method.
Further, the voting method is soft voting.
The invention also provides a human activity recognition device based on topological representation, which comprises:
the system comprises an acquisition module, a display module and a control module, wherein the acquisition module is used for acquiring data of a plurality of sensors, and the sensors are placed at different parts of a human body;
the preprocessing module is used for preprocessing the data of each sensor to obtain the preprocessed data of each sensor;
the characteristic extraction module is used for extracting topological characteristics of the preprocessed data of each sensor to obtain characteristic data of each sensor;
the classification module is used for classifying the characteristic data of each sensor by using the trained classifier to obtain a classification result corresponding to each sensor;
and the fusion module is used for fusing the classification results corresponding to the sensors to obtain the category of the human body activity.
The invention also provides a device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the computer program to implement the human activity recognition method as described above.
The present invention also provides a computer readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the human activity recognition method as described above.
The human body activity recognition method provided by the invention has the advantages that the motion signals of different parts of a human body are collected by the sensors, the topological characteristics of the data of the sensors are extracted, the trained classifier is used for classifying the characteristic data of each sensor, finally, the classification results corresponding to the sensors are fused to obtain the category of the human body activity, the classification results of the sensors are fused to improve the recognition accuracy, in addition, the topological characteristics of the sensor data are used as the input data of the classifier to obtain the key information lost in the conventional statistical analysis, and the recognition accuracy is further improved.
Drawings
The technical solution and other advantages of the present invention will become apparent from the following detailed description of specific embodiments of the present invention, which is to be read in connection with the accompanying drawings.
Fig. 1 is a schematic diagram of a human activity recognition method based on topological features in an embodiment of the present application;
FIG. 2 is a schematic diagram of the preprocessing of data for each sensor in an embodiment of the present application;
FIG. 3 is a schematic diagram illustrating topology feature extraction performed on preprocessed data of each sensor in an embodiment of the present application;
FIG. 4 is a schematic diagram of point cloud data in an embodiment of the present application;
FIG. 5 is a schematic view of a bar code in an embodiment of the present application;
FIG. 6 is a schematic illustration of a sustained tone map in an embodiment of the present application;
FIG. 7 is a schematic diagram illustrating the classification of feature data of each sensor according to an embodiment of the present disclosure;
FIG. 8 is a schematic diagram of a human activity recognition device based on topological features in an embodiment of the present application;
fig. 9 is a schematic diagram of an apparatus in an embodiment of the present application.
Detailed Description
Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. This invention may, however, be embodied in many different forms and should not be construed as limited to the specific embodiments set forth herein. Rather, these embodiments are provided to explain the principles of the invention and its practical application to thereby enable others skilled in the art to understand the invention for various embodiments and with various modifications as are suited to the particular use contemplated. In the drawings, like reference numerals will be used to refer to like elements throughout.
Referring to fig. 1, a human activity recognition method based on topology representation provided in this embodiment includes the following steps:
s1, acquiring data of a plurality of sensors, wherein the plurality of sensors are placed at different parts of a human body;
s2, preprocessing the data of each sensor to obtain preprocessed data of each sensor;
s3, extracting topological features of the preprocessed data of each sensor to obtain feature data of each sensor;
s4, classifying the feature data of each sensor by using the trained classifier to obtain a classification result corresponding to each sensor;
and S5, fusing the classification results corresponding to the sensors to obtain the category of the human body activity.
In step S1, the plurality of sensors are wearable sensors, the types of the sensors may include acceleration sensors, rotation speed sensors, magnetic field direction sensors, and the like, for example, accelerometers, gyroscopes, and magnetometers, and the plurality of sensors are placed at different parts of the human body, and the plurality of sensors may detect the motion experienced by the different parts of the body, so as to capture the posture of the body better and improve the accuracy of recognition. The present embodiment takes as an example a plurality of sensors, which are respectively an accelerometer, a gyroscope and a magnetometer, wherein the accelerometer, the gyroscope and the magnetometer are respectively placed on the chest, the wrist and the ankle of the human body, the accelerometer is used for obtaining acceleration information, the gyroscope is used for obtaining rotation speed information, and the magnetometer is used for obtaining magnetic field direction information.
Referring to fig. 2, in step S2, preprocessing the data of each sensor, and obtaining the preprocessed data of each sensor specifically includes:
s21, denoising the data of each sensor to obtain denoising data of each sensor;
s22, normalizing the de-noising data of each sensor to obtain normalized data of each sensor;
and S23, performing sliding window processing on the normalized data of each sensor to obtain a plurality of window data of each sensor, and taking the plurality of window data of each sensor as the preprocessing data of the sensor.
The frequency of data collected by the sensor is generally 50Hz, a plurality of interference factors exist in the process of human body movement, and in order to avoid the influence of the interference factors on subsequent identification, the data collected by the sensor needs to be denoised, specifically, the data collected by the sensor is subjected to median filtering firstly, and then the data subjected to median filtering is filtered through a three-order low-pass Butterworth filter with the cut-off frequency of 20Hz, so that the noise is reduced as much as possible.
Since the data dimensions of the plurality of sensors are not uniform, it is necessary to uniform the data dimensions of the plurality of sensors, and therefore, in step S22, the data dimensions of the plurality of sensors are uniform by normalizing the denoised data of each sensor, specifically, the data of the plurality of sensors are normalized by the following formula:
wherein x is i Representing de-noised data of the ith sensor, i =1,2,3 i ' normalized data of the i-th sensor, x min Minimum, x, in de-noised data representing multiple sensors max Representing the maximum value in the de-noised data of the plurality of sensors, the de-noised data of the plurality of sensors is positioned at 0,1 after normalization processing]An interval.
In the field of machine learning, the accuracy of a machine learning model is affected by the number of samples and the correlation between the samples, and the accuracy of the machine learning model is higher when the number of samples is larger and the correlation between the samples is larger. Therefore, in order to improve the accuracy of the classifier, in step S23, the normalized data of each sensor is subjected to sliding window processing to obtain a plurality of window data corresponding to each sensor, the plurality of window data are partially overlapped, and the plurality of window data are used as samples of a subsequent classifier, so that the accuracy of the classifier can be effectively improved. Specifically, in the present embodiment, the window length of the sliding window on the time axis selected for the sliding window processing is 128, and multiple pieces of window data are obtained by moving the sliding window on the time axis, where an overlap ratio between two adjacent pieces of window data on the time axis is 50%.
Referring to fig. 3, in step S3, performing topological feature extraction on the preprocessed data of each sensor to obtain feature data of each sensor, that is, performing topological feature extraction on each window data in the multiple window data of each sensor, specifically:
s31, respectively carrying out phase space reconstruction on the plurality of window data of each sensor to obtain a plurality of point cloud data of each sensor;
s32, respectively carrying out continuous coherence on the plurality of point cloud data of each sensor to obtain a continuous coherence map of each point cloud data;
s33, extracting the topological features of each point cloud data according to the continuous concordance diagram of each point cloud data, obtaining a plurality of topological features corresponding to each sensor, and taking the plurality of topological features corresponding to each sensor as the feature data of the sensor.
Specifically, in step S31, the plurality of window data are one-dimensional time series, the phase space reconstruction aims to map the one-dimensional time series to the point cloud in the high-dimensional phase space, and the present embodiment maps the plurality of window data to the point cloud in the high-dimensional phase space by using the time delay embedding method, specifically, S is used here i ,i∈[1,T]Representing a one-dimensional time sequence corresponding to a plurality of window data, wherein T represents the number of the plurality of window data corresponding to each sensor, S i ＝x i (n),n＝[1,M]M represents the number of sample points included in each window data, and since the window length of the sliding window on the time axis selected in this embodiment is 128, each window data includes 128 sample points, that is, M =128, a plurality of window data are mapped to a point cloud in the high-dimensional phase space by the following formula:
k i (a)＝[x i (a),x i (a+τ),......,x i (a+(d-1)τ)],a＝1,2,......,N
wherein k is i (a) Representing any phase point in the point cloud corresponding to the ith window data, representing delay time, representing embedding dimensionality, representing the number of the phase points in the point cloud by d, representing the number of the phase points in the point cloud by N, and a + (d-1) tau is less than or equal to M, then, after all the phase points are obtained, forming the point cloud by all the phase points, and obtaining the point cloud data corresponding to the ith window data as H i ＝{k i (1),k i (2),......,k i (N) }, as shown in FIG. 4, which shows a schematic diagram of point cloud data.
The method comprises the steps of obtaining a plurality of point cloud data corresponding to each sensor through the method, and continuously and synchronously adjusting the plurality of point cloud data of each sensor, wherein the continuous and synchronously adjusting mainly comprises homology and persistence, the homology is used for measuring a specific structure of a simple manifold, the persistence is used for obtaining survival information of different simple manifolds, the survival time of the simple manifold refers to a time period from appearance to disappearance of the simple manifold, the time period with longer survival time is a useful characteristic, and the time period with shorter survival time is noise. In this embodiment, a vitorris-Rips complex (Vietor is-Rips complex) algorithm is used to construct a simple complex, wherein the existence of the simple complex is visualized through a barcode diagram, as shown in fig. 5, a horizontal axis in fig. 5 represents a filtering threshold, an ordinate represents a p-dimensional base number, h0 represents a one-dimensional base number, h1 represents a two-dimensional base number, and h3 represents a three-dimensional base number, and a continuous coherence map is obtained according to the barcode diagram, as shown in fig. 6, a horizontal axis in fig. 6 represents the time when a p-dimensional hole appears, and a vertical axis represents the time when a p-dimensional hole disappears, and finally, a topological feature of each point cloud data is extracted according to the continuous coherence map, and the pure complex always existing in the continuous coherence map is used as the topological feature of each point cloud data.
The topological features of each point cloud data in the plurality of point cloud data corresponding to each sensor can be obtained through the method, so that the plurality of topological features corresponding to each sensor are obtained, the trained classifier is used for classifying the feature data of each sensor, and here, a training set used for training the classifier can be obtained from an existing training set.
Referring to fig. 7, in this embodiment, preferably, a part of data is selected from a plurality of topological features corresponding to each sensor as a training set of a classifier, so that the classifier can better classify according to the topological features, specifically, step S4 includes:
s41, dividing a plurality of topological features corresponding to each sensor into training data and testing data;
s42, training the classifier by using the training data to obtain a trained classifier;
and S43, classifying the test data by using the trained classifier.
Before the classifier carries out classification, the classifier needs to be trained so as to improve the accuracy of the classifier. In this embodiment, a plurality of topological features corresponding to each sensor are divided into training data and test data, where 80% of the topological features are used as the training data, and the remaining 20% of the topological features are used as the test data, then the training data is used to train the classifier to obtain a trained classifier, and then the trained classifier is used to classify the test data to obtain a classification result corresponding to each sensor. Because the random forest classifier can effectively operate on a large data set and can process input samples with high-dimensional characteristics, dimension reduction is not needed, and the method is simple and efficient, the random forest classifier is adopted in the classifier in the embodiment.
The accelerometer is configured to obtain acceleration information of a human body in x, y, and z axes, the gyroscope is configured to obtain rotational speed information of the human body in x, y, and z axes, and the magnetometer is configured to obtain magnetic field direction information of the human body in x, y, and z axes, so that data collected by each sensor has multiple channels, that is, data collected by each sensor includes 3 channels, and thus, each of multiple topological features corresponding to each sensor also includes multiple channels, and in order to fuse data of the multiple channels, step S41 in this embodiment is specifically:
s411, splicing the characteristics of the plurality of channels of each topological characteristic in the plurality of topological characteristics to obtain a plurality of spliced characteristics corresponding to each sensor;
s412, dividing the plurality of spliced features corresponding to each sensor into training data and testing data.
After the classification result corresponding to each sensor is obtained, the classification results of the plurality of sensors are fused by using a voting method, preferably, the voting method is soft voting. The following describes the fusion process in detail, and the results of classifying the test data by using the trained classifier are shown in the following table:
table classification results of a plurality of sensors
After the test data are classified by the accelerometer, the A category accounts for 97%, and the B category accounts for 3%; after the gyroscope classifies the test data, the A category accounts for 92%, and the B category accounts for 8%; the magnetometer classifies the test data, the A category accounts for 94%, the B category accounts for 6%, soft voting is carried out on the classification results of the three sensors, namely the classification results of the three sensors are averaged to obtain the A category which accounts for 94%, the B category which accounts for 6%, so that the category of the human body activity is obtained as the A category, wherein the A/B category comprises sitting, running, walking, going up and down stairs and the like.
The human body activity recognition method provided by the embodiment has the following beneficial effects:
1. the classification results of the sensors are fused, so that the identification accuracy is improved;
2. the key information lost in the conventional statistical analysis can be obtained by taking the topological characteristics of the sensor data as the sample data of the classifier, so that the identification accuracy is further improved, and the robustness of the human body identification method is improved;
3. a part of data is selected from a plurality of topological features corresponding to each sensor and is used as a training set of the classifier, so that the accuracy of identification is further improved;
4. after the multiple topological features of each sensor are obtained, the features of the multiple channels of each topological feature are spliced, and then the spliced features are input into the classifier, so that the features of the multiple channels are fused together, and the accuracy of identification is further improved.
Referring to fig. 8, the present embodiment further provides a human activity recognition apparatus based on topology representation, where the human activity recognition apparatus includes an obtaining module 1, a preprocessing module 2, a feature extraction module 3, a classification module 4, and a fusion module 5.
The acquisition module 1 is used for acquiring data of a plurality of sensors, wherein the plurality of sensors are placed at different parts of a human body. The preprocessing module 2 is used for preprocessing the data of each sensor to obtain the preprocessed data of each sensor. The feature extraction module 3 is configured to perform topological feature extraction on the preprocessed data of each sensor to obtain feature data of each sensor. The classification module 4 is configured to classify the feature data of each sensor by using the trained classifier, and obtain a classification result corresponding to each sensor. The fusion module 5 is used for fusing the classification results corresponding to the plurality of sensors to obtain the category of the human body activity.
The preprocessing module 2 is specifically configured to perform denoising processing on the data of each sensor to obtain denoised data of each sensor, perform normalization processing on the denoised data of each sensor to obtain normalized data of each sensor, perform sliding window processing on the normalized data of each sensor to obtain multiple window data of each sensor, and use the multiple window data of each sensor as the preprocessed data of the sensor.
The feature extraction module 3 is specifically configured to perform phase space reconstruction on the plurality of window data of each sensor respectively to obtain a plurality of point cloud data of each sensor, perform continuous coherence for the plurality of point cloud data of each sensor respectively to obtain a continuous coherence map of each point cloud data, extract a topological feature of each point cloud data according to the continuous coherence map of each point cloud data to obtain a plurality of topological features corresponding to each sensor, and use the plurality of topological features corresponding to each sensor as the feature data of the sensor.
The classification module 4 is specifically configured to classify the plurality of topological features corresponding to each sensor into training data and test data, train a classifier using the training data, and classify the test data using the trained classifier.
The accelerometer is used for obtaining acceleration information of a human body in x, y and z axes, the gyroscope is used for obtaining rotation speed information of the human body in x, y and z axes, the magnetometer is used for obtaining magnetic field direction information of the human body in x, y and z axes, therefore, as data collected by each sensor has a plurality of channels, that is, data collected by each sensor includes 3 channels, each topological feature of a plurality of topological features corresponding to each sensor also includes a plurality of channels, in order to fuse data of the plurality of channels, the classification module 4 is used for classifying the plurality of topological features corresponding to each sensor into training data and test data, specifically: the method comprises the steps of firstly splicing the characteristics of a plurality of channels of each topological characteristic in a plurality of topological characteristics to obtain a plurality of spliced characteristics corresponding to each sensor, and then dividing the plurality of spliced characteristics corresponding to each sensor into training data and testing data.
The fusion module 5 specifically fuses the classification results corresponding to the plurality of sensors by using a voting method, and preferably, the voting method is soft voting.
Referring to fig. 9, the present embodiment provides an apparatus, which includes a memory 100, a processor 200, and a network interface 202, where the memory 100 stores a computer program, and the processor 200 executes the computer program to implement the human activity recognition method in the present embodiment.
The Memory 100 may include a Random Access Memory (RAM) and may also include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory.
The processor 200 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the target detection method in this embodiment may be implemented by integrated logic circuits of hardware in the processor 200 or instructions in the form of software. The Processor 200 may also be a general-purpose Processor including a Central Processing Unit (CPU), a Network Processor (NP), etc., and may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component.
The memory 100 is used for storing a computer program, and the processor 200 executes the computer program to implement the human activity recognition method in the present embodiment after receiving the execution instruction.
The embodiment also provides a computer storage medium, a computer program is stored in the computer storage medium, and the processor 200 is configured to read and execute the computer program stored in the computer storage medium 201 to implement the human activity recognition method in the embodiment.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer storage medium or transmitted from one computer storage medium to another computer storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer storage media may be any available media that can be accessed by a computer or a data storage device, such as a server, data center, etc., that includes an integration of one or more available media. The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a Solid State Disk (SSD)), among others.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus, and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing is directed to embodiments of the present application and it is noted that numerous modifications and adaptations may be made by those skilled in the art without departing from the principles of the present application and are intended to be within the scope of the present application.
Claims (10)
1. A human activity recognition method based on topological characterization is characterized by comprising the following steps:
acquiring data of a plurality of sensors, wherein the plurality of sensors are placed at different parts of a human body;
preprocessing the data of each sensor to obtain the preprocessed data of each sensor;
carrying out topological feature extraction on the preprocessed data of each sensor to obtain feature data of each sensor;
classifying the characteristic data of each sensor by using the trained classifier to obtain a classification result corresponding to each sensor;
and fusing the classification results corresponding to the plurality of sensors to obtain the category of the human body activity.
2. The human activity recognition method of claim 1, wherein preprocessing the data of each sensor to obtain preprocessed data of each sensor comprises:
denoising the data of each sensor to obtain the denoising data of each sensor;
normalizing the de-noising data of each sensor to obtain normalized data of each sensor;
and performing sliding window processing on the normalized data of each sensor to obtain a plurality of window data of each sensor, and taking the plurality of window data of each sensor as the preprocessing data of the sensor.
3. The human activity recognition method of claim 2, wherein the performing topological feature extraction on the preprocessed data of each sensor to obtain feature data of each sensor comprises:
respectively carrying out phase space reconstruction on the window data of each sensor to obtain point cloud data of each sensor;
respectively carrying out continuous coherence on a plurality of point cloud data of each sensor to obtain a continuous coherence map of each point cloud data;
and extracting the topological features of each point cloud data according to the continuous concordance diagram of each point cloud data, obtaining a plurality of topological features corresponding to each sensor, and taking the plurality of topological features corresponding to each sensor as the feature data of the sensor.
4. The human activity recognition method of claim 3, wherein the classifying the feature data of each sensor by using the trained classifier comprises:
dividing a plurality of topological features corresponding to each sensor into training data and testing data;
training a classifier by using the training data to obtain a trained classifier;
and classifying the test data by using the trained classifier.
5. The human activity recognition method of claim 1, wherein each topological feature comprises features of a plurality of channels, and the classifying the plurality of topological features corresponding to each sensor into training data and test data comprises:
splicing the characteristics of the plurality of channels of each topological characteristic in the plurality of topological characteristics to obtain a plurality of spliced characteristics corresponding to each sensor;
and dividing the spliced characteristics corresponding to each sensor into training data and test data.
6. The human activity recognition method according to claim 1, wherein fusing the classification results corresponding to the plurality of sensors includes: and fusing the classification results corresponding to the plurality of sensors by using a voting method.
7. The human activity recognition method according to claim 6, wherein the voting method is soft voting.
8. A human activity recognition device based on topological characterization, the human activity recognition device comprising:
the system comprises an acquisition module, a display module and a control module, wherein the acquisition module is used for acquiring data of a plurality of sensors, and the sensors are placed at different parts of a human body;
the preprocessing module is used for preprocessing the data of each sensor to obtain the preprocessed data of each sensor;
the characteristic extraction module is used for extracting topological characteristics of the preprocessed data of each sensor to obtain characteristic data of each sensor;
the classification module is used for classifying the characteristic data of each sensor by using the trained classifier to obtain a classification result corresponding to each sensor;
and the fusion module is used for fusing the classification results corresponding to the sensors to obtain the category of the human body activity.
9. An apparatus comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the human activity recognition method as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions, when executed by a processor, implement the human activity recognition method according to any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110975148.7A CN115731602A (en) | 2021-08-24 | 2021-08-24 | Human body activity recognition method, device, equipment and storage medium based on topological representation |
| PCT/CN2021/137627 WO2023024334A1 (en) | 2021-08-24 | 2021-12-13 | Human activity recognition method and apparatus based on topological representation, and device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110975148.7A CN115731602A (en) | 2021-08-24 | 2021-08-24 | Human body activity recognition method, device, equipment and storage medium based on topological representation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115731602A true CN115731602A (en) | 2023-03-03 |
Family
ID=85289595
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110975148.7A Pending CN115731602A (en) | 2021-08-24 | 2021-08-24 | Human body activity recognition method, device, equipment and storage medium based on topological representation |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN115731602A (en) |
| WO (1) | WO2023024334A1 (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105678222A (en) * | 2015-12-29 | 2016-06-15 | 浙江大学 | Human behavior identification method based on mobile equipment |
| CN107609501A (en) * | 2017-09-05 | 2018-01-19 | 东软集团股份有限公司 | The close action identification method of human body and device, storage medium, electronic equipment |
| CN109086698A (en) * | 2018-07-20 | 2018-12-25 | 大连理工大学 | A kind of human motion recognition method based on Fusion |
| US20190231231A1 (en) * | 2016-10-04 | 2019-08-01 | The Johns Hopkins University | Measuring patient mobility in the icu using a novel non-invasive sensor |
| CN110659677A (en) * | 2019-09-10 | 2020-01-07 | 电子科技大学 | Human body falling detection method based on movable sensor combination equipment |
| CN111428690A (en) * | 2020-04-21 | 2020-07-17 | 桂林电子科技大学 | Identity authentication method based on gait signal topology analysis |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019010435A1 (en) * | 2017-07-06 | 2019-01-10 | Icuemotion Llc | Systems and methods for data-driven movement skill training |
| US11455383B2 (en) * | 2019-04-30 | 2022-09-27 | TruU, Inc. | Supervised and unsupervised techniques for motion classification |
-
2021
- 2021-08-24 CN CN202110975148.7A patent/CN115731602A/en active Pending
- 2021-12-13 WO PCT/CN2021/137627 patent/WO2023024334A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105678222A (en) * | 2015-12-29 | 2016-06-15 | 浙江大学 | Human behavior identification method based on mobile equipment |
| US20190231231A1 (en) * | 2016-10-04 | 2019-08-01 | The Johns Hopkins University | Measuring patient mobility in the icu using a novel non-invasive sensor |
| CN107609501A (en) * | 2017-09-05 | 2018-01-19 | 东软集团股份有限公司 | The close action identification method of human body and device, storage medium, electronic equipment |
| CN109086698A (en) * | 2018-07-20 | 2018-12-25 | 大连理工大学 | A kind of human motion recognition method based on Fusion |
| CN110659677A (en) * | 2019-09-10 | 2020-01-07 | 电子科技大学 | Human body falling detection method based on movable sensor combination equipment |
| CN111428690A (en) * | 2020-04-21 | 2020-07-17 | 桂林电子科技大学 | Identity authentication method based on gait signal topology analysis |
Non-Patent Citations (3)
| Title |
|---|
| ALIREZA DIRAFZOON 等: "Action classification from motion capture data using topological data analysis", 《2016 IEEE GLOBAL CONFERENCE ON SIGNAL AND INFORMATION PROCESSING (GLOBALSIP)》, pages 1260 - 1264 * |
| 朱国刚 等: "基于Kinect传感器骨骼信息的人体动作识别", 《计算机仿真》, vol. 31, no. 12, pages 329 - 333 * |
| 陈鹏展: "《个体行为的机器识别与决策协同》", 31 July 2018, 知识产权出版社, pages: 28 - 29 * |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2023024334A1 (en) | 2023-03-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ha et al. | Multi-modal convolutional neural networks for activity recognition | |
| Dawar et al. | Action detection and recognition in continuous action streams by deep learning-based sensing fusion | |
| Wang et al. | Human fall detection in surveillance video based on PCANet | |
| CN106846729B (en) | Tumble detection method and system based on convolutional neural network | |
| CN106228200B (en) | Action identification method independent of action information acquisition equipment | |
| US8935195B2 (en) | Method of identification and devices thereof | |
| CN109886068B (en) | Motion data-based action behavior identification method | |
| Ahmad et al. | Inertial sensor data to image encoding for human action recognition | |
| CN110991278A (en) | Human body action recognition method and device in video of computer vision system | |
| CN109902550A (en) | The recognition methods of pedestrian's attribute and device | |
| CN107609501A (en) | The close action identification method of human body and device, storage medium, electronic equipment | |
| Sideridis et al. | Gesturekeeper: Gesture recognition for controlling devices in iot environments | |
| Hafeez et al. | Multi-Sensor-Based Action Monitoring and Recognition via Hybrid Descriptors and Logistic Regression | |
| CN111428690B (en) | Identity authentication method based on gait signal topology analysis | |
| Guan et al. | A video-based fall detection network by spatio-temporal joint-point model on edge devices | |
| Harrou et al. | Statistical control chart and neural network classification for improving human fall detection | |
| Ramzan et al. | Automatic Unusual Activities Recognition Using Deep Learning in Academia. | |
| CN115731602A (en) | Human body activity recognition method, device, equipment and storage medium based on topological representation | |
| CN115886794A (en) | Multi-classifier selection method, frozen gait detection system, device and storage medium | |
| CN111861275B (en) | Household work mode identification method and device | |
| Janardhanan et al. | Vision based Human Activity Recognition using Deep Neural Network Framework | |
| Othmen et al. | A Sparse Representation Classification for Noise Robust Wrist-based Fall Detection. | |
| CN112084889A (en) | Image behavior recognition method and device, computing equipment and storage medium | |
| CN111950575A (en) | Device for fall detection and method thereof | |
| Singh et al. | Gait Recognition Using Activities of Daily Livings and Ensemble Learning Models |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |