CN106228200B - Action identification method independent of action information acquisition equipment - Google Patents

Abstract

The invention relates to an action recognition method that does not depend on motion information collection equipment, and is suitable for different motion information collection equipment; the method includes two stages, namely a model training stage and a model prediction stage, wherein the model training stage refers to A mapping relationship between motion information and actions is established, and the model prediction stage refers to calculating the corresponding action category according to the collected motion information; the present invention deploys an action recognition method on different motion information collection terminal devices. The problem of compatibility , specifically considering the influence of different sampling frequencies, different wearing positions, different accuracy and sensitivity of sensors on the action recognition results; the present invention can be applied to embedded inertial sensor units such as smart phones, tablet computers, wristbands, wrist watches, etc., such as acceleration terminal equipment such as a gyroscope, gyroscope or magnetometer.

Description

A motion recognition method that does not depend on motion information collection equipment

technical field

The invention relates to an action recognition method that does not depend on motion information collection equipment, is suitable for different motion information collection equipment, and can be applied to embedded inertial sensor units such as accelerometers, smart phones, tablet computers, wristbands, wrist watches, etc. Terminal devices such as gyroscopes or magnetometers.

Background technique

In recent years, with the development of MEMS technology, more and more terminal devices (such as smartphones, tablet computers, wristbands, wrist watches, etc.) are embedded with various types of sensors (such as accelerometers, gyroscopes, magnetometers, etc.). , infrared camera, etc.). Correspondingly, various applications around these sensors are emerging, such as in the field of health care, including: body motion recognition, fall detection and alarm, heart rate monitoring, abnormal gait analysis and quantitative evaluation, etc.

However, most of the various applications on the market are only applicable to specific models (brands) of terminal equipment, and cannot be effectively compatible with all types of terminal equipment, essentially because the information collected from different terminal equipment is different. The reasons generally include the following aspects:

(1) The sensor models embedded in different terminal devices are also different, and the sensitivity, accuracy, detection limit and other indicators of the sensor are also different;

(2) The sampling frequencies of sensors set in different terminal devices are also different;

(3) If the terminal device runs other applications while collecting sensor information, the sampling frequency of the sensor will fluctuate;

(4) After the terminal equipment falls and other abnormal conditions, the embedded sensor will drift.

Specific to the application of action recognition, the existing literature has clearly pointed out that when the existing action recognition algorithms are deployed on different terminal devices, the accuracy of action recognition is reduced to some extent. Therefore, how to design a motion recognition method that can be applied to various types of terminal equipment independently of information collection equipment is an urgent problem to be solved at present.

SUMMARY OF THE INVENTION

Aiming at the common problem of the existing action recognition methods relying on information collection equipment, the present invention proposes an action recognition method that does not depend on motion information collection equipment. The method first constructs a standard sampler, normalizes motion information of different sampling rates collected from different terminal devices, and unifies them to a standard sampling frequency, and then constructs a clusterer to distinguish different terminal devices. Wearing position, and finally construct an integrated action recognition method framework composed of multiple weak classifiers to eliminate the influence of different indicators such as accuracy and sensitivity of built-in sensors in different terminal devices.

In order to solve the above-mentioned technical problems, the technical scheme adopted in the present invention is:

An action recognition method that does not depend on motion information collection equipment, including two stages, namely a model training stage and a model prediction stage, the model training stage refers to establishing a mapping relationship between motion information and actions, and the model The prediction stage refers to calculating the corresponding action category according to the collected motion information.

Preferably, the model training phase specifically includes the following steps:

1) The step of motion information collection, wear different motion information collection devices on different positions of the human body, and then record the motion information of the human body when performing different actions;

2) The step of normalizing the sampling frequency, using the down-sampling method to normalize the frequency of the original motion information from different motion information collection devices;

3) The steps of feature extraction and feature selection, use time domain method, frequency domain method or nonlinear analysis method to extract the original motion information, and use mutual information correlation method, genetic algorithm, sparse optimization method or principal component analysis method, etc. The extracted features are screened to select the features that best characterize the motion information;

4) The step of identifying and clustering the wearing position of the terminal equipment, using the features extracted and screened in the above step 3) to construct a terminal equipment wearing position identification clusterer by using the tutored learning method or the unsupervised learning method;

5) The steps of the random forest action recognition model, for each wearing position, use the random forest method to establish a corresponding action recognition model.

Preferably, in step 1), the sports information collection device includes but is not limited to a smartphone, a tablet computer, a wrist watch, and a wristband; the wearing position of the sports information collection device includes but is not limited to the wrist, forearm, upper arm, waist, thigh, calf ; Actions performed by the human body include but are not limited to: sitting, lying down, standing, walking slowly, going up stairs, going down stairs, and running; the collected motion information includes but not limited to the acceleration, angular velocity, magnetic field of the three-dimensional space X, Y, and Z axes strength.

Preferably, the frequency normalization in the step 2) is to resample the motion information whose sampling frequency is higher than 25 Hz at a lower sampling frequency, so that the new sampling frequency of the motion information is 25 Hz.

Preferably, in step 3), the features extracted by the time domain method include but are not limited to motion amplitude, angle, and speed; the features extracted by the frequency domain method include but are not limited to motion frequency and energy; the nonlinear analysis method is used The extracted features include but are not limited to approximate entropy and multi-scale entropy.

Preferably, the tutored learning methods in the step 4) include but are not limited to neural networks, support vectors, and decision trees.

Preferably, the unsupervised learning method in the step 4) includes but is not limited to self-organizing mapping neural network and distance discrimination method.

Preferably, the clusterer in the step 4) refers to firstly identifying the wearing position of the terminal device between motion recognition, and then establishing motion recognition models for different wearing positions respectively.

Preferably, the model prediction stage is specifically: first wear the motion information collection device on a certain part of the human body, then collect the motion information of the human body in the process of completing the action to be recognized, and then pass the original information through the standard sampler, Feature extraction and feature selection, wearing position clusterer, action recognition model and other modules, and finally output the final action recognition result.

Compared with the prior art, the present invention has the following beneficial effects:

The method proposed in the present invention focuses on the compatibility of an action recognition method deployed on different motion information collection terminal devices (including but not limited to: smart phones, tablet computers, wristwatches, wristbands, etc.), and specifically considers different sampling The influence of factors such as frequency, different wearing positions, different accuracy and sensitivity of sensors on the results of motion recognition; this method has the advantages of strong compatibility and high accuracy, which can greatly improve the applicability of motion recognition technology in a wide range of specific applications.

Description of drawings

1 is a system block diagram of the present invention;

Figure 2 is a sampling frequency table of the built-in acceleration sensor of a typical motion information acquisition device;

Fig. 3 is the same action acceleration signal collected by different terminal equipment;

Fig. 4 is the different action acceleration signals collected by the same terminal equipment;

Fig. 5 is a result table of terminal equipment wearing position identification results;

Figure 6 is a comparison table of the accuracy of action recognition.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

As shown in FIG. 1 , an action recognition method that does not depend on a motion information collection device, the action recognition method belongs to a tutored learning method, and includes two stages of model establishment and model prediction.

The model building stage mainly includes the following steps:

(1) Movement information collection. Wear different sports information collection devices (such as smart phones, tablet computers, wristwatches, bracelets, etc.) on different positions of the human body (such as wrist, forearm, upper arm, waist, thigh, calf, etc.), and then record the movement of the human body in Motion information when performing different actions (such as: sitting, lying down, standing, walking slowly, going up stairs, going down stairs, running, etc.) Wait).

(2) Standardize the sampling frequency. For different terminal devices, the sampling frequency of the built-in sensors is also different. Figure 2 lists the maximum sampling frequencies of acceleration sensors supported by several typical terminal devices. It can be seen that from 25 to 200 Hz, there are great differences between different models. From the Nyquist-Shannon sampling theorem, it can be known that motion information collected with different sampling frequencies contains different information components. Therefore, it is necessary to standardize the sampling frequency of information from different terminal devices first. The commonly used methods include interpolation method (up-sampling) and down-sampling method, but considering that the interpolation method will artificially introduce new errors, and for motion recognition applications, the human body usually does not generate motion information greater than 10Hz. Therefore, in the present invention, the down-sampling method is adopted, and the sampling frequency is unified to 25 Hz, that is, for the terminal equipment whose sampling frequency is higher than 25 Hz, the collected motion information needs to be re-sampled.

(3) Feature extraction and feature selection. Since the motion information collected during an effective action cycle of the human body usually contains many data points, it is difficult to directly analyze these raw data. Therefore, it is necessary to perform feature extraction on the original information. Commonly used feature extraction methods include but are not limited to the following methods: time domain method (movement amplitude, angle, speed, etc.), frequency domain method (movement frequency, energy, etc.), nonlinear analysis method (Approximate entropy, multiscale entropy, etc.). At the same time, since many types of sensors (accelerometers, gyroscopes, magnetometers, etc.) are usually built in many terminal devices, and they are multi-axis (two-axis or three-axis) sensors, the feature dimensions that can be extracted are usually high. . Therefore, in practical applications, if it is impossible to determine which features can best characterize each action, feature selection (feature reduction) is often required. Commonly used feature selection methods include but are not limited to the following methods: mutual information correlation method, genetic algorithm, sparse optimization method, principal component analysis method, etc.

(4) Identification and clustering of terminal equipment wearing position. Most of the traditional action recognition methods are only aimed at the situation when the terminal device is worn in the same position, so when the terminal device is worn in other positions, the accuracy of the action recognition will be greatly reduced. Therefore, in order to make the action recognition method compatible with different wearing positions, the present invention constructs a clusterer, which first identifies the wearing position of the terminal device between action recognition, and then targets different wearing positions (wrist, forearm, upper arm, waist, thigh, calf, etc.) to establish action recognition models respectively. Commonly used clusterer construction methods include but are not limited to the following two types of methods: tutored learning methods (neural networks, support vector machines, decision trees, etc.) and unsupervised learning methods (self-organizing mapping neural networks, distance discrimination methods, etc.).

(5) Random forest action recognition model. For each wearing position, a corresponding action recognition model is established. In order to eliminate the influence of different performances such as built-in sensors in different terminal devices, the present invention constructs a random forest action recognition model integrating multiple weak classifiers.

In the model prediction stage, the motion information collection device is firstly worn on a certain part of the human body, and then the motion information of the human body in the process of completing the action to be recognized is collected, and then the original information is sequentially passed through the standard sampler, feature extraction and feature selection, Wear position clusterer, action recognition model and other modules, and finally output the final action recognition result.

The present invention will be further analyzed below through examples in conjunction with FIG. 3 to FIG. 6 .

This embodiment includes 3 different motion information collection devices: Samsung Galaxy Gear, HTCDesire, and Xsens inertial sensor units. The maximum sampling frequencies of the corresponding built-in acceleration sensors are 100Hz, 50Hz, and 200Hz, respectively; this embodiment includes 5 different actions : Sitting, standing, lying down, going up the stairs, going down the stairs; at the same time, the wearing positions of the above three kinds of motion information collection devices are also different, Samsung Galaxy Gear is worn on the wrist, HTC Desire is worn on the upper arm, Xsens inertial sensor unit Wear on the back waist.

First, let the subjects wear each device in the corresponding position, and then complete the above 5 different actions in turn, repeating each action 10 times for each device. Part of the motion information recorded during the experiment is shown in Figures 3 and 4. It can be seen from the figures that the motion information corresponding to the same action collected by different devices when worn in different positions is quite different. There are also obvious differences in different motion information collected by the device.

Secondly, the collected motion information is resampled by using the down-sampling method, so that the frequency of collected information corresponding to all terminal devices is 25Hz.

Then, the feature corresponding to each motion information is extracted by using the time domain method, specifically, including: motion amplitude, speed and angle in the directions of X, Y and Z in the three-dimensional space.

Next, use the support vector machine to construct a terminal equipment wearing position recognition clusterer. In this embodiment, 50 motion signals are collected at each wearing position, of which 40 samples are randomly selected for training and the remaining 10 samples are used for testing. That is, for the entire data set, there are 150 samples in total, of which the training set contains 120 samples and the test set contains 30 samples. The identification results corresponding to the test set are shown in Figure 5. It can be seen that the position worn by the terminal device can be better identified through the constructed identification clusterer.

Finally, for each wearing position, a random forest method is used to establish an action recognition model. Each random forest contains 50 to 100 decision trees, and the final recognition results are summarized by voting method. The comparison between the recognition results of this embodiment and the recognition accuracy obtained by the motion recognition method established only with the data of a single motion information collection device is shown in Figure 6. It can be seen that if only the data collected from a single motion information equipment is used Build an action recognition model that is only applicable to the same terminal device. If the model is applied to other terminal devices, the accuracy of action recognition will be significantly reduced. On the contrary, the action recognition model established by the method of the present invention can be well compatible with different terminal devices. The reason is that the method of the present invention integrates sensor information from all terminal devices in the modeling process, and adds standard samplers, terminal device wearing position recognition clustering, random Modules such as forest multi-weak classifier integration can effectively eliminate the effects of different sampling frequencies of different terminal devices, and different built-in sensor accuracy and sensitivity.

The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to the above-mentioned embodiments, and within the scope of knowledge possessed by those of ordinary skill in the art, various aspects can also be made without departing from the purpose of the present invention. Various changes should be included within the protection scope of the present invention.

Claims (6)

1. A motion recognition method independent of motion information acquisition equipment is characterized by comprising the following steps: the method comprises two stages, namely a model training stage and a model prediction stage, wherein the model training stage is used for establishing a mapping relation between motion information and motions, and the model prediction stage is used for calculating corresponding motion categories according to the collected motion information;

the model training phase specifically comprises the following steps:

1) acquiring motion information, namely wearing different motion information acquisition equipment at different positions of a human body, and then recording the motion information of the human body when different actions are executed;

2) the step of sampling frequency standardization, namely, carrying out frequency normalization on original motion information from different motion information acquisition equipment by using a down-sampling method; the frequency normalization is to perform sampling frequency reduction resampling on the motion information with the sampling frequency higher than 25Hz so that the sampling frequency of the new motion information is 25 Hz;

3) extracting the characteristics of the original motion information by a time domain method, a frequency domain method or a nonlinear analysis method, and screening the extracted characteristics by a mutual information correlation method, a genetic algorithm, a sparse optimization method or a principal component analysis method, so as to select the characteristics which can represent the characteristics of the motion information most;

4) identifying and clustering the wearing positions of the terminal equipment, namely constructing a terminal equipment wearing position identifying and clustering device by using a method of leading a teacher or a method of no leading a teacher according to the characteristics extracted and screened in the step 3); the clustering device is used for firstly identifying the wearing position of the terminal equipment during action identification and then respectively establishing action identification models aiming at different wearing positions;

5) and a step of random forest action recognition model, namely establishing a corresponding action recognition model by using a random forest method aiming at each wearing position.

2. The motion recognition method independent of a motion information collection device according to claim 1, characterized in that: in the step 1), the motion information acquisition equipment comprises a smart phone, a tablet personal computer, a wristwatch and a bracelet; the wearing position of the motion information acquisition equipment comprises a wrist, a forearm, an upper arm, a waist, a thigh and a shank; the actions performed by the human body include: sitting still, lying, standing, walking slowly, going upstairs, going downstairs, running; the acquired motion information includes acceleration, angular velocity, magnetic field strength of the three-dimensional space X, Y, Z axis.

3. The motion recognition method independent of a motion information collection device according to claim 1, characterized in that: in the step 3), the characteristics extracted by using the time domain method comprise motion amplitude, angle and speed; the features extracted by the frequency domain method comprise motion frequency and energy; the features extracted by the nonlinear analysis method comprise approximate entropy and multi-scale entropy.

4. The motion recognition method independent of a motion information collection device according to claim 1, characterized in that: the instructor learning method in the step 4) comprises a neural network, a support vector and a decision tree.

5. The motion recognition method independent of a motion information collection device according to claim 1, characterized in that: the instructor-free learning method in the step 4) comprises a self-organizing mapping neural network and a distance discrimination method.

6. The motion recognition method independent of motion information collection equipment according to claim 1, wherein the model prediction stage is specifically: the motion information recognition method comprises the steps of firstly wearing motion information acquisition equipment on a certain part of a human body, then acquiring motion information of the human body in the process of finishing a motion to be recognized, then sequentially passing the motion information through a standard sampler, feature extraction and feature selection, a wearing position clustering device and a motion recognition model, and finally outputting a final motion recognition result.

Images