CN112163510B - Human body action classification recognition method based on multi-observation variable HMM model - Google Patents

Abstract

The invention belongs to the technical field of human body action classification and recognition, and particularly relates to a human body action classification and recognition method based on a multi-observation variable HMM model. The method firstly extracts the upper envelope and the lower envelope of a human body micro-Doppler signal and a Doppler frequency change sequence of a human body trunk from a micro-Doppler image of human body radar echo data, uses the sequences as three observation variables of an HMM model, and discretizes the state quantity of the observation variables through vector quantization. The hidden state of the HMM model is utilized to present the internal relation between the upper envelope and the lower envelope of the human body micro Doppler signal and the frequency change of the trunk, and the Markov property of the hidden state variable of the model is utilized to represent the randomness of the state change of the moving human body, so that the classification performance of subsequent human body actions is improved, and the effectiveness of the method is verified by an experimental result.

Description

Human body action classification recognition method based on multi-observation variable HMM model

Technical Field

The invention belongs to the technical field of human body action classification and recognition, and particularly relates to a human body action classification and recognition method based on a multi-observation variable HMM model.

Background

With the increasing emphasis on human body safety protection, the non-contact human body motion state identification technology gradually becomes a research hotspot. The technology aims at detecting and identifying and classifying human body actions in a non-contact, non-invasive and remote manner, and has important significance in the aspects of identifying abnormal behavior personnel in safety protection, monitoring activities of the elderly living alone, monitoring the physical state of patients in hospitals and the like.

At present, in the aspect of radar detection and human body action identification, except that imaging radar detection carries out motion state identification through human body imaging and trajectory tracking, time domain decomposition or time frequency analysis is mainly carried out on radar echo signals, so that radar signal time domain characteristics or time domain characteristics are extracted to carry out motion state identification.

The human body motion detection technology based on time-frequency analysis has been developed greatly, and has more meaningful theories and technical achievements in the aspects of detection technology, feature recognition, algorithm classification and the like. The Youngwood Kim and Hao Link utilize the micro Doppler spectrogram of the human body to extract the statistical characteristics of the micro Doppler bandwidth, the four limb movement period, the average Doppler of the trunk and the like to realize the classification of the human body actions. The classification method is based on the statistical characteristics of micro Doppler signals, does not consider the state change in the human motion process, and has poor classification effect on random unstable human motion. Victor C · Chen proposes a motion classification method based on markov property of envelope variation of human micro-doppler images. Although this method uses the state change of the human body motion, the envelope of the human body micro doppler signal is a random combination of each moving part of the human body, and the state change is not markov, so that the classification performance is not satisfactory.

Disclosure of Invention

In order to solve the problems, the invention provides a method for classifying and identifying human body actions based on a constructed multi-observation variable Hidden Markov Model (HMM). The method comprises the steps of firstly extracting the upper envelope and the lower envelope of a human body micro-Doppler signal and a Doppler frequency change sequence of a human body trunk from a micro-Doppler image of human body action, reducing the state quantity of an observation variable through quantification, simplifying a model, and reducing the training time of a subsequent HMM model. And the quantized change sequence data is used as three observation state sequences of the HMM, an HMM model of the observation variable is trained, and finally classification of human body actions is achieved. The Markov property of the hidden state variable of the HMM model can fully describe the randomness of the physical condition change of the moving human body, so that the classification performance of the subsequent human body action is improved.

The technical scheme of the invention is as follows:

a human body action classification recognition method based on a multi-observation variable HMM model comprises the following steps:

s1, the upper envelope frequency, the lower envelope frequency and the trunk doppler frequency of the human micro doppler signal are used as three observation variables of the hidden markov model, so that the observation state sequence of the model is extracted first.

And acquiring observation sequences, namely the upper envelope frequency, the lower envelope frequency and the trunk Doppler frequency of the human body micro Doppler signal, wherein different echo effects can be formed on the human body micro signals in a motion state due to different micro characteristics of different body parts. And acquiring the frequency change condition of the upper envelope and the lower envelope of the body micro Doppler signal in an envelope extraction mode. Since the amplitude of the human body micro-doppler signal in the micro-doppler spectrogram is greater than zero, the variation sequence of the upper envelope frequency and the lower envelope frequency is extracted by the following formula:

wherein F_HFor a sequence of frequency variations of the upper envelope, F_LIs the frequency variation sequence of the lower envelope, Sp (f, t) is the micro Doppler spectrum, f is the micro Doppler frequency, t is the time, [ -f_m,f_m]Is the range of spectrogram doppler frequencies, T is the total duration (total sequence length);

in the process of human body movement, the trunk is an energy concentration area of echo signals. Aiming at a micro Doppler energy distribution result at a certain moment, the Doppler signal intensity formed by trunk movement is represented as a maximum value, so that a trunk Doppler frequency change sequence is extracted through peak detection, namely, a point with the maximum amplitude value at the moment t of a spectrogram is extracted, and the trunk Doppler at the moment is recordedFrequency F_T(t)，F_T(t) satisfies the following formula:

the extracted upper envelope frequency variation sequence F_H(t) lower envelope frequency variation sequence F_L(t) and torso frequency variation sequence F_T(t) three observation state sequences as models.

S2, quantifying the observed variable, comprising:

in order to reduce the state quantity of the observed variable, the training time of a subsequent HMM model is reduced, and the observed variable is quantized.

The codebook is obtained through a learning process, assuming a training set of M vector sources (training samples):

Tr＝{x₁,x₂,...,x_M}；

the source vector is 3-dimensional:

x_m＝(F_H(t_m),F_T(t_m),F_L(t_m))，m＝1,2,...,M

F_H(t_m),F_T(t_m),F_L(t_m) Are each t_mThe frequency of the upper envelope, torso and lower envelope of the time micro-doppler signal; given the number of codevectors N (i.e. the vector space is to be divided into N parts), the codebook (the set of all codevectors) is expressed as: c ═ C₁,c₂,...,c_NEach code-vector is a 3-dimensional vector: c. C_n＝(c_n,h,c_n,t,c_n,l) N is 1,2,.. N, wherein c_n,h,c_n,t,c_n,lThree code words which are respectively the nth code vector correspond to the states of three observation variables; and code vector c_nThe corresponding coding region is denoted S_nThe division of the space is then represented as: p ═ S₁,S₂,...,S_N}; if the source vector x_mAt S_nThen its approximation is c_nDenoted Ap (x)_m)＝c_nThe average distortion factor is used for measurement, and the average distortion factor is expressed as follows:

using LBG algorithm, find D_avMinimum code vector c_nObtaining a codebook C; the LBG algorithm is an iterative algorithm that alternately adjusts P and C such that the average distortion continuously moves toward its local minimum, and finally finds D_avThe smallest code vector.

For any vector x to be quantized (f)_h(t),f_t(t),f_l(t))，f_h(t),f_t(t),f_l(t) the frequencies of the upper envelope, the trunk and the lower envelope of the human micro-Doppler signal at the time t are respectively calculated and associated with each codevector c in the codebook through the following formula (L2 norm)_nEuclidean distance of d_n：

d_n＝||x-c_n||₂

The code vector with the shortest Euclidean distance is the vector x after x quantization_q；

S3, constructing a multi-observation variable HMM model:

for the HMM model for human action classification in this method, the observed variables are multiple, definition Q is the set of all possible hidden states, codebook C is the set of all possible observed states, i.e.:

Q＝{q₁,q₂,...,q_K}，C＝{c₁,c₂,...,c_N}

wherein K is the number of possible hidden states, N is the number of all possible observed states;

for a micro Doppler signal of human body movement, I is defined as a corresponding hidden state sequence, and a corresponding observation sequence is a quantized vector sequence X of upper envelope, trunk and lower envelope frequency combination_qAnd the sequence length is T:

I＝{i₁,i₂,...,i_T}，X_q＝{x_q,1,x_q,2,...,x_q,T}

wherein any hidden state i_tE.g. Q, any observation state x_q,t∈C；

Two assumptions for the HMM model:

(1) homogeneous Markov assumption: if the hidden state at any time t is only dependent on the previous hidden state, i_t＝q_iHidden state i at time t +1_t+1＝q_jThe hidden state transition probability a from time t to time t +1_ijExpressed as:

a_ij＝p(i_t+1＝q_j|i_t＝q_i)

then the hidden state transition matrix a ═ a_ij]_K×K；

(2) Observe independent assumptions: assuming that the observation state at any time only depends on the hidden state at the current time, if the hidden state i at the time t_t＝q_jObserved state x at this time_q,t＝c_nAnd the three observation variables are independent from each other, the probability b of the observation state generated in the hidden state at the moment_j(n) is:

the probability matrix (emission matrix) B ═ B of observation state generation_j(n)]_K×N；

In addition, a set of hidden state probability distributions pi at the initial time needs to be defined as:

Π＝[π(i)]_K

wherein pi (i) ═ p (i)₁＝q_i)；

An HMM model is composed of three parameters of hidden state initial probability distribution, a hidden state transition matrix and an emission matrix, and the HMM model is represented by a triplet lambda (pi, A and B);

s4, performing parameter learning on the HMM model λ (Π, a, B) using the EM algorithm, and assigning a training observation sequence X_q,trainMaking the conditional probability p (X) of the observed sequence under the model_q,train| λ) is the maximum, and model parameters λ are obtained as (Π, a, B), and human body actions are classified and recognized through models.

The invention has the beneficial effects that: the hidden state of the HMM model can be used for presenting the internal relation between the upper envelope and the lower envelope of the human body micro Doppler signal and the frequency change of the trunk, and the randomness of the state change of the moving human body is represented by the Markov property of the hidden state variable of the model, so that the classification performance of the subsequent human body action is improved.

Detailed Description

The utility of the invention is analyzed by combining simulation experiments.

The experiment is based on a frequency modulation continuous wave radar (AWR1642) of TI company, and radar echo data of 6 motion actions of 4 persons 3.5-5 meters away from the radar are respectively collected. The 5 actions are walking, forward tumbling, stooping, squatting and in-situ arm swinging, respectively. 2750 time-frequency spectrograms are acquired in each action, wherein 1500 are used for classifier training, and 1250 are used for testing.

In human body action classification, an HMM model is firstly trained for each class of motion actions, namely a model parameter lambda of the ith motion class is obtained_i. The human body action identification and classification stage is to evaluate the human body action observation sequence lambda to be classified_xI.e. λ of the HMM model in the ith action class_iUnder the condition of pi, A and B, calculating an observation sequence X to be recognized under the model by a forward and backward algorithm_qProbability of occurrence p (X)_q|λ_i). Probability p (X)_q|λ_i) And when the maximum value is reached, the action class corresponding to the given HMM model is the human body action class of the recognition classification result.

The experiment uses other two human body motion classification methods to compare with the method provided by the invention. The results of the experiment are shown in table 1.

TABLE 1 human action Classification results

Type of action	Walking machine	Fall down forward	Stoop down	Squatting down	In-situ swing arm	Total number of
							Number of samples tested	250	250	250	250	250	1250
Correct classification number (method 1)	210	219	204	42	236	1001
							Classification accuracy (%) (method 1)	84.00	87.60	81.60	16.80	94.40	80.08
Correct classification number (method 2)	201	210	226	230	241	1108
							Classification accuracy (%) (method 2)	80.40	84.00	90.40	92.00	96.40	88.64
Correct classification number (invention)	224	225	240	238	247	1174
							Classification accuracy (%) (present invention)	89.60	90.00	96.00	95.20	98.80	93.92

The method 1 is an action classification method based on the envelope variation characteristic of the human body micro Doppler image. The method 2 is a method for extracting the characteristics of micro Doppler bandwidth, four limb movement periods, trunk average Doppler and the like by using a human body micro Doppler spectrogram and then realizing human body action classification by using a Support Vector Machine (SVM).

As can be seen from the classification result data in the table, in the method 2, the body state change in the human body motion process is not considered, the random unstable human body motion classification effect is poor, and therefore, the motion classification accuracy of the walking and forward falling human body postures which are relatively high in randomness is low. Although the method 1 utilizes the state change of human body motion, the extracted state change depends on the envelope, and the micro Doppler signal envelope change of the same motion is also larger, so that the walking and forward and backward classification effect is improved, but the method is not very effective for classifying some motions. The human body action classification method provided by the invention mainly depends on the constructed HMM model of the non-single observation variable, finds the internal relation between the upper envelope and the lower envelope of the human body micro Doppler signal and the frequency change of the trunk through the model, presents the relation through the hidden state of the model, and improves the action classification effect by utilizing the relation. The classification accuracy in the table shows that the human body motion classification method provided by the invention has good classification effect, and further improves the classification of motions (walking and forward falling) with high randomness.

Claims (1)

1. A human body action classification recognition method based on a multi-observation variable HMM model is characterized by comprising the following steps:

s1, obtaining observation sequences which are respectively the upper envelope frequency, the lower envelope frequency and the trunk Doppler frequency of the human body micro Doppler signal, wherein the change sequences of the upper envelope frequency and the lower envelope frequency are extracted through the following formula:

wherein F_HFor a sequence of frequency variations of the upper envelope, F_LIs as followsA sequence of frequency variations of the envelope, Sp (f, t) being the micro-Doppler spectrum, f being the micro-Doppler frequency, t being the time, [ -f_m,f_m]Is the range of spectrogram Doppler frequency, T is the total duration;

extracting the trunk Doppler frequency change sequence through peak detection, namely extracting a point with the maximum amplitude value at the moment t of a spectrogram, and recording the trunk Doppler frequency F at the moment_T(t)，F_T(t) satisfies the following formula:

s2, quantifying the observed variable, comprising:

assume a training set of M vector sources:

Tr＝{x₁,x₂,...,x_M}；

the source vector is 3-dimensional:

x_m＝(F_H(t_m),F_T(t_m),F_L(t_m))，m＝1,2,...,M

F_H(t_m),F_T(t_m),F_L(t_m) Are each t_mThe frequency of the upper envelope, torso and lower envelope of the time micro-doppler signal; setting the number of code vectors as N, setting the set of all code vectors as a code book, wherein the code book is expressed as: c ═ C₁,c₂,...,c_NEach code-vector is a 3-dimensional vector: c. C_n＝(c_n,h,c_n,t,c_n,l) N is 1,2,.. N, wherein c_n,h,c_n,t,c_n,lThree code words which are respectively the nth code vector correspond to the states of three observation variables; and code vector c_nThe corresponding coding region is denoted S_nThe division of the space is then represented as: p ═ S₁,S₂,...,S_N}; if the source vector x_mAt S_nThen its approximation is c_nDenoted Ap (x)_m)＝c_nThe average error distortion is used for measurement, and the average error distortion is expressed as follows:

using LBG algorithm, find D_avMinimum code vector c_nObtaining a codebook C;

for any vector x to be quantized (f)_h(t),f_t(t),f_l(t))，f_h(t),f_t(t),f_l(t) respectively calculating the frequency of the upper envelope, the trunk and the lower envelope of the human body micro Doppler signal at the time t by the following formula_nEuclidean distance of d_n：

d_n＝||x-c_n||₂

The code vector with the shortest Euclidean distance is the vector x after x quantization_q；

S3, constructing a multi-observation variable HMM model:

definition Q is the set of all possible hidden states and codebook C is the set of all possible observed states, i.e.:

Q＝{q₁,q₂,...,q_K}，C＝{c₁,c₂,...,c_N}

wherein K is the number of possible hidden states, N is the number of all possible observed states;

for a micro Doppler signal of human motion, I is defined as a hidden state sequence, and the corresponding observation sequence is a quantized vector sequence X of the frequency combination of an upper envelope, a trunk and a lower envelope_qThe sequence length is the total duration:

I＝{i₁,i₂,...,i_T}，X_q＝{x_q,1,x_q,2,...,x_q,T}

wherein any hidden state i_tE.g. Q, any observation state x_q,t∈C；

If the hidden state at any time t is only dependent on the previous hidden state, i_t＝q_iHidden state i at time t +1_t+1＝q_jFrom time tHidden state transition probability a to time t +1_ijExpressed as:

a_ij＝p(i_t+1＝q_j|i_t＝q_i)

then the hidden state transition matrix a ═ a_ij]_K×K；

Assuming that the observation state at any time only depends on the hidden state at the current time, if the hidden state i at the time t_t＝q_jObserved state x at this time_q,t＝c_nAnd the three observation variables are independent from each other, the probability b of the observation state generated in the hidden state at the moment_j(n) is:

the emission matrix B generated by the observation state is ═ B_j(n)]_K×N；

Defining hidden state probability distribution pi at initial time as:

Π＝[π(i)]_K

wherein pi (i) ═ p (i)₁＝q_i)；

Forming an HMM model by three parameters including hidden state initial probability distribution, a hidden state transition matrix and an emission matrix, wherein the HMM model is lambda (pi, A and B);

s4, performing parameter learning on the HMM model λ (Π, a, B) using the EM algorithm, and assigning a training observation sequence X_q,trainMaking the conditional probability p (X) of the observed sequence under the model_q,train| λ) is the maximum, and model parameters λ are obtained as (Π, a, B), and human body actions are classified and recognized through models.