KR101017449B1 - Method for recognizing motion based on hidden markov model and computer-readable medium having thereon program performing function embodying the same - Google Patents

Abstract

The present invention comprises (a) generating a normal motion symbol sequence based on a change in position of feature points of a normal motion over time, and (b) performing motion pattern modeling using a hidden Markov model for the normal motion symbol sequence. (C) extracting a user action symbol sequence based on a change in position of feature points of a user action over time; and (d) modeling the user action symbol sequence in the step (b). And recognizing the user's motion by comparing the method, and (e) comparing the user's motion symbol sequence with the normal motion symbol sequence and extracting a motion similarity between the user's motion and the normal motion. Markov model-based motion recognition method.

According to the present invention, after extracting the feature points of the motion and performing the motion pattern modeling using the hidden Markov model to change the position of the feature points of the motion over time, motion recognition and motion similarity are extracted by comparing the motion pattern modeling and the user motion. can do.

Hidden Markov, Regular Behavior Symbol Sequence, User Behavior Symbol Sequence, Behavior Pattern Modeling, Behavior Similarity

Description

TECHNICAL FOR RECOGNIZING MOTION BASED ON HIDDEN MARKOV MODEL AND COMPUTER-READABLE MEDIUM HAVING THEREON PROGRAM PERFORMING FUNCTION EMBODYING THE SAME}

The present invention relates to a hidden Markov model-based motion recognition method and a computer-readable recording medium having recorded thereon a program for realizing the same. After performing the motion pattern modeling using the model, a hidden Markov model-based motion recognition method for extracting motion recognition and motion similarity by comparing the motion pattern modeling and the user motion, and a computer-readable recording medium recording a program for realizing the motion pattern model It is about.

Recently, with the development of ubiquitous technology and virtual reality based technology, motion recognition processing technologies have been studied. Motion recognition processing technology is, for example, a technology related to human-computer interface (HCI) related to electronic devices and computer control using hand gestures, and technology that recognizes movement using data transmitted through a sensor attached to a person and utilizes it for computer animation. Etc.

The motion recognition processing technology can be applied to various fields such as a system technology that analyzes an image input through a camera to recognize or track a motion, or a surveillance system technology that automatically finds dangerous motions in a place requiring security or surveillance. . In addition, motion recognition processing technology allows users to control virtual instruments and equipment using their bodies in the game and entertainment field, and also recognizes motion in therapeutic areas such as posture correction and rehabilitation training, which imply meaning to the exercise itself. Processing techniques can be used.

According to the conventional motion recognition technology, in order to recognize a motion, a lot of operations are required in a process of extracting a user's body features in a continuous frame, and a process of recognizing a pattern of a task in the extracted feature vector is required. Pattern recognition of motion mainly uses gesture recognition. Gesture recognition means extracting what shape change each part of the human body has on the time axis. However, since the human body has a very complex three-dimensional joint structure, it is very difficult to recognize a gesture. Gesture recognition therefore requires that a human hand, arm, etc., be measured by a machine.

In order to recognize a specific motion of a person such as a hand gesture or a body movement, research on a video sequence image based on vision is being actively conducted.

The motion recognition method using vision-based image information is a non-contact recognition method that recognizes gestures based on appearance features using an image obtained through a camera without attaching any device to a human body. In the vision-based motion recognition method, motion classification and classification are performed to compare motions with models, and various feature information is used for comparison. Such feature information includes geometric features, statistical features, and features extracted from time changes.

The first method of using geometric features is to find the edges, corners, and so on, to prepare data for a regular model and compare it with the data obtained in the same way. The use of geometric features makes it difficult to extract features for many complex motions and requires a lot of preprocessing to reduce recognition speed and recognition rate.

The second statistical feature is a preprocessing method that attempts matching by analyzing brightness information and location information having statistically high probability values for many image sets. It has a relatively good recognition rate and is effective for real time application.

Third, a method of using feature point extraction information of an image according to a change of time is a method of classifying or classifying an operation according to a pattern by analyzing a pattern of an operation according to time. The method of using the feature extraction information has the advantage of easily extracting the operability even in a general environment, but has the disadvantage of learning using a large number of images and modeling an appropriate pattern.

An object of the present invention is to extract the feature points of the motion, perform the motion pattern modeling using the hidden Markov model of the positional changes of the feature points of the motion, and then compare the motion pattern modeling and user motion to extract motion recognition and motion similarity To provide a hidden Markov model-based motion recognition method.

Another object of the present invention is to provide a computer readable recording medium having recorded thereon a program for realizing each step of the hidden Markov model-based motion recognition method.

In order to achieve the above technical problem, the present invention comprises the steps of (a) generating a normal motion symbol sequence based on the change in position of the feature points of the normal operation over time, and (b) a hidden Markov model for the normal motion symbol sequence. (C) extracting a user motion symbol sequence based on a change in position of feature points of the user motion over time using the method; and (d) converting the user motion symbol sequence to the step ( b) recognizing the user's motion by comparing the motion pattern modeling in (b), and (e) comparing the user's motion symbol sequence with the normal motion symbol sequence to determine the motion similarity between the user's motion and the normal motion. It provides a hidden Markov model-based motion recognition method further comprising the step of extracting.

In the Hidden Markov Model-based Motion Recognition Method according to the present invention, the step (a) may include (a-1) the normal motion symbol according to a space including a direction vector of positional change over time of the feature points of the normal motion. Generating a sequence, wherein step (c) comprises: (c-1) generating the user gesture symbol sequence according to a space in which a direction vector of positional change over time of the feature points of the user gesture is included; It may be to include a step.

In the hidden Markov model-based motion recognition method according to the present invention, the step (a-1) may include (a-2) a direction vector between the center of gravity of the initial feature points of the normal motion and the initial feature points of the normal motion. And determining an initial value of the normal operation sign sequence according to a space in which the step is included. The step (c-1) may include (c-2) the center of gravity of the initial feature points of the user motion and the user motion. The method may include determining an initial value of the user gesture symbol sequence according to a space in which a direction vector between initial feature points of U is included.

및

를 반복적으로 추출하는 단계와,

,

[

는 상태 w _i (t-1)에서 상태 w _j (t)로의 전이 기대 확률, v _k 는 정규 동작 기호 시퀀스 내의 기호를 나타냄] (b-3) 최종적으로 추출한

및

를 전이 확률 a _ij 와 관측 확률 b _jk 로 지정하는 단계를 포함하는 것일 수 있다.In the hidden Markov model-based motion recognition method according to the present invention, the step (b) includes (b-1) initializing the transition probability a _ij and the observation probability b _jk of the state constituting the motion pattern modeling; , (b-2) the difference between a _ij (z) and a _ij (z-1), b _jk (z) and b _jk (z-1) for z less than or equal to the total length T , Estimates Using Equations

And

Extracting it repeatedly;

,

[

Is the expected probability of transition from state w _i (t-1 ) to state w _j (t) , and v _k represents a symbol in the sequence of normal motion symbols] (b-3)

And

It may include the step of designating the transition probability a _ij and the observation probability b _jk .

In the hidden Markov model-based motion recognition method according to the present invention, the step (d) may include (d-1) matching probabilities of the user motion symbol sequence and the normal motion symbol sequence based on the motion pattern modeling. And (d-2) recognizing the user's motion as the normal motion when the difference of the matching probability is equal to or less than a predetermined reference value.

delete

In addition, in the hidden Markov model-based motion recognition method according to the present invention, the step (e) subtracts the cost incurred in (e-1) matching the user motion symbol sequence with the normal motion symbol sequence at a predetermined reference value. It may include extracting a value to the operation similarity.

In addition, in the hidden Markov model-based gesture recognition method according to the present invention, the user gesture symbol sequence and the regular gesture symbol sequence may include a space.

을 기초로 추출될수 있다. In addition, in the hidden Markov model-based motion recognition method according to the present invention, the cost (A _cost ),

Can be extracted based on.

[Wherein S ₁ ′ (i) and S ₂ ′ (i) indicate the i th symbol of the normal operation symbol sequence and the user operation symbol sequence each including a space, and s (S ₁ '(i), S ₂ '(i)) represents a function for outputting a cost value for aligning the i-th symbol of the regular motion sign sequence and the user motion sign sequence, and l indicates an alignment length]

In addition, in the hidden Markov model-based motion recognition method according to the present invention, in the s (S ₁ '(i), S ₂ ' (i)), the i-th symbol of the normal operation symbol sequence is not blank, the user operation An insertion cost incurred when the i-th symbol of the symbol sequence is blank, a deletion cost incurred when the i-th symbol in the regular operation symbol sequence is blank and the non-blank in the i-operation symbol sequence of the user sequence; Replacement cost incurred when the i th symbol of the operation symbol sequence and the i th symbol of the user operation symbol sequence are not blank, and the i th symbol of the regular operation symbol sequence and the i th symbol of the user operation symbol sequence match It may be to output any one of the matching costs that occur.

In addition, in the hidden Markov model-based motion recognition method according to the present invention, the step (e) is (e-2) when the same symbol is repeated in the user motion symbol sequence or the normal motion symbol sequence, the repeated symbol Converting to display only one time, and comparing the extracted user motion with the normal motion.

In addition, in the hidden Markov model-based motion recognition method according to the present invention, the step (e) is (e-3) when the user motion is a partial motion by comparing the user motion symbol sequence and the normal motion symbol sequence And extracting a partial motion similarity between the user motion and the normal motion.

The present invention also provides a computer-readable recording medium having recorded thereon a program for realizing each step of the above-described hidden Markov model-based motion recognition method.

According to the present invention, after extracting the feature points of the motion and performing the motion pattern modeling using the hidden Markov model of the positional change of the feature points of the motion over time, motion recognition and motion similarity can be extracted by comparing the motion pattern modeling and the user motion. Can be.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT Hereinafter, a hidden markov model-based motion recognition method of the present invention and an embodiment of a computer-readable recording medium recording a program for realizing the same will be described in more detail with reference to the accompanying drawings.

1 is an exemplary flowchart of a hidden Markov model based motion recognition method in accordance with the present invention.

First, a normal operation symbol sequence is generated based on a change in position of feature points of the normal operation over time (S110).

The generation of the regular operation symbol sequence may be performed as follows.

First, the feature points of the human body such as arms and legs of the human body are extracted.

For example, infrared LEDs and image processing are used to extract positional coordinates for five parts of the head, both wrists, and both ankles.

Thereafter, the positional change according to the time change is classified into each of the extracted positional coordinates into a predetermined number of sections, for example, eight sections.

2 to 3 are diagrams illustrating an example of generating a normal motion symbol sequence using a position change of a feature point and a position according to time in a hidden Markov model-based motion recognition method according to the present invention.

Referring to FIG. 2, in the hidden Markov model-based motion recognition method according to the present invention, a positional change of a feature point over time, that is, a positional change from a reference point to a change point may be expressed as a direction vector. A normal motion symbol sequence may be generated based on a space including a direction vector of a position change of a feature point of the normal motion over time.

For example, referring to FIG. 2, the direction vector is disposed in any one of the space divided into eight sections, and a section including the motion vector over time is designated as a symbol of the corresponding direction vector. Here, the sign of the direction vector is represented by a number between 1 and 8 corresponding to the space divided into eight sections for the convenience of calculation.

Referring to FIG. 3, a change from a position at t = 0 to a position at t = 1 corresponds to 8 sections of FIG. 2. Therefore, the symbol for the change can be set to 8. In addition, the change from the position of t = 1 to the position of t = 2 corresponds to the section 7 of FIG. 2. Therefore, the symbol for the change can be set to 7.

On the other hand, the symbol for the first starting position is not generated through space division because there is no change of position.

Therefore, the symbol for the first starting position is to create all polygons using the Convex Hull technique, obtain the center of gravity of the polygon, and then rearrange the feature points in the space with the center of gravity as the reference point. You can specify that the symbol of to be the symbol for the starting position.

2 and 3, assuming that the feature point is in the second section at the position t = 0, the symbol for the first start position may be set to two.

Accordingly, the normal operation symbol sequence for the situation of FIG. 3 may be represented by {2, 8, 7}.

A set of positional changes according to time changes, that is, a normal operation symbol sequence generated through the method described with reference to FIGS. 2 to 3, may be used as pattern data for an operation.

Subsequently, motion pattern modeling is performed on the normal motion symbol sequence generated in S110 using a hidden Markov model (S130).

In order to recognize a motion, motion pattern modeling is required to pattern motion-specific data and to structure the patterned data for use.

Behavior pattern modeling techniques generally use a stochastic approach. For the behavior pattern modeling, the present invention uses a hidden Markov model used for speech recognition, hand gesture recognition, and the like. Hidden Markov models are models of double stochastic structures that model stochastic structures of patterns that are difficult to model through clearly expressible symbols. The hidden Markov model expresses the pattern as a Probabilistic finite state automata structure, and the finite states of the pattern can be expressed as a joint probability over time.

Therefore, the symbol set for modeling should be in the form of including the relationship between the inherent characteristics of the pattern and the time change. In the present invention, such modelable symbols, that is, a set of positional changes over time as the unique data, that is, the above-described normal motion symbol sequence, are used.

First, in order to express the relationship between the pattern data and time, the finite states of the pattern are represented with respect to time t , and the moment t _n is completed. The state of t + 1 is expressed as the influence of t state in order to model the process that evolves up to. If the state at t is w (t) , the specific state w _i in the state sequence of the entire length T can be expressed as Equation 1 below.

Probability that the time condition w _j (t + 1) of the t + 1 when for state w _i (t) of time t when can be represented by the transition probabilities [Transition Probability] a _ij, as shown in the following equation (2).

If there is a full set model θ of a _ij and a state sequence of full length T , then the probability that the model θ generates this state sequence can be represented by multiplying the transition probability between the series of states.

If the symbol at time t is v (t) , the specific symbol v _k in the symbol sequence of the full length T may be expressed as in Equation 3 below.

If the symbol that can be observed in state w _j , that is, the symbol for motion vector information of the motion, is v _k , then the probability that the symbol v _k (t) will occur in any state w _j (t) at time t is P (v _k (t) | w _j (t))

That is, the entire state sequence can be modeled by finding a state w _j (t) having a high probability that a symbol v _k (t ) for time t is given when a symbol set for the entire length T is given.

The state transition probability distribution (A) according to the state of the model and the number of symbols and the probability occurrence probability distribution (B) of each state are summarized as in Equation 4 below.

을 만족하고, 모든 j에 대해서

를 만족한다. 또한 W^T는 T개의 원소를 가질 수 있으며, V^T도 T개의 원소를 가질 수 있다. In Equation 4, c is the number of states, k is the number of symbols, W ^T is a sequence of states, V ^T is a sequence of symbols, for all i

For all j

. In addition, W ^T may have T elements, and V ^T may also have T elements.

Based on Equation 4, the values of the parameter sets A and B for the motion model θ, that is, the state transition probability distribution (A) and the symbol occurrence probability distribution (B) in each state are calculated using a symbol sequence of the motion to be modeled. Can be obtained through the learning process.

The parameter value calculation uses the following backward algorithm.

When the model θ is in state w _i (t) , the probability of generating the state sequence up to t is α _i (t) , and the probability of generating the remaining state sequences from t + 1 → T is β _i (t) . It can be expressed as 5.

If β _i (T-1) is obtained using Equation 5, it may be expressed by Equation 6.

Β _i (T-2) can be obtained using β _i (T-1) obtained through Equation 6, and β _i (t) can be obtained by repeating this process in the reverse direction.

4 is a diagram illustrating an example of obtaining β _i (t) in the hidden Markov model-based motion recognition method according to the present invention.

Referring to FIG. 4, β _i (t) is obtained for each state based on the total length T , and maximum values at that time, that is, β _MAX (1) to β _MAX (T) are shown.

,

은 학습을 위한 기호 시퀀스 V ^T 를 통하여 구할 수 있다. Estimated value of transition probability a _ij and observation probability b _jk to be determined in Equation 6

,

Can be obtained through the symbol sequence V ^T for learning.

라 하면,

는 다음 수학식 7과 같이 나타낼 수 있다. Expected probability of transition from state w _i (t-1 ) to state w _j (t)

Say,

May be expressed as in Equation 7 below.

로 나타낼 수 있으며, w _i 로부터의 임의의 전이 전체 기대 수는

로 나타낼 수 있다. The expected number of transitions from state w _i (t-1) to state w _j (t) is

And the total expected number of any transition from w _i is

It can be represented as.

는 w _i 에서 w _j 로의 전이 기대 수와 w _i 로부터의 임의의 전이의 대한 전체 기대 수간의 비율로 찾아 낼 수 있으며, 같은 방법으로 상태 w _j 에서 특정 v _k 가 관측되는 빈도와 임의의 기호에 대한 빈도 간의 비율을 계산하여 추정

를 구할 수 있다.Thus estimated

Can be found as the ratio between the expected number of transitions from w _i to w _j and the total expected number of arbitrary transitions from w _i , and in the same way depends on the frequency at which particular v _k is observed in state w _j and on any symbol. By calculating the ratio between frequencies for

Can be obtained.

및

를 각각 나타낸다.Equations 8 to 9 are estimated values of the transition probability a _ij and the observation probability b _jk ,

And

Respectively.

Using Equations 8 and 9, the values of parameter sets A and B for the motion model θ, namely, the state transition probability distribution A and the symbol occurrence probability distribution B in each state can be obtained.

5 is a diagram illustrating an example of an algorithm for extracting a state transition probability distribution A and a symbol occurrence probability distribution B in each state in the hidden Markov model-based motion recognition method according to the present invention. .

Referring to FIG. 5, a state transition probability distribution A and a symbol occurrence probability distribution B in each state may be extracted using a learning algorithm.

및

를 반복적으로 구한다.First, the transition probability a _ij and the observation probability b _jk are initialized and the learning sequence, that is, the symbol sequence V ^T is learned. If the difference between a _ij (z) and a _ij (z-1), b _jk (z) and b _jk (z-1) is less than or equal to the reference value ε,

And

Get it repeatedly.

및

를 전이 확률 a _ij 와 관측 확률 b _jk 로 지정하는 것에 의해서 상태 전이 확률 분포(A)와 각 상태에서의 기호 발생 확률 분포(B)를 구할 수 있다.After the final extraction

And

By specifying the transition probability a _ij and the observation probability b _jk , the state transition probability distribution A and the symbol occurrence probability distribution B in each state can be obtained.

Operation pattern modeling is completed through the above-described process.

Thereafter, the user's motion symbol sequence is extracted based on the change in position of the feature points of the user's motion over time (S150).

Since the user operation symbol sequence is the same as the extraction of the normal operation symbol sequence described with reference to FIGS. 2 to 3, a detailed description thereof will be omitted.

Thereafter, the user motion symbol sequence extracted in step S150 is compared with the motion pattern modeling in step S130 to recognize the user motion (S170).

Through the operation pattern modeling process in step S130, the operation pattern model θ may be expressed by the obtained parameter sets A and B.

In the present invention, after each matching probability is obtained by using the user's motion symbol sequence and the normal motion symbol sequence used for learning with respect to the model θ obtained through the learning process in order to recognize the motion, when the difference between the two probabilities is less than a predetermined motion Use behaviors that designate model-like behavior as actions.

The method of calculating the matching probability for the model generates an optimal state sequence using the relationship between the symbol sequence and the state sequence used in the modeling, and the state transition probability and the symbol for the entire length T using the state sequence. A method of multiplying the occurrence probability of the sequence may be used.

In other words, the state transition path (Path) in the regular model θ is obtained through the user's motion symbol sequence V ^T , and the sum Sum (X ₁ = A _Path ) of the path value (transition probability value) of t → T and the normal used in modeling. The closest motion can be recognized by the difference between Sum (X ₂ = A _Path ) values, which can be obtained as the action symbol sequence.

The search of the state transition path may be performed as follows.

6 is a diagram illustrating an example of an algorithm for searching for an optimal path in the hidden Markov model-based motion recognition method according to the present invention.

6, the model θ is α _j (t) at that time and then calculating the t α _j (t) status sequence, the probability of occurrence of until the condition w _i (t) with respect to the following T t Adds to the path if has the maximum value. This operation can be repeated up to T to search for the optimal path.

FIG. 7 is a diagram illustrating an optimal path generated using the algorithm of FIG. 6 in the hidden Markov model-based motion recognition method according to the present invention.

Referring to FIG. 7, when α _j (t) has a maximum value at that time, that is , an optimal path is searched using α _MAX (1) to α _MAX (T) .

Since the optimal path is the path with the largest sum of the path values up to T, the path search should use global search, not local search.

Through this process, the user's motion can be recognized.

Meanwhile, the hidden Markov model-based motion recognition method according to the present invention may further include extracting motion similarity between the user motion and the normal motion by comparing the user motion symbol sequence with the normal motion symbol sequence.

The motion similarity is a measure of the degree of correspondence between two motions to be compared. However, since the hidden Markov model performs a probabilistic approach, it calculates the matching probabilities for the entire time T for the two motions to be compared and extracts the matching probabilities for the two motions. There is a disadvantage that can not be represented.

In the present invention, a symbol sequence is used to extract motion similarity for partial motion.

When the symbol sequence of the normal operation used for learning is called S ₁ , and the symbol sequence of the user's action is S ₂ , the two sequences S ₁ using a score matrix at a reference value when the two sequences match perfectly. The motion similarity can be extracted by subtracting the sum of costs incurred from matching S ₂ to S ₂ .

More specifically, when the set of sequences included in S ₁ and S ₂ is called Λ and the set with spaces added to Λ is Λ ', the cost incurred when aligning arbitrary elements x and y in Λ' Assume that s (x, y) is a function that extracts.

In this case, the total cost incurred in matching two sequences of S ₁ and S ₂ may be expressed as in Equation 10 below.

Here, S ₁ '(i), S ₂ ' (i) represents the i-th symbol of S ₁ and S ₂ in consideration of white space, and l represents the length of the alignment.

Meanwhile, s (S ₁ '(i), S ₂ ' (i)) may have the following values.

The insertion cost (insCost) is a cost incurred when S ₁ '(i) is not empty and S ₂ ' (i) is empty. In the present invention, the insertion cost can be designated as 5, for example.

The delete cost (delCost) is a cost incurred when S ₁ '(i) is blank and S ₂ ' (i) is not blank. In the present invention, the deletion cost can be specified as, for example, five.

ChangeCost is a cost incurred when S ₁ '(i) and S ₂ ' (i) are both non-blank and not equal. In the present invention, the replacement cost can be designated, for example.

On the other hand, when S ₁ '(i) and S ₂ ' (i) is the same, s (S ₁ '(i), S ₂ ' (i)) can be assigned a value, such as 0, the matching cost.

Assuming that S ₁ = {6, 2, 7, 6, 3, 6}, and S ₂ = {6, 2, 3, 2, 3, 7}, Λ = {2, 3, 6, 7} Can be.

On the other hand, suppose S ₁ 'is {6, 2, 7,-, 6, 3, 6} and S ₂ ' is {6, 2, 3, 2,-, 3, 7} with spaces in mind, ( '-' Represents a space).

In this case, the score matrix of Λ 'may be set based on the following Table 1.

S 2 3 6 7 - 2 0 7 7 7 5 3 0 7 7 5 6 0 7 5 7 0 5 - 0

In the score matrix, if x and y match, s (x, y) is zero, and if it does not match, s (x, y) has a value greater than zero. In the case of non-matching, the insertion cost, the deletion cost, and the replacement cost were set as described above. In Table 1, the insertion cost was 5, the deletion cost was 5, and the replacement cost was 7.

In this case, the total cost is calculated using Equation 10 as follows.

The first symbol of S ₁ ′ corresponds to the first symbol of S ₂ ′, so s (S ₁ '(1), S ₂ ' (1)) is 0.

S ₁ 'of the second symbol S _2', so match the second symbol of the _{s (S 1 '(2)} , S 2' (2)) is zero.

S ₁ 'of the third symbol S _2' does not match the sign of the _{third, s (S 1 '(3} ), S 2' (3)) has a value of 7, the replacement cost.

Since the fourth symbol of S ₁ ′ is blank, s (S ₁ '(4), S ₂ ' (4)) has a value of 5, which is the deletion cost.

Since the fifth symbol of S ₁ 'is not blank and the fifth symbol of S ₂ ' is blank, s (S ₁ '(5), S ₂ ' (5)) has a value of 5, which is an insertion cost.

The sixth symbol of S ₁ 'corresponds to the sixth symbol of S ₂ ', so s (S ₁ '(6), S ₂ ' (6)) is zero.

S ₁ 'of the seventh symbol S _2' does not match the sign of the _{seventh, s (S 1 '(7} ), S 2' (7)) has a value of 7, the replacement cost.

The total cost has 24, the sum of each cost.

Thus, assuming a reference value of 100, for example, the operational similarity has a value of 76 minus the total cost from the reference value.

Meanwhile, the aforementioned symbol sequences S ₁ and S ₂ have a direction information format expressed as a motion vector per unit time. If the same symbol is repeated to convert the symbol sequence independently of the time axis, the repeated symbol can be reduced to one. The form of the symbol sequence from which the repeatability is removed has a data format indicating the form of the operation while removing the information on the time interval. In order to measure the aforementioned motion similarity, the amount of computation may be minimized by using a symbol sequence from which repeatability is removed.

In particular, in order to extract the motion similarity of the partial motion, the overall cost having the smallest possible value may be extracted using the aforementioned score matrix, and the motion similarity may be extracted by subtracting the total cost extracted from the reference value.

On the other hand, for the experiment of the hidden Markov model-based motion recognition method according to the present invention, a system was constructed and experimented using one host computer for computation processing such as image processing and motion recognition and one computer for user display.

Experimental method firstly calculates the motion recognition rate by 10, 20, and 30 repetition experiments at distances of 2, 3m for each of the three sample motions, and divides one motion into 2 to 5 sections. The motion similarity was extracted by the partial motion experiment for.

Sample operations are shown in Table 2 below.

division Action Characteristic A Sitting in a chair, lifting both hands alternately Arm movement, half-circle movement of the arm around the body B Waking up from a chair Up and down movement C Move 3 steps left Leg movement

The extracted motion recognition rates are shown in Table 3 below.

division 10th 20 times 30 times 2 m 3m 2 m 3m 2 m 3m A 100% 100% 95% 95% 95% 95% B 95% 95% 95% 90% 90% 90% C 90% 90% 90% 90% 93% 93%

As a result, the recognition rate was more than 95% for the frontal movement based on the camera position. In addition, about the sample motion C, when the user's action is moved to the side, the center of gravity is moved every time the user moves.

On the other hand, in order to experiment the motion similarity with respect to the partial motions, the motions were classified into 5 steps for the sample motion A, and the motion similarity results were tested when only some step motions were performed without performing the entire motion.

8 is a diagram illustrating a step operation for extracting a motion similarity of a partial motion in the hidden Markov model-based motion recognition method according to the present invention.

Referring to Figure 8, the operation of Sample A was divided into (a) lifting the left hand (b) lifting the right hand (c) lowering both hands (d) lifting both hands (e) lowering both hands.

Table 4 shows the motion similarity for the extracted partial motion.

division 2 m 3m a ~ e 100 100 a ~ d 72 70 a ~ c 51 51 d ~ e 55 52 d 10 0

As a result of extracting the motion similarity for partial motion, it was shown that the normal motion and the user motion exactly matched regardless of the distance when all the partial step motions were performed, and the motion similarity proportional to the overall motion when some steps of the partial motions were performed. Indicated. In particular, when the initial start of the operation is different, for example, even if only d-e or d is performed, the partial motion similarity can be measured.

The hidden Markov model-based motion recognition method according to the present invention can be applied to various fields. For example, it can be used for motion recognition based games, repetitive training and measurement of training achievement, and posture recognition using motion recognition. In addition, since the similarity to partial motion can be measured, it can be used for rehabilitation training for the disabled through comparison with regular motion.

The present invention also provides a computer-readable recording medium having recorded thereon a program for realizing each step of the above-described hidden Markov model-based motion recognition method according to the present invention.

A computer readable recording medium refers to any kind of recording device that stores data, that is, data in code or program form, so that it can be read by a computer system. Such computer-readable recording media include, for example, memories such as ROM and RAM, storage media such as CD-ROM and DVD-ROM, magnetic storage media such as magnetic tape and floppy disk, optical data storage devices, and the like. It also includes a case where it is implemented in the form of transmission via. Such computer readable recording media can also be distributed over network coupled computer systems so that the computer readable data is stored and executed in a distributed fashion.

However, a detailed description of such a computer-readable recording medium is omitted because it overlaps with the hidden Markov model-based motion recognition method according to the present invention described with reference to FIGS.

Although the configuration of the present invention has been described in detail, these are merely illustrative of the present invention, and various modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. This will be possible.

Therefore, the embodiments disclosed in the present specification are intended to illustrate rather than limit the present invention, and the scope and spirit of the present invention are not limited by these embodiments. It is intended that the scope of the invention be interpreted by the following claims, and that all descriptions within the scope equivalent thereto will be construed as being included in the scope of the present invention.

As described above, according to the present invention, after extracting the feature points of the motion and performing the motion pattern modeling using the hidden Markov model of the positional change of the feature points of the motion over time, the motion pattern modeling and the user motion are compared and the motion recognition and Motion similarity can be extracted. For example, it can be used for motion recognition based games, repetitive training and measurement of training achievement, and posture recognition using motion recognition. In addition, since the similarity to partial motion can be measured, it can be used for rehabilitation training for the disabled through comparison with regular motion.

1 is an exemplary flow diagram of a hidden Markov model based motion recognition method in accordance with the present invention.

2 to 3 are diagrams illustrating an example of generating a normal motion symbol sequence using a position change of a feature point and a position according to time in a hidden Markov model-based motion recognition method according to the present invention.

4 is a diagram illustrating an example of obtaining β _i (t) in the hidden Markov model-based motion recognition method according to the present invention.

5 is a diagram illustrating an example of an algorithm for extracting a state transition probability distribution A and a symbol occurrence probability distribution B in each state using estimated values in a hidden Markov model-based motion recognition method according to the present invention.

6 illustrates an example of an algorithm for searching for an optimal path in a hidden Markov model-based motion recognition method according to the present invention.

7 is a diagram illustrating an exemplary optimal path generated using the algorithm of FIG. 6 in a hidden Markov model-based motion recognition method according to the present invention.

8 is a diagram illustrating a step operation for extracting a motion similarity of a partial motion in a hidden Markov model-based motion recognition method according to the present invention.

Claims (13)

(a) generating a normal motion symbol sequence based on a change in position of the feature points of the normal motion over time; (b) performing motion pattern modeling on the regular motion symbol sequence using a hidden Markov model; (c) extracting a user action symbol sequence based on a change in position of feature points of the user action over time; (d) comparing the motion pattern modeling in the step (b) with the user motion symbol sequence to recognize the user motion Including, (e) extracting a motion similarity between the user motion and the normal motion by comparing the user motion symbol sequence with the normal motion symbol sequence; Hidden Markov model based motion recognition method further comprising. The method of claim 1, The step (a) includes the step of (a-1) generating the normal motion symbol sequence according to a space including a direction vector of the positional change over time of the feature points of the normal motion, The step (c) may include (c-1) generating the user gesture symbol sequence according to a space including a direction vector of the positional change over time of the feature points of the user gesture. Motion recognition method. The method of claim 2, In the step (a-1), (a-2) an initial value of the normal motion symbol sequence according to a space including a direction vector between the center of gravity of the initial feature points of the normal operation and the initial feature points of the normal operation is included. Making a decision, In the step (c-1), (c-2) the initial value of the user gesture symbol sequence may be determined according to a space including a direction vector between the centers of gravity of the initial feature points of the user motion and the initial feature points of the user motion. And determining the hidden Markov model based motion recognition method. The method of claim 1, wherein step (b) (b-1) initializing transition probabilities a _ij and observation probabilities b _jk of the states constituting the motion pattern modeling; (b-2) The difference between a _ij (z) and a _ij (z-1), b _jk (z) and b _jk (z-1) is less than or equal to the reference value ε for z less than or equal to the total length T Estimate using

And

Extracting it repeatedly;

,

[

Is the expected probability of transition from state w _i (t-1 ) to state w _j (t) , and v _k represents a symbol in the sequence of normal behavior symbols] (b-3) finally extracted

And

Specifying the transition probability a _ij and the observation probability b _jk Hidden Markov model based motion recognition method comprising a. The method of claim 1, wherein step (d) (d-1) extracting matching probabilities of the user motion symbol sequence and the normal motion symbol sequence based on the motion pattern modeling; (d-2) recognizing the user's motion as the normal motion when the difference of the matching probability is equal to or less than a predetermined reference value; Hidden Markov model based motion recognition method comprising a. delete The method of claim 1, In the step (e), (e-1) extracting a value obtained by subtracting the cost incurred to match the user action sign sequence and the normal action sign sequence from a predetermined reference value as the motion similarity. Hidden Markov model based motion recognition method comprising a. The method of claim 1, And the user action symbol sequence and the normal action symbol sequence include a space. The method of claim 7, wherein The cost (A _cost ),

[Wherein S ₁ ′ (i) and S ₂ ′ (i) indicate the i th symbol of the normal operation symbol sequence and the user operation symbol sequence each including a space, and s (S ₁ '(i), S ₂ '(i)) represents a function for outputting a cost value for aligning the i-th symbol of the regular motion sign sequence and the user motion sign sequence, and l indicates an alignment length] Hidden Markov model based motion recognition method that is extracted based on. 10. The method of claim 9, The s (S ₁ '(i), S ₂ ' (i)), An insertion cost incurred when the i th symbol of the regular operation symbol sequence is not blank and the i th symbol of the user operation symbol sequence is blank; a deletion cost incurred when the i th symbol is not empty, a replacement cost incurred when both the i th symbol of the regular operation symbol sequence and the i th symbol of the user operation symbol sequence are not empty, and the normal operation symbol sequence And outputting any one of matching costs incurred when the i-th symbol of i and the i-th symbol of the user motion symbol sequence coincide with each other. The method of claim 1, In the step (e), if the same symbol is repeated in the user action symbol sequence or the normal action symbol sequence, the step (e) is converted to display the repeated symbol only once and compared with the user action and the normal action. Extracting motion similarity of Hidden Markov model based motion recognition method comprising a. The method of claim 1, In the step (e), (e-3) extracting a partial motion similarity between the user motion and the normal motion by comparing the user motion symbol sequence with the normal motion symbol sequence when the user motion is a partial motion. Hidden Markov model based motion recognition method comprising a. A computer-readable recording medium having recorded thereon a program for realizing each step of the hidden Markov model-based motion recognition method according to any one of claims 1 to 5 and 13.

Images