TW200832237A - Human activity recognition method by combining temple posture matching and fuzzy rule reasoning - Google Patents

Abstract

The present invention relates to a human activity recognition method, particularly to a method which uses temple posture matching and fuzzy rule reasoning to recognize an action. First of all, a foreground subject is extracted and converted to a binary image by a statistical background model based on frame ratio, which is robust to illumination changes. For better efficiency and separability, the binary image is then transformed to a new space by eigenspace and canonical space transformation, and recognition is done in canonical space. At least two image frame sequence is converted to a posture sequence by template matching. The posture sequence is classified to an action by fuzzy rule inference. Fuzzy rule approach can not only combine temporal sequence information for recognition, but also be tolerant to variation of action done by different people. Therefore, the present invention is able to achieve higher recognition accuracy than the prior art.

Description

200832237 IX. Description of the invention: [Technical field to which the invention belongs] = The main purpose of the invention is to provide a method for recognizing human motion, in particular with respect to one kind. The heart order money, _ ageless paradox to the silk man _ made identification. [Prior Art]

Human actions rely on automatic surveillance _, human face, home security, and smart home ring _ _ possessed a major position. Many human action surfaces use a single-image _ to the action. However, at time _, the relationship of posture state transition is important information for identifying other people's actions. Recently, there have been many methods for recognizing human motions. The method of human motion recognition can be based on the features used in the method. The knife is used in two categories. The first class uses the momentum feature (m〇ti〇n—to (four) Gam (10)). In "Cong Trans. Pattern Anal, ν〇ΐ· 23, η〇·3, Lan,, the paper, B〇bick and the momentum energy (IQne resistance) and the previous momentum statistics using the template in the plate 4 Motion-history to discern human movements. In "pr〇c. c〇nf,

Comput. Vision

Pattern Recog·, ν〇1· 4, ρρ· 38-45, 2〇〇3” In the paper, R Hamid et al. used features in time and space, such as the attachment of two hands to the distance and between the two hands. The relative speed 'in addition to the dynamic Bayesian netw〇rks to complete the human motion recognition work. On the other hand, the use of two-dimensional and three-dimensional shape features to identify human actions. In ΪΕΕΕ Comput· Soc· Workshop Models versus Exemplars in Comput· Vision, pp. 263-270, 2003” uses the data obtained by the Canny edge detector to represent the contour shape of the motion and defines for each motion. Key frame. "IEEE Int· Workshop on Anal· Modeling of 200832237

Faces and Gestures, ρρ· 74-81, 2003” The authors of the paper proposed the use of SVM and the two-dimensional shape feature description of the angle of view to achieve the classification and recognition of human actions. The shape-based feature information is used to complete the motion recognition work. The timing information is not taken into account, so there are still many actions that cannot be clearly identified. Therefore, this prompts us to design a stable and robust method to utilize humans. Implicit inherent timing information in the action to achieve more accurate motion recognition. With enMark〇vM〇dei ( _ )

Ability to process timing data and to provide the ability to identify the effects of scale changes on the scale. • Hidden Markov Model ^ o MM is cost effective and must collect large amounts of data and spend a lot of time in estimating HMM Parameters. The present invention extracts features in an image by using feature space conversion (Eigenspace transf_ti〇n) and standard spatial transform transform (因而onical space transformati〇n), and thus can convert temporal sequences into posture sequences by using these spatially transformed vectors. It can be represented by a combination of index values for the template template category. Because if there is less difference in the change of Luna _» _ money in a short interval, the red is limited by the inherent natural freshness of the person's age, and the frequency is not high; therefore, we use the reduction Sampling the frequency image without using all the pictures for identification. In addition, the present invention further proposes a method for inferring using a fuzzy rule. This method can not only combine the sequence information of human motion, but also the fuzzy rule can cover the slight differences in postures of different people when performing the same action, and the stability of the recognition effect. SUMMARY OF THE INVENTION The main object of the present invention is to provide a method for identifying people, which is related to a kind of 200832237, (the combination of the timing and the contrast of the silky people.) The person of the present invention makes a step of identifying the green containing town. First, a photographing device is set up at a point to capture the original picture of the t point and a learning-background picture is created, and then the image of the _« 定 定 频率 , , , , , Reduce the sampling frequency = the sequence of the & like the input screen, each of the input screens in the sequence and compare it with the background surface to extract the foreground image. Then take each (four) foreground image Performing the domain swapping, and then transforming the foreground image after the implementation of the vacant _ two & air-converted into a sequence of singular order. Finally, the group of timing hybrids (four) _ _ _ _ _ _ Into a certain person's action _. This method can not only combine the position sequence of the person's gestures, but also the ambiguous logic method can cover the differences in the postures of different people when doing the same action. The advantages of the fuzzy rule also include the scale. Pose mode _ hidden mode The purpose of the present invention, the technical contents, the features and the effects achieved by the present invention will be more readily understood by the following detailed description of the embodiments of the present invention. [Embodiment] The main object of the present invention is to provide - A method of combining the time series (4) _ and _ law to complete the recognition of human motion. The first figure is the system architecture diagram of the present invention, the main Wei structure can be divided into three wires. The first process S110 is the extraction of foreground image characters The second process S120 is to spatially transform the image onto a space with a smaller dimension and more easily recognizing the gesture. The third process S130 is to classify the pose of the single-image and identify the female sequence. The category to which the unknown action belongs. The first step of the human motion recognition system is to extract the foreground character part. In the first flow 8 200832237, the first step S111 is to construct a background model for the foreground character extraction. Use the connected image division method to describe a statistical background model. Calculate the maximum ratio of the largest and smallest grayscale values and the grayscale values of the image. It is possible to obtain a background model. The present invention proposes an image division ratio method that has been confirmed to be less sensitive to illumination changes, without causing the female aberration method to establish a background module. The image intensity can be expressed by Equation 1. I^x^)-Sl(x, y)ri(x9y) (1) where S represents a certain pixel position smoothness, and one represents a certain pixel position reflectance, and 2 Represents the index value in the image sequence. If you only shoot the background and keep the camera stable, the effects of reflectivity will exist. However, if image separation is used, the effect of reflectivity can be eliminated. In addition to the two shadows, they can write their pixel strong values as equations 2 ° log

A-xi^y) log ^Ms^y)h\〇g{sM(s9y)) (2) Therefore, the present invention proposes a ratio of side image division to construct a background flip. Each pixel of the background image is represented by three statistical values: the minimum grayscale intensity value, for example, the maximum grayscale intensity value (4) and the maximum tree image fineness. The three health_lin methods are shown in Equation 3. 9 (3)200832237 max {it (x9 y)} min {/, (jc,^)} max {/,(x, less)//b|(x,j〇} I . otherwise max {/^i Min {h{x9y)} max {/,_! (x, ^)//^ (x, j)} .* . After establishing the background model, steps S112 to S113 are to make the foreground character part available from each Separated from the image. The pixels in an image are classified as foreground or background, mainly based on the results obtained in Equation 4. 0, a background pixel if ^ =* h (xty)/m{x,y)<kd(x,y) or h {x,y)/n{x,y) < kd{x,y) (4) 1, a foreground pixel otherwise where the equation B represents a binarized image after the foreground character is separated, and the 々 value is an adjustable parameter. In general, k=l·4. The block of the foreground character can use the projection statistics of the X axis and the γ axis to set a threshold value. According to the boundary position of the statistical value, the image cut out of the foreground portion can be adjusted to the same size. If the two images are captured in a short interval, the difference in posture between the two images is not too great. In addition, the human body is about a rigid body, so it has its natural frequency; in other words, when it is a certain action, it has a natural limit on the speed of action. In our method, we take an image at regular intervals, called the basic template image. The second picture shows an example of selecting a sample image. The detail is five sets of basic left 200832237 to the right walking miscellaneous record at the time interval (4). The basic view of the miscellaneous axis feature lang and the standard space conversion projection onto a new space as shown in the two processes sl2〇 are done in the standard space. The entire process of identification, in video and image processing, the dimensions of image data are usually very large. Since there are a lot of parts in these images, the way to solve these redundant parts is to project the image onto a new space via space conversion. Most of these conversion methods use a smaller number of dimensions to get the original image. In the second flow measurement, the conversion method of the present invention combines the secret space conversion of the 5121 (Ei_space t_f〇 and the "commutation of the work" (Ca_lcal space transf_ti〇n). The feature space conversion has been effective. It is used in the system of automatic face recognition and transportation, and after the feature space is converted, the secret space is exchanged to reduce the separability between the two dimensions and the optimized two categories. And improve the performance of the identification system. The identification is done in the standard space. Suppose there are a total of e__ learning materials. Each face represents the posture pattern of a certain action in the entire training image data. The ith image of the i-th class, the total number of images in the entire training set can be expressed as ~^code+code. The entire training set--, the text is written as x丨~4, each of which contains η The pixel's parameters. First, the intensity of each image will be normalized as shown in Equation 5. Ά.

Ml (5) The average pixel value of each image can be obtained by Equation 5 and expressed by Equation 6. (6) 200832237 7

The same % training 'can set can be rewritten as _ one dimension is _ two turns, that is, the equation gas = ί XL 1,1 -1ϋχ,···,χ1,ΛΓι—πιχ,···,χ〇,Λ ^-Ιη 1 T (7) Assuming that the maturity of the matrix xxT is κ, the matrix χχΤ will have a non-zero eigenvalue K of K, and its corresponding eigenvector, W” W and in accordance with Equation 8. (8) 4U, Ru/, / = 1, 2,..., κ Equation 8 (4) R = XXT, ^ face is - touch _ bar is relatively small, the shadow is generally small _ ruler nucleus, this lion will be non-fA, cut and scale operation is very large复^Μ^(8ίη^Γ value decomp〇siti〇n the〇ry)^^ The eigenvalue and eigenvector are obtained by counting the matrix and expressed by Equation 9. R = xTx (9) The dimension of the state matrix is one. It is much smaller than the _ of the original R matrix. This tilt matrix is also 曰/K non-zero special f plants, with a, and the corresponding feature vector, with stone, then Use the persimmon 10 to obtain the R moment _ eigenvalue with the ship matrix. λ / = λ / (10) e ^ ^ pXe, , in the formula μ 'κ, then κ represents κ feature vectors for mutual positive Crossed vector. Based on Principal component analysis (10) ncipaI called nent anaiysis) The result of each theory can be derived from the axis. These 12 200832237 and can paste equation 11 to extend the original k signs to a new space, the image is projected into a new space Yij=[ei,e2,-,eA]TXi_ where /= 1, 2,...,c and hi, 2d change the vector. (11) We call the [~~·Α]Τ matrix The space is calculated using Equation 12 Ντ ΣΣ ytj (12) where ί=1'2'...',=u'.'. The average vector for each category can be calculated using Equation 13. m<=忐Σ~. y/ Sickness (13) • After the heart has an average vector, _ can be the following three numbers: St for the total scatter _Ηχ, & for the intra-group scatter matrix (coffee), Sb for inter-category distribution The matrix plus the order %%. These three variables can be calculated by the following equations.

St=right (eight)Ί~)τ 1 c Σ6,,factory 1 »c_

Sb=^L^(m/-«n,)(m/-,nj/)T 13 200832237 The main purpose of standard space conversion is to minimize the scatter matrix in the group and the scatter matrix between classes. To get this result, it can be calculated by Equation 14. J(w)=^w. WTSwW (14) The solution is to choose a maximal solution W to conform to Equation 15.

It is assumed that W* is the best solution, and the <vector is the feature vector obtained with respect to the i-th solid maximum eigenvalue. Equation 15 can be rewritten into equation i6 according to the theory mentioned in the book "/Step Wall Ice, 1990".

Sbw* =XZ-Sww* (16) After the solution, you can get η non-zero eigenvalues and their corresponding eigenvectors [heart, ~]. To thin this base, we can project a point on the feature space to another point on the standard space, as shown in Equation 17. Άι'ν2,.(10) , ^^^^Κν2,...,νε_ι]ΤΜι]^„^ΓΒΐ# Change matrix. ...~,5方&11 and 17 'Mother Zhang Jing multi-image can pass the equation Β 投景 has a new space in c- 1 dimension. Z/j=HX/j.. (18) 14 200832237

The third process S130 is mainly for the identification process. In the time image sequence, the transition relationship between different postures is a rewarding human action - a very important feeding 4. We only use the - image to make a fine-grained basis, then Lai's silk material is wrong, __ different movements may be similar to the posture image. The single-pose of human action is often ambiguous, so in the footsteps to the Qing, we propose a fuzzy method to listen to human motion recognition, not only this enough, 'Ό σ time series information, but also can tolerate different people - The difference in actions. The papers related to the theory of reliance are as follows. In the paper "臓Trans.fine., age η, ν〇1. 22' no. 6' (10) and this Xie proposed to learn from the examples to produce fuzzy ΊΕΕΕ Trans. Sys., Man Cybern, A, vol. 30 , no. 2« 中' Su proposed to use the method of grading the county base _ method to identify the position of the vocabulary. False '^ 疋 the first space category of the first model image of the space transformation vector, and the tile is ^- ^.1 0 ^iri#.j^,^g^^^#t(membership If the sister 010 comes to the silk--Zhang M-Yu--the possibility of reluctance, this membership function can be expressed by Equation 19. Exp 2 a (19) J represents the covariance matrix of a <c〇variance pulse (4) and the heart represents the average vector. The dimensions of my Ha and heart are independent of each other, then the equation can be rewritten as square 20.

15 (20) 200832237 /, where m represents the index of the dimension, ~ is the standard deviation of the first dimension obtained using all the average vectors, and r represents the membership of the image threat, and the membership of the health_. Finally, the pose category to which this image belongs is expected and is shown by Equation 21. / = arg max r 1 1 (21) The parent image will be replaced by a text symbol 1 to represent it. The i here refers to the index value of the pose category. In order to contain timing information, we combine three consecutive reduced sampling frequency images into one, and as (k>, 13). If this _ group of data uses too many images, then the action of _ a quick completion of the action cycle is easily affected by its phase; if the wire is used too little _ image, the information of the sequence will be insufficient. First, the three images are projected onto a standard (four) towel, and then converted into a sequence of gestures based on the value of the membership function. Then, the action sequence corresponding to the gesture sequence is (U k h, D) ′ D is the action (Action Category). Such a posture order 冓 冓 糊 法 系统 系统 系统 系统 系统 系统 系统 系统 系统 系统 系统 系统 系统 系统 系统And the system through this set of input rounds the _ pair 'generates the corresponding rule of the group, its form is as follows IF antecedent conditions hold, mm consequent conditions For example, as shown in Figure 2, the pose sequence of the first image It belongs to ^, the second 丨 belongs to P19, 帛 two sheets of f彡 like threats p2(), and its

It will be calculated by the equation w, the coffee paste, and the table H (/p/2?/3; D) = (/&gt;85/, 9) P2〇; ^ 200832237 This will produce the following rule IF the activity's Ii is Pi8 AND its h is Pi9 AND its L· is p2{), then the activity is Wlr” Figure 3 shows the structure of the system classification algorithm. First, every time a fixed time interval, go to the green to take an image such as S300. The image is spatially converted into the standard space and its corresponding membership function values are calculated as S305 to S310. Then, as shown in steps S3I5 to S325, the membership function values of each of the three images are grouped into one group. The fuzzy law database obtained by the training finds a set of the most similar pose sequences. This image sequence is classified into the action categories recorded in the most similar rules. Our experimental environment is a classroom. The light source is a stable fluorescent lamp. Use a simple background with only one table. The camera is fixed at one location and the same scene is taken without moving. The camera will shoot 30 images of 640x480 pixels per second. There are six actors, each doing the same. The following six actions: "Left to the right" (5), "From right to left" (5), "Jump", "Crgh", "Cup" and "攸下" ((5). Five of the films are for training, and the rest are for testing. Each person's film will be taken for testing. Figure 4a is - Zhang captured by the camera, the first picture is the foreground The result of the separation of the characters and the conversion to binarized images. Detailed _ six _ basic (four) sample images for "from left to right", "from right to left" and "squat", five to "crawl" "The next three classes are for "Kneeling" and the two classes are for "jumping". The total is 28 categories. After all the test videos have passed the above training, we must set a threshold value, which is used to eliminate the money. The rule that the number of occurrences of certain posture sequences is relatively small, the height of the threshold of 200832237 will affect the number of rules. Figure 5 is a comparison of the difference between the value of the value and the identification result. Figure village coffee, tf still handsome _ solution Green, in our experiment, 'we use three thresholds, because if the gate is used Too low, then the combination is too ^ then (10) the general inspection w, t - some contradictory, i generated, we choose the law that occurs more frequently during training. Table - shows the recognition rate of our system. The motion recognition system is currently in the offline test state 'also _ 彳 彳 。 。. 目 5: 丨 New Zealand sampled image steps, we read the different starting timing position, i to 5 position in the test Taking images and identifying them is similarly considered. This is the same as when we are training fuzzy rules, and it is similar to what happens when instant recognition occurs. For example, the younger, second, third, fourth, or fifth image in our movie begins to be read for identification. In the paper "IEEECVPR, PP. 379-385, 1992", Yamat〇 and 〇_ proposed the use of HiddenMarkQvMQdel as a reward. With enMarkQvMQdei (ugly) is the probability of a kind of conversion, usually used for timing (four) material analysis. In the experiment we compare the recognition rate of the fuzzy rule method with the HMM method. Action recognition rate of all actor 200832237

- ---- Inner, &quot;eight w 曰 、 w, j r hundred paragraphs â€ΗΜΜ Each must be trained to produce the posture conversion probability parameter that best represents a certain type of action. In the experiment, the fine BaUm-Weleh calculus produced _ _ off parameters, and we used the forward linkage method (f_rd-chaining), the number of states was set to 28, and the length of the last observation sequence was set to three. . After the training materials have been learned, six groups of actions can be obtained. When identifying the observed phase axis of an unknown action, the unknown action is classified into a HMM class that can generate the maximum probability value from the six HMMs, that is, the closest. We use a forward algorithm to calculate the probability value. A comparison of the recognition rates between the HMM algorithm and the algorithm based on the fuzzy rule is shown in 200832237 in Table 2. The method of calculating the fuzzy law is that the fuzzy law is higher in the human 貊μ, and the spoon is still 2.4%. This __ is called coffee recognition effect.

The experiment, I fruit does not show the total recognition rate of I and seed movements can reach 9178% without reference to any position, moving path and moving speed. Compared with the painting method, the fuzzy law algorithm improves the accuracy of about 2.4%. The above description is only a preferred embodiment of the invention and is not intended to limit the scope of the invention. Therefore, any changes or modifications in the shape, structure, and spirit of the shapes described in the scope of the present invention are included in the scope of the invention. 20 200832237 [Simple description of the diagram] Fig. 1 is a schematic diagram of the system architecture of the present invention. Figure 2 is a schematic diagram of an example of selecting a sample image. Figure 3 is a schematic diagram of the structure of the system classification algorithm. Figure 4 is a schematic diagram of a sample of prospective characters. Figure 5 is a comparison of different threshold values and identification results. [Main component symbol description] Lu Wu

twenty one

Claims (1)

200832237 X. Applying for a patent garden: 1· A human motion recognition method, including: taking a device at a point to capture the original image of the fixed point and establishing a learning_Northern picture; the month is not equal to the camera as the fixed frequency The time interval is taken to capture the image, and after the sampling frequency is lowered, the sequence h image is rounded into the picture, and each input in the sequence is input and compared with the background picture to take out the foreground image; _ the foreground of the parent piece is taken out The image performs spatial transformation, and then converts at least two consecutive foreground-converted foreground images into a set of time-series pose sequences; and classifies the set of time-series poses into a certain action based on the human motion recognition based on the fuzzy rule inference. category. 2. The human motion recognition method of claim i, wherein the spatial transformation comprises feature space conversion and standard space conversion. 3. The human motion recognition method according to claim i, wherein the extracted foreground φ image is a binarized image. 4. If you apply for a patent scope! The human motion recognition method according to the item, wherein the modular law recognizes the fuzzy rule by using a lower sampling frequency of at least one image sequence as an input and combining at least one of the basic human poses. 5. * Apply for the special female encyclopedia _ _ people _ as the identification method, which can be combined with the time-sequence information 'and can tolerate the difference between different people doing the same action. 6. The human motion recognition method according to claim 1, wherein the fuzzy rule comprises a human motion hidden mode in which a gesture transition relationship can be learned. 22 200832237 7. The patent as claimed in claim 1 wherein the human motion recognition is done in a standard space. 8. The human motion recognition method of claim 1, wherein the background model uses a connected image division method to describe a statistical background model. 9. The human motion recognition method according to claim 8, wherein the statistical background model transmits, calculates a statistical maximum and minimum gray scale value and a maximum ratio % value of the connected image gray scale value to obtain a background. model. • 1〇·The human motion recognition method according to item 8 of the patent application scope, wherein the moon image pixels of the statistical background model are represented by two statistical values: a minimum gray scale intensity value and a maximum gray scale intensity value. Divides the ratio of the maximum connected image grayscale value. 11. The human motion recognition method of claim 2, wherein the human motion recognition utilizes a Gaussian membership function to represent the likelihood of each image relative to each gesture category. ♦ 23