CN112906438B - Human body action behavior prediction method and computer equipment - Google Patents

Abstract

The invention provides a prediction method of human body action behaviors and computer equipment, wherein the prediction method comprises the following steps: frame-dividing and sampling the human body action period image information to obtain an image sequence with reduced frame number; sequentially aiming at the images of a plurality of frames of the image sequence according to the ordering of the frames, establishing a skeleton model aiming at each frame of image, and determining the actual values of a plurality of characteristic angles of the skeleton model; according to the reference values of a plurality of characteristic angles corresponding to a plurality of preset human body action behavior categories obtained through pre-training and learning, actual values of a plurality of characteristic angles of the skeleton model are compared with the reference values of a plurality of characteristic angles corresponding to the preset human body action behavior categories; and determining the human action behavior category of each frame of image according to the comparison result. The invention can improve the recognition accuracy of human motion behavior prediction.

Description

Prediction method and computer equipment of human action behavior

technical field

The invention relates to the fields of computer vision technology and pattern recognition, in particular to a method for predicting human action behavior and computer equipment.

Background technique

The rapid development of artificial intelligence has made computer vision technology widely used in many fields such as video surveillance, motion retrieval, human-computer interaction, smart home, and healthcare. Human behavior recognition technology has achieved certain results, which can accurately identify and classify simple human behaviors, such as walking, running, bending, waving and other actions. In the process of action behavior detection, it is often hoped to quickly obtain what one needs and reduce unnecessary waste of time. Therefore, the key frame retrieval of action behavior in video sequences is of great significance. The prediction of human action behavior is one of the important foundations.

Human action prediction is to guess the category of the action that has not yet completed the overall action as much as possible. Compared with the traditional human action recognition, it lacks the timing structure of the complete action. Ryoo et al. extracted the middle-level features related to the temporal structure of actions through improved two bag-of-words models, and took the lead in proposing the problem of action prediction. Kong et al. proposed the concept of multi-time scales, divided the video into a fixed number of segments, extracted the dynamic features of the behavior one by one from the video segments, and completed the ability to discriminate incomplete actions by constraining the temporal relationship of actions. Xu et al. used the idea of automatic completion in information retrieval to propose an automatic action completion method. Each segmented video segment is equivalent to a single character of a sentence, and the incomplete video constitutes the prefix of the sentence. Prediction of actions in videos. Known human action behavior prediction algorithms have low recognition accuracy.

Contents of the invention

In view of this, the present invention provides a method for predicting human action behavior and computer equipment, so as to improve the recognition accuracy of human action behavior prediction.

On the one hand, the present invention provides a method for predicting human action behavior, including: performing frame-by-frame sampling on the image information of the human action cycle to obtain an image sequence with a reduced number of frames; The image of frame carries out following operation: set up skeleton model for each frame image, and determine the actual value of a plurality of characteristic angles of described skeleton model; According to the multiple characteristic angles corresponding to a plurality of predetermined human action behavior categories that pre-training learning obtains The reference value is to compare the actual values of the multiple characteristic angles of the skeleton model with the reference values of the multiple characteristic angles corresponding to the predetermined human action category; determine the human action category to which each frame of image belongs according to the comparison result.

Further, the above-mentioned establishment of a skeleton model for each frame of image, and determining the actual values of multiple characteristic angles of the skeleton model, includes: establishment of a skeleton model for each frame of image, and the skeleton model includes an upper limb part, a lower limb part and a middle part; the upper body part includes: left hand, left elbow, left shoulder, right hand, right elbow, and right shoulder; the lower body part includes: left foot, left knee, right foot, and right knee; the middle part includes the crotch ;

Determine the actual value of the first characteristic angle formed by the left hand, left elbow and left shoulder, the actual value of the second characteristic angle formed by the left elbow, left shoulder and crotch, and the third characteristic angle formed by the right hand, right elbow and right shoulder The actual value of the angle, the actual value of the fourth characteristic angle formed by the right elbow, right shoulder and hip, the actual value of the fifth characteristic angle formed by the left foot, left knee and hip, the actual value of the right foot, right knee and hip The actual value of the 6th characteristic angle formed by the upper body, the actual value of the 7th characteristic angle formed by the left knee, the crotch and the right knee; wherein, the first to the fourth characteristic angles are derived from the upper limb part , the fifth to sixth characteristic angles originate from the lower limb part, and the seventh characteristic angle originates from the middle part.

Further, according to the reference values of the multiple characteristic angles corresponding to the multiple predetermined human action behavior categories obtained through pre-training, the actual values of the multiple characteristic angles of the skeleton model and the multiple predetermined human action behavior categories correspond to Comparing the reference values of the characteristic angles includes: sequentially judging whether the actual value of the nth characteristic angle is within the preset deviation range of the reference value of the nth characteristic angle corresponding to the predetermined human action category, where n is 1-7 Integers between; For each frame of image, count the number of each characteristic angle within the preset deviation range of the reference value of the 1st to the 7th characteristic angle of a specific human action behavior; the number is the comparison Result; the specific human motion behavior is any one of the predetermined human motion behavior categories.

Further, the above-mentioned determination of the human body action category to which each frame of image belongs according to the comparison result includes: if the number is greater than or equal to 5, initially determining the human body action category to which the image of the corresponding frame belongs to the specific human action Behavior; for each frame of image, the number of characteristic angles that are not within the preset deviation range of the reference value of each characteristic angle of a specific human motion behavior is two, and are not located in the reference value of each characteristic angle of a specific human motion behavior When the two characteristic angles within the preset deviation range are only from any one of the upper limb part, the lower limb part and the middle part, the human action behavior category to which the image of the corresponding frame belongs is finally determined as the specific human action behavior .

Further, the above-mentioned skeleton model is a tree structure model connecting the left hand, left elbow, left shoulder, right hand, right elbow, right shoulder, left foot, left knee, right foot, right knee and crotch.

Further, the above-mentioned preset deviation range of the reference value of the nth characteristic angle is the range of degrees of the reference value of the nth characteristic angle.

Further, the above-mentioned benchmark values of multiple characteristic angles corresponding to the predetermined human action behavior categories obtained through pre-training and learning through the following operations specifically include: inputting action behavior training and learning data sets; extracting feature information of the training and learning data sets; Inputting the feature information into a support vector machine for classification to obtain the action behavior category of each training and learning data; the action behavior category of each training and learning data is a predetermined human action behavior category; according to the action behavior corresponding to each type of predetermined action behavior category The training learning data determines the reference values of the plurality of characteristic angles corresponding to the predetermined action category.

Further, the above-mentioned frame-by-frame sampling of the image information of the human body motion cycle to obtain an image sequence with a reduced number of frames includes: for the image information of the human body motion cycle, the first five frames are fused into one frame, and sampling is performed at intervals of ten frames. The five frames are fused to obtain an image sequence with a reduced number of frames.

On the other hand, the present invention also provides a computer device, which includes a processor, and the processor implements the above-mentioned method for predicting human action behavior when executed.

In yet another aspect, the present invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the above-mentioned method for predicting human action behavior is realized.

The method for predicting human action behavior and the computer equipment of the present invention can reduce the calculation burden by performing frame-by-frame sampling on the image information of the human action cycle to obtain an image sequence with a reduced number of frames; The image of each frame is carried out as follows: set up skeleton model for each frame image, and determine the actual value of a plurality of characteristic angles of described skeleton model; According to a plurality of characteristic angles corresponding to a plurality of predetermined human action behavior categories that pre-training learning obtains Comparing the actual values of multiple characteristic angles of the skeleton model with the reference values of multiple characteristic angles corresponding to the predetermined human action category; determining the human action category to which each frame of image belongs according to the comparison result . The reference values of multiple characteristic angles corresponding to the predetermined human action behavior categories obtained through pre-training and learning are determined to determine more accurate reference values of multiple characteristic angles, and then the characteristic behavior is performed by comparing the results with the reference values of multiple characteristic angles. Prediction to improve the recognition accuracy of human action behavior prediction.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. Those of ordinary skill in the art can also obtain other drawings based on these drawings without any creative effort.

FIG. 1 is a flowchart of a method for predicting human action behavior according to an exemplary first embodiment of the present invention;

Fig. 2 is a flowchart of a method for predicting human action behavior according to an exemplary second embodiment of the present invention;

Fig. 3 is the flow chart that utilizes SVM classification to obtain each predetermined action behavior category in Fig. 2;

Fig. 4 is the flow chart of predicting the human action behavior category to which each frame image belongs among Fig. 2;

Figure 5a and Figure 5b are divided into schematic diagrams of the skeleton refinement of the 15th frame and the 25th frame;

Fig. 6 is the skeleton model schematic diagram that each frame image is established;

Fig. 7 is a schematic structural diagram of a computer device according to an exemplary third embodiment of the present invention.

Detailed ways

Embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings.

It should be noted that, in the case of no conflict, the following embodiments and the features in the embodiments can be combined with each other; and, based on the embodiments in the present disclosure, those of ordinary skill in the art obtained without creative work All other embodiments belong to the protection scope of the present disclosure.

It is noted that the following describes various aspects of the embodiments that are within the scope of the appended claims. It should be apparent that the aspects described herein may be embodied in a wide variety of forms and that any specific structure and/or function described herein is illustrative only. Based on the present disclosure one skilled in the art should appreciate that an aspect described herein may be implemented independently of any other aspects and that two or more of these aspects may be combined in various ways. For example, any number of the aspects set forth herein can be used to implement an apparatus and/or practice a method. In addition, such an apparatus may be implemented and/or such a method practiced using other structure and/or functionality than one or more of the aspects set forth herein.

As shown in Fig. 1, a kind of prediction method of human action behavior of the present invention comprises:

Step 101: Perform frame-by-frame sampling on the image information of the human body movement cycle to obtain an image sequence with a reduced number of frames;

Step 102: according to the ordering of each frame, the following operations are performed for the images of a plurality of frames of the image sequence:

Step 102a: establishing a skeleton model for each frame of image, and determining actual values of multiple characteristic angles of the skeleton model;

Step 102b: According to the reference values of multiple characteristic angles corresponding to multiple predetermined human action behavior categories obtained in pre-training, compare the actual values of the multiple characteristic angles of the skeleton model with the multiple features corresponding to the predetermined human action behavior categories The benchmark value of the angle is compared;

Step 102c: According to the comparison result, determine the human action category to which each frame of image belongs.

This embodiment obtains the image sequence after the number of frames is reduced by performing frame-by-frame sampling on the image information of the human body movement cycle, which can reduce the computational burden; according to the ordering of each frame, the following operations are performed on the images of a plurality of frames of the image sequence: Establish a skeleton model for each frame of image, and determine the actual values of a plurality of characteristic angles of the skeleton model; The actual values of the multiple characteristic angles are compared with the reference values of the multiple characteristic angles corresponding to the predetermined human action category; and the human action category to which each frame of image belongs is determined according to the comparison result. The reference values of multiple characteristic angles corresponding to the predetermined human action behavior categories obtained through pre-training and learning are determined to determine more accurate reference values of multiple characteristic angles, and then the characteristic behavior is performed by comparing the results with the reference values of multiple characteristic angles. Prediction to improve the recognition accuracy of human action behavior prediction.

Figure 2 provides a preferred implementation of a method for predicting human action behaviors in the present invention. This embodiment mainly includes two major processes. The first process is the multiple characteristic angles corresponding to the predetermined human action behavior categories obtained through pre-training and learning. The reference value, the specific steps correspond to step 21-step 32 in FIG. 2; the second process is a process of specifically identifying the image information to be predicted according to the reference values of multiple characteristic angles. As shown in FIG. 2-FIG. 6, another method for predicting human action behavior in the embodiment of the present invention includes:

Firstly, combined with SVM, the benchmark values of multiple characteristic angles corresponding to predetermined human action categories are obtained by modeling. In this embodiment, SVM is used to identify daily human action behaviors from a large number of public action behavior data sets, so as to obtain predetermined categories of human action actions. Due to the high accuracy of SVM in the recognition of various human daily actions, according to the recognition results of SVM, it is possible to carry out feature modeling for each predetermined human action category, and obtain the benchmark of multiple feature angles corresponding to the predetermined human action category value.

For the process of identifying predetermined human action behavior categories according to SVM, refer to step 21-step 24 for details, and also refer to step 301-step 304 in the process shown in Figure 3;

Step 21: Input the image information of human body motion cycle, which includes but not limited to video;

Step 22: Preprocessing, such as filtering and denoising and other signal routine processing, will not be carried out here;

Step 23: Extract feature information from the image information of the human motion cycle; specifically, the HOG (Histogram of Oriented Gradient) method can be used to extract the feature of the action behavior, and the horizontal gradient and vertical gradient of the pixels in the image can be calculated respectively , find the gradient magnitude and gradient direction, and use the histogram for feature extraction. For example, the horizontal and vertical gradients of pixels (x, y) in the image are:

D _x (x,y)=H(x+1,y)-H(x-1,y);

D _y (x,y)=H(x,y+1)-H(x,y-1);

Among them, H(x, y) is the pixel value of the pixel point in the input image at (x, y); the gradient magnitude and gradient direction at the pixel point (x, y) are:

α(x,y)=tan ^-1 (G _y (x,y)/G _x (x,y));

Step 24: Input the feature information into a support vector machine (SVM) to classify actions, and obtain classification results of the predetermined action categories.

The essence of SVM is to transform the problem of maximum geometric compartmentalization into a convex function for mathematical solution optimization. This embodiment uses multiple binary classifiers to construct a multi-classification method to realize the classification of various human action behaviors. The recognition results of SVM in the two action behavior datasets of KTH and Weizmann:

Step 25: Perform frame-by-frame extraction on correctly classified motion behavior sequences;

Considering that action training often has data redundancy due to high-speed acquisition, this embodiment preferably adopts the processing method of sampling and dividing into frames to reorganize the action sequence, reduce the amount of calculation of follow-up work, and reduce the complexity of the process. The first five frames can be fused into one frame, sampled every ten frames, and the five sampled frames are fused, and finally the behavior sequence is divided into image sequences with fewer frames; for example, a class of action behavior video segmentation An action sequence of 750 frames is divided into 50 basic units according to each 15-frame unit, and the first 5 frames of each basic unit are sampled and fused into 1 frame of image, and so on, the regularized The sequence is a new sequence containing 50 frames of images.

Step 26: Carry out skeleton thinning processing to the human body appearance model in the image (the model shown in Figure 5a and Figure 5b, wherein the left picture in Figure 5a and Figure 5b is a two-dimensional image, and the right picture is an image skeleton) (such as As shown in Figure 6);

Considering that the human body is a non-rigid structure, for the execution of action behaviors, the most significant action features mainly focus on the movement of the limbs of the human body. There is also a certain correlation between the changes. For example, when running, the bending angle of the knee has a certain correlation with the angle of the hands and shoulders; the angle of the hands and shoulders has a certain correlation with the bending angle of the elbow wait. Therefore, this embodiment extracts the angle change of the limbs of the human body as the main angle feature. Specifically, as shown in FIG. 6 , the skeletonized image can be divided into three parts according to the graphic structure: the upper limb part, the lower limb part and the middle part. The upper limb part includes: left hand, left elbow, left shoulder, right hand, Right elbow, right shoulder; lower limbs include: left foot, left knee, right foot, right knee; middle part is crotch;

Step 27: Calculate the feature angle to realize modeling;

First, extract the angle feature angle of independent joint points, which are divided into left hand-left elbow-left shoulder, left elbow-left shoulder-crotch, right hand-right elbow-right shoulder, right elbow-right shoulder-crotch; left foot-left Knee-crotch, right foot-right knee-crotch; left knee-crotch-right knee seven characteristic angles;

Secondly, set the threshold range of the characteristic angle, establish the human skeleton angle characteristic model of this type of action behavior, and establish the angle characteristic model of various action behaviors in turn;

After a large number of experiments, it is concluded that the threshold setting of the selected characteristic angle and the allowable range of the error are selected, and the selection deviation of the error size is 5°C. For example, the action behavior characteristic angle threshold setting and allowable error of running are as follows:

After obtaining the reference value of the characteristic angle of each action behavior through the above-mentioned step 21-step 27 modeling, calculate the size of the characteristic angle of each frame image through the following steps 28-210 for each data to be tested, wherein step 28-step 30 basically Corresponding to the processing of steps 25-207. For the flow of steps 28-31, refer to the detailed flow in FIG. 4 .

Step 28: Perform frame-by-frame sampling on the image information of the human body movement cycle to obtain an image sequence with a reduced number of frames;

Step 29: set up skeleton model (thinning processing) for each frame image; Described skeleton model comprises upper limb part, lower limb part and middle part; Described upper limb part comprises: left hand, left elbow, left shoulder, right hand, right elbow and right hand shoulder; said lower body portion includes: left foot, left knee, right foot, and right knee; said middle portion includes crotch;

Step 30: Calculate the size of the characteristic angle, specifically determine the actual value of the first characteristic angle formed by the left hand, left elbow and left shoulder, the actual value of the second characteristic angle formed by the left elbow, left shoulder and crotch, the right hand and right elbow and the actual value of the third characteristic angle formed by the right shoulder, the actual value of the fourth characteristic angle formed by the right elbow, right shoulder and hip, and the actual value of the fifth characteristic angle formed by the left foot, left knee and hip value, the actual value of the sixth characteristic angle formed by the right foot, right knee and crotch, and the actual value of the seventh characteristic angle formed by the left knee, crotch and right knee;

Wherein, the first to fourth characteristic angles are derived from the upper limb, the fifth to sixth characteristic angles are derived from the lower limb, and the seventh characteristic angle is derived from the middle part.

Step 31: According to the reference value determined in step 27, determine the error range between the actual value of each characteristic angle and the reference value. That is: sequentially determine whether the actual value of the nth characteristic angle is within the preset deviation range of the reference value of the nth characteristic angle corresponding to the predetermined human action category, and n is an integer between 1-7;

For each frame of image, count the number of each characteristic angle within the preset deviation range of the reference value of the 1st to the 7th characteristic angle of a specific human action behavior; the number is the comparison result; the specific The human action behavior is any one of predetermined categories of human action actions.

Step 32: Perform action prediction according to the pre-set rules, and then predict the corresponding human action behavior for the next frame of image.

Specifically, the action behavior prediction can be performed through the template matching method. The specific steps include: the first step, rough judgment, compare the extracted angle features of the sequence to be tested with the template, and if 5 of the 7 features are at the threshold Within the variation range, this type of action behavior can be preliminarily predicted; the second step is fine judgment, and the angle feature matching is performed on the upper limb part, the lower limb part and the central part respectively. If there are two local angle features in the Within the threshold variation range, this type of action behavior can be accurately predicted. The rough judgment only compares 7 independent angle features, and the fine judgment is made on the basis of the rough judgment, that is, the overall feature angle judgment is added to the partial combination.

In this embodiment, the action behavior of running is taken as an example to demonstrate the predictive effect. First, extract the images of the 10th, 15th, 25th, 35th, 45th, and 50th frames in the action cycle, and then perform skeleton thinning processing on the extracted images to obtain the skeleton model. 5a and 5b are skeleton refinement diagrams of key frames; wherein, FIG. 5a is a skeleton refinement diagram of the 15th frame, and FIG. 5b is a skeleton refinement diagram of the 25th frame. The extracted feature angles are as follows:

Fine judgment: that is, the comparison of local overall feature angles. In this embodiment, three local overalls are set up: left hand-elbow-left shoulder, left elbow-left shoulder-middle (i.e. crotch), right hand-elbow-right shoulder, right elbow- Take the right shoulder-middle as a whole, that is, the upper limb part, and perform the first partial overall angle change comparison; take the left foot-left knee-middle, right foot-right knee-middle as a whole, that is, the lower limb part, and perform the second comparison. A local overall angle change comparison; the left knee-middle-right knee is taken as a whole, that is, the central part, and the crotch angle change is compared. In the fine judgment, the three local overall angle changes are all within the threshold range as the constraint condition. If the requirements are met, the prediction of this type of action behavior can be realized, and the key frame of the action in the video sequence can also be determined.

Through the screening of simple human daily actions through the public action behavior data set, the experimental prediction of 6 types of daily actions (running, walking, clapping, bending, waving, and upward waving) is carried out. The results are shown in the following table.

It can be seen that the prediction effect of running and walking is better than that of the other four types of behaviors. Among them, the specific behaviors of running and walking can be completely judged within one-half cycle of completion; while clapping, bending, waving upwards and The effect of waving on anticipatory behavior was only apparent after most of the behavioral cycle had completed. There are two reasons for this phenomenon: first, running and walking have a larger range of body movement than the other four types of behaviors during exercise, that is, the change of characteristic angle is more obvious; second, the predictive effect of running and walking behaviors The significant area is concentrated at the half-period moment, because the moment when the action execution is half completed is the moment when the behavior changes the most, and the change range of the characteristic angle is the most obvious.

In this embodiment, at first, the human action behavior in the selected data set is identified and classified, and the correct action behavior classification is carried out by using the method of HOG feature extraction and SVM classifier; then a skeleton model is established for each type of action behavior, and the specific steps include: This type of behavior is sampled at intervals in the action cycle, and the data set of this type of behavior is reconstructed, and the target in the image is processed by the skeleton thinning image processing method, and 7 independent angle features are extracted respectively, and the change of the feature angle is set. Threshold, establish a skeleton model, and complete model training; then extract feature angles from the action behavior data set to be tested, and make two judgments through independent angles and partial overall angles to achieve the purpose of prediction and the determination of action behavior key frames in this embodiment. The human skeleton model proposed in this embodiment can effectively represent the human body's action posture with a lower feature dimension; at the same time, the interval sampling and fusion frame algorithm can better remove redundant information and describe key actions more effectively, compared with the existing schemes. The frame-by-frame feature extraction requires less computation; in addition, through the secondary angle feature template matching, the classification accuracy of action behavior and the accurate prediction of key frames of action behavior can be further improved.

As shown in FIG. 7 , an embodiment of a computer device includes a processor, and when executed, the processor implements the method for predicting human action behavior shown in FIG. 1 or FIG. 2 above. Wherein, executing the method shown in Figure 2 specifically includes:

(1) Select the public dataset of action behaviors for action recognition, and use the HOG feature extraction and SVM classifier to correctly classify the actions;

(2) Firstly, frame-by-frame extraction is carried out on the correctly classified actions and behavior sequences. In this embodiment, frame-by-frame processing is carried out by using the interval frame-by-frame method, and the first five frames are fused into one frame, and sampling is performed every ten frames. Five frames are fused, and finally the action sequence is divided into image sequences with fewer frames;

(3) Then carry out skeleton thinning processing to the human appearance model in the image;

(4) The image of the skeleton is refined, and the human skeleton is divided into three parts according to the graphic structure: the upper limb part, the lower limb part and the central part. The upper limb part includes: left hand, left elbow, left shoulder, right hand, right elbow, right shoulder; lower limb Parts include: left foot, left knee, right foot, right knee; the middle part is the crotch;

(5) Angle feature extraction of independent joint points, specifically divided into left hand-elbow-left shoulder, left elbow-left shoulder-middle, right hand-elbow-right shoulder, right elbow-right shoulder-middle; left foot-left knee-middle, right Foot-right knee-middle; left knee-middle-right knee seven characteristic angles;

(6) Set the threshold range of the characteristic angle, establish the human skeleton angle characteristic model of this type of action behavior, and establish the angle characteristic model of various action behaviors in turn;

(7) Utilize the method in step (2) to step (6) to carry out segmentation and angle feature extraction to test sequence;

(8) Predict action behavior by template matching method. The specific steps include: the first step, rough judgment, compare the extracted angle features of the sequence to be tested with the template, if 5 of the 7 features are in the If it is within the range of the threshold value change, this type of action behavior can be preliminarily predicted; the second step is fine judgment, and the angle feature matching is performed on the three local parts of the upper body part, the lower body part and the central part respectively. If there are two local angle features If it is within the threshold variation range, this type of action behavior can be accurately predicted;

(9) Through the prediction step of the action behavior, determine the key frame of the action behavior.

In this embodiment, action key frame retrieval and behavior prediction classification can be performed on simple human daily action behaviors, which has high classification accuracy. Based on the human skeleton information extracted from the video sequence, through the multi-angle feature matching model, the key frame of the action is retrieved and the action behavior at the subsequent moment is predicted.

The present invention also provides an embodiment of a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the above-mentioned method for predicting human action behavior is realized.

The embodiment of the computer-readable storage medium storing the computer program of the present invention has the corresponding technical effects of the above-mentioned embodiment of the method for predicting human action behavior, which will not be repeated here.

The above is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Anyone skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present invention. All should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (8)

1. A method for predicting human action behavior, comprising:

frame-dividing and sampling the human body action period image information to obtain an image sequence with reduced frame number;

sequentially aiming at the images of a plurality of frames of the image sequence according to the ordering of the frames, carrying out the following operations:

establishing a skeleton model for each frame of image, and determining actual values of a plurality of characteristic angles of the skeleton model;

according to the reference values of the plurality of characteristic angles corresponding to the plurality of preset human action behavior categories obtained through pre-training and learning, the actual values of the plurality of characteristic angles of the skeleton model are compared with the reference values of the plurality of characteristic angles corresponding to the preset human action behavior categories, and the method comprises the following steps:

sequentially judging whether the actual value of the nth characteristic angle is within a preset deviation range of a reference value of the nth characteristic angle corresponding to a preset human body action behavior category, wherein n is an integer between 1 and 7;

counting the number of the characteristic angles within a preset deviation range of the reference values of the 1 st to 7 th characteristic angles of the specific human action behaviors for each frame of image; the number is the comparison result; the specific human action behavior is any one of preset human action behavior categories;

determining the human action behavior category of each frame of image according to the comparison result, wherein the human action behavior category comprises:

if the number is greater than or equal to 5, preliminarily determining the human body action behavior category to which the image of the corresponding frame belongs as the specific human body action behavior;

for each frame of image, if the number of the characteristic angles which are not located in the preset deviation range of the reference value of each characteristic angle of the specific human motion behavior is two, and the two characteristic angles which are not located in the preset deviation range of the reference value of each characteristic angle of the specific human motion behavior are respectively sourced from any one of the upper limb part, the lower limb part and the middle part, the human motion behavior category to which the image of the corresponding frame belongs is finally determined as the specific human motion behavior.

2. The method of predicting human action behavior according to claim 1, wherein building a skeleton model for each frame of images in the plurality of frames and determining actual values of a plurality of feature angles of the skeleton model comprises:

establishing a skeleton model for each frame of image, wherein the skeleton model comprises an upper limb part, a lower limb part and a middle part; the upper limb portion includes: left hand, left elbow, left shoulder, right hand, right elbow, and right shoulder; the lower limb portion includes: left foot, left knee, right foot, right knee; the intermediate portion comprising a crotch portion;

respectively determining actual values of 1 st characteristic angles formed by a left hand, a left elbow and a left shoulder, actual values of 2 nd characteristic angles formed by a left elbow, a left shoulder and a crotch, actual values of 3 rd characteristic angles formed by a right hand, a right elbow and a right shoulder, actual values of 4 th characteristic angles formed by a right elbow, a right shoulder and a crotch, actual values of 5 th characteristic angles formed by a left foot, a left knee and a crotch, actual values of 6 th characteristic angles formed by a right foot, a right knee and a crotch, and actual values of 7 th characteristic angles formed by a left knee, a crotch and a right knee;

wherein the 1 st to 4 th characteristic angles originate from the upper limb portion, the 5 th to 6 th characteristic angles originate from the lower limb portion, and the 7 th characteristic angle originates from the intermediate portion.

3. The method of claim 1 or 2, wherein the skeletal model is a tree model of left hand, left elbow, left shoulder, right hand, right elbow, right shoulder, left foot, left knee, right foot, right knee, and crotch connection.

4. The prediction method of human motion behavior according to claim 1, wherein the preset deviation range of the reference value of the nth characteristic angle is a range of ±5 degrees of the reference value of the nth characteristic angle.

5. The method for predicting human action behavior according to claim 1, wherein the reference values of the plurality of feature angles corresponding to the predetermined human action behavior class obtained by training and learning in advance are obtained by:

inputting an action training learning data set;

extracting characteristic information of the training learning data set;

inputting the characteristic information into a support vector machine for classification to obtain action behavior categories of each training learning data; the action behavior type of each training learning data is a preset human action behavior type;

and determining reference values of a plurality of characteristic angles of the corresponding preset action behavior categories according to the action behavior training learning data corresponding to each preset action behavior category.

6. The method for predicting human motion behavior according to claim 1, wherein the step of sampling the human motion cycle image information in frames to obtain the image sequence with reduced frame number comprises:

and merging the first five frames into one frame according to the human motion period image information, sampling every ten frames, and merging the sampled five frames to obtain an image sequence with reduced frame number.

7. A computer device comprising a processor which when executed implements a method of predicting human action behaviour according to any one of claims 1 to 6.

8. A computer readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the method of predicting human action behavior according to any one of claims 1 to 6.

Images