CN112036324A - A method and system for human posture determination for complex multi-person scenes - Google Patents

Abstract

The invention discloses a human body posture judgment method and system for a complex multi-person scene. The method firstly divides the obtained key points of the human body into a plurality of sets, such as: a key point set of the upper half of the human body, a key point set of the lower half of the human body and the like; then extracting the statistical characteristics of the geometric distribution of the human body key points in each set, such as: minimum circumscribed rectangles, convex hulls, and the like; and finally, according to the calculated statistical characteristics, such as: and judging the posture of the human body according to the rotation angle of the minimum circumscribed rectangle, the horizontal included angle of the convex hull and the like. The invention has the advantages that the human body posture can be rapidly and accurately judged under the conditions that the positions of key points of the human body are inaccurate or partial key points are missing, and the like, and the abnormal behaviors are as follows: detection of falls, collisions, etc. provides accurate information.

Description

A method and system for human posture determination for complex multi-person scenes

technical field

The invention belongs to the technical field of pattern recognition and artificial intelligence, in particular to a robust human body posture determination method, in particular to a human body posture determination method and system for complex multi-person scenes.

Background technique

Video surveillance equipment is widely deployed in various public places, such as: kindergartens, nursing homes, stations, shopping malls, etc. These devices generate massive amounts of surveillance data in their daily operations, and it is obviously impractical to monitor video scenes manually. The semantic description of human posture in monitoring scenes, that is, determining different human postures, can help people quickly understand the state of individuals in the scene and the events that occur, which is of great significance for real-time detection of accidents and emergencies.

The traditional manual monitoring method is not only inefficient, but also difficult to respond quickly to some emergencies in time. With the development of artificial intelligence and pattern recognition technology, some methods of human posture recognition have emerged. According to whether the human body is regarded as a "link" model, human pose recognition methods can be roughly divided into two categories: methods based on video feature extraction and methods based on human body key point extraction. Both types of methods are discussed below.

In the method based on video feature extraction, the document "Action recognition by densetrajectories[C].IEEE Conference on Computer Vision and Pattern Recognition" proposes to use optical flow field to track dense sampling points to obtain the trajectory of human motion, and then complete the detection of human posture. identify. This method requires real-time tracking of sampling points to obtain human motion trajectories, and this tracking is prone to errors in scenes where pedestrians intersect. The document "Human posture recognition based on projection histogram and SupportVector Machine" uses an ellipse to fit the outline of the human body, then constructs a histogram along the long and short axes of the ellipse to describe the shape of the human body, and finally uses a support vector machine to recognize the human posture. For situations where the bounding box cannot be accurately extracted, such as multiple people occluding or occluding each other, the shape of the human body extracted by this method will change severely, resulting in a sharp drop in the performance of gesture recognition. The document "A bio-inspired event-based size and position invariant humanposture recognition algorithm" proposes a method for human posture recognition based on simplified line segment Hausdorff distance. The method takes two consecutive image frames in a video sequence as input, obtains the moving object by comparing the difference between the two frames, and decomposes the outline of the moving object into vector line segments. This method simplifies the calculation and improves the efficiency, but the inability to detect the moving body parts between the frames leads to a decrease in the accuracy of the recognition. The paper "Posture recognition invariant to background, cloth textures, body size, and camera distance using morphological geometry" uses the length and width of the extracted human silhouette to recognize human posture, avoiding the influence of detailed information such as wearing background. Since the length and width of the human body outline can only roughly describe the human posture, the accuracy of the method in identifying the posture is not high. The method based on video feature extraction needs to input the video to track the target before judging the human posture, or first extract the movement process of the target through the tracking algorithm, which makes it difficult for such methods to be suitable for complex scenes with a large number of people who have difficulty in target tracking.

In the method based on human body key point extraction, the document "Realtime Multi-person 2D PoseEstimation Using Part Affinity Fields" proposes a model that simultaneously predicts the position of body parts and the positional relationship of each part to extract human key points; the document "A Coarse- Fine Network for Keypoint Localization" proposes a coarse-to-fine multi-level supervised network CFN (Coarse-Fine Network) for extracting human key points. This kind of method does not need to track the target, so it can be suitable for human gesture recognition in multi-person scenes. The document "Neural Network Approach for 2-Dimension Person Pose Estimation With EncodedMask and Keypoint Detection" uses convolutional deep neural network to extract key points from image segmentation masks, learn the mutual connection between key points, and realizes the image segmentation and A bottom-up strategy is combined to extract human keypoints. Most of the methods based on human body key point extraction only extract feature points, but do not make a judgment of human body posture. However, in practical applications, it is often not enough to only obtain the feature points of the human body. Giving the judgment result of the posture can help people quickly understand the state of the individual in the scene and the events that have occurred, and it is also the intelligent monitoring system that automatically makes decisions about the situation. important basis for further analysis and judgment. Therefore, there are some methods to determine the human pose based on the key point information. The document "Human posture classification using skeleton information" uses the distance between the key points of the human body and the angle between the key points to determine the posture of the human body. However, in practical applications, there are often problems such as errors in the extracted key points and missing key points due to occlusion, which will make the performance of the existing attitude determination methods drop sharply or even fail.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a method and system for judging human body posture in a complex multi-person scene, so as to quickly understand the state of each individual in the scene and the events that occur.

The technical solution to achieve the purpose of the present invention is: a method for determining human body posture for complex multi-person scenes, the method comprises the following steps:

Step 1, detect human key points, and divide the key point set;

Step 2, extracting the statistical features of the geometric distribution of the human body key points in each set;

Step 3: Determine the posture of the human body according to the statistical features.

Further, the human body key points are detected in step 1, and the key point set is divided, and the specific process includes:

Step 1-1, construct a sample training set, the set includes several human body images marked with key points of the human body;

Step 1-2, using the sample training set to train a deep convolutional neural network;

Steps 1-3, using the trained deep convolutional neural network to detect key points in the human image to be detected;

Steps 1-4, divide the set of key points, specifically:

Eliminate key points of non-human torso, including hands, to form a set of whole body key points;

Divide the key points of the human torso into a set of upper body key points, a set of lower body key points, a set of left arm key points and a set of right arm key points;

The key points of the lower body are further divided into a set of key points of the left leg, a set of key points of the right leg, a set of key points of the left thigh, and a set of key points of the right thigh.

Further, in step 2, the statistical features of the geometric distribution of the key points of the human body in each set are extracted, and the specific process includes:

Step 2-1-1, calculate the convex hull of each key point set;

Step 2-1-2, find the two key points with the largest distance from the key points forming the convex hull;

Step 2-1-3: Calculate the angle between the clockwise rotation and the horizontal direction of the connecting line of the two key points as the horizontal angle of the convex hull.

Further, in step 3, determining the posture of the human body according to the statistical features is specifically: according to the horizontal angle of the convex hull of each key point set, determining the posture of the arms, legs and the whole body:

Assume that the horizontal included angle of the convex hull of the whole body key point set is α ₀ ; the horizontal included angle of the upper body key point set convex hull is α ₁ ; the horizontal included angle of the lower body key point set convex hull is α ₂ ; the left arm key point set convex hull is α 2 ; The horizontal included angle is α ₃ ; the horizontal included angle of the convex hull of the right arm key point set is α ₄ ; the horizontal included angle of the left leg key point set convex hull is α ₅ ; the horizontal included angle of the right leg key point set convex hull is α ₆ ; the horizontal included angle of the convex hull of the left thigh key point set is α ₇ ; the horizontal included angle of the convex hull of the right thigh key point set is α ₈ ;

If |tanα ₃ |∈(a ₃ ,b ₃ ) or |tanα ₄ |∈(a ₃ ,b ₃ ), it is judged to be a raised arm posture; if |tanα ₅ |∈(a ₅ ,b ₅ ) or |tanα ₆ |∈(a ₅ ,b ₅ ), it is determined as a kicking posture; if |tanα ₀ |∈(a ₀ ,b ₀ ), |tanα ₁ |∈(a ₁ ,b ₁ ) and |tanα ₂ |∈ (a ₂ , b ₂ ), it is judged as standing posture; if |tanα ₇ |∈[a ₇ ,b ₇ ] or |tanα ₈ |∈[a ₇ ,b ₇ ], it is judged as squatting posture; if | tanα ₀ |∈[c ₀ ,d ₀ ], |tanα ₁ |∈[c ₁ ,d ₁ ] and |tanα ₂ |∈[c ₂ ,d ₂ ], then it is determined to be a lying posture.

Further, in step 2, the statistical features of the geometric distribution of the key points of the human body in each set are extracted, and the specific process includes:

Step 2-2-1, calculate the minimum circumscribed rectangle of each key point set;

Step 2-2-2, respectively calculate the angle formed by the clockwise rotation of the long side and the short side of the minimum circumscribed rectangle and the horizontal direction;

Step 2-2-3, take the smaller of the two included angles as the rotation angle of the minimum circumscribed rectangle.

Further, in step 3, determining the posture of the human body according to the statistical features is specifically: determining the postures of the arms, legs and the whole body according to the rotation angle of the minimum circumscribed rectangle of each key point set:

Assume that the horizontal included angle of the minimum circumscribed rectangle of the whole body key point set is β ₀ ; the horizontal included angle of the minimum circumscribed rectangle of the upper body key point set is β ₁ ; the horizontal included angle of the minimum circumscribed rectangle of the lower body key point set is β ₂ ; the left arm key point set The horizontal included angle of the minimum circumscribed rectangle is β ₃ ; the horizontal included angle of the minimum circumscribed rectangle of the right arm key point set is β ₄ ; the horizontal included angle of the minimum circumscribed rectangle of the left leg key point set is β ₅ ; The horizontal included angle of the rectangle is β ₆ ; the horizontal included angle of the minimum circumscribed rectangle of the left thigh key point set is β ₇ ; the horizontal included angle of the minimum circumscribed rectangle of the right thigh key point set is β ₈ ;

或

则判定为抬臂姿态；若

或

则判定为踢腿姿态；若

且

则判定为站立姿态；若β₁|∈[λ₁,γ₁]，则判定为弯腰姿态；若

或

则判定为蹲坐姿态；若|β₀|∈[θ₀,Θ₀]、β₁|∈[θ₁,Θ₁]且β₂|∈[θ₂,Θ₂]，则判定为平躺姿态。like

or

It is judged to be the arm raising posture; if

or

It is judged as kicking posture; if

and

It is judged as standing posture; if β ₁ |∈[λ ₁ ,γ ₁ ], it is judged as bending posture;

or

If |β ₀ |∈[θ ₀ ,Θ ₀ ], β ₁ |∈[θ ₁ ,Θ ₁ ] and β ₂ |∈[θ ₂ ,Θ ₂ ], then it is determined as lying down attitude.

A human body posture determination system for complex multi-person scenes, the system includes:

The key point division module is used to detect the key points of the human body and divide the set of key points;

The statistical feature extraction module is used to extract the statistical features of the geometric distribution of the key points of the human body in each set;

The posture determination module is used to determine the posture of the human body according to the statistical features.

Compared with the prior art, the present invention has the following significant advantages: 1) The present invention adopts the idea of dividing the human body key point set, compared with the method based on video feature extraction, it avoids the difficult problem of target extraction, and can be applied to multi-person scenarios 2) Compared with the existing method based on human body key point extraction, the problem of human body posture determination is converted into the statistical relationship of the key point set, so that the present invention is insensitive to the deletion and displacement of some key points, Even if some of the body is occluded and the key points are inaccurate, the correct attitude determination result can still be obtained according to the statistical features of the set, thereby greatly improving the robustness of attitude determination; 3) The determination speed of the present invention is relatively high. Fast, it can realize real-time determination of human body posture within the monitoring range, and help people quickly understand the individual status and events in the current scene.

The present invention will be described in further detail below with reference to the accompanying drawings.

Description of drawings

FIG. 1 is a flowchart of a method for determining human posture in a complex multi-person scene in one embodiment.

FIG. 2 is a schematic diagram of division of a human body key point set in one embodiment.

FIG. 3 is a schematic diagram of a horizontal angle of calculating a convex hull of a set of key points in one embodiment.

FIG. 4 is a schematic diagram of calculating the rotation angle of the minimum circumscribed rectangle of the key point set in one embodiment.

Fig. 5 is the judgment result diagram of the basic posture of a single person in one embodiment; wherein, the key points of the human body are marked with dots; the judgment result of the posture is marked on the head of the person, and Figures (a) to (c) are standing postures respectively. , squatting posture and bending posture.

Fig. 6 is the judgment result diagram of the complex posture of a single person in one embodiment, wherein, the key points of the human body are marked with dots, and the judgment result of the posture is marked on the head of the person, and Figures (a) to (c) are standing and raising arms respectively. Stance, Standing Kick Stance, and Standing Arm Kick Stance.

FIG. 7 is a diagram showing the judgment result of complex postures of multiple people in one embodiment, wherein the key points of the human body are marked with dots, and the judgment result of the posture is marked on the head of the person.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

It should be noted that if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for description purposes, and should not be understood as instructions or Implicit their relative importance or implicitly indicate the number of technical features indicated. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In addition, the technical solutions between the various embodiments can be combined with each other, but must be based on the realization by those of ordinary skill in the art. When the combination of technical solutions is contradictory or cannot be realized, it should be considered that the combination of such technical solutions does not exist. , is not within the scope of protection required by the present invention.

In one embodiment, with reference to FIG. 1 , a method for determining human posture in a complex multi-person scene is provided, and the method includes the following steps:

Step 1, detect human key points, and divide the key point set;

Step 2, extracting the statistical features of the geometric distribution of the human body key points in each set;

Step 3: Determine the posture of the human body according to the statistical features.

Further, in one of the embodiments, the human body key points are detected in step 1, and the key point set is divided, and the specific process includes:

Step 1-1, construct a sample training set, the set includes several human body images marked with key points of the human body;

Step 1-2, using the sample training set to train a deep convolutional neural network;

Steps 1-3, using the trained deep convolutional neural network to detect key points in the human image to be detected;

Steps 1-4, combined with Figure 2, divide the set of key points, specifically:

Eliminate key points of non-human torso, including hands, to form a set of whole body key points;

Divide the key points of the human torso into a set of upper body key points, a set of lower body key points, a set of left arm key points and a set of right arm key points;

The key points of the lower body are further divided into a set of key points of the left leg, a set of key points of the right leg, a set of key points of the left thigh, and a set of key points of the right thigh.

Here, the method in the document "Realtime Multi-person 2D Pose Estimation Using PartAffinity Fields" is used to extract human key points.

Here, the set of key points of the human torso includes ten key points in total: left eye, right eye, mouth, neck, left hip joint, right hip joint, left knee joint, right knee joint, left ankle joint, and right ankle joint;

The upper body key point collection includes six key points: left eye, right eye, mouth, neck, left hip joint, and right hip joint;

The lower body key point set includes six key points in total: left hip joint, right hip joint, left knee joint, right knee joint, left ankle joint, and right ankle joint;

The set of key points of the left arm includes the left wrist joint, the left elbow joint, and the left shoulder joint, a total of three key points;

The right arm key point set includes the right wrist joint, right elbow joint, right shoulder joint, a total of three key points;

The set of key points of the left leg includes the left ankle joint, the left knee joint, and the left hip joint, a total of three key points;

The set of key points of the right leg includes the right ankle joint, the right knee joint, and the right hip joint, a total of three key points;

The set of key points of the left thigh includes the left knee joint and the left hip joint, a total of two key points;

The right thigh key point set includes the right knee joint and the right hip joint, a total of two key points.

Further, in one of the embodiments, with reference to FIG. 3 , in step 2, the statistical features of the geometric distribution of the human body key points in each set are extracted, and the specific process includes:

Step 2-1-1, calculate the convex hull of each key point set;

Step 2-1-2, find the two key points with the largest distance from the key points forming the convex hull;

Step 2-1-3: Calculate the angle between the clockwise rotation and the horizontal direction of the connecting line of the two key points as the horizontal angle of the convex hull.

Further, in one of the embodiments, the step 3 determines the posture of the human body according to the statistical features, specifically: according to the horizontal angle of the convex hull of each key point set, the posture of the arm, the leg and the whole body is determined:

Assume that the horizontal included angle of the convex hull of the whole body key point set is α ₀ ; the horizontal included angle of the upper body key point set convex hull is α ₁ ; the horizontal included angle of the lower body key point set convex hull is α ₂ ; the left arm key point set convex hull is α 2 ; The horizontal included angle is α ₃ ; the horizontal included angle of the convex hull of the right arm key point set is α ₄ ; the horizontal included angle of the left leg key point set convex hull is α ₅ ; the horizontal included angle of the right leg key point set convex hull is α ₆ ; the horizontal included angle of the convex hull of the left thigh key point set is α ₇ ; the horizontal included angle of the convex hull of the right thigh key point set is α ₈ ;

If |tanα ₃ |∈(a ₃ ,b ₃ ) or |tanα ₄ |∈(a ₃ ,b ₃ ), it is judged to be a raised arm posture; if |tanα ₅ |∈(a ₅ ,b ₅ ) or |tanα ₆ |∈(a ₅ ,b ₅ ), it is determined as a kicking posture; if |tanα ₀ |∈(a ₀ ,b ₀ ), |tanα ₁ |∈(a ₁ ,b ₁ ) and |tanα ₂ |∈ (a ₂ , b ₂ ), it is judged as standing posture; if |tanα ₇ |∈[a ₇ ,b ₇ ] or |tanα ₈ |∈[a ₇ ,b ₇ ], it is judged as squatting posture; if | tanα ₀ |∈[c ₀ ,d ₀ ], |tanα ₁ |∈[c ₁ ,d ₁ ] and |tanα ₂ |∈[c ₂ ,d ₂ ], then it is determined to be a lying posture.

Here preferably, (a ₃ ,b ₃ )=(0.25,+∞), (a ₅ ,b ₅ )=(0.25,5), (a ₀ ,b ₀ )=(a ₁ ,b ₁ )=( a ₂ ,b ₂ )=(4,+∞), [a ₇ ,b ₇ ]=[0,0.25], [c ₀ ,d ₀ ]=[c ₁ ,d ₁ ]=[c ₂ ,d ₂ ]=[0,0.25].

Further, in one of the embodiments, with reference to FIG. 4 , in step 2, the statistical features of the geometric distribution of the human body key points in each set are extracted, and the specific process includes:

Step 2-2-1, calculate the minimum circumscribed rectangle of each key point set;

Step 2-2-2, respectively calculate the angle formed by the clockwise rotation of the long side and the short side of the minimum circumscribed rectangle and the horizontal direction;

Step 2-2-3, take the smaller of the two included angles as the rotation angle of the minimum circumscribed rectangle.

Further, in one of the embodiments, the step 3 determines the posture of the human body according to the statistical features, specifically: according to the rotation angle of the minimum circumscribed rectangle of each key point set, the posture of the arm, the leg and the whole body is determined:

Assume that the horizontal included angle of the minimum circumscribed rectangle of the whole body key point set is β ₀ ; the horizontal included angle of the minimum circumscribed rectangle of the upper body key point set is β ₁ ; the horizontal included angle of the minimum circumscribed rectangle of the lower body key point set is β ₂ ; the left arm key point set The horizontal included angle of the minimum circumscribed rectangle is β ₃ ; the horizontal included angle of the minimum circumscribed rectangle of the right arm key point set is β ₄ ; the horizontal included angle of the minimum circumscribed rectangle of the left leg key point set is β ₅ ; The horizontal included angle of the rectangle is β ₆ ; the horizontal included angle of the minimum circumscribed rectangle of the left thigh key point set is β ₇ ; the horizontal included angle of the minimum circumscribed rectangle of the right thigh key point set is β ₈ ;

或

则判定为抬臂姿态；若

或

则判定为踢腿姿态；若

且

则判定为站立姿态；若β₁|∈[λ₁,γ₁]，则判定为弯腰姿态；若

或

则判定为蹲坐姿态；若|β₀|∈[θ₀,Θ₀]、β₁|∈[θ₁,Θ₁]且β₂|∈[θ₂,Θ₂]，则判定为平躺姿态。like

or

It is judged to be the arm raising posture; if

or

It is judged as kicking posture; if

and

It is judged as standing posture; if β ₁ |∈[λ ₁ ,γ ₁ ], it is judged as bending posture;

or

If |β ₀ |∈[θ ₀ ,Θ ₀ ], β ₁ |∈[θ ₁ ,Θ ₁ ] and β ₂ |∈[θ ₂ ,Θ ₂ ], then it is determined as lying down attitude.

[λ₁,γ₁]＝[0°,25°]，

[θ₀,Θ₀]＝[θ₁,Θ₁]＝[θ₂,Θ₂]＝[0°,25°]。Here preferably,

[λ ₁ ,γ ₁ ]=[0°, 25°],

[θ ₀ , Θ ₀ ]=[θ ₁ , Θ ₁ ]=[θ ₂ , Θ ₂ ]=[0°, 25°].

In one embodiment, a system for determining human posture in a complex multi-person scene is provided, the system comprising:

The key point division module is used to detect the key points of the human body and divide the set of key points;

The statistical feature extraction module is used to extract the statistical features of the geometric distribution of the key points of the human body in each set;

The posture determination module is used to determine the posture of the human body according to the statistical features.

Further, in one of the embodiments, the key point division module includes:

The training set construction unit is used to construct a sample training set, the set includes several human body images marked with key points of the human body;

a training unit for training a deep convolutional neural network using the sample training set;

The key point detection unit uses the trained deep convolutional neural network to detect the key points in the human image to be detected;

The division unit is used to divide the set of key points, including:

The first division subunit is used to eliminate key points of non-human torso, including hands, to form a set of key points of the whole body;

The second division subunit is used to divide the key points of the human torso into a set of key points of the upper body, a set of key points of the lower body, a set of key points of the left arm and a set of key points of the right arm;

The third division subunit is used to further divide the key points of the lower body into a set of key points of the left leg, a set of key points of the right leg, a set of key points of the left thigh, and a set of key points of the right thigh.

Further, in one of the embodiments, the statistical feature extraction module includes:

a first calculation unit, used to calculate the convex hull of each key point set;

The key point screening unit is used to find the two key points with the largest distance from the key points that constitute the convex hull;

The second calculation unit is used to calculate the angle between the clockwise rotation and the horizontal direction of the connecting line of the two key points, as the horizontal angle of the convex hull;

The attitude determination module is used to determine the attitude of the arms, legs and the whole body according to the horizontal angle of the convex hull of each key point set:

Assume that the horizontal included angle of the convex hull of the whole body key point set is α ₀ ; the horizontal included angle of the upper body key point set convex hull is α ₁ ; the horizontal included angle of the lower body key point set convex hull is α ₂ ; the left arm key point set convex hull is α 2 ; The horizontal included angle is α ₃ ; the horizontal included angle of the convex hull of the right arm key point set is α ₄ ; the horizontal included angle of the left leg key point set convex hull is α ₅ ; the horizontal included angle of the right leg key point set convex hull is α ₆ ; the horizontal included angle of the convex hull of the left thigh key point set is α ₇ ; the horizontal included angle of the convex hull of the right thigh key point set is α ₈ ;

If |tanα ₃ |∈(a ₃ ,b ₃ ) or |tanα ₄ |∈(a ₃ ,b ₃ ), it is judged to be a raised arm posture; if |tanα ₅ |∈(a ₅ ,b ₅ ) or |tanα ₆ |∈(a ₅ ,b ₅ ), it is determined as a kicking posture; if |tanα ₀ |∈(a ₀ ,b ₀ ), |tanα ₁ |∈(a ₁ ,b ₁ ) and |tanα ₂ |∈ (a ₂ , b ₂ ), it is judged as standing posture; if |tanα ₇ |∈[a ₇ ,b ₇ ] or |tanα ₈ |∈[a ₇ ,b ₇ ], it is judged as squatting posture; if | tanα ₀ |∈[c ₀ ,d ₀ ], |tanα ₁ |∈[c ₁ ,d ₁ ] and |tanα ₂ |∈[c ₂ ,d ₂ ], then it is determined to be a lying posture.

Further, in one of the embodiments, the statistical feature extraction module includes:

The third calculation unit is used to calculate the minimum circumscribed rectangle of each key point set;

The fourth calculation unit is used to calculate the angle formed by the clockwise rotation of the long side and the short side of the minimum circumscribed rectangle and the horizontal direction respectively;

The fifth calculation unit is used to take the smaller value of the two included angles as the rotation angle of the minimum circumscribed rectangle;

The attitude determination module is used to determine the attitude of the arm, the leg and the whole body according to the rotation angle of the minimum circumscribed rectangle of each key point set:

Assume that the horizontal included angle of the minimum circumscribed rectangle of the whole body key point set is β ₀ ; the horizontal included angle of the minimum circumscribed rectangle of the upper body key point set is β ₁ ; the horizontal included angle of the minimum circumscribed rectangle of the lower body key point set is β ₂ ; the left arm key point set The horizontal included angle of the minimum circumscribed rectangle is β ₃ ; the horizontal included angle of the minimum circumscribed rectangle of the right arm key point set is β ₄ ; the horizontal included angle of the minimum circumscribed rectangle of the left leg key point set is β ₅ ; The horizontal included angle of the rectangle is β ₆ ; the horizontal included angle of the minimum circumscribed rectangle of the left thigh key point set is β ₇ ; the horizontal included angle of the minimum circumscribed rectangle of the right thigh key point set is β ₈ ;

或

则判定为抬臂姿态；若

或

则判定为踢腿姿态；若

且

则判定为站立姿态；若β₁|∈[λ₁,γ₁]，则判定为弯腰姿态；若

或

则判定为蹲坐姿态；若|β₀|∈[θ₀,Θ₀]、β₁|∈[θ₁,Θ₁]且β₂|∈[θ₂,Θ₂]，则判定为平躺姿态。like

or

It is judged to be the arm raising posture; if

or

It is judged as kicking posture; if

and

It is judged as standing posture; if β ₁ |∈[λ ₁ ,γ ₁ ], it is judged as bending posture;

or

If |β ₀ |∈[θ ₀ ,Θ ₀ ], β ₁ |∈[θ ₁ ,Θ ₁ ] and β ₂ |∈[θ ₂ ,Θ ₂ ], then it is determined as lying down attitude.

For the specific limitation of the human body posture determination system used in the complex multi-person scene, please refer to the definition of the human body posture determination method used in the complex multi-person scene above, which will not be repeated here. Each module in the above-mentioned human posture determination system for complex multi-person scenarios can be implemented in whole or in part by software, hardware and combinations thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

As a specific example, in one of the embodiments, the present invention is further verified and explained, the above-mentioned method of minimum external rectangle is used to determine the human body posture in the surveillance video, and the test results are shown in Figure 5, Figure 6, Figure 7 and Table 1 .

Table 1 Statistics of human posture determination results

It can be seen from the judgment results of multiple basic postures of a single person in FIG. 5 and the judgment results of multiple complex postures of a single person in FIG. 6 that the present invention can more accurately determine the posture of a single person. It can be seen from the determination result of complex postures of multiple people with mutual occlusion in FIG. 7 that the present invention can simultaneously determine the postures of multiple people, and is insensitive to influences such as occlusion. As shown in Table 1, the present invention has an average correct rate of 92.833% in determining the human body posture. Among them, the correct rate for judging postures with obvious distinctions such as standing, squatting, raising arms, and lying down is higher, while for bending over , kicking and other confusing gestures, the accuracy rate is slightly lower.

The foregoing has shown and described the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above-mentioned embodiments, and the descriptions in the above-mentioned embodiments and the description are only to illustrate the principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have Various changes and modifications fall within the scope of the claimed invention. The claimed scope of the present invention is defined by the appended claims and their equivalents.

Claims (10)

1. A human body posture judgment method for a complex multi-person scene is characterized by comprising the following steps:

step 1, detecting key points of a human body, and dividing a key point set;

step 2, extracting the statistical characteristics of the geometric distribution of the human body key points in each set;

and 3, judging the posture of the human body according to the statistical characteristics.

2. The method for determining the human body posture in the complex multi-person scene according to claim 1, wherein the step 1 of detecting the human body key points and dividing the key point set comprises the following specific processes:

step 1-1, constructing a sample training set, wherein the set comprises a plurality of human body images marked with human body key points;

step 1-2, training a deep convolutional neural network by using the sample training set;

step 1-3, detecting key points in a human body image to be detected by using a trained deep convolution neural network;

step 1-4, dividing a key point set, specifically:

removing key points of non-human body trunk, including hands, to form a whole body key point set;

dividing the key points of the human body trunk into an upper body key point set, a lower body key point set, a left arm key point set and a right arm key point set;

and continuously dividing the lower half key points into a left leg key point set, a right leg key point set, a left thigh key point set and a right thigh key point set.

3. The method for determining human body posture in a complex multi-person scene according to claim 1 or 2, wherein the step 2 of extracting statistical features of the geometric distribution of the human body key points in each set comprises the following specific steps:

step 2-1-1, calculating convex hulls of each key point set;

step 2-1-2, finding out two key points with the largest distance from the key points forming the convex hull;

and 2-1-3, calculating an included angle between the clockwise rotation and the horizontal direction of the connecting line of the two key points, and taking the included angle as the horizontal angle of the convex hull.

4. The method for determining human body posture in a complex multi-person scene as claimed in claim 3, wherein the step 3 of determining human body posture according to the statistical features specifically comprises: judging the postures of the arms, the legs and the whole body according to the horizontal angle of the convex hull of each key point set:

assuming that the horizontal included angle of the convex hull of the whole body key point set is alpha₀(ii) a The horizontal included angle of the convex hull of the upper half key point set is alpha₁(ii) a The horizontal included angle of the convex hull of the lower half key point set is alpha₂(ii) a The horizontal included angle of the convex hull of the left arm key point set is alpha₃(ii) a The horizontal included angle of the right arm key point set convex hull is alpha₄(ii) a The horizontal included angle of the convex hull of the left leg key point set is alpha₅(ii) a The horizontal included angle of the right leg key point set convex hull is alpha₆(ii) a The horizontal included angle of the convex hull of the left thigh key point set is alpha₇(ii) a The horizontal included angle of the right thigh key point set convex hull is alpha₈；

If | tan α₃|∈(a₃,b₃) Or | tan α₄|∈(a₃,b₃) Judging the posture of the arm lifting; if | tan α₅|∈(a₅,b₅) Or | tan α₆|∈(a₅,b₅) Then it is determined asA kicking posture; if | tan α₀|∈(a₀,b₀)、|tanα₁|∈(a₁,b₁) And | tan α₂|∈(a₂,b₂) Judging the posture to be a standing posture; if | tan α₇|∈[a₇,b₇]Or | tan α₈|∈[a₇,b₇]Judging the squatting posture; if | tan α₀|∈[c₀,d₀]、|tanα₁|∈[c₁,d₁]And | tan α₂|∈[c₂,d₂]Then, the user is determined to be in the lying posture.

5. The method for determining human body posture in a complex multi-person scene according to claim 1 or 2, wherein the step 2 of extracting statistical features of the geometric distribution of the human body key points in each set comprises the following specific steps:

step 2-2-1, calculating the minimum circumscribed rectangle of each key point set;

step 2-2-2, respectively calculating included angles formed by the rotation of the long side and the short side of the minimum circumscribed rectangle along the clockwise direction and the horizontal direction;

and 2-2-3, taking the smaller value of the two included angles as the rotation angle of the minimum circumscribed rectangle.

6. The method for determining human body posture in a complex multi-person scene as claimed in claim 5, wherein the step 3 of determining human body posture according to the statistical features specifically comprises: judging the postures of the arms, the legs and the whole body according to the rotation angle of the minimum circumscribed rectangle of each key point set:

assuming that the horizontal included angle of the minimum circumscribed rectangle of the whole body key point set is beta₀(ii) a The minimum external rectangle horizontal included angle of the upper half body key point set is beta₁(ii) a The horizontal included angle of the minimum external rectangle of the lower half key point set is beta₂(ii) a The horizontal included angle of the minimum circumscribed rectangle of the left arm key point set is beta₃(ii) a The horizontal included angle of the minimum circumscribed rectangle of the right arm key point set is beta₄(ii) a The horizontal included angle of the minimum circumscribed rectangle of the left leg key point set is beta₅(ii) a Right leg jointThe horizontal included angle of the minimum circumscribed rectangle of the key point set is beta₆(ii) a The horizontal included angle of the minimum circumscribed rectangle of the left thigh key point set is beta₇(ii) a The horizontal included angle of the minimum circumscribed rectangle of the right thigh key point set is beta₈；

If it is

Or

Judging the posture of the arm lifting; if it is

Or

Judging the posture of the kicking leg; if it is

And is

Judging the posture of standing; if beta is₁|∈[λ₁,γ₁]Judging the posture of bending waist; if it is

Or

Judging the squatting posture; if beta₀|∈[θ₀,Θ₀]、β₁|∈[θ₁,Θ₁]And beta is₂|∈[θ₂,Θ₂]Then, the user is determined to be in the lying posture.

7. A body pose determination system for a complex multi-person scene, the system comprising:

the key point dividing module is used for detecting key points of a human body and dividing a key point set;

the statistical feature extraction module is used for extracting the statistical features of the geometric distribution of the human body key points in each set;

and the posture judgment module is used for judging the human body posture according to the statistical characteristics.

8. The system of claim 7, wherein the keypoint segmentation module comprises:

the training set constructing unit is used for constructing a sample training set, and the set comprises a plurality of human body images marked with human body key points;

a training unit for training a deep convolutional neural network using the sample training set;

the key point detection unit is used for detecting key points in the human body image to be detected by using the trained deep convolutional neural network;

the dividing unit is used for dividing the key point set, and specifically comprises:

the first dividing unit is used for removing key points of the trunk of the non-human body, including hands, and forming a whole body key point set;

the second dividing subunit is used for dividing the key points of the human body into an upper body key point set, a lower body key point set, a left arm key point set and a right arm key point set;

and the third dividing subunit is used for continuously dividing the lower body key points into a left leg key point set, a right leg key point set, a left thigh key point set and a right thigh key point set.

9. The system of claim 8, wherein the statistical feature extraction module comprises:

the first calculating unit is used for calculating convex hulls of all the key point sets;

the key point screening unit is used for finding out two key points with the largest distance from the key points forming the convex hull;

the second calculation unit is used for calculating an included angle between the clockwise rotation and the horizontal direction of the connecting line of the two key points as the horizontal angle of the convex hull;

the gesture judging module is used for judging the gestures of the arms, the legs and the whole body according to the horizontal angles of the convex hulls of the key point sets:

assuming that the horizontal included angle of the convex hull of the whole body key point set is alpha₀(ii) a The horizontal included angle of the convex hull of the upper half key point set is alpha₁(ii) a The horizontal included angle of the convex hull of the lower half key point set is alpha₂(ii) a The horizontal included angle of the convex hull of the left arm key point set is alpha₃(ii) a The horizontal included angle of the right arm key point set convex hull is alpha₄(ii) a The horizontal included angle of the convex hull of the left leg key point set is alpha₅(ii) a The horizontal included angle of the right leg key point set convex hull is alpha₆(ii) a The horizontal included angle of the convex hull of the left thigh key point set is alpha₇(ii) a The horizontal included angle of the right thigh key point set convex hull is alpha₈；

If | tan α₃|∈(a₃,b₃) Or | tan α₄|∈(a₃,b₃) Judging the posture of the arm lifting; if | tan α₅|∈(a₅,b₅) Or | tan α₆|∈(a₅,b₅) Judging the posture of the kicking leg; if | tan α₀|∈(a₀,b₀)、|tanα₁|∈(a₁,b₁) And | tan α₂|∈(a₂,b₂) Judging the posture to be a standing posture; if | tan α₇|∈[a₇,b₇]Or | tan α₈|∈[a₇,b₇]Judging the squatting posture; if | tan α₀|∈[c₀,d₀]、|tanα₁|∈[c₁,d₁]And | tan α₂|∈[c₂,d₂]Then, the user is determined to be in the lying posture.

10. The system of claim 8, wherein the statistical feature extraction module comprises:

the third calculation unit is used for calculating the minimum circumscribed rectangle of each key point set;

the fourth calculation unit is used for calculating included angles formed by the rotation of the long side and the short side of the minimum circumscribed rectangle along the clockwise direction and the horizontal direction respectively;

the fifth calculation unit is used for taking the smaller value of the two included angles as the rotation angle of the minimum circumscribed rectangle;

the gesture judging module is used for judging the gestures of the arms, the legs and the whole body according to the rotation angle of the minimum circumscribed rectangle of each key point set:

assuming that the horizontal included angle of the minimum circumscribed rectangle of the whole body key point set is beta₀(ii) a The minimum external rectangle horizontal included angle of the upper half body key point set is beta₁(ii) a The horizontal included angle of the minimum external rectangle of the lower half key point set is beta₂(ii) a The horizontal included angle of the minimum circumscribed rectangle of the left arm key point set is beta₃(ii) a The horizontal included angle of the minimum circumscribed rectangle of the right arm key point set is beta₄(ii) a The horizontal included angle of the minimum circumscribed rectangle of the left leg key point set is beta₅(ii) a The horizontal included angle of the minimum circumscribed rectangle of the right leg key point set is beta₆(ii) a The horizontal included angle of the minimum circumscribed rectangle of the left thigh key point set is beta₇(ii) a The horizontal included angle of the minimum circumscribed rectangle of the right thigh key point set is beta₈；

If it is

Or

Judging the posture of the arm lifting; if it is

Or

Judging the posture of the kicking leg; if it is

And is

Judging the posture of standing; if beta is₁|∈[λ₁,γ₁]Judging the posture of bending waist; if it isOr

Judging the squatting posture; if beta₀|∈[θ₀,Θ₀]、β₁|∈[θ₁,Θ₁]And beta is₂|∈[θ₂,Θ₂]Then, the user is determined to be in the lying posture.

Images