CN112036324A - Human body posture judgment method and system for complex multi-person scene - Google Patents

Human body posture judgment method and system for complex multi-person scene Download PDF

Info

Publication number
CN112036324A
CN112036324A CN202010905107.6A CN202010905107A CN112036324A CN 112036324 A CN112036324 A CN 112036324A CN 202010905107 A CN202010905107 A CN 202010905107A CN 112036324 A CN112036324 A CN 112036324A
Authority
CN
China
Prior art keywords
key point
point set
posture
included angle
human body
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010905107.6A
Other languages
Chinese (zh)
Inventor
张新峰
杨文强
李斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yangzhou University
Original Assignee
Yangzhou University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yangzhou University filed Critical Yangzhou University
Priority to CN202010905107.6A priority Critical patent/CN112036324A/en
Publication of CN112036324A publication Critical patent/CN112036324A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a human body posture judgment method and system for a complex multi-person scene. The method firstly divides the obtained key points of the human body into a plurality of sets, such as: a key point set of the upper half of the human body, a key point set of the lower half of the human body and the like; then extracting the statistical characteristics of the geometric distribution of the human body key points in each set, such as: minimum circumscribed rectangles, convex hulls, and the like; and finally, according to the calculated statistical characteristics, such as: and judging the posture of the human body according to the rotation angle of the minimum circumscribed rectangle, the horizontal included angle of the convex hull and the like. The invention has the advantages that the human body posture can be rapidly and accurately judged under the conditions that the positions of key points of the human body are inaccurate or partial key points are missing, and the like, and the abnormal behaviors are as follows: detection of falls, collisions, etc. provides accurate information.

Description

Human body posture judgment method and system for complex multi-person scene
Technical Field
The invention belongs to the technical field of pattern recognition and artificial intelligence, particularly relates to a robust human body posture judgment method, and particularly relates to a human body posture judgment method and system for a complex multi-person scene.
Background
Video monitoring equipment is widely deployed in various public places, such as: kindergarten, old home, station, market, etc. These devices generate a huge amount of monitoring data during daily operation, and it is obviously impractical to monitor video scenes manually. The semantic description of the human body posture in the scene is monitored, namely different human body postures are judged, so that people can be helped to quickly know the state of individuals and events in the scene, and the method has great significance for detecting accidents and emergency situations in real time.
The traditional manual monitoring mode is not only low in efficiency, but also difficult to make rapid response to some emergency situations in time, and with the development of artificial intelligence and pattern recognition technology, some human posture recognition methods appear. Human posture recognition methods can be roughly divided into two categories according to whether a human body is regarded as a "connecting rod" model: a method based on video feature extraction and a method based on human body key point extraction. These two types of methods will be discussed in greater detail below.
In the method based on video feature extraction, the literature "Action Recognition by dense objects [ C ]. IEEE Conference on Computer Vision and Pattern Recognition" proposes to obtain the trajectory of human motion by tracking dense sampling points using an optical flow field, and then to complete the Recognition of human posture. The method needs to track sampling points in real time for acquiring the motion trail of the human body, and the tracking is easy to make mistakes in the scene of pedestrian crossing. The literature, "Human position recognition based on projection history and Support Vector Machine" fits a Human body contour by using an ellipse, then constructs a histogram along the directions of the long axis and the short axis of the ellipse for describing the shape of the Human body, and finally identifies the Human body posture by using a Support Vector Machine. For the case that the bounding box cannot be accurately extracted, for example: when a plurality of people are occluded or shielded, the shape of the human body extracted by the method can be seriously changed, so that the gesture recognition performance is sharply reduced. The document "A bio-influenced event-based size and position innovative human position recognition algorithm" proposes a human posture recognition method based on a simplified line segment Hausdorff distance. The method takes two continuous image frames in a video sequence as input, obtains a moving object by comparing the difference between the two frames, and decomposes the outline of the moving object into vector line segments. The method simplifies calculation and improves efficiency, but the accuracy of recognition is reduced because a body part which does not move between the front frame and the rear frame cannot be detected. The document "position recognition in variable to background, contour, body size, and camera distance using morphological geometry" identifies the body pose by using the length and width of the extracted body contour, and avoids the influence of detail information such as the wearing background. Because the length and the width of the human body outline can only roughly describe the human body posture, the method has low accuracy in posture recognition. The method based on video feature extraction needs to input video before judging the human body posture, the video is tracked and shot for a target, or the motion process of the target is extracted through a tracking algorithm, so that the method is difficult to be applied to complex scenes with more difficult target tracking people.
In a method based on human key point extraction, a document 'real Multi-person 2D position Estimation Using Part Affinity Fields' proposes a model for simultaneously predicting the position of a body Part and the position relation of each Part to extract human key points; the document "a Coarse-Fine Network for Keypoint Localization" proposes a Coarse-Fine multi-level supervised Network CFN (Coarse-Fine Network) for extracting key points of a human body. The method does not need to track the target, so the method can be suitable for human body posture recognition in a multi-person scene. The literature 'Neural Network Approach for 2-Dimension Person position Estimation With Encoded Mask and Keypoint Detection' extracts key points from an image segmentation Mask by using a convolution depth Neural Network, learns the interconnection relationship among the key points, and realizes the purpose of extracting human key points by combining image segmentation and bottom-up strategy. Most methods based on human body key point extraction only extract feature points, but do not make determination of human body posture. However, in practical application, it is often not enough to obtain only the characteristic points of the human body, and the given determination result of the posture can help people to quickly know the state of the individual in the scene and the occurred event, and is also an important basis for the intelligent monitoring system to automatically further analyze and judge the state of the event. Therefore, some methods for determining the posture of the human body based on the key point information have appeared. The document "Human position classification using sketch information" determines the Human body posture by using the distance between the key points of the Human body and the included angle of the key point connecting lines. However, in practical applications, there are often problems of errors in extracted key points, and missing key points due to occlusion, which may cause a sharp decrease in performance or even failure of the existing pose determination method.
Disclosure of Invention
The present invention provides a method and a system for determining a human body posture in a complex multi-person scene, so as to quickly know the state and occurrence of each individual in the scene.
The technical solution for realizing the purpose of the invention is as follows: a human body posture determination method for a complex multi-person scene, the method comprising the steps of:
step 1, detecting key points of a human body, and dividing a key point set;
step 2, extracting the statistical characteristics of the geometric distribution of the human body key points in each set;
and 3, judging the posture of the human body according to the statistical characteristics.
Further, step 1, detecting key points of the human body and dividing a key point set, wherein the specific process comprises:
step 1-1, constructing a sample training set, wherein the set comprises a plurality of human body images marked with human body key points;
step 1-2, training a deep convolutional neural network by using the sample training set;
step 1-3, detecting key points in a human body image to be detected by using a trained deep convolution neural network;
step 1-4, dividing a key point set, specifically:
removing key points of non-human body trunk, including hands, to form a whole body key point set;
dividing the key points of the human body trunk into an upper body key point set, a lower body key point set, a left arm key point set and a right arm key point set;
and continuously dividing the lower half key points into a left leg key point set, a right leg key point set, a left thigh key point set and a right thigh key point set.
Further, step 2, extracting the statistical characteristics of the geometric distribution of the human body key points in each set, wherein the specific process comprises:
step 2-1-1, calculating convex hulls of each key point set;
step 2-1-2, finding out two key points with the largest distance from the key points forming the convex hull;
and 2-1-3, calculating an included angle between the clockwise rotation and the horizontal direction of the connecting line of the two key points, and taking the included angle as the horizontal angle of the convex hull.
Further, the step 3 of determining the posture of the human body according to the statistical characteristics specifically includes: judging the postures of the arms, the legs and the whole body according to the horizontal angle of the convex hull of each key point set:
assuming that the horizontal included angle of the convex hull of the whole body key point set is alpha0(ii) a The horizontal included angle of the convex hull of the upper half key point set is alpha1(ii) a The horizontal included angle of the convex hull of the lower half key point set is alpha2(ii) a The horizontal included angle of the convex hull of the left arm key point set is alpha3(ii) a The horizontal included angle of the right arm key point set convex hull is alpha4(ii) a The horizontal included angle of the convex hull of the left leg key point set is alpha5(ii) a The horizontal included angle of the right leg key point set convex hull is alpha6(ii) a The horizontal included angle of the convex hull of the left thigh key point set is alpha7(ii) a The horizontal included angle of the right thigh key point set convex hull is alpha8；
If | tan α3|∈(a3,b3) Or | tan α4|∈(a3,b3) Judging the posture of the arm lifting; if | tan α5|∈(a5,b5) Or | tan α6|∈(a5,b5) Judging the posture of the kicking leg; if | tan α0|∈(a0,b0)、|tanα1|∈(a1,b1) And | tan α2|∈(a2,b2) Judging the posture to be a standing posture; if | tan α7|∈[a7,b7]Or | tan α8|∈[a7,b7]Judging the squatting posture; if | tan α0|∈[c0,d0]、|tanα1|∈[c1,d1]And | tan α2|∈[c2,d2]Then, the user is determined to be in the lying posture.
Further, step 2, extracting the statistical characteristics of the geometric distribution of the human body key points in each set, wherein the specific process comprises:
step 2-2-1, calculating the minimum circumscribed rectangle of each key point set;
step 2-2-2, respectively calculating included angles formed by the rotation of the long side and the short side of the minimum circumscribed rectangle along the clockwise direction and the horizontal direction;
and 2-2-3, taking the smaller value of the two included angles as the rotation angle of the minimum circumscribed rectangle.
Further, the step 3 of determining the posture of the human body according to the statistical characteristics specifically includes: judging the postures of the arms, the legs and the whole body according to the rotation angle of the minimum circumscribed rectangle of each key point set:
assuming that the horizontal included angle of the minimum circumscribed rectangle of the whole body key point set is beta0(ii) a The minimum external rectangle horizontal included angle of the upper half body key point set is beta1(ii) a The horizontal included angle of the minimum external rectangle of the lower half key point set is beta2(ii) a The horizontal included angle of the minimum circumscribed rectangle of the left arm key point set is beta3(ii) a The horizontal included angle of the minimum circumscribed rectangle of the right arm key point set is beta4(ii) a The horizontal included angle of the minimum circumscribed rectangle of the left leg key point set is beta5(ii) a The horizontal included angle of the minimum circumscribed rectangle of the right leg key point set is beta6(ii) a The horizontal included angle of the minimum circumscribed rectangle of the left thigh key point set is beta7(ii) a The horizontal included angle of the minimum circumscribed rectangle of the right thigh key point set is beta8；
If it is
Figure BDA0002661141280000041
Or
Figure BDA0002661141280000042
Then it is determined as the arm lifting postureState; if it is
Figure BDA0002661141280000043
Or
Figure BDA0002661141280000044
Judging the posture of the kicking leg; if it is
Figure BDA0002661141280000045
And is
Figure BDA0002661141280000046
Judging the posture of standing; if beta is1|∈[λ1,γ1]Judging the posture of bending waist; if it is
Figure BDA0002661141280000047
Or
Figure BDA0002661141280000048
Judging the squatting posture; if beta0|∈[θ0,Θ0]、β1|∈[θ1,Θ1]And beta is2|∈[θ2,Θ2]Then, the user is determined to be in the lying posture.
A body pose determination system for a complex multi-person scene, the system comprising:
the key point dividing module is used for detecting key points of a human body and dividing a key point set;
the statistical feature extraction module is used for extracting the statistical features of the geometric distribution of the human body key points in each set;
and the posture judgment module is used for judging the human body posture according to the statistical characteristics.
Compared with the prior art, the invention has the following remarkable advantages: 1) the method adopts the idea of dividing the human body key point set, avoids the problem of target extraction compared with a method based on video feature extraction, and can be suitable for judging the human body posture in a multi-person scene; 2) compared with the existing method based on human body key point extraction, the method converts the human body posture judgment problem into the statistical relationship of the key point set, so that the method is insensitive to the deletion and the displacement of partial key points, and can still obtain a correct posture judgment result according to several statistical characteristics of the set even under the condition that the key points are inaccurate due to the fact that partial body is shielded, thereby greatly improving the robustness of posture judgment; 3) the method has high judgment speed, can realize real-time judgment of the human body posture in the monitoring range, and helps people to quickly know the individual state and the occurrence event in the current scene.
The present invention is described in further detail below with reference to the attached drawing figures.
Drawings
FIG. 1 is a flow diagram of a method for human pose determination for a complex multi-person scene in one embodiment.
FIG. 2 is a diagram illustrating the partitioning of a set of human key points according to an embodiment.
FIG. 3 is a diagram illustrating the calculation of horizontal angles of convex hulls of a set of keypoints, according to an embodiment.
FIG. 4 is a diagram illustrating the rotation angle of the minimum bounding rectangle of the set of key points according to one embodiment.
FIG. 5 is a diagram of a determination of a basic pose of an individual person in one embodiment; wherein, the key points of the human body are marked by round points; the judgment result of the posture is marked on the head of the person, and the pictures (a) to (c) are respectively a standing posture, a squatting posture and a stooping posture.
Fig. 6 is a diagram illustrating the determination result of a single person in a complex posture in an embodiment, in which key points of a human body are marked with dots, the determination result of the posture is marked on the head of the person, and the diagrams (a) to (c) are respectively a standing arm-raising posture, a standing leg-kicking posture and a standing arm-raising leg-kicking posture.
FIG. 7 is a diagram illustrating the determination result of a multi-person complex pose in an embodiment, wherein key points of a human body are marked with dots, and the determination result of the pose is marked on the head of the person.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
It should be noted that if the description of "first", "second", etc. is provided in the embodiment of the present invention, the description of "first", "second", etc. is only for descriptive purposes and is not to be construed as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In addition, technical solutions between various embodiments may be combined with each other, but must be realized by a person skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present invention.
In one embodiment, in conjunction with fig. 1, there is provided a human body posture determination method for a complex multi-person scene, the method comprising the steps of:
step 1, detecting key points of a human body, and dividing a key point set;
step 2, extracting the statistical characteristics of the geometric distribution of the human body key points in each set;
and 3, judging the posture of the human body according to the statistical characteristics.
Further, in one embodiment, the step 1 of detecting key points of a human body and dividing a key point set includes:
step 1-1, constructing a sample training set, wherein the set comprises a plurality of human body images marked with human body key points;
step 1-2, training a deep convolutional neural network by using the sample training set;
step 1-3, detecting key points in a human body image to be detected by using a trained deep convolution neural network;
step 1-4, with reference to fig. 2, dividing a key point set, specifically:
removing key points of non-human body trunk, including hands, to form a whole body key point set;
dividing the key points of the human body trunk into an upper body key point set, a lower body key point set, a left arm key point set and a right arm key point set;
and continuously dividing the lower half key points into a left leg key point set, a right leg key point set, a left thigh key point set and a right thigh key point set.
Here, human keypoints are extracted Using the method in the document "real Multi-person 2D position Estimation Using Part Affinity Fields".
Here, the human trunk key point set includes ten key points of a left eye, a right eye, a mouth, a neck, a left hip joint, a right hip joint, a left knee joint, a right knee joint, a left ankle joint and a right ankle joint;
the upper half key point set comprises six key points of a left eye, a right eye, a mouth, a neck, a left hip joint and a right hip joint;
the lower half key point set comprises six key points, namely a left hip joint, a right hip joint, a left knee joint, a right knee joint, a left ankle joint and a right ankle joint;
the left arm key point set comprises a left wrist joint, a left elbow joint and a left shoulder joint, and the three key points are total;
the right arm key point set comprises a right wrist joint, a right elbow joint and a right shoulder joint, and the three key points are all included;
the left leg key point set comprises a left ankle joint, a left knee joint and a left hip joint, and the three key points are total;
the right leg key point set comprises a right ankle joint, a right knee joint and a right hip joint, and the three key points are all included;
the left thigh key point set comprises a left knee joint and a left hip joint, and the two key points are total;
the right thigh key point set comprises a right knee joint and a right hip joint, and the two key points are total.
Further, in one embodiment, with reference to fig. 3, the step 2 of extracting statistical features of geometric distribution of the human body key points in each set includes:
step 2-1-1, calculating convex hulls of each key point set;
step 2-1-2, finding out two key points with the largest distance from the key points forming the convex hull;
and 2-1-3, calculating an included angle between the clockwise rotation and the horizontal direction of the connecting line of the two key points, and taking the included angle as the horizontal angle of the convex hull.
Further, in one embodiment, the step 3 of determining the posture of the human body according to the statistical features specifically includes: judging the postures of the arms, the legs and the whole body according to the horizontal angle of the convex hull of each key point set:
assuming that the horizontal included angle of the convex hull of the whole body key point set is alpha0(ii) a The horizontal included angle of the convex hull of the upper half key point set is alpha1(ii) a The horizontal included angle of the convex hull of the lower half key point set is alpha2(ii) a The horizontal included angle of the convex hull of the left arm key point set is alpha3(ii) a The horizontal included angle of the right arm key point set convex hull is alpha4(ii) a The horizontal included angle of the convex hull of the left leg key point set is alpha5(ii) a The horizontal included angle of the right leg key point set convex hull is alpha6(ii) a The horizontal included angle of the convex hull of the left thigh key point set is alpha7(ii) a The horizontal included angle of the right thigh key point set convex hull is alpha8；
If | tan α3|∈(a3,b3) Or | tan α4|∈(a3,b3) Judging the posture of the arm lifting; if | tan α5|∈(a5,b5) Or | tan α6|∈(a5,b5) Judging the posture of the kicking leg; if | tan α0|∈(a0,b0)、|tanα1|∈(a1,b1) And | tan α2|∈(a2,b2) Judging the posture to be a standing posture; if | tan α7|∈[a7,b7]Or | tan α8|∈[a7,b7]Judging the squatting posture; if | tan α0|∈[c0,d0]、|tanα1|∈[c1,d1]And | tan α2|∈[c2,d2]Then, the user is determined to be in the lying posture.
Preferably here (a)3,b3)＝(0.25,+∞)，(a5,b5)＝(0.25,5)，(a0,b0)＝(a1,b1)＝(a2,b2)＝(4,+∞)，[a7,b7]＝[0,0.25]，[c0,d0]＝[c1,d1]＝[c2,d2]＝[0,0.25]。
Further, in one embodiment, with reference to fig. 4, the step 2 of extracting statistical features of geometric distribution of the human body key points in each set includes:
step 2-2-1, calculating the minimum circumscribed rectangle of each key point set;
step 2-2-2, respectively calculating included angles formed by the rotation of the long side and the short side of the minimum circumscribed rectangle along the clockwise direction and the horizontal direction;
and 2-2-3, taking the smaller value of the two included angles as the rotation angle of the minimum circumscribed rectangle.
Further, in one embodiment, the step 3 of determining the posture of the human body according to the statistical features specifically includes: judging the postures of the arms, the legs and the whole body according to the rotation angle of the minimum circumscribed rectangle of each key point set:
assuming that the horizontal included angle of the minimum circumscribed rectangle of the whole body key point set is beta0(ii) a The minimum external rectangle horizontal included angle of the upper half body key point set is beta1(ii) a The horizontal included angle of the minimum external rectangle of the lower half key point set is beta2(ii) a The horizontal included angle of the minimum circumscribed rectangle of the left arm key point set is beta3(ii) a The horizontal included angle of the minimum circumscribed rectangle of the right arm key point set is beta4(ii) a The horizontal included angle of the minimum circumscribed rectangle of the left leg key point set is beta5(ii) a The horizontal included angle of the minimum circumscribed rectangle of the right leg key point set is beta6(ii) a The horizontal included angle of the minimum circumscribed rectangle of the left thigh key point set is beta7(ii) a The horizontal included angle of the minimum circumscribed rectangle of the right thigh key point set is beta8；
If it is
Figure BDA0002661141280000081
Or
Figure BDA0002661141280000082
Judging the posture of the arm lifting; if it is
Figure BDA0002661141280000083
Or
Figure BDA0002661141280000084
Judging the posture of the kicking leg; if it is
Figure BDA0002661141280000085
And is
Figure BDA0002661141280000086
Judging the posture of standing; if beta is1|∈[λ1,γ1]Judging the posture of bending waist; if it is
Figure BDA0002661141280000087
Or
Figure BDA0002661141280000088
Judging the squatting posture; if beta0|∈[θ0,Θ0]、β1|∈[θ1,Θ1]And beta is2|∈[θ2,Θ2]Then, the user is determined to be in the lying posture.
It is preferable here that the first and second parts,
Figure BDA0002661141280000089
[λ1,γ1]＝[0°,25°]，
Figure BDA00026611412800000810
[θ0,Θ0]＝[θ1,Θ1]＝[θ2,Θ2]＝[0°,25°]。
in one embodiment, there is provided a human pose determination system for a complex multi-person scene, the system comprising:
the key point dividing module is used for detecting key points of a human body and dividing a key point set;
the statistical feature extraction module is used for extracting the statistical features of the geometric distribution of the human body key points in each set;
and the posture judgment module is used for judging the human body posture according to the statistical characteristics.
Further, in one embodiment, the keypoint splitting module comprises:
the training set constructing unit is used for constructing a sample training set, and the set comprises a plurality of human body images marked with human body key points;
a training unit for training a deep convolutional neural network using the sample training set;
the key point detection unit is used for detecting key points in the human body image to be detected by using the trained deep convolutional neural network;
the dividing unit is used for dividing the key point set, and specifically comprises:
the first dividing unit is used for removing key points of the trunk of the non-human body, including hands, and forming a whole body key point set;
the second dividing subunit is used for dividing the key points of the human body into an upper body key point set, a lower body key point set, a left arm key point set and a right arm key point set;
and the third dividing subunit is used for continuously dividing the lower body key points into a left leg key point set, a right leg key point set, a left thigh key point set and a right thigh key point set.
Further, in one embodiment, the statistical feature extraction module includes:
the first calculating unit is used for calculating convex hulls of all the key point sets;
the key point screening unit is used for finding out two key points with the largest distance from the key points forming the convex hull;
the second calculation unit is used for calculating an included angle between the clockwise rotation and the horizontal direction of the connecting line of the two key points as the horizontal angle of the convex hull;
the gesture judging module is used for judging the gestures of the arms, the legs and the whole body according to the horizontal angles of the convex hulls of the key point sets:
assuming that the horizontal included angle of the convex hull of the whole body key point set is alpha0(ii) a The horizontal included angle of the convex hull of the upper half key point set is alpha1(ii) a The horizontal included angle of the convex hull of the lower half key point set is alpha2(ii) a The horizontal included angle of the convex hull of the left arm key point set is alpha3(ii) a The horizontal included angle of the right arm key point set convex hull is alpha4(ii) a The horizontal included angle of the convex hull of the left leg key point set is alpha5(ii) a The horizontal included angle of the right leg key point set convex hull is alpha6(ii) a The horizontal included angle of the convex hull of the left thigh key point set is alpha7(ii) a The horizontal included angle of the right thigh key point set convex hull is alpha8；
If | tan α3|∈(a3,b3) Or | tan α4|∈(a3,b3) Judging the posture of the arm lifting; if | tan α5|∈(a5,b5) Or | tan α6|∈(a5,b5) Judging the posture of the kicking leg; if | tan α0|∈(a0,b0)、|tanα1|∈(a1,b1) And | tan α2|∈(a2,b2) Judging the posture to be a standing posture; if | tan α7|∈[a7,b7]Or | tan α8|∈[a7,b7]Judging the squatting posture; if | tan α0|∈[c0,d0]、|tanα1|∈[c1,d1]And | tan α2|∈[c2,d2]Then, the user is determined to be in the lying posture.
Further, in one embodiment, the statistical feature extraction module includes:
the third calculation unit is used for calculating the minimum circumscribed rectangle of each key point set;
the fourth calculation unit is used for calculating included angles formed by the rotation of the long side and the short side of the minimum circumscribed rectangle along the clockwise direction and the horizontal direction respectively;
the fifth calculation unit is used for taking the smaller value of the two included angles as the rotation angle of the minimum circumscribed rectangle;
the gesture judging module is used for judging the gestures of the arms, the legs and the whole body according to the rotation angle of the minimum circumscribed rectangle of each key point set:
assuming that the horizontal included angle of the minimum circumscribed rectangle of the whole body key point set is beta0(ii) a The minimum external rectangle horizontal included angle of the upper half body key point set is beta1(ii) a The horizontal included angle of the minimum external rectangle of the lower half key point set is beta2(ii) a The horizontal included angle of the minimum circumscribed rectangle of the left arm key point set is beta3(ii) a The horizontal included angle of the minimum circumscribed rectangle of the right arm key point set is beta4(ii) a The horizontal included angle of the minimum circumscribed rectangle of the left leg key point set is beta5(ii) a The horizontal included angle of the minimum circumscribed rectangle of the right leg key point set is beta6(ii) a The horizontal included angle of the minimum circumscribed rectangle of the left thigh key point set is beta7(ii) a The horizontal included angle of the minimum circumscribed rectangle of the right thigh key point set is beta8；
If it is
Figure BDA0002661141280000101
Or
Figure BDA0002661141280000102
Judging the posture of the arm lifting; if it is
Figure BDA0002661141280000103
Or
Figure BDA0002661141280000104
Judging the posture of the kicking leg; if it is
Figure BDA0002661141280000105
And is
Figure BDA0002661141280000106
Judging the posture of standing; if beta is1|∈[λ1,γ1]Judging the posture of bending waist; if it is
Figure BDA0002661141280000107
Or
Figure BDA0002661141280000108
Judging the squatting posture; if beta0|∈[θ0,Θ0]、β1|∈[θ1,Θ1]And beta is2|∈[θ2,Θ2]Then, the user is determined to be in the lying posture.
For specific limitations of the human body posture determination system for a complex multi-person scene, reference may be made to the above limitations on the human body posture determination method for a complex multi-person scene, and details are not repeated here. All modules in the human body posture determination system for the complex multi-person scene can be completely or partially realized through software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
As a specific example, in one embodiment, the present invention is further verified and explained, in which the human body posture in the surveillance video is determined by using the above-mentioned method of minimum external rectangle, and the test results are shown in fig. 5, fig. 6, fig. 7 and table 1.
TABLE 1 statistics of human body posture determination results
Figure BDA0002661141280000109
As can be seen from the determination results of multiple basic postures of a single person in FIG. 5 and the determination results of multiple complex postures of a single person in FIG. 6, the present invention can determine the postures of a single person more accurately. As can be seen from the determination result of the multi-person complex gesture with mutual occlusion in FIG. 7, the invention can determine the gestures of multiple persons at the same time, and is insensitive to the occlusion and other influences. As shown in table 1, the average accuracy of the present invention for determining the posture of the human body is 92.833%, wherein the accuracy for determining the posture with obvious discrimination such as standing, squatting, arm raising, lying and the like is higher, and the accuracy for determining the posture with confusion such as bending, kicking and the like is slightly lower.
The foregoing illustrates and describes the principles, general features, and advantages of the present invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.

Claims (10)

1. A human body posture judgment method for a complex multi-person scene is characterized by comprising the following steps:
step 1, detecting key points of a human body, and dividing a key point set;
step 2, extracting the statistical characteristics of the geometric distribution of the human body key points in each set;
and 3, judging the posture of the human body according to the statistical characteristics.
2. The method for determining the human body posture in the complex multi-person scene according to claim 1, wherein the step 1 of detecting the human body key points and dividing the key point set comprises the following specific processes:
step 1-1, constructing a sample training set, wherein the set comprises a plurality of human body images marked with human body key points;
step 1-2, training a deep convolutional neural network by using the sample training set;
step 1-3, detecting key points in a human body image to be detected by using a trained deep convolution neural network;
step 1-4, dividing a key point set, specifically:
removing key points of non-human body trunk, including hands, to form a whole body key point set;
dividing the key points of the human body trunk into an upper body key point set, a lower body key point set, a left arm key point set and a right arm key point set;
and continuously dividing the lower half key points into a left leg key point set, a right leg key point set, a left thigh key point set and a right thigh key point set.
3. The method for determining human body posture in a complex multi-person scene according to claim 1 or 2, wherein the step 2 of extracting statistical features of the geometric distribution of the human body key points in each set comprises the following specific steps:
step 2-1-1, calculating convex hulls of each key point set;
step 2-1-2, finding out two key points with the largest distance from the key points forming the convex hull;
and 2-1-3, calculating an included angle between the clockwise rotation and the horizontal direction of the connecting line of the two key points, and taking the included angle as the horizontal angle of the convex hull.
4. The method for determining human body posture in a complex multi-person scene as claimed in claim 3, wherein the step 3 of determining human body posture according to the statistical features specifically comprises: judging the postures of the arms, the legs and the whole body according to the horizontal angle of the convex hull of each key point set:
assuming that the horizontal included angle of the convex hull of the whole body key point set is alpha0(ii) a The horizontal included angle of the convex hull of the upper half key point set is alpha1(ii) a The horizontal included angle of the convex hull of the lower half key point set is alpha2(ii) a The horizontal included angle of the convex hull of the left arm key point set is alpha3(ii) a The horizontal included angle of the right arm key point set convex hull is alpha4(ii) a The horizontal included angle of the convex hull of the left leg key point set is alpha5(ii) a The horizontal included angle of the right leg key point set convex hull is alpha6(ii) a The horizontal included angle of the convex hull of the left thigh key point set is alpha7(ii) a The horizontal included angle of the right thigh key point set convex hull is alpha8；
If | tan α3|∈(a3,b3) Or | tan α4|∈(a3,b3) Judging the posture of the arm lifting; if | tan α5|∈(a5,b5) Or | tan α6|∈(a5,b5) Then it is determined asA kicking posture; if | tan α0|∈(a0,b0)、|tanα1|∈(a1,b1) And | tan α2|∈(a2,b2) Judging the posture to be a standing posture; if | tan α7|∈[a7,b7]Or | tan α8|∈[a7,b7]Judging the squatting posture; if | tan α0|∈[c0,d0]、|tanα1|∈[c1,d1]And | tan α2|∈[c2,d2]Then, the user is determined to be in the lying posture.
5. The method for determining human body posture in a complex multi-person scene according to claim 1 or 2, wherein the step 2 of extracting statistical features of the geometric distribution of the human body key points in each set comprises the following specific steps:
step 2-2-1, calculating the minimum circumscribed rectangle of each key point set;
step 2-2-2, respectively calculating included angles formed by the rotation of the long side and the short side of the minimum circumscribed rectangle along the clockwise direction and the horizontal direction;
and 2-2-3, taking the smaller value of the two included angles as the rotation angle of the minimum circumscribed rectangle.
6. The method for determining human body posture in a complex multi-person scene as claimed in claim 5, wherein the step 3 of determining human body posture according to the statistical features specifically comprises: judging the postures of the arms, the legs and the whole body according to the rotation angle of the minimum circumscribed rectangle of each key point set:
assuming that the horizontal included angle of the minimum circumscribed rectangle of the whole body key point set is beta0(ii) a The minimum external rectangle horizontal included angle of the upper half body key point set is beta1(ii) a The horizontal included angle of the minimum external rectangle of the lower half key point set is beta2(ii) a The horizontal included angle of the minimum circumscribed rectangle of the left arm key point set is beta3(ii) a The horizontal included angle of the minimum circumscribed rectangle of the right arm key point set is beta4(ii) a The horizontal included angle of the minimum circumscribed rectangle of the left leg key point set is beta5(ii) a Right leg jointThe horizontal included angle of the minimum circumscribed rectangle of the key point set is beta6(ii) a The horizontal included angle of the minimum circumscribed rectangle of the left thigh key point set is beta7(ii) a The horizontal included angle of the minimum circumscribed rectangle of the right thigh key point set is beta8；
If it is
Figure FDA0002661141270000021
Or
Figure FDA0002661141270000022
Judging the posture of the arm lifting; if it is
Figure FDA0002661141270000023
Or
Figure FDA0002661141270000024
Judging the posture of the kicking leg; if it is
Figure FDA0002661141270000025
And is
Figure FDA0002661141270000026
Judging the posture of standing; if beta is1|∈[λ1,γ1]Judging the posture of bending waist; if it is
Figure FDA0002661141270000027
Or
Figure FDA0002661141270000028
Judging the squatting posture; if beta0|∈[θ0,Θ0]、β1|∈[θ1,Θ1]And beta is2|∈[θ2,Θ2]Then, the user is determined to be in the lying posture.
7. A body pose determination system for a complex multi-person scene, the system comprising:
the key point dividing module is used for detecting key points of a human body and dividing a key point set;
the statistical feature extraction module is used for extracting the statistical features of the geometric distribution of the human body key points in each set;
and the posture judgment module is used for judging the human body posture according to the statistical characteristics.
8. The system of claim 7, wherein the keypoint segmentation module comprises:
the training set constructing unit is used for constructing a sample training set, and the set comprises a plurality of human body images marked with human body key points;
a training unit for training a deep convolutional neural network using the sample training set;
the key point detection unit is used for detecting key points in the human body image to be detected by using the trained deep convolutional neural network;
the dividing unit is used for dividing the key point set, and specifically comprises:
the first dividing unit is used for removing key points of the trunk of the non-human body, including hands, and forming a whole body key point set;
the second dividing subunit is used for dividing the key points of the human body into an upper body key point set, a lower body key point set, a left arm key point set and a right arm key point set;
and the third dividing subunit is used for continuously dividing the lower body key points into a left leg key point set, a right leg key point set, a left thigh key point set and a right thigh key point set.
9. The system of claim 8, wherein the statistical feature extraction module comprises:
the first calculating unit is used for calculating convex hulls of all the key point sets;
the key point screening unit is used for finding out two key points with the largest distance from the key points forming the convex hull;
the second calculation unit is used for calculating an included angle between the clockwise rotation and the horizontal direction of the connecting line of the two key points as the horizontal angle of the convex hull;
the gesture judging module is used for judging the gestures of the arms, the legs and the whole body according to the horizontal angles of the convex hulls of the key point sets:
assuming that the horizontal included angle of the convex hull of the whole body key point set is alpha0(ii) a The horizontal included angle of the convex hull of the upper half key point set is alpha1(ii) a The horizontal included angle of the convex hull of the lower half key point set is alpha2(ii) a The horizontal included angle of the convex hull of the left arm key point set is alpha3(ii) a The horizontal included angle of the right arm key point set convex hull is alpha4(ii) a The horizontal included angle of the convex hull of the left leg key point set is alpha5(ii) a The horizontal included angle of the right leg key point set convex hull is alpha6(ii) a The horizontal included angle of the convex hull of the left thigh key point set is alpha7(ii) a The horizontal included angle of the right thigh key point set convex hull is alpha8；
If | tan α3|∈(a3,b3) Or | tan α4|∈(a3,b3) Judging the posture of the arm lifting; if | tan α5|∈(a5,b5) Or | tan α6|∈(a5,b5) Judging the posture of the kicking leg; if | tan α0|∈(a0,b0)、|tanα1|∈(a1,b1) And | tan α2|∈(a2,b2) Judging the posture to be a standing posture; if | tan α7|∈[a7,b7]Or | tan α8|∈[a7,b7]Judging the squatting posture; if | tan α0|∈[c0,d0]、|tanα1|∈[c1,d1]And | tan α2|∈[c2,d2]Then, the user is determined to be in the lying posture.
10. The system of claim 8, wherein the statistical feature extraction module comprises:
the third calculation unit is used for calculating the minimum circumscribed rectangle of each key point set;
the fourth calculation unit is used for calculating included angles formed by the rotation of the long side and the short side of the minimum circumscribed rectangle along the clockwise direction and the horizontal direction respectively;
the fifth calculation unit is used for taking the smaller value of the two included angles as the rotation angle of the minimum circumscribed rectangle;
the gesture judging module is used for judging the gestures of the arms, the legs and the whole body according to the rotation angle of the minimum circumscribed rectangle of each key point set:
assuming that the horizontal included angle of the minimum circumscribed rectangle of the whole body key point set is beta0(ii) a The minimum external rectangle horizontal included angle of the upper half body key point set is beta1(ii) a The horizontal included angle of the minimum external rectangle of the lower half key point set is beta2(ii) a The horizontal included angle of the minimum circumscribed rectangle of the left arm key point set is beta3(ii) a The horizontal included angle of the minimum circumscribed rectangle of the right arm key point set is beta4(ii) a The horizontal included angle of the minimum circumscribed rectangle of the left leg key point set is beta5(ii) a The horizontal included angle of the minimum circumscribed rectangle of the right leg key point set is beta6(ii) a The horizontal included angle of the minimum circumscribed rectangle of the left thigh key point set is beta7(ii) a The horizontal included angle of the minimum circumscribed rectangle of the right thigh key point set is beta8；
If it is
Figure FDA0002661141270000041
Or
Figure FDA0002661141270000042
Judging the posture of the arm lifting; if it is
Figure FDA0002661141270000043
Or
Figure FDA0002661141270000044
Judging the posture of the kicking leg; if it is
Figure FDA0002661141270000045
And is
Figure FDA0002661141270000046
Judging the posture of standing; if beta is1|∈[λ1,γ1]Judging the posture of bending waist; if it isOr
Figure FDA0002661141270000048
Judging the squatting posture; if beta0|∈[θ0,Θ0]、β1|∈[θ1,Θ1]And beta is2|∈[θ2,Θ2]Then, the user is determined to be in the lying posture.
CN202010905107.6A 2020-09-01 2020-09-01 Human body posture judgment method and system for complex multi-person scene Pending CN112036324A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010905107.6A CN112036324A (en) 2020-09-01 2020-09-01 Human body posture judgment method and system for complex multi-person scene

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010905107.6A CN112036324A (en) 2020-09-01 2020-09-01 Human body posture judgment method and system for complex multi-person scene

Publications (1)

Publication Number Publication Date
CN112036324A true CN112036324A (en) 2020-12-04

Family

ID=73590874

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010905107.6A Pending CN112036324A (en) 2020-09-01 2020-09-01 Human body posture judgment method and system for complex multi-person scene

Country Status (1)

Country Link
CN (1) CN112036324A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113537234A (en) * 2021-06-10 2021-10-22 浙江大华技术股份有限公司 Quantity counting method and device, electronic device and computer equipment
CN114155518A (en) * 2021-11-08 2022-03-08 西安西光产业发展有限公司 Expressway visor inclination identification method based on deep semantic segmentation network and image correction

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113537234A (en) * 2021-06-10 2021-10-22 浙江大华技术股份有限公司 Quantity counting method and device, electronic device and computer equipment
CN114155518A (en) * 2021-11-08 2022-03-08 西安西光产业发展有限公司 Expressway visor inclination identification method based on deep semantic segmentation network and image correction
CN114155518B (en) * 2021-11-08 2024-09-27 西安西光产业发展有限公司 Highway light shield inclination recognition method based on depth semantic segmentation network and image correction

Similar Documents

Publication Publication Date Title
Parameswaran et al. View invariance for human action recognition
WO2020042419A1 (en) Gait-based identity recognition method and apparatus, and electronic device
Bobick et al. The recognition of human movement using temporal templates
Iwasawa et al. Real-time estimation of human body posture from monocular thermal images
Jojic et al. Tracking self-occluding articulated objects in dense disparity maps
CN114067358A (en) Human body posture recognition method and system based on key point detection technology
Roche et al. A multimodal data processing system for LiDAR-based human activity recognition
Singh et al. Action recognition in cluttered dynamic scenes using pose-specific part models
CN110008913A (en) Pedestrian re-identification method based on fusion of attitude estimation and viewpoint mechanism
CN108875586B (en) Functional limb rehabilitation training detection method based on depth image and skeleton data multi-feature fusion
Pellegrini et al. Human posture tracking and classification through stereo vision and 3d model matching
CN112036324A (en) Human body posture judgment method and system for complex multi-person scene
CN112966628A (en) Visual angle self-adaptive multi-target tumble detection method based on graph convolution neural network
Chen et al. A video‐based human fall detection system for smart homes
Nguyen et al. Combined YOLOv5 and HRNet for high accuracy 2D keypoint and human pose estimation
Chan Detection of swimmer using dense optical flow motion map and intensity information
Haker et al. Self-organizing maps for pose estimation with a time-of-flight camera
Iwasawa et al. Real-time human posture estimation using monocular thermal images
Zarka et al. Real-time human motion detection and tracking
Medioni et al. Non-cooperative persons identification at a distance with 3D face modeling
JP2017097549A (en) Image processing apparatus, method, and program
Lv et al. Cheating behavior detection based-on pictorial structure model
CN114170685A (en) RGBD image-based detection method for falling behavior of pedestrian riding escalator
Hazra et al. A pilot study for investigating gait signatures in multi-scenario applications
Tian et al. Multi-scale human pose tracking in 2D monocular images

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination