CN115690895A - Human skeleton point detection-based multi-person motion detection method and device - Google Patents
Human skeleton point detection-based multi-person motion detection method and device Download PDFInfo
- Publication number
- CN115690895A CN115690895A CN202210963067.XA CN202210963067A CN115690895A CN 115690895 A CN115690895 A CN 115690895A CN 202210963067 A CN202210963067 A CN 202210963067A CN 115690895 A CN115690895 A CN 115690895A
- Authority
- CN
- China
- Prior art keywords
- human
- key
- skeleton
- motion
- points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention discloses a method and a device for detecting multi-person motion based on human body bone point detection. The method comprises the following steps: s100: sending a starting signal, and simultaneously starting video recording and countdown; s200: extracting action images of the time sequence, calculating the confidence and the affinity of the key points of the human skeleton, and drawing a connection diagram of the key points of the human skeleton; s300: calculating the time-space domain characteristics of key skeletal points, and identifying effective actions through action classification and motion tracking; s400: counting the motion of the human body based on the motion classification result; s500: sending out an end signal, stopping video recording, and ending counting; s600: and returning a counting result. The invention is based on the human skeleton point detection technology, analyzes the change characteristics of the time domain and the space domain of the human skeleton key points in the motion process, identifies effective actions through action classification and motion tracking, and can improve the accuracy of motion counting.
Description
Technical Field
The invention relates to a multi-person motion detection method based on human body bone point detection, and also relates to a corresponding multi-person motion detection device, belonging to the technical field of computer vision.
Background
At present, a group of students is divided into two groups for rope skipping test in primary and secondary schools, wherein a plurality of persons in one group skip ropes at the same time, and a plurality of persons in the other group skip rope actions for counting. Although this method is simple and easy to implement, it has a large error and is liable to cause unnecessary disputes. In the prior art, a skipping rope counting handle with a mechanical structure or an electronic sensor or a carpet with a touch switch type can be used for realizing intelligent counting, but the intelligent counting mode needs to purchase a large number of skipping rope devices with the functions, so that the investment cost is high, and the skipping rope devices are easy to damage if the skipping rope counting device is not used properly. Another method is to use wearable wrist-type smart devices, such as smart bands, smart watches, etc., and determine the count using motion sensors in these smart devices. However, these smart devices cannot accurately determine a movement such as shaking an arm or standing on tiptoe, and thus have a problem that the count is not accurate enough. Therefore, the existing intelligent counting mode is not suitable for rope skipping examinations of primary and secondary school students.
In a Chinese invention patent with the patent number ZL 202010887939.X, a skipping rope counting method based on multi-target tracking is disclosed, and comprises the following steps: 1) Acquiring original video data of rope skipping actions, and extracting image data from the original video data; 2) Performing single-frame processing on the image data to obtain a group of sequentially arranged single-frame images; 3) Acquiring initial positions and size information of all face targets; 4) Tracking each human face target in each frame of image respectively, and displaying the tracking results of all the human face targets on the same frame of image; 5) Judging and identifying rope skipping persons and non-rope skipping persons in the single-frame image; 6) Carrying out rope skipping counting on all rope skipping personnel in the single-frame image; 7) And outputting and displaying the result. The rope skipping and non-rope skipping personnel are identified by identifying and tracking the height position change of the face, the rope skipping times of a plurality of rope skipping personnel are judged, the method is accurate, manual counting is not needed, the video can be back-molded, and the method has high application value. However, the skipping rope counting method requires skipping rope in situ, and cannot move horizontally back and forth, left and right. Meanwhile, the method cannot effectively judge invalid actions such as rope treading, no jumping and the like.
Disclosure of Invention
The invention aims to solve the primary technical problem of providing a multi-person motion detection method based on human body bone point detection.
Another technical problem to be solved by the present invention is to provide a multi-user motion detection device based on human body bone point detection.
In order to achieve the purpose, the invention adopts the following technical scheme:
according to a first aspect of the embodiments of the present invention, there is provided a method for detecting motion of multiple persons based on human skeleton point detection, including the following steps:
s100: sending a starting signal, and simultaneously starting video recording and countdown;
s200: extracting action images of the time sequence, calculating the confidence coefficient and affinity of the key points of the human skeleton, and drawing a connection graph of the key points of the human skeleton;
s300: calculating the time-space domain characteristics of key skeletal points, and identifying effective actions through action classification and motion tracking;
s400: counting the motion of the human body based on the motion classification result;
s500: sending an end signal, stopping video recording, and ending counting;
s600: and returning a counting result.
Preferably, the step S200 includes the following sub-steps:
s201: acquiring an original video of multi-person movement;
s202: extracting action images based on time sequences from an original video to detect key skeleton points of a human body;
s203: detecting the motion image by using an OpenPose model to obtain a point bitmap of at least one human body key bone point, and acquiring parameters of the obtained human body key bone point;
s204: respectively calculating the confidence coefficient and the affinity of each bone key point according to the human bone key point parameters obtained in the step S203, calculating the connection weight of the human bone key points and drawing a human bone key point connection graph;
s205: and drawing a skeleton key point connection diagram of each human body according to the connection relation of the human body skeleton key points in the S204.
Preferably, the step S300 includes the following sub-steps:
s301: calculating the time-space domain characteristics of the key skeleton points;
s302: establishing a data set of key points of human bones;
s303: building a stacking model and training;
s304: performing action classification by using the stack model;
s305: and identifying effective rope skipping actions according to the consistency of the actions.
Preferentially, the connection weight of the key points of the human skeletons is calculated through Hungarian algorithm, and a connection diagram of the key points of the human skeletons is drawn.
Preferably, in step S203, a point bitmap of 16 human key skeletal points is obtained.
Preferably, in step S203, the obtained 16 key skeleton points of the human body include a head, a neck, a left shoulder, a left elbow, a left wrist, a right shoulder, a right elbow, a right wrist, a spine, a hip center point, a right hip, a left hip, a right knee, a left knee, a right ankle and a left ankle.
Preferably, the bone key point parameters obtained in step S203 are normalized.
Preferably, the time-space domain features of the key skeletal points are calculated based on the normalized parameters.
According to a second aspect of the embodiments of the present invention, there is provided a multi-person motion detection apparatus based on human skeleton point detection, comprising a processor, a memory, a display screen and a speaker,
wherein the memory stores a computer program for executing the above-mentioned multi-person motion detection method based on human skeleton point detection;
the processor calls the computer program to realize a multi-person motion detection method based on human body bone point detection;
the display screen provides a human-computer interaction interface.
Preferably, the multi-person motion detection device is a smart phone, a tablet computer or a personal computer.
Compared with the prior art, the invention has the following technical effects:
(1) Based on the human skeleton point detection technology, the change characteristics of the time domain and the space domain of the human skeleton key points in the rope skipping process are analyzed, effective rope skipping actions are identified through action classification and motion tracking, and the accuracy of rope skipping counting can be improved.
(2) The method adopts a plurality of human body skeleton key points to draw the human body skeleton key point connection diagram by Hungary algorithm, is favorable for distinguishing human body skeleton key point overlap in multi-person motion, and improves the accuracy of identifying multi-person motion.
(3) The skipping rope counting is carried out by a computer vision method, so that the cost can be reduced; the examination efficiency can be improved, and the simultaneous counting of multiple persons can be realized.
Drawings
FIG. 1 is a schematic structural diagram of a multi-user motion detection apparatus based on human skeleton point detection according to an embodiment of the present invention;
FIG. 2 is a flowchart of a multi-user motion detection method based on human skeleton point detection according to an embodiment of the present invention;
FIG. 3 is a flowchart of an embodiment of the present invention, illustrating a skeleton key point connection diagram;
FIG. 4 is a schematic diagram of a skeleton key point line graph in an embodiment of the present invention;
FIG. 5 is a flow chart of action recognition according to an embodiment of the present invention;
FIG. 6 is a diagram illustrating training results of a training set and a test set according to an embodiment of the present invention;
FIG. 7 is a schematic diagram illustrating rope skipping operation recognition according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a connecting line of bone key points in motion according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of coordinates and angles of key points of the bone shown by the dashed circle in FIG. 8
FIG. 10 is a diagram illustrating accuracy of motion recognition according to an embodiment of the present invention.
Detailed Description
The technical contents of the invention are specifically described in the following with reference to the accompanying drawings and specific embodiments.
The multi-person movement counting method based on the skeleton key point detection is particularly suitable for a scene of counting the movement of multiple persons at the same time. For convenience of understanding, the following description will be given only by taking the counting of the rope skipping at the same time by the same person as an example. However, it can be understood by those skilled in the art that the method for counting movements of multiple persons based on skeletal key point detection provided by the present invention can also be used for counting or detecting other movements, for example, detecting movements of multiple kicks to analyze the normative of the movements, or detecting and analyzing movements of multiple persons swimming.
Fig. 1 is a diagram illustrating a multi-person motion counting apparatus for detecting skeletal key points according to an embodiment of the present invention. The device can be equipment with an image processing function, such as a smart phone, a tablet personal computer or a personal computer, and can also be special electronic equipment. The device includes a processor, a memory, a display screen, and a speaker. The memory stores a computer program for executing the multi-person motion counting method for detecting the skeletal key points, and the computer program is called by the processor. The display screen provides a man-machine interaction interface and can realize the functions of starting, setting, ending, score inquiry, data export and the like; the speaker is used to give an indication of starting or ending a movement, etc.
As shown in fig. 2, the embodiment of the invention discloses a multi-person motion detection method based on human skeleton point detection, which comprises the following steps:
s100: and sending a starting signal, and simultaneously starting video recording and countdown.
Specifically, when the rope skipping test is started, a teacher clicks a start button on a display screen of the terminal device, the terminal device sends a starting voice signal after receiving a start signal, the timer starts a 1-minute countdown program, the camera starts to record rope skipping videos, the counter starts to count the number of rope skipping, and meanwhile, students start rope skipping after hearing the voice signal.
Specifically, since the rope skipping test is to examine the number of rope skipping of students within 1 minute, before the start, the counter of the terminal device is in a zero clearing state, that is, the rope skipping count of the students starts from zero. If the number of the skipping ropes represented by the skipping rope counting signal is more than 1 when the voice signal is received, the student is indicated to rush.
Specifically, before the examination, the teacher needs to test whether the terminal device is in a normal state, such as: whether the current count is zero, whether the equipment power is sufficient, whether the camera is usable, whether the countdown function is usable, and the like.
S200: and extracting the action images of the time sequence, calculating the confidence coefficient and the affinity of the key points of the human skeleton, and drawing a connection graph of the key points of the human skeleton. The key skeleton points of each student can be distinguished in the step, and the counting of multiple persons and rope skipping at the same time is realized.
As shown in fig. 3, step S200 specifically includes sub-steps S201 to S205:
s201: acquiring an original video of a plurality of people.
Specifically, students stand facing teachers in sequence according to the school numbers, and the teachers record video streams obtained in the process of rope skipping of the students by using cameras of terminal equipment such as smart phones and the like, namely the original videos.
S202: and extracting motion images based on time series from the original video to detect key skeleton points of the human body.
The extracted motion image may be a motion image of each frame in the video stream or a motion image of each n frames, and may be determined according to real-time requirements and system processing capability.
S203: and detecting the motion image by using an OpenPose model to obtain a point bitmap of at least one human body key bone point, and acquiring parameters of the obtained human body key bone point.
Specifically, the openpos model can realize posture estimation of human body actions of one person or a plurality of persons and the like, and the method comprises the steps of detecting all key points in an image, clustering the key points to form a plurality of human body skeleton maps, wherein each human body skeleton map corresponds to one person, and different human body skeleton maps correspond to different person individuals. Wherein, human skeleton key point mainly includes: 16 skeleton key points such as the head, the neck, the left shoulder, the left elbow, the left wrist, the right shoulder, the right elbow, the right wrist, the spine, the hip central point, the right hip, the left hip, the right knee, the left knee, the right ankle, the left ankle and the like. As shown in fig. 4, the human skeleton pattern formed in the embodiment of the present invention is substantially T-shaped,
and inputting the motion image obtained in the S202 into an OpenPose model, so that the coordinate parameters of the key points of the human body frame and the human body skeleton of the rope skipping person in the image can be extracted. The upper left corner of the picture is taken as the origin of coordinates, the right side is taken as the x-axis, and the lower side is taken (refer to fig. 9). The distance from the key skeleton point of the human body to the x axis is the ordinate, and the distance from the key skeleton point of the human body to the y axis is the abscissa. Thus, the coordinate values of the key points of the human bones can be obtained in each frame.
Before calculating parameters of key points of human bones, detecting the obtained parameters, if abnormal points are found, carrying out interpolation calculation by combining corresponding key points of bones of a previous frame and a next frame, and replacing the abnormal points by using interpolation calculation results; if a skeletal keypoint is found missing, it can be supplemented with the corresponding skeletal keypoint of the previous frame.
S204: and respectively calculating the confidence coefficient and the affinity of each bone key point according to the parameters of the human bone key points obtained in the step S203, calculating the connection weight of the human bone key points and drawing a connection graph of the human bone key points.
Specifically, the confidence of the bone key points represents the probability that the pixel points in the specific region in the image are human bone key points, and the expression is as follows:
wherein, X j,k Represents the jth skeletal key point of the kth individual in the training picture data set, P is a pixelThe coordinates of the point. The confidence coefficient S follows normal distribution, and the maximum value in the normal distribution is taken as the peak value of the bone key point when a plurality of human bodies participate in calculation.
Specifically, the affinity of the skeletal key points represents the direction of the pixel points in the specific region in the image in the human body, and the expression is as follows:
wherein d is j1 、d j2 Being a skeletal key point, L c Denotes d j1 、d j2 The distance between them. E is the projection length on the connecting line of the key points of the skeleton, and the larger the value is, the higher the connection effectiveness of the key points of the skeleton is, and the greater the affinity is. Therefore, the obtained affinity can be used as the weight of the connection effectiveness of the key point to evaluate the reliability degree of the connection effectiveness.
The Hungarian algorithm is adopted to optimally match the key point connection weights, the attribution of skeleton key points can be determined, namely whether the two key points are in a connection relation or not, and the expression is as follows:
wherein the content of the first and second substances,
representing key points of the j classM critical point, E mn Representing the connection weights of the mth keypoint and the nth keypoint,
and (3) representing whether the mth key point and the nth key point are connected or not, wherein the value 1 is connected, and the value 0 is disconnected.
S205: and drawing a skeleton key point connection graph of each student according to the connection relation of the skeleton key points determined by the Hungarian algorithm in the S204.
As shown in fig. 4, in this step, by extracting key points of bones of a human body and connecting the key points of bones of different individuals, the key bone points of each student can be accurately distinguished, and a function of posture analysis for recognizing simultaneous rope skipping of multiple persons is achieved. As shown in fig. 4, the arms of the two people on the left side are overlapped, but because a T-shaped (or cross-shaped) human body skeleton key point connection diagram formed by 16 skeleton key points is adopted, the key skeleton points of different people can be correctly distinguished. Compared with the method that the persons are tracked only by using a small number of human skeleton key points such as wrists, arms and ankles, the Hungarian algorithm is adopted to draw the connection graph of 16 skeleton key points, so that the overlapping among a plurality of persons can be more accurately distinguished, and the method is more suitable for monitoring the movement of the plurality of persons at the same time.
S300: and calculating the time-space domain characteristics of key skeleton points, and identifying effective rope skipping actions through action classification and motion tracking.
As shown in fig. 5, the method specifically includes substeps S301 to S305:
s301: and calculating the time-space domain characteristics of the key skeleton points.
Specifically, the raw data of the time-space domain feature calculation is from the bone key point parameters acquired in step S203. Before data acquisition, a normalization mechanism needs to be established, so that the influence of differences of height, weight and the like on a counting result is reduced.
Because the length of the skeleton segment of the human body is related to the height, the difference of the height needs to be considered when extracting the key points of the skeleton. Therefore, the skeleton key point parameters obtained in step S203 are normalized to eliminate the influence of height on the extraction of the skeleton key points of the human body, and the calculation formula is as follows:
wherein H is the absolute length from the head to the spine,
for the time-space domain feature, the final motion space position feature is obtained as
And calculating the time-space domain characteristics of the key skeleton points of the human body based on the normalized parameters. The time-space domain features mainly comprise time domain features and space domain features. The time domain features refer to the relative motion of the human skeletal key points between the current frame and the previous frame, and the position change of other points relative to the previous frame. The spatial domain characteristics comprise three indexes of spatial position difference characteristics, spatial angle characteristics and joint included angle characteristics of human bones. Connecting the time-space domain features as the preliminarily extracted action sequence features F P 。
Before data calculation and analysis, the motion sequence characteristics F obtained from the four characteristics can be analyzed by Principal Component Analysis (PCA) P The dimension reduction processing is carried out, namely, the original data is replaced by low-dimensional data obtained by projection on the most important coordinate axes, so that the original data can be well summarized, and the complexity of calculation can be reduced.
Specifically, the spatial motion feature of human skeleton mainly refers to the change of the skeleton key points of each frame relative to the previous frame, and the motion feature vector difference between the 16 skeleton key points of each frame and the 16 skeleton key points of the previous frame is calculated by the following calculation formula:
wherein the content of the first and second substances,
for the motion feature vector of the current frame bone key point,
the motion feature vector of the previous frame bone key point. And (5) describing the spatial motion characteristics of the human skeleton key points in the time domain by using the motion characteristic vector difference.
Specifically, the spatial position difference feature of the human skeleton key point mainly describes the change of the spatial positions of each part of the human body in the same frame image, and the calculation method comprises the following steps: respectively calculating three coordinate axes of other 15 skeleton key points by taking the central point of the hip as a central origin
Coordinates of hip center point
Difference of (2)
Concatenated into a position feature vector for the current frame
One motion can be represented as the concatenation of the position feature vectors of all images, i.e., F = [ F = [) 1 ,f 2 ,...,f m ]And m represents the number of frames of the moving picture. Each frame image obtains a 15 × 3 =45-dimensional positional feature vector.
As can be seen, equation (6) describes the spatial position difference features of the key points of the human skeleton in the time domain by using the multi-dimensional position feature vector variation.
Specifically, with reference to fig. 8 and 9, the spatial angle feature of the human skeleton key points mainly describes a three-dimensional spatial angle between two key points of the human body in the same frame of image, and the calculation method thereof is as follows: the three-dimensional spatial angle between the two specified skeletal keypoints is computed in a single frame of motion image. The method mainly calculates the space angle characteristics of 8 groups of bone key points of a left shoulder-left elbow, a left elbow-left wrist, a right shoulder-right elbow, a right elbow-right wrist, a left hip-left knee, a left knee-left ankle, a right hip-right knee and a right knee-right ankle, and calculates the direction angle alpha and the elevation angle beta of the 8 groups of bone key points by taking a straight line connecting the left hip to the right hip as an X axis and the trunk as a Y axis. In a coordinate plane taking an X-axis-Y-axis as a coordinate, a direction angle alpha is an included angle between a connecting line of two skeleton key points and the X-axis; the elevation angle beta is an included angle between a connecting line of two skeleton key points and the Y axis; the included angle theta is the included angle between the connecting lines of two adjacent bone key points. Spatial angle A of each set of bone key points n ＝(a n ,β n ) In the ith frame, the spatial angle features of all the skeletal key points are as follows:
specifically, the joint included angle characteristic of the key points of human bones refers to the included angle of two adjacent bones in the same frame image, the invention mainly calculates the included angles of 6 human body joints such as a left shoulder joint, a right shoulder joint, a left elbow joint, a right elbow joint, a left knee joint, a right knee joint, a left ankle joint, a right ankle joint and the like, and the calculation formula is as follows:
bone key point P 1 i Has spatial coordinates of
Key points of skeleton
Has spatial coordinates of
Key points of skeleton
Has spatial coordinates of
Representing skeletal key points P 1 i And skeletal key points
The spatial vector of (a) is determined,
representing skeletal keypoints
And skeletal key points
The spatial vector of (2). In frame i, the resulting bone space angles are:
through the calculation of the space domain characteristics, the motion tracking of the key human skeleton points can be realized, the overlapping condition of the key human skeleton points can be effectively distinguished through a connecting line diagram of the key human skeleton points, the rope skipping action is accurately identified, and the counting is accurate.
S302: and establishing a data set of key points of human skeletons.
The action recognition data set is used for collecting different action types in real time, randomly selecting a plurality of students with different heights and body types, collecting image data of rope skipping actions according to the standards of multi-posture, multi-quantity and multi-angle, and dividing the image data into a training set and a testing set.
In specific implementation, 60 pupils with different heights and body types can be randomly selected, 6000 image data of rope skipping actions are collected according to the standards of multi-posture, large quantity and multi-angle, 692 pieces of prepared data, 524 pieces of standing data, 1649 pieces of jumping data, 1632 pieces of flying data, 1503 pieces of landing data are provided, 70% of each type of image data are divided into training sets, and the rest 30% of the image data are used as testing sets.
S303: and building a stacking model and training.
Specifically, a stacking model is built by a build method, data are trained by back propagation to complete parameter tuning, and then the accuracy of the model on motion recognition is evaluated by adopting a confusion matrix.
As shown in fig. 6, the loss of the training set and the verification set gradually approaches 0, the accuracy gradually approaches 1, and the curves of the training set and the verification set are very close, which shows that the training effect of the stack model adopted by the invention is very good.
S304: and (5) classifying the actions.
The complete rope skipping action is divided into 5 actions of preparation, standing, take-off, landing, soaring and the like according to the time-space domain characteristics of the human skeleton points, as shown in fig. 7.
The prepared movement is that in a section of continuous movement data, the arms extend forwards, the left hand and the right hand are extended straight and vertical to the body, and the two legs are closed and stand.
The jumping motion is that in a section of continuous motion data, the direction angle alpha and the elevation angle beta of the left shoulder-left elbow, the left elbow-left wrist, the right shoulder-right elbow and the right elbow-right wrist are greatly changed to reach a preset threshold value; the included angles theta between the left shoulder and the left elbow and between the right shoulder and the right elbow are acute angles, the included angles theta between the left hip and the left knee and between the left knee and the left ankle are changed from 180 degrees to acute angles, and the included angles theta between the right hip and the right knee and between the right knee and the right ankle are changed from right angles to acute angles.
The soaring action is that in a section of continuous action data, the direction angle alpha and the elevation angle beta of a left elbow-left wrist and a right elbow-right wrist are greatly changed to reach a preset threshold value; the included angles theta between the left elbow and the left wrist and between the right elbow and the right wrist are increased, the included angles theta between the left hip and the left knee and between the right hip and the right knee are changed from acute angles to 180 degrees, and the included angles theta between the left knee and the left ankle and between the right knee and the right ankle are changed from acute angles to obtuse angles. Compared with the previous frame, the left wrist and the right wrist move downwards, and the key points of the rest human skeletons are upwards to the highest.
The landing movement is that in a section of continuous movement data, the direction angle alpha and the elevation angle beta of a left elbow-left wrist and a right elbow-right wrist are greatly changed to reach a preset threshold value; the included angle between the right knee and the right ankle and the included angle between the left knee and the left ankle are changed from an obtuse angle to a right angle. The human skeletal keypoints are dropped to the lowest compared to the previous frame.
The standing motion is that key skeletal points of a human body are lowered to the lowest in a section of continuous motion data, and two feet stand naturally.
As shown in fig. 7, in order to distinguish the key point detection results of different limbs, the same limb portion is marked with the same color and different portions are marked with different colors.
As shown in fig. 8, it is verified that, according to the scheme of the present invention, the recognition accuracy of the stack model built and trained in S303 for recognizing five actions reaches more than 95%, and the requirements of actual use are met.
S305: valid actions are identified based on the consistency of the actions.
Specifically, when the preliminary operation is recognized, a count is prepared. If the cyclic actions of take-off, emptying and landing are continuously identified, adding 1 to the rope skipping times of the target associated personnel; otherwise, the count is unchanged. When the timer reaches 1 minute, the countdown is finished, and the rope skipping counting is stopped.
Specifically, a line graph of key points of human bones is taken as a counting unit in a counting link.
Specifically, the counting condition includes that there is a complete rope skipping period between the current video frame and the frame which is counted successfully last time.
S400: and sending an end signal, stopping recording and ending counting.
Specifically, when the time reaches 1 minute, the countdown is finished, the terminal device automatically generates an end signal, a stop voice signal is sent out based on the end signal, the camera stops shooting the video, and the counter stops counting. Meanwhile, the student stops skipping the rope after hearing the voice signal.
S500: and returning a counting result.
And after counting is finished, displaying the rope skipping quantity of each student on a display screen of the terminal equipment.
In summary, the counting method for the multi-person simultaneous rope skipping test based on the skeletal key point detection provided by the embodiment of the invention realizes automatic and accurate counting of simultaneous rope skipping of multiple students by tracking the motion rules of multiple skeletal key points of a human target through motion, and the method mainly starts to shoot rope skipping videos and time by a terminal device through a signal that a user starts a rope skipping test at the terminal device, so that the students start rope skipping; then, extracting action images based on time sequences from the original video, and drawing a skeletal key point connection diagram of the student; then tracking the effective rope skipping action through the movement by extracting the time-space domain characteristics of the key skeleton points; and finally, counting the rope skipping actions of the target associated personnel based on the rope skipping action recognition result.
As mentioned above, the embodiment of the present invention further provides a multi-person motion detection apparatus based on human skeleton point detection, which includes a processor, a memory, a display screen, and a speaker. Wherein, the memory stores a computer program for executing the multi-person motion detection method based on human body skeleton point detection; the processor calls the computer program to realize a multi-person motion detection method based on human skeleton point detection; the display screen provides a human-computer interaction interface.
Compared with the prior art, the invention has the following advantages:
(1) Based on a human skeleton point detection technology, the change characteristics of a time domain and a space domain of a human skeleton key point in the motion process are analyzed, effective actions are identified through action classification and motion tracking, and the accuracy of motion counting can be improved;
(2) The Hungarian algorithm is carried out on the plurality of human skeleton key points to draw a human skeleton key point connection diagram, so that the overlapping of the human skeleton key points in the motion of multiple persons can be distinguished, and the accuracy of identifying the motion of the multiple persons is improved;
(3) The skipping rope counting is carried out by a computer vision method, so that the cost can be reduced; the examination efficiency can be improved, and the simultaneous counting of multiple persons can be realized.
The method and device for detecting multi-person motion based on human skeleton point detection provided by the invention are explained in detail above. It will be apparent to those skilled in the art that any obvious modifications thereof can be made without departing from the spirit of the invention, which infringes the patent right of the invention and bears the corresponding legal responsibility.
Claims (10)
1. A multi-person motion detection method based on human skeleton point detection is characterized by comprising the following steps:
s100: sending a starting signal, and simultaneously starting video recording and countdown;
s200: extracting action images of the time sequence, calculating the confidence and the affinity of the key points of the human skeleton, and drawing a connection diagram of the key points of the human skeleton;
s300: calculating the time-space domain characteristics of key skeletal points, and identifying effective actions through action classification and motion tracking;
s400: counting the motion of the human body based on the motion classification result;
s500: sending out an end signal, stopping video recording, and ending counting;
s600: and returning a counting result.
2. The method for detecting motion of multiple persons based on human skeletal point detection as claimed in claim 1, wherein said step S200 comprises the following sub-steps:
s201: acquiring an original video of multi-person movement;
s202: extracting a motion image based on a time sequence from an original video to detect key skeleton points of a human body;
s203: detecting the action image by using an OpenPose model to obtain a point bitmap of at least one human body key bone point, and acquiring parameters of the obtained human body key bone point;
s204: respectively calculating the confidence coefficient and the affinity of each bone key point according to the human bone key point parameters obtained in the step S203, calculating the connection weight of the human bone key points and drawing a human bone key point connection graph;
s205: and drawing a skeleton key point connection diagram of each human body according to the connection relation of the human body skeleton key points in the S204.
3. The method for detecting motion of multiple persons based on human skeletal point detection as claimed in claim 1, wherein said step S300 comprises the following sub-steps:
s301: calculating the time-space domain characteristics of key skeleton points;
s302: establishing a data set of human skeleton key points;
s303: building a stacking model and training;
s304: performing action classification by using the stacking model;
s305: and identifying effective rope skipping actions according to the consistency of the actions.
4. The method of claim 1, wherein the human skeletal point detection-based multi-person motion detection comprises:
calculating the connection weight of the key points of the human skeleton through a Hungarian algorithm, and drawing a connection graph of the key points of the human skeleton.
5. The method of claim 4, wherein the human skeletal point detection-based multi-person motion detection method comprises:
in step S203, a point map of 16 human key skeletal points is obtained.
6. The method of claim 5, wherein the human skeletal point detection-based multi-person motion detection method comprises:
in step S203, the obtained 16 key human skeletal points include head, neck, left shoulder, left elbow, left wrist, right shoulder, right elbow, right wrist, spine, hip center point, right hip, left hip, right knee, left knee, right ankle, and left ankle.
7. The method of claim 6, wherein the human skeletal point detection-based multi-person motion detection method comprises:
the bone key point parameters acquired in step S203 are normalized and calculated.
8. The method of claim 7, wherein the human skeletal point detection-based multi-person motion detection method comprises:
and calculating the time-space domain characteristics of the key skeleton points based on the normalized parameters.
9. A multi-person motion detection device based on human body skeleton point detection is characterized by comprising a processor, a memory, a display screen and a loudspeaker,
wherein the memory stores a computer program for executing the method for detecting a multi-person motion based on human skeletal point detection according to any one of claims 1 to 8;
the processor calls the computer program to realize a multi-person motion detection method based on human skeleton point detection;
the display screen provides a human-computer interaction interface.
10. The multi-person motion detection apparatus according to claim 9, characterized in that:
the multi-person motion detection device is a smart phone, a tablet computer or a personal computer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210963067.XA CN115690895A (en) | 2022-08-11 | 2022-08-11 | Human skeleton point detection-based multi-person motion detection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210963067.XA CN115690895A (en) | 2022-08-11 | 2022-08-11 | Human skeleton point detection-based multi-person motion detection method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115690895A true CN115690895A (en) | 2023-02-03 |
Family
ID=85061231
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210963067.XA Pending CN115690895A (en) | 2022-08-11 | 2022-08-11 | Human skeleton point detection-based multi-person motion detection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115690895A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117193534A (en) * | 2023-09-13 | 2023-12-08 | 北京小米机器人技术有限公司 | Motion interaction method and device, electronic equipment and storage medium |
-
2022
- 2022-08-11 CN CN202210963067.XA patent/CN115690895A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117193534A (en) * | 2023-09-13 | 2023-12-08 | 北京小米机器人技术有限公司 | Motion interaction method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109191588B (en) | Motion teaching method, motion teaching device, storage medium and electronic equipment | |
| CN111368810B (en) | Sit-up detection system and method based on human body and skeleton key point identification | |
| Islam et al. | Yoga posture recognition by detecting human joint points in real time using microsoft kinect | |
| CN107301370B (en) | Kinect three-dimensional skeleton model-based limb action identification method | |
| CN111437583B (en) | Badminton basic action auxiliary training system based on Kinect | |
| CN110490109B (en) | Monocular vision-based online human body rehabilitation action recognition method | |
| CN110544301A (en) | Three-dimensional human body action reconstruction system, method and action training system | |
| CN110711374A (en) | Multi-modal dance action evaluation method | |
| CN110448870B (en) | Human body posture training method | |
| CN107930048B (en) | Space somatosensory recognition motion analysis system and motion analysis method | |
| WO2012117392A1 (en) | Device, system and method for determining compliance with an instruction by a figure in an image | |
| CN110298218B (en) | Interactive fitness device and interactive fitness system | |
| WO2017161734A1 (en) | Correction of human body movements via television and motion-sensing accessory and system | |
| CN113856186B (en) | Pull-up action judging and counting method, system and device | |
| CN110084192A (en) | Quick dynamic hand gesture recognition system and method based on target detection | |
| KR102320960B1 (en) | Personalized home training behavior guidance and correction system | |
| CN115116127A (en) | Fall detection method based on computer vision and artificial intelligence | |
| CN115690895A (en) | Human skeleton point detection-based multi-person motion detection method and device | |
| CN115227234A (en) | Cardiopulmonary resuscitation pressing action evaluation method and system based on camera | |
| CN114618142A (en) | Auxiliary training system and method for table tennis sports | |
| CN116271757A (en) | Auxiliary system and method for basketball practice based on AI technology | |
| CN114639168B (en) | Method and system for recognizing running gesture | |
| CN115331304A (en) | Running identification method | |
| Pai et al. | Home Fitness and Rehabilitation Support System Implemented by Combining Deep Images and Machine Learning Using Unity Game Engine. | |
| CN117423166B (en) | Motion recognition method and system according to human body posture image data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |