CN116978127A - Body-building posture correction method, device, equipment and medium based on posture estimation - Google Patents
Body-building posture correction method, device, equipment and medium based on posture estimation Download PDFInfo
- Publication number
- CN116978127A CN116978127A CN202311007231.0A CN202311007231A CN116978127A CN 116978127 A CN116978127 A CN 116978127A CN 202311007231 A CN202311007231 A CN 202311007231A CN 116978127 A CN116978127 A CN 116978127A
- Authority
- CN
- China
- Prior art keywords
- human body
- estimation
- posture
- building
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012937 correction Methods 0.000 title claims abstract description 73
- 238000000034 method Methods 0.000 title claims abstract description 47
- 230000009471 action Effects 0.000 claims abstract description 33
- 238000001514 detection method Methods 0.000 claims abstract description 31
- 238000012545 processing Methods 0.000 claims abstract description 29
- 238000000605 extraction Methods 0.000 claims description 27
- 238000004590 computer program Methods 0.000 claims description 16
- 238000010606 normalization Methods 0.000 claims description 13
- 238000011176 pooling Methods 0.000 claims description 9
- 238000005070 sampling Methods 0.000 claims description 9
- 230000000694 effects Effects 0.000 abstract description 7
- 230000036544 posture Effects 0.000 description 125
- 238000010586 diagram Methods 0.000 description 9
- 230000004044 response Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 230000036541 health Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 208000025978 Athletic injury Diseases 0.000 description 1
- 206010023204 Joint dislocation Diseases 0.000 description 1
- 206010050031 Muscle strain Diseases 0.000 description 1
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 210000004394 hip joint Anatomy 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 230000002427 irreversible effect Effects 0.000 description 1
- 230000004630 mental health Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000008450 motivation Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/23—Recognition of whole body movements, e.g. for sport training
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
- G16H20/30—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Human Computer Interaction (AREA)
- Social Psychology (AREA)
- Psychiatry (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Physical Education & Sports Medicine (AREA)
- Epidemiology (AREA)
- Primary Health Care (AREA)
- Public Health (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of human body posture estimation, and discloses a body-building posture correction method, device, equipment and medium based on posture estimation. The method comprises the following steps: acquiring a human body posture image, and inputting the human body posture image into a target detection network for detection to obtain a target human body image; inputting the target human body image into a root joint point network for operation processing to obtain a root joint point three-dimensional coordinate containing depth information; inputting the three-dimensional coordinates of the root joint point and the target human body image into a gesture estimation network to perform gesture estimation to obtain 3D gesture estimation to be identified; and comparing the 3D gesture estimation to be identified with a preset standard 3D gesture estimation to obtain a comparison result, and outputting a first body-building gesture correction suggestion according to the comparison result. The embodiment of the invention can effectively analyze the body-building actions of the body-building person, and give accurate correction suggestions to nonstandard body-building actions so as to improve the body-building effect and body-building safety.
Description
Technical Field
The embodiment of the invention relates to the technical field of human body posture estimation, in particular to a body-building posture correction method, device, equipment and medium based on posture estimation.
Background
Along with the heat of healthy life theory, people's health consciousness is stronger and stronger, and more people also can utilize leisure time to temper the health at home more to, house body-building only needs simple sports equipment, does not receive time and place restriction, has the advantage that participates in the threshold low. Although the participation threshold of the body building on the home line is low, which is beneficial to keeping physical and mental health, the incorrect body building action can also cause muscle strain, joint dislocation and other sports injuries, and cause irreversible injuries to body building participants; moreover, the body building on the home line lacks the guidance of a professional body building coach, and the problems of invalid exercise and the like caused by incorrect body building actions, nonstandard body building actions and low action completion degree can occur to the body builder.
Most of the existing body-building posture correcting methods need to wear professional correcting equipment to monitor body-building actions during body-building, and can bring wearing burden or inadaptation to a certain extent to a body-building person during body-building, so that the actual body-building training effect of the body-building person is affected, and the body-building posture of the body-building person cannot be corrected well; moreover, the traditional gesture recognition utilizes a monocular camera to obtain the depth information of the image, only two-dimensional skeleton point information can be obtained, the depth information of skeleton points can not be obtained, partial actions are not analyzed in place during the guiding training, and the correction effect is poor.
Disclosure of Invention
The embodiment of the invention provides a body-building posture correction method, device, equipment and medium based on posture estimation, and aims to solve the problem that the correction effect of the existing body-building posture correction method is poor.
In a first aspect, an embodiment of the present invention provides a posture correction method for exercise based on posture estimation, including:
acquiring a human body posture image, and inputting the human body posture image into a target detection network for detection to obtain a target human body image;
inputting the target human body image into a root joint point network for operation processing to obtain a root joint point three-dimensional coordinate containing depth information;
inputting the three-dimensional coordinates of the root joint point and the target human body image into a gesture estimation network to perform gesture estimation to obtain 3D gesture estimation to be identified;
and comparing the 3D gesture estimation to be identified with a preset standard 3D gesture estimation to obtain a comparison result, and outputting a first body-building gesture correction suggestion according to the comparison result.
In a second aspect, embodiments of the present invention further provide a posture-estimating-based fitness posture correcting apparatus, including:
the detection unit is used for acquiring a human body posture image, inputting the human body posture image into a target detection network for detection, and obtaining a target human body image;
the operation unit is used for inputting the target human body image into a root joint point network to perform operation processing to obtain a root joint point three-dimensional coordinate containing depth information;
the estimation unit is used for inputting the three-dimensional coordinates of the root joint points and the target human body image into a gesture estimation network to perform gesture estimation to obtain 3D gesture estimation to be identified;
and the comparison unit is used for comparing the 3D gesture estimation to be identified with a preset standard 3D gesture estimation to obtain a comparison result, and outputting a first body-building gesture correction suggestion according to the comparison result.
In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory and a processor, where the memory stores a computer program, and the processor implements the method when executing the computer program.
In a fourth aspect, embodiments of the present invention also provide a computer readable storage medium storing a computer program which, when executed by a processor, implements the above method.
The embodiment of the invention provides a body-building posture correction method, device, equipment and medium based on posture estimation. Wherein the method comprises the following steps: acquiring a human body posture image, and inputting the human body posture image into a target detection network for detection to obtain a target human body image; inputting the target human body image into a root joint point network for operation processing to obtain a root joint point three-dimensional coordinate containing depth information; inputting the three-dimensional coordinates of the root joint point and the target human body image into a gesture estimation network to perform gesture estimation to obtain 3D gesture estimation to be identified; and comparing the 3D gesture estimation to be identified with a preset standard 3D gesture estimation to obtain a comparison result, and outputting a first body-building gesture correction suggestion according to the comparison result. According to the technical scheme, the three-dimensional coordinates of the root joint points in the target human body image are identified through the root joint point network, the three-dimensional coordinates of the root joint points comprise depth information of the root joint points, the 3D gesture estimation to be identified is obtained through the three-dimensional coordinates of the root joint points and the target human body image, the 3D gesture estimation to be identified is compared with the standard 3D gesture estimation, the body-building gesture correction suggestion is output, the body-building action of a body-building person can be effectively analyzed through the depth information contained in the 3D gesture estimation to be identified, and accurate correction suggestion is given to nonstandard body-building actions, so that body-building effect and body-building safety are improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a body-building posture correction method based on posture estimation according to an embodiment of the present invention;
fig. 2 is a display diagram of a target human body image detected by a target detection network of a body-building posture correction method based on posture estimation according to an embodiment of the present invention;
fig. 3 is an exhibition diagram of three-dimensional coordinates of a root joint point calculated by a root joint point network of a body-building posture correction method based on posture estimation according to an embodiment of the present invention;
fig. 4 is a schematic sub-flowchart of a body-building posture correction method based on posture estimation according to an embodiment of the present invention;
FIG. 5 is a schematic sub-flowchart of a body-building posture correction method based on posture estimation according to an embodiment of the present invention;
FIG. 6 is a schematic sub-flowchart of a body-building posture correction method based on posture estimation according to an embodiment of the present invention;
FIG. 7 is a view showing a root joint point network and a scene application of a posture estimation network of a posture estimation-based fitness posture correction method according to an embodiment of the present invention;
FIG. 8 is a schematic sub-flowchart of a body-building posture correction method based on posture estimation according to an embodiment of the present invention;
FIG. 9 is a schematic block diagram of a body-building posture correction device based on posture estimation according to an embodiment of the present invention;
fig. 10 is a schematic block diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be understood that the terms "comprises" and "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
As used in this specification and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".
Referring to fig. 1, fig. 1 is a schematic flow chart of a posture correction method based on posture estimation according to an embodiment of the present invention. The body-building posture correcting method based on posture estimation is described in detail below. As shown in fig. 1, the method includes the following steps S100 to S130.
S100, acquiring a human body posture image, and inputting the human body posture image into a target detection network for detection to obtain a target human body image.
In the embodiment of the invention, the human body posture image is firstly acquired, wherein the acquired human body posture image can be a photo containing human body posture or a video containing human body posture, and the human body posture image can be extracted from the video. Then, the human body posture image is input into a target detection network, and the human body image contained in the human body posture image is detected to obtain a target human body image. The target detection network is Mask R-CNN. As shown in fig. 2, fig. 2 is a display diagram of a target human body image detected by the target detection network. And inputting a human body posture image containing two human body postures into a target detection network for detection to obtain two target human body images, wherein the two target human body images are target human body images.
S110, inputting the target human body image into a root joint point network for operation processing to obtain the three-dimensional coordinates of the root joint point containing depth information.
In the embodiment of the invention, after the target human body image is obtained, the target human body image is used as the input of the root node network and is input into the root node network to identify the root node in the target human body image. Specifically, referring to fig. 3, fig. 3 is a display diagram of a root node three-dimensional coordinate calculated by a root node network, a target human body image is input into the root node network, a 2D coordinate (i.e., a two-dimensional coordinate) and depth information of a root node are calculated respectively, and the root node two-dimensional coordinate and the root node depth information are obtained respectively, wherein the root node two-dimensional coordinate and the root node depth information are the three-dimensional coordinate of the root node, and the three-dimensional coordinate of the root node contains the depth information. In this embodiment, the depth information indicates a distance between the camera and the root joint point.
Referring to fig. 4, in an embodiment, for example, in the embodiment of the present invention, the step S110 includes the following steps S111-S114.
S111, inputting the target human body image into the feature extraction network, and extracting human body posture features of the target human body image to obtain a human body posture feature map, wherein the human body posture feature map comprises root nodes.
In the embodiment of the present invention, the root node network includes 3 different networks, which are a feature extraction network, a convolution network, and a depth information extraction network, respectively. The feature extraction network can extract human body posture features in the target human body image, the target human body image is used as input of the feature extraction network, the input is input into the feature extraction network to extract human body posture features, and a human body posture feature map is output, wherein the human body posture feature map comprises root nodes. In this embodiment, the feature extraction network uses the res net50 network as a backbone network to extract features in the human body image.
And S112, inputting the human body posture feature map into the convolution network for operation processing to obtain the two-dimensional coordinates of the root node.
In the embodiment of the present invention, the convolution network may be used to calculate two-dimensional coordinates (may also be referred to as 2D coordinates), and in this embodiment, the two-dimensional coordinates of the root node are calculated, where a human body posture feature map including the root node is input into the convolution network as an input of the convolution network, and is subjected to operation processing, and the two-dimensional coordinates of the root node are output.
S113, inputting the human body posture feature map into the depth information extraction network to extract depth information, and obtaining depth information of the root joint point;
in the embodiment of the invention, the depth information extraction network can be used for calculating the depth information, and in the embodiment, the depth information of the root node is calculated, wherein the human body posture feature map containing the root node is used as the input of the depth information extraction network and is input into the depth information extraction network to calculate and output the depth information of the root node.
And S114, taking the two-dimensional coordinates of the root node and the depth information of the root node as the three-dimensional coordinates of the root node.
And finally, taking the two-dimensional coordinates and depth information of the root joint point as the three-dimensional coordinates of the root joint point, namely the three-dimensional coordinates of the root joint point. In this embodiment, the root node is a certain node in a predefined target human body image, and in this embodiment, a hip joint in the target human body image is used as the root node.
Referring to fig. 5, in an embodiment, for example, in the embodiment of the present invention, the step S112 includes the following steps S1121-S1123.
S1121, inputting the human body posture feature map into the batch normalization layer for normalization processing to obtain a normalized feature map;
s1122, inputting the normalized feature map into the deconvolution layer for up-sampling treatment to obtain a sampled feature map;
s1123, performing convolution operation on the sampling feature map to generate two-dimensional coordinates of the root node.
In an embodiment of the invention, the convolution network comprises a batch normalization layer and a deconvolution layer, wherein the deconvolution layer comprises 3 convolution layers. The human body posture feature map is input into a batch normalization layer and three deconvolution layers for up-sampling, and two-dimensional coordinates of the root joint point are generated through the convolution layers of 1*1. Specifically, the human body posture feature map is input as a batch normalization layer, is input into the batch normalization layer for normalization processing to obtain a normalization feature map, is subjected to up-sampling processing by using 3 deconvolution layers to obtain a sampling feature map, and is subjected to convolution operation by a convolution layer of 1*1 to generate the two-dimensional coordinates of the root node.
Referring to fig. 6, in an embodiment, for example, in the embodiment of the present invention, the step S113 includes the following steps S1131-S1133.
S1131, inputting the human body posture feature map into the pooling layer for average pooling treatment to obtain a pooled human body posture feature map;
s1132, inputting the pooled human body posture feature map into the convolution layer to carry out convolution operation, so as to obtain correction parameters;
s1133, obtaining area parameters of the target human body image, and calculating the product of the area parameters and the correction parameters to obtain the depth information of the root joint point.
In an embodiment of the present invention, the depth information extraction networkThe human body posture feature map is input into the pooling layer for average pooling treatment to obtain pooled human body posture features, the pooled human body posture feature map is input into the 1*1 convolution layer for convolution operation, and a parameter gamma is output, and the parameter gamma is taken as a punishment parameter and can punish bounding boxes of target human body images under different conditions. The parameter gamma is correspondingly changed to obtain a correction parameterAnd obtaining an area parameter k of the target human body image, and calculating the area parameter k and the correction parameterThe product of->Obtaining depth information of root joint point->In the present embodiment, the calibration parameters are used to calibrate the target human body image because the human body image is actually a far-near human body image in the human body posture image, the two target human body images obtained after the detection of the target detection network are on the same plane, the far-near relationship of the target human body image is lacking, and the correction parameters are used>The far-near relationship of the target human body image can be re-expressed. However, since the areas of the target human body images are different, the depth information is also different, for example, the target human body image with a large area is perceived to be displayed closer, and the target human body image with a small area is perceived to be displayed farther, so that the correction parameter needs to be multiplied according to the difference of the areas to prevent the deviation of the calculated depth information from being too large.
And S120, inputting the three-dimensional coordinates of the root joint point and the target human body image into a gesture estimation network to perform gesture estimation, and obtaining 3D gesture estimation to be identified.
In an embodiment of the present invention, as shown in fig. 7, fig. 7 is a view showing a scene application of the joint point network and the pose estimation network. After the three-dimensional coordinates of the root joint point are obtained, the three-dimensional coordinates of the root joint point and the target human body image are used as input of a gesture estimation network, the input is input into the gesture estimation network for gesture estimation, and the three-dimensional coordinates of the rest joint points except the root joint point in the target human body image are calculated. Specifically, the root node is taken as a reference point, the depth information and the two-dimensional coordinates of the remaining node can be obtained by calculating the distance between the remaining node and the root node, and the three-dimensional coordinates of the remaining node can be obtained by combining the two-dimensional coordinates and the depth information. And then taking the three-dimensional coordinates of the rest joint points and the three-dimensional coordinates of the root joint points as 3D gesture estimation to be identified. In this embodiment, three-dimensional coordinates are obtained, and the three-dimensional coordinates of each joint point are connected to form a 3D pose estimation of the human body with the human body pose. In this embodiment, the posture estimation network uses a stacked hourglass network model as a reference model.
S130, comparing the 3D gesture estimation to be recognized with a preset standard 3D gesture estimation to obtain a comparison result, and outputting a first body-building gesture correction suggestion according to the comparison result.
In the embodiment of the invention, before comparing the 3D gesture estimation to be identified with the preset standard 3D gesture estimation, the method further comprises the following steps: the method comprises the steps of obtaining standard body-building action images and/or standard body-building action videos, inputting the standard body-building action images and/or standard body-building action videos into a gesture estimation network for recognition to obtain standard body-building 3D gesture estimation, and storing the 3D gesture estimation in the 3D gesture estimation network as preset standard 3D gesture estimation. In practical application, the 3D pose estimation to be identified is obtained through a camera, for example, when the exerciser moves, the camera arranged right in front of the exerciser shoots a moving picture of the exerciser, and the moving picture is processed through the target detection network and the root joint point network and then is input into the pose estimation network, so as to obtain the 3D pose estimation to be identified. Comparing the 3D gesture estimation to be identified with a preset standard body-building 3D gesture estimation in a gesture estimation network to obtain a comparison result, wherein the comparison result is the difference of the two gesture estimation, and outputting a first body-building gesture correction suggestion according to the comparison result, wherein the first body-building gesture correction suggestion is specifically an optimization suggestion for a certain body-building action.
Referring to fig. 8, in an embodiment, for example, in the embodiment of the present invention, the step S130 includes the following steps S131-S134.
S131, connecting joint points in the 3D gesture estimation to be identified to obtain joint point connection lines to be identified;
s132, connecting joint points corresponding to the 3D gesture estimation to be identified in the preset standard 3D gesture estimation to obtain standard joint point connection lines;
s133, comparing the angle direction of the joint point connecting line to be identified with the angle direction of the standard joint point connecting line;
and S134, outputting the first body-building posture correction suggestion if the angle direction of the joint point connecting line to be identified is different from the angle direction of the standard joint point connecting line.
In the embodiment of the invention, the joint points in the 3D gesture estimation to be recognized corresponding to a certain body-building action are connected to obtain the joint point connection to be recognized, and the joint points corresponding to the 3D gesture estimation to be recognized are found out from the preset standard 3D gesture estimation to be connected to obtain the standard joint point connection. For example, if the connected nodes in the 3D pose estimation to be recognized are a and B, the connected nodes in the standard 3D pose estimation are a 'and B', and the two pose estimates correspond to the same joint points in the human body. And comparing the angle direction of the joint point connecting line to be identified with the angle direction of the standard joint point connecting line, if the direction angle of the connecting line is different, indicating that the body-building action of the body-building person is not standard enough, and if the posture requirement of a certain body-building action is not met and correction is needed, outputting a first body-building posture correction suggestion, namely a correction guidance suggestion corresponding to the certain body-building action. As can be appreciated, if there is no difference between the angle direction of the joint point connecting line to be identified and the angle direction of the standard joint point connecting line, it indicates that the exercise of the exerciser reaches the standard action level without correction, so that a second exercise posture correction suggestion is output, where the second exercise posture correction suggestion can be a posture with a higher standard level of exercise formulated according to the height, weight, etc. of the exerciser, so as to encourage the exerciser to make guidance with higher requirements, higher difficulty and better exercise effect; the second exercise posture correction advice may also be advice that encourages the exerciser to adhere to the exercise, continue to hold waiting for inflation. As can be appreciated, the first exercise posture correction suggestion is a correction suggestion output when the exercise action of the exerciser is different from the standard exercise action, and different exercise actions correspondingly output different exercise action suggestions; the second body-building posture correction suggestion is a personalized correction suggestion or body-building motivation which is output when the body-building action of the body-building person is not different from the standard body-building action.
According to the body-building posture correction method based on posture estimation, the limitation of traditional body-building guidance is eliminated, professional equipment for monitoring is not needed to be worn during body-building, 3D posture estimation of a body-building person can be calculated on the basis of shooting human body images by using a monocular camera in a traditional way without adding other auxiliary equipment, the 3D posture estimation of the body-building person can be compared with standard actions, and proper optimization suggestions or guidance comments are given for nonstandard body-building actions, so that body-building safety of the body-building person can be improved, body-building efficiency can be effectively improved, and body-building correction effects are improved.
Fig. 9 is a schematic block diagram of an exercise posture correction device 200 based on posture estimation according to an embodiment of the present invention. As shown in fig. 9, the present invention further provides an exercise posture correction device 200 based on posture estimation, corresponding to the above exercise posture correction method based on posture estimation. The posture-estimation-based body-building posture correction apparatus 200 includes means for performing the posture-estimation-based body-building posture correction method described above, which may be configured in an electronic device. Specifically, referring to fig. 9, the posture-estimation-based exercise posture correction apparatus 200 includes a detection unit 201, an operation unit 202, an estimation unit 203, and an alignment unit 204.
The detection unit 201 is configured to obtain a human body posture image, input the human body posture image into a target detection network, and detect the human body posture image to obtain a target human body image; the operation unit 202 is configured to input the target human body image into a root joint point network for operation processing, so as to obtain a root joint point three-dimensional coordinate containing depth information; the estimation unit 203 is configured to input the three-dimensional coordinates of the root joint point and the target human body image into a gesture estimation network to perform gesture estimation, so as to obtain a 3D gesture estimation to be identified; the comparison unit 204 is configured to compare the 3D posture estimation to be identified with a preset standard 3D posture estimation, obtain a comparison result, and output a first exercise posture correction suggestion according to the comparison result.
In some embodiments, for example, the operation unit 202 includes a first extraction subunit, a first operation subunit, a second extraction subunit, and a first extraction subunit.
The first extraction subunit is configured to input the target human body image into the feature extraction network, and extract human body posture features of the target human body image to obtain a human body posture feature map, where the human body posture feature map includes root nodes; the first operation subunit is used for inputting the human body posture feature map into the convolution network to perform operation processing to obtain the two-dimensional coordinates of the root node; the second extraction subunit is used for inputting the human body posture feature map into the depth information extraction network to extract depth information, so as to obtain the depth information of the root joint point; the first serving subunit is configured to use the two-dimensional coordinate of the root node and the depth information of the root node as the three-dimensional coordinate of the root node.
In some embodiments, for example, the first operator unit includes a first processing subunit, a second processing subunit, and a second operator unit.
The first processing subunit is used for inputting the human body posture feature images into the batch normalization layer for normalization processing to obtain normalized feature images; the second processing subunit is used for inputting the normalized feature map into the deconvolution layer for up-sampling processing to obtain a sampled feature map; and the second operation subunit is used for carrying out convolution operation on the sampling feature map to generate the two-dimensional coordinates of the root node.
In some embodiments, for example, the second extraction subunit includes a third processing subunit, a third operation subunit, and a fourth operation subunit.
The third processing subunit is used for inputting the human body posture feature map into the pooling layer to perform average pooling processing to obtain a pooled human body posture feature map; the third operation subunit is used for inputting the pooled human body posture feature map into the convolution layer to carry out convolution operation so as to obtain correction parameters; the fourth operation subunit is used for inputting the pooled human body posture feature map into the convolution layer to carry out convolution operation so as to obtain correction parameters; the fourth operation subunit is configured to obtain an area parameter of the target human body image, calculate a product of the area parameter and the correction parameter, and obtain depth information of the root node.
In some embodiments, for example, the estimation unit 203 includes a fifth operation subunit and a second operation subunit.
The fifth operation subunit is configured to input the three-dimensional coordinates of the root joint point and the target human body image into the gesture estimation network, and calculate a distance between a remaining joint point in the target human body image and the root joint point, so as to obtain the three-dimensional coordinates of the remaining joint point; the second serving subunit is configured to take the three-dimensional coordinates of the remaining joint points and the three-dimensional coordinates of the root joint point as the 3D pose estimation to be identified.
In some embodiments, for example, the comparison unit 204 includes an acquisition subunit and an identification subunit.
Wherein the comparison unit 204 is configured to obtain a standard exercise image and/or a standard exercise video; the recognition subunit is used for inputting the standard body-building action image and/or the standard body-building action video into the gesture estimation network for recognition to obtain the preset standard body-building 3D gesture estimation.
In some embodiments, for example, the comparing unit 204 includes a first connection subunit, a second connection subunit, a comparing subunit, and an output subunit.
The first connection subunit is used for connecting the joint points in the 3D gesture estimation to be identified to obtain joint point connection to be identified; the second connection subunit is configured to connect, in the preset standard 3D pose estimation, a joint point corresponding to the 3D pose estimation to be identified, so as to obtain a standard joint point connection; the comparison subunit is used for comparing the angle direction of the joint point connecting line to be identified with the angle direction of the standard joint point connecting line; and the output subunit is used for outputting the first body-building posture correction suggestion if the angle direction of the joint point connecting line to be identified is different from the angle direction of the standard joint point connecting line.
The above-described exercise posture correction apparatus based on posture estimation may be implemented in the form of a computer program that can be run on an electronic device as shown in fig. 10.
Referring to fig. 10, fig. 10 is a schematic block diagram of an electronic device according to an embodiment of the present invention. The electronic device 300 is a display device having a body-building posture correcting function.
Referring to fig. 10, the electronic device 300 includes a processor 302, a memory, and a network interface 305, which are connected by a system bus 301, wherein the memory may include a non-volatile storage medium 303 and an internal memory 304.
The non-volatile storage medium 303 may store an operating system 3031 and a computer program 3032. The computer program 3032, when executed, may cause the processor 302 to perform a method of body-building posture correction based on posture estimation.
The processor 302 is used to provide computing and control capabilities to support the operation of the overall electronic device 300.
The internal memory 304 provides an environment for the execution of a computer program 3032 in the non-volatile storage medium 303, which computer program 3032, when executed by the processor 302, causes the processor 302 to perform a method of exercise posture correction based on posture estimation.
The network interface 305 is used for network communication with other devices. It will be appreciated by those skilled in the art that the structure shown in fig. 10 is merely a block diagram of a portion of the structure associated with the present inventive arrangements and is not limiting of the electronic device 300 to which the present inventive arrangements are applied, and that a particular electronic device 300 may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
It should be appreciated that in embodiments of the present invention, the processor 302 may be a central processing unit (Central Processing Unit, CPU), the processor 302 may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSPs), application specific integrated circuits (Application Specific Integrated Circuit, ASICs), field programmable gate arrays (Field-Programmable Gate Array, FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. Wherein the general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
Those skilled in the art will appreciate that all or part of the flow in a method embodying the above described embodiments may be accomplished by computer programs instructing the relevant hardware. The computer program may be stored in a storage medium that is a computer readable storage medium. The computer program is executed by at least one processor in the computer system to implement the flow steps of the embodiments of the method described above.
Accordingly, the present invention also provides a storage medium. The storage medium may be a computer readable storage medium. The storage medium stores a computer program. The computer program, when executed by a processor, causes the processor to perform any of the embodiments of the exercise posture correction method described above based on posture estimation.
The storage medium may be a U-disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, or an optical disk, or other various computer-readable storage media that can store program codes.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps described in connection with the embodiments disclosed herein may be embodied in electronic hardware, in computer software, or in a combination of the two, and that the elements and steps of the examples have been generally described in terms of function in the foregoing description to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of each unit is only one logic function division, and there may be another division manner in actual implementation. For example, multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed.
The steps in the method of the embodiment of the invention can be sequentially adjusted, combined and deleted according to actual needs. The units in the device of the embodiment of the invention can be combined, divided and deleted according to actual needs. In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The integrated unit may be stored in a storage medium if implemented in the form of a software functional unit and sold or used as a stand-alone product. Based on such understanding, the technical solution of the present invention is essentially or a part contributing to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, comprising several instructions for causing an electronic device (which may be a personal computer, a terminal, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention.
In the foregoing embodiments, the descriptions of the embodiments are focused on, and for those portions of one embodiment that are not described in detail, reference may be made to the related descriptions of other embodiments.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
While the invention has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.
Claims (10)
1. A body-building posture correction method based on posture estimation, comprising:
acquiring a human body posture image, and inputting the human body posture image into a target detection network for detection to obtain a target human body image;
inputting the target human body image into a root joint point network for operation processing to obtain a root joint point three-dimensional coordinate containing depth information;
inputting the three-dimensional coordinates of the root joint point and the target human body image into a gesture estimation network to perform gesture estimation to obtain 3D gesture estimation to be identified;
and comparing the 3D gesture estimation to be identified with a preset standard 3D gesture estimation to obtain a comparison result, and outputting a first body-building gesture correction suggestion according to the comparison result.
2. The method according to claim 1, wherein the root node network includes a feature extraction network, a convolution network, and a depth information extraction network, the inputting the target human body image into the root node network is performed with an operation process, to obtain three-dimensional coordinates of the root node including depth information, including:
inputting the target human body image into the feature extraction network, and extracting human body posture features of the target human body image to obtain a human body posture feature map, wherein the human body posture feature map comprises root nodes;
inputting the human body posture feature map into the convolution network for operation processing to obtain the two-dimensional coordinates of the root node;
inputting the human body posture feature map into the depth information extraction network to extract depth information, so as to obtain the depth information of the root joint point;
and taking the two-dimensional coordinates of the root node and the depth information of the root node as the three-dimensional coordinates of the root node.
3. The method according to claim 2, wherein the convolution network includes a batch normalization layer and a deconvolution layer, and the inputting the human body posture feature map into the convolution network for operation processing, to obtain the two-dimensional coordinates of the root node, includes:
inputting the human body posture feature map into the batch normalization layer for normalization processing to obtain a normalized feature map;
inputting the normalized feature map into the deconvolution layer for up-sampling treatment to obtain a sampled feature map;
and carrying out convolution operation on the sampling feature map to generate the two-dimensional coordinates of the root node.
4. The method according to claim 2, wherein the depth information extraction network includes a pooling layer and a convolution layer, the inputting the human body posture feature map into the depth information extraction network performs depth information extraction, and obtaining depth information of the root joint point includes:
inputting the human body posture feature map into the pooling layer for carrying out average pooling treatment to obtain a pooled human body posture feature map;
inputting the pooled human body posture feature map into the convolution layer to carry out convolution operation to obtain correction parameters;
and acquiring the area parameter of the target human body image, and calculating the product of the area parameter and the correction parameter to obtain the depth information of the root node.
5. The method according to claim 1, wherein inputting the three-dimensional coordinates of the root joint point and the target human body image into a pose estimation network for pose estimation to obtain a 3D pose estimation to be identified, comprises:
inputting the three-dimensional coordinates of the root joint point and the target human body image into the gesture estimation network, and calculating the distance between the rest joint point in the target human body image and the root joint point to obtain the three-dimensional coordinates of the rest joint point;
and taking the three-dimensional coordinates of the rest joint points and the three-dimensional coordinates of the root joint points as the 3D gesture estimation to be identified.
6. The method of claim 1, wherein prior to comparing the 3D pose estimate to be identified with a preset standard 3D pose estimate, further comprising:
acquiring standard body-building action images and/or standard body-building action videos;
and inputting the standard body-building action image and/or the standard body-building action video into the gesture estimation network for recognition to obtain the preset standard body-building 3D gesture estimation.
7. The method according to claim 1, wherein comparing the 3D pose estimation to be identified with a preset standard 3D pose estimation to obtain a comparison result, and outputting a first body-building pose correction suggestion according to the comparison result, comprises:
connecting the joint points in the 3D gesture estimation to be identified to obtain a joint point connection to be identified;
connecting joint points corresponding to the 3D gesture estimation to be identified in the preset standard 3D gesture estimation to obtain a standard joint point connecting line;
comparing the angle direction of the joint point connecting line to be identified with the angle direction of the standard joint point connecting line;
and if the angle direction of the joint point connecting line to be identified is different from the angle direction of the standard joint point connecting line, outputting the first body-building posture correction suggestion.
8. A body-building posture correction device based on posture estimation, characterized by comprising:
the detection unit is used for acquiring a human body posture image, inputting the human body posture image into a target detection network for detection, and obtaining a target human body image;
the operation unit is used for inputting the target human body image into a root joint point network to perform operation processing to obtain a root joint point three-dimensional coordinate containing depth information;
the estimation unit is used for inputting the three-dimensional coordinates of the root joint points and the target human body image into a gesture estimation network to perform gesture estimation to obtain 3D gesture estimation to be identified;
and the comparison unit is used for comparing the 3D gesture estimation to be identified with a preset standard 3D gesture estimation to obtain a comparison result, and outputting a first body-building gesture correction suggestion according to the comparison result.
9. An electronic device comprising a memory and a processor, the memory having a computer program stored thereon, the processor implementing the method of any of claims 1-7 when executing the computer program.
10. A computer readable storage medium, characterized in that the storage medium stores a computer program which, when executed by a processor, implements the method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311007231.0A CN116978127A (en) | 2023-08-10 | 2023-08-10 | Body-building posture correction method, device, equipment and medium based on posture estimation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311007231.0A CN116978127A (en) | 2023-08-10 | 2023-08-10 | Body-building posture correction method, device, equipment and medium based on posture estimation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116978127A true CN116978127A (en) | 2023-10-31 |
Family
ID=88481346
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311007231.0A Pending CN116978127A (en) | 2023-08-10 | 2023-08-10 | Body-building posture correction method, device, equipment and medium based on posture estimation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116978127A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117994819A (en) * | 2024-04-01 | 2024-05-07 | 南昌市小核桃科技有限公司 | Human body posture monitoring system based on image data analysis |
-
2023
- 2023-08-10 CN CN202311007231.0A patent/CN116978127A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117994819A (en) * | 2024-04-01 | 2024-05-07 | 南昌市小核桃科技有限公司 | Human body posture monitoring system based on image data analysis |
| CN117994819B (en) * | 2024-04-01 | 2024-06-07 | 南昌市小核桃科技有限公司 | Human body posture monitoring system based on image data analysis |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021000401A1 (en) | Posture assessment method, electronic apparatus, computer device, and storage medium | |
| US10898755B2 (en) | Method for providing posture guide and apparatus thereof | |
| CN113272852A (en) | Method for acquiring photograph for measuring body size, and body size measuring method, server, and program using same | |
| Gaber et al. | Quantifying facial paralysis using the kinect v2 | |
| US20220198834A1 (en) | Skeleton recognition method, storage medium, and information processing device | |
| US20220207921A1 (en) | Motion recognition method, storage medium, and information processing device | |
| CN116978127A (en) | Body-building posture correction method, device, equipment and medium based on posture estimation | |
| CN111178280A (en) | Human body sitting posture identification method, device, equipment and storage medium | |
| CN112568898A (en) | Method, device and equipment for automatically evaluating injury risk and correcting motion of human body motion based on visual image | |
| CN112464793A (en) | Method, system and storage medium for detecting cheating behaviors in online examination | |
| CN111461042A (en) | Fall detection method and system | |
| KR101636171B1 (en) | Skeleton tracking method and keleton tracking system using the method | |
| KR20230085901A (en) | Method and device for providing alopecia information | |
| JP2017047105A (en) | Exercise menu providing system and exercise menu providing method | |
| CN114973048A (en) | Method and device for correcting rehabilitation action, electronic equipment and readable medium | |
| JP2016004354A (en) | Determination method of body pose | |
| EP4040386A1 (en) | Motion recognition method, motion recognition program, and information processing device | |
| CN117503099A (en) | Human body composition analyzer, measuring method, measuring device, apparatus and storage medium | |
| US10094731B2 (en) | Standing position evaluation apparatus, standing position evaluation method, and non-transitory computer readable medium | |
| CN117115922A (en) | Seat body forward-bending evaluation method, system, electronic equipment and storage medium | |
| US20220078339A1 (en) | Method for obtaining picture for measuring body size and body size measurement method, server, and program using same | |
| CN112541382A (en) | Method and system for assisting movement and identification terminal equipment | |
| KR102035919B1 (en) | posture analysis application providing method for correct posture management | |
| CN115240856A (en) | Exercise health assessment method, system and equipment based on exercise posture | |
| CN113808207B (en) | Binocular range finding method and system for aiming at children |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |