CN115273219A - Yoga action evaluation method and system, storage medium and electronic equipment - Google Patents

Yoga action evaluation method and system, storage medium and electronic equipment Download PDF

Info

Publication number
CN115273219A
CN115273219A CN202210666543.1A CN202210666543A CN115273219A CN 115273219 A CN115273219 A CN 115273219A CN 202210666543 A CN202210666543 A CN 202210666543A CN 115273219 A CN115273219 A CN 115273219A
Authority
CN
China
Prior art keywords
yoga
data
yoga action
evaluated
human body
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210666543.1A
Other languages
Chinese (zh)
Inventor
李建伟
胡海晴
李金阳
王思琦
沈燕飞
曹润
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Sport University
Original Assignee
Beijing Sport University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Sport University filed Critical Beijing Sport University
Priority to CN202210666543.1A priority Critical patent/CN115273219A/en
Publication of CN115273219A publication Critical patent/CN115273219A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/23—Recognition of whole body movements, e.g. for sport training
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/273—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion removing elements interfering with the pattern to be recognised
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/34—Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Human Computer Interaction (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention relates to a yoga action evaluation method, a yoga action evaluation system, a storage medium and electronic equipment, wherein the yoga action evaluation method comprises the following steps: performing fusion optimization on the yoga action data to be evaluated at different viewing angles by adopting a three-dimensional skeleton information fusion optimization algorithm to obtain three-dimensional human body skeleton data subjected to shielding completion; identifying the three-dimensional human body skeleton data subjected to shielding completion by using the trained hierarchical cascade graph convolution network model to obtain a yoga action category and a yoga action completion score corresponding to the yoga action data to be evaluated; and acquiring a difficulty coefficient corresponding to the yoga action category, and obtaining an evaluation result of the yoga action data to be evaluated according to the difficulty coefficient and the yoga action completion score. The yoga actions are identified and evaluated by constructing the cascade diagram convolutional neural network model, the yoga posture classification and the accurate evaluation of the yoga actions to be evaluated are realized, and powerful support is provided for the user to specifically improve the yoga actions according to the evaluation result in the follow-up process.

Description

Yoga action evaluation method and system, storage medium and electronic equipment
Technical Field
The invention relates to the technical field of image recognition, in particular to a yoga action evaluation method and system, a storage medium and electronic equipment.
Background
Currently, human motion analysis based on vision aims to identify and evaluate human motion in an image, and mainly includes tasks of human motion detection, human motion classification, motion quality evaluation and the like. Deep learning based on human skeletal information is the mainstream method of human motion analysis at present, wherein the most widely used models are a Recurrent Neural Network (RNN) and a Convolutional Neural Network (CNN). RNN-based methods generally model skeletal data as a sequence of coordinate vectors along the spatial and temporal dimensions, where each vector represents a human joint. CNN-based methods typically model skeletal data as an image according to manually designed transformation rules.
Neither RNN-based nor CNN-based methods in the prior art are able to fully represent the structure of the skeletal data, because the skeletal data is embedded in the form of a graph, rather than a vector sequence or a two-dimensional grid. In recent years, graph convolution networks have generalized convolution to graphs, and dynamic skeletal models ST-GCN can automatically learn spatial and temporal patterns from images, and have been successfully applied in the field of motion analysis, but deep learning methods rely on a large amount of labeled data for model training. Compared with the traditional human body motion analysis task, the yoga motion analysis difficulty based on vision is that the yoga motion types are various, postures are uncommon, and the yoga image data acquired by a single visual angle has a serious shielding problem and is difficult to effectively identify and evaluate.
Disclosure of Invention
In order to solve the technical problem, the invention provides a yoga action evaluation method, a yoga action evaluation system, a storage medium and electronic equipment.
The technical scheme of the yoga action evaluation method is as follows:
performing fusion optimization on the yoga action data to be evaluated at different viewing angles by adopting a three-dimensional skeleton information fusion optimization algorithm to obtain three-dimensional human body skeleton data subjected to shielding completion;
identifying the sheltered and completed three-dimensional human body skeleton data by using the trained hierarchical cascade diagram convolution network model to obtain a yoga action category and a yoga action completion score corresponding to the yoga action data to be evaluated;
and acquiring a difficulty coefficient corresponding to the yoga action category, and acquiring an evaluation result of the yoga action data to be evaluated according to the difficulty coefficient and the yoga action completion score.
The yoga action evaluation method has the following beneficial effects:
according to the method, the yoga action is identified and evaluated by constructing the cascade diagram convolutional neural network model, the yoga posture classification and accurate evaluation of the yoga action to be evaluated are realized, and powerful support is provided for the user to improve the yoga action in a targeted manner according to the evaluation result.
On the basis of the scheme, the yoga action evaluation method can be further improved as follows.
Further, adopt three-dimensional skeleton information fusion optimization algorithm to carry out the fusion optimization to the yoga action data of waiting to assess of different visual angles, obtain sheltering from full three-dimensional human skeleton data, include:
registering the yoga action data to be evaluated at different viewing angles by adopting an iterative closest point algorithm to obtain a transformation matrix between different viewing angles;
fusing the yoga action data to be evaluated at different visual angles according to the transformation matrix to obtain original three-dimensional human body skeleton data;
and optimizing the original three-dimensional human body skeleton data according to a parameterized human body model to obtain the sheltered and supplemented three-dimensional human body skeleton data.
Further, the yoga action data to be evaluated at different viewing angles includes: the yoga action data to be evaluated at the main view angle and the yoga action data to be evaluated at the side view angle; according to the transformation matrix is to the yoga action data of waiting to assess of different visual angles fuses, obtains original three-dimensional human skeleton data, includes:
fusing the yoga action data to be evaluated at the main view angle and the yoga action data to be evaluated at the side view angle according to a preset fusion transformation formula and the transformation matrix to obtain the original three-dimensional human body skeleton data;
wherein the preset fusion transformation formula is as follows:
Figure BDA0003690583460000031
Figure BDA0003690583460000032
f represents the yoga action data to be evaluated of the main view angle, s represents the yoga action data to be evaluated of the side view angle,
Figure BDA0003690583460000033
joint coordinates, T, for the ith joint of the original three-dimensional body skeleton datasfFor the transformation matrix, SfiJoint coordinates of the ith joint of the yoga action data to be evaluated at the main view angle, SsiJoint coordinates W of the ith joint of the yoga action data to be evaluated at the side view anglefiIs SfiCorresponding weight, WsiIs SsiThe corresponding weight of the weight is set to be,
Figure BDA0003690583460000034
θi∈(0，90)，ziis the depth value corresponding to the ith joint, thetaiThe joint angle corresponding to the ith joint, ciConfidence coefficient corresponding to i-th joint, WiThe weight corresponding to the ith joint.
Further, the optimizing the original three-dimensional human body skeleton data according to the parameterized human body model to obtain the sheltered and supplemented three-dimensional human body skeleton data comprises:
optimizing the original three-dimensional human body skeleton data according to the parameterized human body model, and obtaining the three-dimensional human body skeleton data subjected to shielding completion when an objective function reaches the minimum;
wherein the objective function is: efused(θ，β)＝ωproEpro+ωshapeEshape，EproRepresenting a function aligning a two-dimensional projection of said original three-dimensional body skeleton data onto a three-dimensional joint, EshapeRepresenting a prior constraint function, omega, of the human body morphologyproIs EproIs preset to balance the weight, ωshapeIs Eshapeθ represents the length of the control skeleton, and β represents the pose of each joint.
Further, still include: acquiring original yoga action data of different view angles by adopting a calibration camera, and screening key frames of the original yoga action data of each view angle to obtain the yoga action data to be evaluated at different view angles.
Further, still include:
acquiring multiple yoga action training data at different view angles, and obtaining three-dimensional human body skeleton training data corresponding to the yoga action training data at each different view angle by adopting the three-dimensional skeleton information fusion optimization algorithm;
and performing hierarchical action category marking and action quality scoring on each three-dimensional human body skeleton training data, and training an original hierarchical cascade graph convolution network model by adopting all marked three-dimensional human body skeleton training data to obtain the trained hierarchical cascade graph convolution network model.
Further, the trained hierarchical cascade graph convolution network model includes: a first cascade network, a second cascade network and a third cascade network; utilize the layering cascade map convolution network model after the training right shelter from the three-dimensional human skeleton data of completion and discern, obtain treat that aassessment yoga action classification and the score is accomplished in the yoga action that yoga action data corresponds, include:
performing coarse-grained and fine-grained action recognition on the sheltered and completed three-dimensional human body skeleton data by using the first-level cascade network and the second-level cascade network to obtain the yoga action category;
and utilizing the third-level cascade network to evaluate the movement of the sheltered and completed three-dimensional human body skeleton data to obtain the yoga movement completion score.
The technical scheme of the yoga action evaluation system is as follows:
the method comprises the following steps: the device comprises a processing module, an identification module and an evaluation module;
the processing module is used for: performing fusion optimization on the yoga action data to be evaluated at different viewing angles by adopting a three-dimensional skeleton information fusion optimization algorithm to obtain three-dimensional human body skeleton data subjected to shielding completion;
the identification module is configured to: identifying the sheltered and completed three-dimensional human body skeleton data by using the trained layered cascade graph convolution network model to obtain a yoga action category and a yoga action completion score corresponding to the yoga action data to be evaluated;
the evaluation module is to: and acquiring a difficulty coefficient corresponding to the yoga action category, and acquiring an evaluation result of the yoga action data to be evaluated according to the difficulty coefficient and the yoga action completion score.
The yoga action evaluation system has the following beneficial effects:
the system identifies and evaluates the yoga actions by constructing the cascade diagram convolutional neural network model, realizes the yoga posture classification and accurate evaluation of the yoga actions to be evaluated, and provides powerful support for the user to improve the yoga actions in a targeted manner according to the evaluation result.
The technical scheme of the storage medium of the invention is as follows:
the storage medium stores instructions, and when the instructions are read by the computer, the computer executes the steps of the yoga action evaluation method.
The technical scheme of the electronic equipment is as follows:
the yoga movement evaluation method comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, and is characterized in that when the processor executes the computer program, the computer is enabled to execute the steps of the yoga movement evaluation method.
Drawings
Fig. 1 is a schematic flow chart of a yoga action evaluation method according to an embodiment of the present invention;
fig. 2 is a schematic view illustrating yoga action categories in the yoga action evaluation method according to the embodiment of the present invention;
fig. 3 is a schematic view of identification accuracy of a yoga action in the yoga action evaluation method according to the embodiment of the present invention;
fig. 4 is a schematic diagram illustrating evaluation results of yoga actions in the yoga action evaluation method according to the embodiment of the present invention;
fig. 5 is a schematic structural diagram of a yoga action evaluation system according to an embodiment of the present invention.
Detailed Description
As shown in fig. 1, the yoga action evaluation method according to the embodiment of the present invention includes the following steps:
s1, performing fusion optimization on yoga action data to be evaluated at different visual angles by adopting a three-dimensional skeleton information fusion optimization algorithm to obtain sheltered and completed three-dimensional human body skeleton data.
The three-dimensional skeleton information fusion optimization algorithm comprises the following steps: acquiring images and skeleton sequences with different lengths of a certain yoga action from different view angles, and selecting a plurality of key frame data after space-time alignment to describe the yoga action; and for the key frame skeleton data under each view angle, performing alignment fusion of three-dimensional human body skeleton postures by using an iterative closest point algorithm (ICP algorithm), and optimizing the human body skeleton data by embedding a parameterized human body model (SMPL), thereby obtaining the shielded and supplemented three-dimensional human body skeleton data corresponding to the yoga action.
And the yoga action data to be evaluated is image data of the preprocessed yoga action.
The three-dimensional human body skeleton data subjected to shielding completion is obtained after optimization of a three-dimensional skeleton information fusion optimization algorithm.
And S2, identifying the sheltered and completed three-dimensional human body skeleton data by using the trained hierarchical cascade diagram convolution network model to obtain the yoga action category and the yoga action completion score corresponding to the yoga action data to be evaluated.
The trained hierarchical cascade graph convolution network model is obtained by labeling and training a large amount of training yoga action data with hierarchical category labels and score labels.
Wherein, yoga action classification includes the two-stage classification, specifically includes: a first level and a second level. The grade names of each grade are named according to professional yoga action classification standards, the first grade is coarse-grained classification, and the second grade is fine-grained classification. As shown in fig. 2, for example: the first level includes, but is not limited to: inversion III and balance VIII, etc.; the second level includes, but is not limited to: canine-17 under one leg and cross-balanced I formula-15, etc.
The score for finishing the yoga action is divided into four levels (0-3), and the higher the score is, the closer the yoga action data to be evaluated is to the standard action data.
And S3, acquiring a difficulty coefficient corresponding to the yoga action category, and obtaining an evaluation result of the yoga action data to be evaluated according to the difficulty coefficient and the yoga action completion score.
Wherein, each yoga action classification all corresponds a degree of difficulty coefficient. The difficulty factor is: reference is made to the fitness yoga posture standard published on the official website of the national health yoga directive committee. The evaluation result is formed by multiplying the difficulty coefficient by the yoga action completion score.
Preferably, the S1 includes:
and S11, registering the yoga action data to be evaluated at different viewing angles by adopting an iterative closest point algorithm to obtain a transformation matrix between different viewing angles.
Of these, the iterative closest point algorithm (ICP algorithm) is prior art. Specifically, the ICP algorithm is first used to map the different viewing angles (the present embodiment uses dominant view)Machine position and side machine position) and taking the three-dimensional coordinates of the registration result as initial values; and calculating three-dimensional points through the corresponding depth image and the intrinsic parameters of the camera. After ICP algorithm registration, a transformation matrix T between two views (from a side machine position to a main machine position) is obtainedsf。
And S12, fusing the yoga action data to be evaluated at different visual angles according to the transformation matrix to obtain original three-dimensional human body skeleton data.
Specifically, the joint depth values, the angle values and the recognition confidence degrees of the yoga action data to be evaluated at different view angles are fused through the transformation matrix.
And S13, optimizing the original three-dimensional human body skeleton data according to the parameterized human body model to obtain the sheltered and supplemented three-dimensional human body skeleton data.
Among these, the parameterized human body model (SMPL) is prior art. Specifically, a parameterized human body model (SMPL) is adopted to adjust the posture of each human body joint and the position of the whole human body for the original three-dimensional human body skeleton data, so as to obtain the three-dimensional human body skeleton data subjected to shielding completion.
Preferably, the yoga action data to be evaluated at different viewing angles includes: the yoga action data to be evaluated at the main view angle and the yoga action data to be evaluated at the side view angle; the S12 includes:
fusing the yoga action data to be evaluated at the main view angle and the yoga action data to be evaluated at the side view angle according to a preset fusion transformation formula and the transformation matrix to obtain the original three-dimensional human body skeleton data;
wherein the preset fusion transformation formula is as follows:
Figure BDA0003690583460000071
Figure BDA0003690583460000072
f represents the yoga action data to be evaluated of the main view angle, s represents the yoga action data to be evaluated of the side view angle,
Figure BDA0003690583460000073
joint coordinates, T, for the ith joint of the original three-dimensional body skeleton datasfFor the transformation matrix, SfiJoint coordinates of the ith joint of the yoga action data to be evaluated at the main view angle SsiJoint coordinates W of the ith joint of the yoga action data to be evaluated at the side view anglefiIs SfiCorresponding weight, WsiIs SsiThe corresponding weight of the weight is set to be,
Figure BDA0003690583460000074
θi∈(0，90)，ziis the depth value corresponding to the ith joint, thetaiThe joint angle corresponding to the ith joint, ciConfidence coefficient corresponding to i-th joint, WiThe weight corresponding to the ith joint.
Preferably, the S13 includes:
optimizing the original three-dimensional human body skeleton data according to the parameterized human body model, and obtaining the three-dimensional human body skeleton data subjected to shielding completion when an objective function reaches the minimum;
wherein the objective function is: efused(θ，β)＝ωproEpro+ωshapeEshape，EproRepresenting a function aligning a two-dimensional projection of said original three-dimensional body skeleton data onto a three-dimensional joint, EshapeRepresenting a prior constraint function, omega, of the human body morphologyproIs EproPreset balance weight of, omegashapeIs Eshapeθ represents the length of the control skeleton, and β represents the pose of each joint.
Preferably, the method further comprises the following steps: acquiring original yoga action data of different view angles by adopting a calibration camera, and screening key frames of the original yoga action data of each view angle to obtain the yoga action data to be evaluated at different view angles.
The camera calibration method comprises the following steps: obtaining internal parameters of each camera of the master side position through the KinectAzure SDK, and calculating the geometrical relationship between the main view and the side view of the two cameras through a Matlab stereo camera calibration tool. The key frame screening process comprises the following steps: during screening, the human body skeleton data is mapped to the RGB image and visualized, the qualified key frame data needs to be full in key point quantity and basically accurate in position, and each key frame stores three different types of data, namely color images, depth images and three-dimensional human body skeleton data.
Preferably, the method further comprises the following steps:
and acquiring yoga action training data at a plurality of different viewing angles, and obtaining three-dimensional human body skeleton training data corresponding to the yoga action training data at each different viewing angle by adopting a three-dimensional skeleton information fusion optimization algorithm.
The method for optimizing the yoga action training data by adopting the three-dimensional skeleton information fusion optimization algorithm is consistent with the algorithm.
And performing hierarchical action category marking and action quality scoring on each three-dimensional human body skeleton training data, and training an original hierarchical cascade graph convolution network model by adopting all marked three-dimensional human body skeleton training data to obtain the trained hierarchical cascade graph convolution network model.
The original hierarchical cascade graph convolution network is obtained based on a double-flow graph convolution model (2S-AGCN) cascade improvement, and based on a triple cascade structure of the improved double-flow graph convolution model, the yoga action classification from coarse granularity to fine granularity and the evaluation of the specific yoga posture quality are realized.
Specifically, firstly, two cameras of a main machine position and a side machine position are adopted to shoot a plurality of yoga actions, and the cameras are calibrated before every yoga action shooting. In each calibration, 100 left and right checkerboard images were taken, with a 9 × 12 grid on the checkerboard, each grid having a true side length of 10cm. An internal parameter matrix of each camera is obtained through a Kinect Azure SDK, the geometric relationship between the two cameras at the main machine position and the side machine position is calculated by a Matlab stereo camera calibration tool, and the average re-projection error of the stereo calibration of the cameras is 2.81 pixels.
Secondly, key frame screening: and performing frame extraction processing on each action fragment, reserving 5 frames of key frame RGB images for each action, mapping skeleton data onto the RGB images by using a Kinect SDK and visualizing, and finally reserving 2 key frames and 10-20 frames of dynamic actions for each static action through manual screening. The screened key frame data needs the whole number of key points and basically accurate positions, and the left and right sides of the upper limbs and the lower limbs of the human body are accurately identified without the condition of left and right reversal. And storing three different types of data, namely color map, depth map and three-dimensional skeleton data, in each key frame. The color map is used for describing texture information, the depth map is used for describing distance information, and the three-dimensional skeleton data is used for describing the posture of the human skeleton.
And finally, marking and hierarchically storing the three-dimensional human skeleton training data acquired in different scenes. Folders of 7 male subjects are denoted M01, M02,. M07, folders of 15 female subjects are denoted F01, F02,. F15. The actions are then classified according to their first class classification, which is numbered a01, a 02. Detailed secondary classification is performed under the primary classification, and the pose ID (a 01, a 03.. A117) represents the folder name of the secondary classification. And each action respectively stores the posture data of the main machine position and the side machine position, and each sub-posture folder corresponding to each action fragment respectively comprises a color image, a depth image and a human body skeleton data file. And improving a double-flow graph convolution model (2S-AGCN) to obtain an action recognition model of the graph convolution network in the hierarchical cascade. And constructing a yoga posture recognition task by utilizing the first level and the second level of the cascade network to obtain the results of the first level and the second level. And after the first-stage classification is finished through the 2S-AGCN, filtering out error samples, and inputting each type of action into a corresponding second-stage network to identify a specific action name. This has the advantage that, since the main network and the branch networks are in one unified framework, the feature maps extracted at the beginning can be shared throughout the network, rather than collecting features at each level of the network from the raw data.
For example, fig. 3 shows motion recognition accuracy analysis on a constructed three-dimensional human skeletal training dataset, which can reach 84.03% accuracy when FC (full convolutional layer in deep neural network) = 256.
Preferably, the trained hierarchical cascade graph convolutional network model comprises: the cascade network comprises a first-stage cascade network, a second-stage cascade network and a third-stage cascade network.
The first-level cascade network is used for identifying the large-class action category (coarse granularity) of the yoga action. The second-level cascade network is used for identifying attitude names (fine granularity) of the yoga actions, and the third-level cascade network is used for acquiring action completion scores of the yoga actions.
The S2 comprises the following steps:
and S21, performing coarse-grained and fine-grained action recognition on the sheltered and completed three-dimensional human body skeleton data by utilizing the first-level cascade network and the second-level cascade network to obtain the yoga action category.
The yoga action types are obtained through a first-stage cascade network of a 2S-AGCN model, and then the yoga actions of each type are input into a corresponding second-stage cascade network to identify specific gesture names.
And S22, performing action evaluation on the sheltered and completed three-dimensional human body skeleton data by utilizing the third-level cascade network to obtain the yoga action completion score.
Wherein a third level cascaded network of the 2S-AGCN model is used to predict the completion score for each yoga posture.
Specifically, as shown in fig. 4, the action quality evaluation effect of the three-dimensional human body skeleton training data set on the constructed yoga fitness data set takes action numbers 15 and 83 as examples, and shows the situation that the target hierarchical cascade graph convolution network model automatically scores according to the actions of four different subjects.
According to the technical scheme, the yoga actions are identified and evaluated by constructing the cascade diagram convolutional neural network model, the yoga posture classification and accurate evaluation of the yoga actions to be evaluated are realized, and powerful support is provided for the user to perform targeted improvement on the yoga actions according to evaluation results in the follow-up process.
In another embodiment of the yoga movement evaluation method of the present invention, the method includes:
and S01, performing fusion optimization on the yoga action data to be evaluated at different visual angles by adopting a three-dimensional skeleton information fusion optimization algorithm to obtain the three-dimensional human body skeleton data subjected to shielding completion.
And S02, identifying the sheltered and completed three-dimensional human body skeleton data by using the trained hierarchical cascade graph convolution network model to obtain the yoga action category and the yoga action completion score corresponding to the yoga action data to be evaluated.
And S03, acquiring a difficulty coefficient corresponding to the yoga action category, and obtaining an evaluation result of the yoga action data to be evaluated according to the difficulty coefficient and the yoga action completion score.
And S04, obtaining target difference data according to the difference value between the evaluation result and the corresponding standard result, judging whether the target difference data is larger than preset difference data, if so, optimizing the yoga action data to be evaluated according to the target difference data, and returning to execute the step S1 until the target difference data is smaller than the preset difference data, so as to obtain the evaluation result of the yoga action data to be evaluated.
In this embodiment, a preset difference data may be set, for example, the preset difference data of a certain yoga action is 4, the standard result is 15, when the evaluation result of the subject is 10, the target difference data is 5 (greater than the preset difference data 4), at this time, the subject can know where the difference point between the yoga action to be evaluated and the corresponding standard yoga action is, and the subject continuously adjusts the action according to the difference point, so as to re-evaluate the yoga action, and finally obtain the evaluation result meeting the standard. By adopting the steps, the difference between the standard action and the action of different subjects can be evaluated, the subjects are promoted to continuously perfect and optimize the actions of the subjects while the actions are accurately identified, and the purposes of accurate prediction and effective training are achieved.
Fig. 5 shows that, the yoga action evaluation system 200 according to the embodiment of the present invention includes: a processing module 210, a recognition module 220, and an evaluation module 230;
the processing module 210 is configured to: performing fusion optimization on the yoga action data to be evaluated at different viewing angles by adopting a three-dimensional skeleton information fusion optimization algorithm to obtain three-dimensional human body skeleton data subjected to shielding completion;
the identification module 220 is configured to: identifying the sheltered and completed three-dimensional human body skeleton data by using the trained layered cascade graph convolution network model to obtain a yoga action category and a yoga action completion score corresponding to the yoga action data to be evaluated;
the evaluation module 230 is configured to: and acquiring a difficulty coefficient corresponding to the yoga action category, and obtaining an evaluation result of the yoga action data to be evaluated according to the difficulty coefficient and the yoga action completion score.
Preferably, the processing module 210 includes: the system comprises a first processing module, a second processing module and a third processing module;
the first processing module is configured to: and registering the yoga action data to be evaluated at different viewing angles by adopting an iterative closest point algorithm to obtain a transformation matrix between different viewing angles.
The second processing module is configured to: and fusing the yoga action data to be evaluated at different view angles according to the transformation matrix to obtain original three-dimensional human body skeleton data.
The third processing module is configured to: and optimizing the original three-dimensional human body skeleton data according to a parameterized human body model to obtain the sheltered and supplemented three-dimensional human body skeleton data.
Preferably, the yoga action data to be evaluated at different viewing angles includes: the yoga action data to be evaluated at the main view angle and the yoga action data to be evaluated at the side view angle; the second processing module is specifically configured to:
fusing the yoga action data to be evaluated at the main view angle and the yoga action data to be evaluated at the side view angle according to a preset fusion transformation formula and the transformation matrix to obtain the original three-dimensional human body skeleton data;
wherein the preset fusion transformation formula is as follows:
Figure BDA0003690583460000121
Figure BDA0003690583460000122
f represents the yoga action data to be evaluated of the main view angle, s represents the yoga action data to be evaluated of the side view angle,
Figure BDA0003690583460000123
joint coordinates, T, for the ith joint of the original three-dimensional body skeleton datasfFor the transformation matrix, SfiJoint coordinates of the ith joint of the yoga action data to be evaluated at the main view angle SsiJoint coordinates W of the ith joint of the yoga action data to be evaluated at the side view anglefiIs SfiCorresponding weight, WsiIs SsiThe corresponding weight of the weight is set to be,
Figure BDA0003690583460000131
θi∈(0，90)，ziis the depth value corresponding to the ith joint, thetaiThe joint angle corresponding to the ith joint, ciConfidence coefficient corresponding to i-th joint, WiThe weight corresponding to the ith joint.
Preferably, the third processing module is specifically configured to:
optimizing the original three-dimensional human body skeleton data according to the parameterized human body model, and obtaining the three-dimensional human body skeleton data subjected to shielding completion when an objective function reaches the minimum;
wherein the objective function is: efused(θ，β)＝ωproEpro+ωshapeEshape，EproRepresenting a function aligning a two-dimensional projection of said original three-dimensional body skeleton data onto a three-dimensional joint, EshapeRepresenting a prior constraint function, omega, of the human body morphologyproIs EproIs preset to balance the weight, ωshapeIs EshapeTheta denotes the length of the control skeleton, beta denotes the posture of each joint to be controlled。
Preferably, the method further comprises the following steps: a preprocessing module;
the preprocessing module is used for: acquiring original yoga action data of different view angles by adopting a calibration camera, and screening key frames of the original yoga action data of each view angle to obtain the yoga action data to be evaluated at different view angles.
Preferably, the method further comprises the following steps: a training module; the training module is configured to:
acquiring yoga action training data of a plurality of different viewing angles, and obtaining three-dimensional human body skeleton training data corresponding to the yoga action training data of each different viewing angle by adopting the three-dimensional skeleton information fusion optimization algorithm;
and performing hierarchical action category marking and action quality scoring on each three-dimensional human body skeleton training data, and training an original hierarchical cascade graph convolution network model by adopting all marked three-dimensional human body skeleton training data to obtain the trained hierarchical cascade graph convolution network model.
Preferably, the target hierarchical cascade graph convolutional network model comprises: a first cascade network, a second cascade network and a third cascade network;
the identification module 220 is specifically configured to:
performing coarse-grained and fine-grained action recognition on the sheltered and completed three-dimensional human body skeleton data by using the first-level cascade network and the second-level cascade network to obtain the yoga action category;
and utilizing the third-level cascade network to evaluate the movement of the sheltered and completed three-dimensional human body skeleton data to obtain the yoga movement completion score.
According to the technical scheme, the yoga action is identified and evaluated by constructing the cascade diagram convolutional neural network model, the yoga action to be evaluated is classified and accurately evaluated, and powerful support is provided for the user to subsequently improve the yoga action pertinently according to the evaluation result.
The above steps for implementing the corresponding functions of each parameter and each module in the yoga motion evaluation system 200 according to the present invention may refer to each parameter and step in the above embodiments of a yoga motion evaluation method, which are not described herein again.
An embodiment of the present invention provides a storage medium, including: the storage medium stores instructions, and when the instructions are read by a computer, the computer is caused to execute the steps of the yoga action evaluation method as described above, which may specifically refer to each parameter and step in an embodiment of the yoga action evaluation method described above, and are not described herein again.
Computer storage media such as: flash disks, portable hard disks, and the like.
An electronic device provided in an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the computer program is executed by the processor, so that the computer executes the steps of the yoga motion evaluation method as described above, and specifically, reference may be made to each parameter and step in the above embodiment of the yoga motion evaluation method, which is not described herein again.
Those skilled in the art will appreciate that the present invention may be embodied as methods, systems, storage media and electronic devices.
Thus, the present invention may be embodied in the form of: may be embodied entirely in hardware, entirely in software (including firmware, resident software, micro-code, etc.) or in a combination of hardware and software, and may be referred to herein generally as a "circuit," module "or" system. Furthermore, in some embodiments, the invention may also be embodied in the form of a computer program product in one or more computer-readable media having computer-readable program code embodied in the medium. Any combination of one or more computer-readable media may be employed. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.

Claims (10)

1. A yoga action assessment method, comprising:
performing fusion optimization on the yoga action data to be evaluated at different view angles by adopting a three-dimensional skeleton information fusion optimization algorithm to obtain sheltered and supplemented three-dimensional human body skeleton data;
identifying the sheltered and completed three-dimensional human body skeleton data by using the trained layered cascade graph convolution network model to obtain a yoga action category and a yoga action completion score corresponding to the yoga action data to be evaluated;
and acquiring a difficulty coefficient corresponding to the yoga action category, and obtaining an evaluation result of the yoga action data to be evaluated according to the difficulty coefficient and the yoga action completion score.
2. The yoga action evaluation method of claim 1, wherein the fusion optimization of the yoga action data to be evaluated at different viewing angles is performed by using a three-dimensional skeleton information fusion optimization algorithm to obtain the three-dimensional human body skeleton data with complete occlusion, and the method comprises:
registering the yoga action data to be evaluated at different viewing angles by adopting an iterative closest point algorithm to obtain a transformation matrix between different viewing angles;
fusing the yoga action data to be evaluated at different visual angles according to the transformation matrix to obtain original three-dimensional human body skeleton data;
and optimizing the original three-dimensional human body skeleton data according to a parameterized human body model to obtain the sheltered and supplemented three-dimensional human body skeleton data.
3. The yoga action evaluation method of claim 2, wherein the yoga action data to be evaluated at different viewing angles comprises: the yoga action data to be evaluated at the main view angle and the yoga action data to be evaluated at the side view angle; according to the transformation matrix is to the yoga action data of waiting to assess of different visual angles fuses, obtains original three-dimensional human skeleton data, includes:
fusing the yoga action data to be evaluated at the main view angle and the yoga action data to be evaluated at the side view angle according to a preset fusion transformation formula and the transformation matrix to obtain the original three-dimensional human body skeleton data;
wherein the preset fusion transformation formula is as follows:
Figure FDA0003690583450000021
Figure FDA0003690583450000022
f represents the yoga action data to be evaluated of the main view angle, s represents the yoga action data to be evaluated of the side view angle,
Figure FDA0003690583450000023
joint coordinates, T, for the ith joint of the original three-dimensional body skeleton datasfFor the transformation matrix, SfiJoint coordinates of the ith joint of the yoga action data to be evaluated at the main view angle, SsiJoint coordinates W of the ith joint of the yoga action data to be evaluated at the side view anglefiIs SfiCorresponding weight, WsiIs SsiThe corresponding weight of the weight is set to be,
Figure FDA0003690583450000024
ziis the depth value corresponding to the ith joint, thetaiThe joint angle corresponding to the ith joint, ciConfidence coefficient, W, for the ith jointiThe weight corresponding to the ith joint.
4. The yoga movement evaluation method of claim 2, wherein the optimizing the original three-dimensional human skeleton data according to the parameterized human model to obtain the occlusion-supplemented three-dimensional human skeleton data comprises:
optimizing the original three-dimensional human body skeleton data according to the parameterized human body model, and obtaining the three-dimensional human body skeleton data subjected to shielding completion when an objective function reaches the minimum;
wherein the objective function is: efused(θ，β)＝ωproEpro+ωshapeEshape，EproRepresenting a function aligning a two-dimensional projection of said original three-dimensional body skeleton data onto a three-dimensional joint, EshapeRepresenting a prior constraint function, omega, of the human body morphologyproIs EproIs preset to balance the weight, ωshapeIs Eshapeθ represents the length of the control skeleton, and β represents the pose of each joint.
5. The yoga action assessment method of claim 1, further comprising: acquiring original yoga action data of different view angles by adopting a calibration camera, and screening key frames of the original yoga action data of each view angle to obtain the yoga action data to be evaluated at different view angles.
6. The yoga action assessment method of claim 1, further comprising:
acquiring yoga action training data of a plurality of different viewing angles, and obtaining three-dimensional human body skeleton training data corresponding to the yoga action training data of each different viewing angle by adopting the three-dimensional skeleton information fusion optimization algorithm;
and performing hierarchical action category marking and action quality scoring on each three-dimensional human body skeleton training data, and training an original hierarchical cascade graph convolution network model by adopting all marked three-dimensional human body skeleton training data to obtain the trained hierarchical cascade graph convolution network model.
7. The yoga movement evaluation method of claim 6, wherein the trained hierarchical cascade graph convolutional network model comprises: a first cascade network, a second cascade network and a third cascade network; utilize the layering cascade map convolution network model after the training right shelter from the three-dimensional human skeleton data of completion and discern, obtain treat that aassessment yoga action classification and the score is accomplished in the yoga action that yoga action data corresponds, include:
performing coarse-grained and fine-grained action recognition on the sheltered and completed three-dimensional human body skeleton data by using the first-level cascade network and the second-level cascade network to obtain the yoga action category;
and utilizing the third-level cascade network to evaluate the movement of the sheltered and completed three-dimensional human body skeleton data to obtain the yoga movement completion score.
8. A yoga action evaluation system, comprising: the device comprises a processing module, an identification module and an evaluation module;
the processing module is used for: performing fusion optimization on the yoga action data to be evaluated at different viewing angles by adopting a three-dimensional skeleton information fusion optimization algorithm to obtain three-dimensional human body skeleton data subjected to shielding completion;
the identification module is configured to: identifying the sheltered and completed three-dimensional human body skeleton data by using the trained layered cascade graph convolution network model to obtain a yoga action category and a yoga action completion score corresponding to the yoga action data to be evaluated;
the evaluation module is to: and acquiring a difficulty coefficient corresponding to the yoga action category, and obtaining an evaluation result of the yoga action data to be evaluated according to the difficulty coefficient and the yoga action completion score.
9. A storage medium having instructions stored therein, which when read by a computer, cause the computer to perform a yoga movement evaluation method according to any one of claims 1 to 7.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor, when executing the computer program, causes the computer to perform a yoga action evaluation method according to any one of claims 1 to 7.
CN202210666543.1A 2022-06-13 2022-06-13 Yoga action evaluation method and system, storage medium and electronic equipment Pending CN115273219A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210666543.1A CN115273219A (en) 2022-06-13 2022-06-13 Yoga action evaluation method and system, storage medium and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210666543.1A CN115273219A (en) 2022-06-13 2022-06-13 Yoga action evaluation method and system, storage medium and electronic equipment

Publications (1)

Publication Number Publication Date
CN115273219A true CN115273219A (en) 2022-11-01

Family

ID=83758880

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210666543.1A Pending CN115273219A (en) 2022-06-13 2022-06-13 Yoga action evaluation method and system, storage medium and electronic equipment

Country Status (1)

Country Link
CN (1) CN115273219A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116525061A (en) * 2023-03-09 2023-08-01 北京体育大学 Training monitoring method and system based on remote human body posture assessment

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116525061A (en) * 2023-03-09 2023-08-01 北京体育大学 Training monitoring method and system based on remote human body posture assessment
CN116525061B (en) * 2023-03-09 2024-04-02 北京体育大学 Training monitoring method and system based on remote human body posture assessment

Similar Documents

Publication Publication Date Title
CN112270249A (en) Target pose estimation method fusing RGB-D visual features
CN110264444B (en) Damage detection method and device based on weak segmentation
CN112446270A (en) Training method of pedestrian re-identification network, and pedestrian re-identification method and device
CN108805016B (en) Head and shoulder area detection method and device
CN114663502A (en) Object posture estimation and image processing method and related equipment
CN112200056B (en) Face living body detection method and device, electronic equipment and storage medium
CN110910445B (en) Object size detection method, device, detection equipment and storage medium
CN113569598A (en) Image processing method and image processing apparatus
CN112836625A (en) Face living body detection method and device and electronic equipment
CN115049821A (en) Three-dimensional environment target detection method based on multi-sensor fusion
CN112336342A (en) Hand key point detection method and device and terminal equipment
CN114219855A (en) Point cloud normal vector estimation method and device, computer equipment and storage medium
CN113781519A (en) Target tracking method and target tracking device
CN113012157A (en) Visual detection method and system for equipment defects
CN113706579A (en) Prawn multi-target tracking system and method based on industrial culture
CN113487610B (en) Herpes image recognition method and device, computer equipment and storage medium
CN115273219A (en) Yoga action evaluation method and system, storage medium and electronic equipment
CN113496148A (en) Multi-source data fusion method and system
CN111531546B (en) Robot pose estimation method, device, equipment and storage medium
CN111738061A (en) Binocular vision stereo matching method based on regional feature extraction and storage medium
CN112329736B (en) Face recognition method and financial system
CN114612545A (en) Image analysis method and training method, device, equipment and medium of related model
CN110728222B (en) Pose estimation method for target object in mechanical arm grabbing system
Zhang et al. Object detection based on deep learning and b-spline level set in color images
CN111681270A (en) Method, device and storage medium for realizing registration between image frames

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination