WO2023108842A1

WO2023108842A1 - Motion evaluation method and system based on fitness teaching training

Info

Publication number: WO2023108842A1
Application number: PCT/CN2022/070026
Authority: WO
Inventors: 曾晓嘉; 刘易; 薛立君
Original assignee: 成都拟合未来科技有限公司
Priority date: 2021-12-14
Filing date: 2022-01-04
Publication date: 2023-06-22

Abstract

A motion evaluation method, system, and apparatus based on fitness teaching training, and a medium, relating to the field of fitness. The method comprises: obtaining a fitness video, a standard posture being preset in the fitness video; obtaining a time period corresponding to the standard posture in the fitness video; obtaining image frames at continuous time points corresponding to the fitness video within the time period; identifying a plurality of user postures corresponding to the image frames at the continuous time points within the time period; and comparing the plurality of user postures with the standard posture, and determining whether a user posture and the standard posture are of the same type. During comparison, a model is constructed by means of a convolutional neural network, and identification is implemented by means of 16 skeleton key points of the human body and respective two-dimensional position coordinates thereof, such that the accuracy is high, whether a user posture is standard can be obtained quickly, and the use efficiency is improved.

Description

Movement evaluation method and system based on fitness teaching and training

technical field

The present disclosure relates to the field of fitness, in particular to an action evaluation method, system, device and medium based on fitness teaching and training.

Background technique

In recent years, the health literacy of the people has been continuously improved, and the demand for fitness exercises has also been increasing. The market for the fitness industry is huge. All kinds of intelligent fitness equipment are developing rapidly. The existing mirror fitness equipment is displayed and/or mirrored by the front screen through the built-in various equipment in the body. Users can perform fitness training according to the displayed fitness content. , fitness coaches can conduct fitness teaching to multiple users through live video or pre-recorded video, wherein, when fitness coaches conduct fitness teaching, they are usually based on the approximate fitness level of the multiple users, such as flow yoga 2 Class fitness level, etc., to carry out unified fitness teaching. However, there are many problems in such a fitness training teaching method. For example, timely feedback cannot be obtained on the user's follow-up situation.

In the process of teaching, it is necessary to monitor and evaluate the user's actions to see whether the user's actions meet the standards. At present, commonly used human behavior detection methods include myoelectric detection, airbag sensor information acquisition, and visual image methods. The method of myoelectric detection is to use the bio-myoelectric signal generated by human body movement to identify human body movements, but the user needs to wear the sensor when using it, which is mostly used for scientific research in specific scenarios and does not meet the needs of ordinary fitness. The method of the airbag is similar to the electromyographic detection method, and it is also necessary to wear relevant sensors during the user's exercise to obtain its motion information. Therefore, whether it is myoelectric detection method or the method based on the use of wearing sensors such as airbags, the user needs to wear the sensor, which is not suitable for the user's posture correction in daily fitness exercises. Visual image methods, such methods need to use images captured by user equipment to implement user profile estimation, user skeleton map estimation, etc. to estimate the user's posture and actions. The main application is Openpose, which uses a large amount of human activity data and labels to train a graph neural network to recognize human body poses. However, in actual use, the ability to recognize the posture of the human body is poor, and it cannot better detect whether the user's action is standard.

Contents of the invention

The purpose of the present disclosure is to better obtain the user's training situation and user posture when performing fitness training for the user through the fitness video, and judge whether the user's posture is up to standard, thereby ensuring the fitness effect of the fitness video.

In order to achieve the above disclosed purpose, the present disclosure provides an action evaluation method based on fitness teaching and training, including:

Obtain fitness videos, in which standard postures are preset;

Obtain the time period corresponding to the standard posture in the fitness video;

Acquire frame images of continuous moments corresponding to the fitness video during the time period;

Identify several user gestures corresponding to frame images at consecutive moments in the time period;

Compare several user postures with standard postures, and determine whether the user postures and standard postures are of the same type.

For this disclosure, when evaluating whether the user's actions are of the same type, not every standard posture of the coach in the fitness video is compared with the user's posture, but only by comparing the preset standard postures, and During the time period when the standard posture appears in the fitness video, the user posture is obtained for comparison, and the standard posture and the user posture are both static postures, so it is easier to operate and use when comparing.

At the same time, the standard posture of the present disclosure appears at a certain time point of the fitness video, but when recognizing the user posture, the present disclosure acquires the user posture in the interval before and after the time point, that is, in a time period, and uses this The user posture corresponding to each frame of image in the time period is compared with the standard posture. This is because the user follows the video training. If a novice has taken the course every time, his actions are lagging behind the course. If he is already familiar with the course Lesson learned, the user may perform actions ahead of time, so when in use, this disclosure expands the time point when the standard posture appears into a time period, and when making a comparison, the basis is accurate and the efficiency is higher.

Further, the present disclosure compares several user postures with standard postures, and judges whether the user postures and standard postures are of the same type, specifically through the twin neural network model for comparison, specifically including:

Train the twin neural network model to get the trained standard gesture recognition model;

Input the user pose and the standard pose into the standard pose recognition model, and judge whether the user pose and the standard pose are of the same type.

Among them, when training the twin neural network, the model training process of the present disclosure is trained through larger samples, such as gesture samples such as raising hands and legs, so that the model training learns how to extract gesture features, that is, mapping from 32 dimensions to 100 dimensions the process of.

Further, input the user posture and the standard posture into the standard posture recognition model, and judge whether the user posture and the standard posture are of the same type, specifically including:

Get the bone key points of the standard pose and user pose and the position coordinates corresponding to each bone key point;

Input the position coordinates corresponding to each bone key point of the standard pose and the user pose into the trained standard pose recognition model, and obtain the output vector V1 of the standard pose and the output vector V2 of the user pose respectively;

Calculate the Euclidean distance between the output vector V1 of the standard posture and the output vector V2 of the user posture;

It is judged whether the user posture and the standard posture are of the same type by the Euclidean distance between the output vector V1 of the standard posture and the output vector V2 of the user posture.

Among them, in this disclosure, a human body posture has a total of 16 bone points with two-dimensional coordinates, and each bone point has x and y coordinate components, so a human body posture can be abstracted into a 32-dimensional bone point vector, namely [x1, y1 ,x2,y2,x3,y3,...,x16,y16]. After passing the trained pose recognition model, the 32-dimensional bone point vector will be mapped into a higher-dimensional vector. The output vector in this disclosure is 100-dimensional, that is, the output vector V1 of the standard pose and the V1 of the user pose The output vectors V2 are all 100-dimensional, namely [a1, a2, a3,..., a100]. When performing pose comparison, the trained models of the standard pose and the user pose will each be mapped into a 100-dimensional vector, namely V1 and V2, and then the Euclidean distance between V1 and V2 will be calculated.

This disclosure uses a deep neural network, which accepts a 32-dimensional vector, that is, a human body pose in this disclosure, and then undergoes a series of intermediate layer operations, such as nonlinear correction, full connection, etc., and finally outputs a 100-dimensional vector. This 100-dimensional vector is a highly abstract feature; in the end, if the two poses are very similar, the Euclidean distance between the two 100-dimensional vectors output by the network is very small, otherwise, the Euclidean distance is very large.

Our neural network has a total of 4 layers. The number of nodes in each layer from input to output is 32->64->128->100, that is, a 32-dimensional vector is input and a 100-dimensional vector is output. The n-dimensional Euclidean The distance calculation formula, this disclosure maps to 100 dimensions, ie n=100:

Input the user pose and the standard pose into the standard pose recognition model, and judge whether the user pose and the standard pose are of the same type. Among them, the user pose and standard pose are input into the standard pose recognition model, and the acquaintance score between the user pose and the standard pose is output, and the user pose with the highest acquaintance score is obtained as the scoring result, and the scoring result is used to judge whether the user pose and the standard pose are of the same type .

Preferably, the Euclidean distance threshold T is obtained based on the standard gesture recognition model, and the threshold T is used to judge whether the user gesture and the standard gesture are of the same type, wherein, if the Euclidean distance output by the standard gesture recognition model is greater than or equal to the threshold T, then the user gesture and the standard gesture The standard poses are of the same type. If the Euclidean distance output by the standard pose recognition model is less than or equal to the threshold T, the user pose and the standard pose are of different types.

Furthermore, after the Siamese network model is trained, it will look for a threshold T of the Euclidean distance on our test set: if the Euclidean distance of the two poses exceeds T, it is considered not to be of the same type; otherwise, it is considered to be the same model.

For each threshold T, the ROC curve can be drawn. The area under the ROC curve, called AUC, is a value from 0 to 1. The larger the AUC, the better the model performance. Find an optimal threshold T-best that maximizes AUC on the test set. In layman's terms, the largest AUC means that the model will judge as many poses that originally belong to the same class as the same class as much as possible, and at the same time, it will try to misjudge two poses that do not belong to the same class as the same class as little as possible. After obtaining the optimal distance threshold T-best, set a critical score according to actual business needs, such as 40 points, which means that at this time, the model believes that the two postures are just at the critical point of similarity and dissimilarity. The mapping relationship is as follows: when the actual distance t is in the interval [0, T-best], the similarity score s is [100, 40]; when the actual distance t is in (T-best, infinity), the similarity score s is (40 ,0).

Moreover, both the standard pose and the user pose include 16 bone key points, and the 16 bone key points correspond to a two-dimensional position coordinate. Among them, 16 bone key points include the top of the head, the bottom of the head, the neck, the right shoulder, the right elbow, the right hand, the left shoulder, the left elbow, the left hand, the right hip, the right knee, the right foot, the left hip, the left knee, the left foot, and the patella .

The present disclosure also provides a fitness training method based on a fitness device, the fitness training method is based on the above-mentioned exercise evaluation method based on fitness teaching training, and the fitness training method includes:

Obtaining the first user's posture according to the first fitness video, and judging whether the user is performing fitness training;

Acquiring the second user's posture according to the second fitness video, scoring the second user's posture, and judging whether the user follows the second fitness video for fitness training according to the scoring result;

The first user posture is acquired according to the first fitness video, the first user posture is scored, and the scoring result is fed back to the user.

For this disclosure, the fitness video is played three times, and the first time mainly serves as a demonstration. By playing the fitness video, the user can understand the training content. In the process, it is only judged whether the user took an action, but it does not score the action that occurred, and does not judge whether it has done the same action. When playing for the second time, it is played in slow motion of the first fitness video. The second fitness video can be played in slow motion for the fitness video played for the first time, or can be decomposed for the action of the first fitness video. When playing, monitor Whether the user performs follow-up according to the second exercise video, and scores the user's actions to determine whether the user has performed the same action as the exercise video, and then determines whether the user has performed follow-up. When playing for the third time, the video played for the third time is the same as the video played for the first time, but when the video is played for the third time, the user's actions will be recognized, and at the same time, the score will be used to judge whether the user's actions are up to standard , thereby improving the fitness effect of the user. Corresponding to this disclosure, the video played for the first time, the video played for the second time, and the video played for the third time are all videos with the same content, except that the video played for the second time is a slower version of the video played for the first time. Release or action decomposition.

Wherein, according to the first fitness video, the posture of the first user is obtained, and whether the user performs fitness training is judged, specifically including:

Obtain the first fitness video for fitness; when acquiring the first fitness video, in order to directly play the first fitness video in the fitness device, the fitness device can be a smart fitness mirror or other fitness devices that can play videos;

When playing the first fitness video, the fitness device recognizes the target fitness area of the first fitness video, and the user is exercising in the target fitness area at this time, so when performing feature extraction on the target fitness area, the first user posture can be obtained;

In the process of judging whether the user is performing fitness training according to the first user posture, it is to determine whether the user has an action. If there is an action, the user is performing fitness training. No action is generated, and the effect of playing the video for the first time is relatively poor, and it cannot be guaranteed that the user can quickly grasp the action when playing the video for the second or third time.

After playing the video for the first time, the user has a preliminary familiarity with the actions in the fitness video. After familiarization, the second user's posture is obtained according to the second fitness video, and the second user's posture is scored. According to the scoring result, it is judged whether the user follows the first Two fitness videos for fitness training, including:

Presetting a second standard posture according to the second fitness video;

Identifying the target fitness area of the second fitness video, performing feature extraction on the target fitness area, and obtaining the second user posture;

Comparing the second standard posture and the second user posture, the acquaintance score of the second user posture based on the second standard posture is obtained, and judging whether the user follows the second fitness video for fitness training according to the scoring result.

Among them, identifying the target fitness area of the second fitness video, performing feature extraction on the target fitness area, and obtaining the second user posture, specifically including:

Obtain the first time period when the second standard posture appears in the second fitness video;

Obtain the video segment corresponding to the second fitness video in the first time period, and process the video segment into frames to obtain frame images corresponding to several consecutive moments of the video segment;

Identifying the target fitness area of the second fitness video in the first time period, performing feature extraction on the target fitness area, and obtaining a plurality of second user postures corresponding to the frame images one-to-one;

Comparing corresponding several frame images and several second user postures, obtaining the acquaintance score of each second user posture based on the corresponding frame images;

Obtain the second user pose with the highest acquaintance score as the scoring result.

When the fitness video is played for the second time, at this stage, the user's actions should be scored to determine whether the user follows the second fitness video. Each frame is scored and judged, but the second standard posture is preset, and the second standard posture corresponding to each frame image in the first time period is obtained by comparing the first time period when the second standard posture appears in the second fitness video. For the user pose, compare the corresponding several frame images with the several second user poses to obtain the acquaintance score of each second user pose based on the corresponding frame images. When judging whether the user is following the action, it is better to obtain the second user posture of the user in the corresponding time period through the preset second standard posture, and compare each frame image with the corresponding second user posture. To determine whether the user is following.

For this disclosure, the specific scoring process includes:

Inputting the second user pose and the second standard pose into the standard pose recognition model to obtain an acquaintance score;

If the scoring result is greater than or equal to the scoring threshold, then the second standard posture and the second user posture are of the same type, and the user performs fitness training;

If the scoring result is less than the scoring threshold, the second standard posture and the second user posture are not in the same category, and the user has not performed fitness training.

On the third play, when the first workout video is played, it includes:

Obtain the first fitness video for fitness, that is, play the first fitness video on the fitness device, and preset the first standard posture according to the first fitness video;

Identifying the target fitness area of the first fitness video, performing feature extraction on the target fitness area, and obtaining the first user posture;

Comparing the first standard posture with the first user posture, obtaining the acquaintance score of the first user posture based on the first standard posture, and feeding back the scoring result to the user.

Among them, identifying the target fitness area of the first fitness video, performing feature extraction on the target fitness area, and obtaining the first user posture, specifically including:

Obtain the second time period when the first standard posture appears in the first fitness video;

Identifying the target fitness area of the first fitness video in the second time period, performing feature extraction on the target fitness area, and obtaining several first user postures;

Comparing the first standard posture and several first user postures, obtaining the acquaintance score of each first user posture based on the first standard posture;

Obtain the first user pose with the highest acquaintance score as the scoring result.

When the video is played for the second time and the video is played for the third time, when the video is played for the second time, each frame image in the first time period is compared with the corresponding second user gesture, and when the video is played for the third time, it is The first user posture in the second time period is compared with the first standard posture, so that when the video is played for the third time, it is possible to better confirm whether the actions followed by the user meet the standards and improve the efficiency of fitness teaching and training.

Corresponding to the action evaluation method based on fitness teaching and training in the present disclosure, the present disclosure also provides an action evaluation system based on fitness teaching and training, including:

The obtaining module is used to obtain the fitness video, and obtain the time period corresponding to the preset standard posture in the fitness video, and obtain the frame images of the continuous moments corresponding to the fitness video in the time period;

An identification module, configured to identify several user gestures corresponding to the frame images at consecutive moments in the above time period;

A comparison module is used to compare several user postures and standard postures to obtain comparison results;

The judging module is used to judge whether the user posture and the standard posture are of the same type according to the comparison result.

Further, the comparison module compares several user postures and standard postures, specifically including training a twin neural network model to obtain a trained standard posture recognition model;

The judging module judges whether the user posture and the standard posture are of the same type according to the comparison result, specifically including inputting the user posture and the standard posture into the standard posture recognition model, and judging whether the user posture and the standard posture are of the same type.

Further, the judging module inputs the user posture and the standard posture into the standard posture recognition model, and judges whether the user posture and the standard posture are of the same type, specifically including:

Further, the judgment module obtains the Euclidean distance threshold T based on the standard posture recognition model, and the threshold T is used to judge whether the user posture and the standard posture are of the same type, wherein, if the Euclidean distance output by the standard posture recognition model is less than or equal to the threshold T, Then the user pose is of the same type as the standard pose, and if the Euclidean distance output by the standard pose recognition model is greater than the threshold T, then the user pose and the standard pose are of a different type.

Further, both the standard posture and the user posture include 16 skeletal key points, each of which corresponds to a two-dimensional position coordinate, and the 16 skeletal key points include the top of the head, the bottom of the head, the neck, the right shoulder, the right elbow, and the right hand. , left shoulder, left elbow, left hand, right hip, right knee, right foot, left hip, left knee, left foot, patella.

Further, the judging module inputs the user posture and the standard posture into the standard posture recognition model, outputs the acquaintance score between the user posture and the standard posture, obtains the user posture with the highest acquaintance score as the scoring result, and judges the user posture and the standard posture based on the scoring result. Whether the gestures are of the same type.

Corresponding to the fitness training method based on the fitness device in the disclosure, the disclosure also provides a fitness training system based on the fitness device, the fitness training system is based on the above-mentioned exercise evaluation system based on fitness teaching and training, and the fitness training system executes:

The present disclosure also provides another fitness training system based on a fitness device. Based on the above-mentioned action evaluation system based on fitness teaching and training, the fitness training system includes:

The obtaining module is used to obtain the fitness video and process the fitness video;

The identification module is used to identify the target fitness area of the fitness video, and extracts the features of the target fitness area to obtain the user's posture;

A comparison module is used to compare the user's posture with a preset standard posture to obtain a comparison result;

The judging module is used to judge whether the user performs fitness training or whether the action meets the standard according to the comparison result. Corresponding to the action evaluation method based on fitness teaching training in the present disclosure, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and operable on the processor, When the processor executes the computer program, the above-mentioned action evaluation method based on fitness teaching and training is realized.

Corresponding to the fitness training method based on the fitness device in the present disclosure, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and operable on the processor, the When the processor executes the computer program, the above fitness device-based fitness training method is realized.

Corresponding to the action evaluation method based on fitness teaching training in this disclosure, this disclosure also provides a storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned fitness-based Action evaluation methods for teaching and training.

Corresponding to the fitness training method based on the fitness device in the present disclosure, the present disclosure also provides a storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above fitness device-based training method is implemented. fitness training methods.

One or more technical solutions provided by the present disclosure have at least the following technical effects or advantages:

In this disclosure, when evaluating whether the user's actions are of the same type, it does not compare and judge every standard posture of the coach in the fitness video with the user's posture, but only compares the preset standard postures, and the standard postures in the fitness In the time period that appears in the video, the user pose is obtained for comparison, and the standard pose and the user pose are both static poses, so it is easier to operate and use when comparing. Secondly, the preset standard posture can be a fixed-point movement, which can more accurately grasp the user's fitness training situation when evaluating the user's movement.

Secondly, when making a comparison in the present disclosure, the model is constructed through a convolutional neural network, and the 16 key points of the skeleton of the human body and their corresponding two-dimensional position coordinates are used for identification, which has higher accuracy and can quickly determine whether the user's posture is standard or not. , improve efficiency.

In addition, when performing fitness teaching and training in this disclosure, the user is given fitness training action demonstration, slow motion teaching, and normal follow-up through three stages. Whether there is an action in the first stage, whether it is followed up in the second stage, whether the user’s action is up to the standard in the third stage, and feedback the scoring result to the user.

In addition, when the present disclosure performs identification and comparison on user actions, the identification and comparison is performed according to the first time period or the second time period corresponding to the preset first standard posture or the second standard posture, and it is not necessary to perform identification and comparison on the entire video. The effect is better, the result is obtained faster, the user's fitness effect is effectively guaranteed, and the user's fitness situation can be obtained at each stage, which improves the efficiency of fitness use and is more convenient for long-term use.

Description of drawings

The drawings described here are used to provide further understanding of the embodiments of the present disclosure, constitute a part of the application, and do not limit the embodiments of the present disclosure. In the attached picture:

Fig. 1 is the schematic flow chart of the action evaluation method based on fitness teaching training;

Fig. 2 is a schematic diagram when two postures belong to different types;

Figure 3 is a schematic diagram when two postures belong to the same type;

Fig. 4 is ROC curve;

Fig. 5 is a schematic flow chart of a fitness training method based on a fitness device;

Fig. 6 is a schematic composition diagram of an action evaluation system based on fitness teaching and training or a fitness training system based on a fitness device.

Detailed ways

In order to better understand the above objects, features and advantages of the present disclosure, the present disclosure will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, under the condition of not conflicting with each other, the embodiments of the present disclosure and the features in the embodiments can be combined with each other.

In the following description, many specific details are set forth in order to fully understand the present disclosure. However, the present disclosure can also be implemented in other ways different from the scope of this description. Therefore, the protection scope of the present disclosure is not limited by the following disclosure. The limitations of specific examples.

Those skilled in the art should understand that, in the disclosure of the present disclosure, the terms "vertical", "transverse", "upper", "lower", "front", "rear", "left", "right", " The orientation or positional relationship indicated by "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, which are only for the convenience of describing the present disclosure and The above terms are not to be construed as limiting the present disclosure because the description is simplified rather than indicating or implying that the device or element referred to must have a particular orientation, be constructed, and operate in a particular orientation.

It can be understood that the term "a" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element can be one, while in another embodiment, the number of the element The quantity can be multiple, and the term "a" cannot be understood as a limitation on the quantity.

Embodiment one

Please refer to FIG. 1. FIG. 1 is a schematic flowchart of an action evaluation method based on fitness teaching and training. The present disclosure provides an action evaluation method based on fitness teaching and training. The method includes:

Obtain fitness videos, in which standard postures are preset;

Compare several user postures with standard postures, and determine whether the user postures and standard postures are of the same type. Input the user pose and standard pose into the standard pose recognition model, output the acquaintance score between the user pose and the standard pose, obtain the user pose with the highest acquaintance score as the scoring result, and judge whether the user pose and the standard pose are of the same type through the scoring result.

Among them, comparing several user postures and standard postures, and judging whether the user postures and standard postures are of the same type, specifically include:

Input the user posture and standard posture into the standard posture recognition model, and judge whether the user posture and standard posture are of the same type, including:

Among them, the standard pose and the user pose both include 16 bone key points, and the 16 bone key points correspond to a two-dimensional position coordinate. The 16 bone key points include the top of the head, the bottom of the head, the neck, the right shoulder, the right elbow, the right hand, the left shoulder, the left elbow, the left hand, the right hip, the right knee, the right foot, the left hip, the left knee, the left foot, and the patella.

The following is an introduction to the action evaluation method based on fitness teaching and training in the present disclosure in conjunction with specific examples:

Operation 1 obtains a fitness video, and a standard posture is preset in the fitness video; in this embodiment, the fitness device is a fitness mirror, and the fitness video is played on the mirror surface of the fitness mirror;

Operation 2 obtains the time period corresponding to the standard posture in the fitness video, and obtains the frame images of the continuous moments corresponding to the fitness video in this time period;

Operation 3 recognizes several user gestures corresponding to the frame images at consecutive moments in the time period;

Operation 3.1 The fitness mirror recognizes the user's actions in the target fitness area, extracts the features of the target fitness area, and obtains the user's posture;

Operation 4 compares several user postures with standard postures, and determines whether the user postures and standard postures are of the same type;

Operation 4.1 train the twin neural network model to obtain the trained standard gesture recognition model;

Operation 4.2 Obtain the bone key points of the standard pose and the user pose and the position coordinates corresponding to each bone key point. Among them, the standard pose and the user pose both include 16 bone key points, and the 16 bone key points correspond to a two-dimensional position coordinate ;16 bone key points include top of head, bottom of head, neck, right shoulder, right elbow, right hand, left shoulder, left elbow, left hand, right hip, right knee, right foot, left hip, left knee, left foot, patella;

Operation 4.3 Input the position coordinates corresponding to each bone key point of the standard posture and the user posture into the trained standard posture recognition model, and obtain the output vector V1 of the standard posture and the output vector V2 of the user posture respectively;

Among them: the neural network has a total of 4 layers, and the number of nodes in each layer from input to output is 32->64->128->100, that is, a 32-dimensional vector is input, a 100-dimensional vector is output, and the European style of n-dimensional space The distance calculation formula, this disclosure maps to 100 dimensions, ie n=100:

The standard gesture recognition model outputs two high-dimensional vectors, which in this embodiment are 100-dimensional vectors. If these two gestures belong to different types, as shown in Figure 2, then the two gestures are mapped to the high-dimensional space The Euclidean distance of the point will be very far; conversely, if the two poses belong to the same type as shown in Figure 3, the Euclidean distance of the points where the two poses are mapped to the high-dimensional space will be very close.

Operation 4.4 Obtain the Euclidean distance threshold T based on the standard gesture recognition model, and the threshold T is used to judge whether the user gesture and the standard gesture are of the same type;

If the Euclidean distance of the scoring result output by the standard gesture recognition model is greater than or equal to the threshold T, the user gesture is of the same type as the standard gesture; if the Euclidean distance of the scoring result output by the standard gesture recognition model is less than or equal to the threshold T, then the user gesture and The standard poses are of different types;

Operation 4.5 converts the Euclidean distance into the acquaintance score between the user pose and the standard pose, and obtains the user pose with the highest acquaintance score as the scoring result.

Specifically, if the Euclidean distance of the two poses exceeds the threshold T, they are considered not to be of the same type; otherwise, they are considered to be of the same model. For each threshold T, the ROC curve can be drawn, as shown in Figure 4, the area under the ROC curve, called AUC, is a value of 0-1, the larger the AUC, the better the model performance. Find an optimal threshold T-best that maximizes AUC on the test set. If the AUC is the largest, the model will judge as many poses that originally belonged to the same class as the same class as much as possible, and at the same time, it will try to misjudge two poses that do not belong to the same class as the same class as little as possible. After obtaining the optimal distance threshold T-best, we set a critical score according to actual business needs, such as 40 points, which means that at this time, the model believes that the two postures are just at the critical point of similarity and dissimilarity. Then the mapping relationship is as follows: when the actual distance t is in the interval [0, T-best], the similarity score s is [100, 40]; when the actual distance t is in (T-best, infinity), the similarity score s is (40 ,0). The threshold T is 40 in this embodiment.

Embodiment two

On the basis of Embodiment 1, the action evaluation method based on fitness teaching and training in the present disclosure will be introduced in conjunction with specific examples below: In the actual use process, the action evaluation method of the present disclosure uses a fitness training based on a fitness device The method is to carry out fitness training action demonstration, action teaching and normal follow-up to the user through three stages respectively. In this process, the user's action is recognized to judge whether the user has an action in the first stage, and the second In the third stage, whether the user's action is up to the standard, and the scoring result is fed back to the user. In addition, when the present disclosure performs identification and comparison on user actions, the identification and comparison is performed according to the first time period or the second time period corresponding to the preset first standard posture or the second standard posture, and it is not necessary to perform identification and comparison on the entire video. The effect is better, the result is obtained faster, the user's fitness effect is effectively guaranteed, and the user's fitness situation can be obtained at each stage, which improves the efficiency of fitness use and is more convenient for long-term use. Specifically include:

Operation 1 obtains the first body-building video for body-building, and in the present embodiment, the body-building device is a body-building mirror, and the first body-building video is played on the mirror surface of the body-building mirror;

Operation 2 Obtain the first user's posture according to the first fitness video, and determine whether the user is performing fitness training;

Operation 2.1 The user follows the first fitness video in the target fitness area of the fitness mirror;

Operation 2.2 Preset the first standard posture in the first fitness video;

Operation 2.3 acquires the second time period when the first standard posture appears in the first fitness video;

Operation 2.4 In the second time period, the fitness mirror recognizes the user's actions in the target fitness area, and performs feature extraction on the target fitness area. The feature extraction includes 16 bone key points, and the 16 bone key points correspond to a two-dimensional position Coordinates; 16 bone key points include top of head, bottom of head, neck, right shoulder, right elbow, right hand, left shoulder, left elbow, left hand, right hip, right knee, right foot, left hip, left knee, left foot, patella , get the first user pose;

Operation 2.5 If the first user’s posture obtained within the second time period has different actions, it means that the user is doing follow-up, that is, the user is performing fitness training; if the first user is not obtained within the second time period posture, or the first user posture does not produce any action, then the user does not follow up, that is, the user does not perform fitness training. In operation 2, during the second time period, the user's action is recognized, but no scoring is performed during this process.

Operation 3 obtains the posture of the second user according to the second fitness video, scores the posture of the second user, and judges whether the user follows the second fitness video for fitness training according to the scoring result;

Operation 3.1 Play the second fitness video on the mirror surface of the fitness mirror;

Operation 3.2 Preset the second standard posture according to the second fitness video;

Operation 3.3 Obtain the first time period when the second standard posture appears in the second fitness video;

Operation 3.4 In the first time period, obtain the video segment corresponding to the second fitness video in the first time period, perform frame processing on the video segment, and obtain frame images corresponding to several consecutive moments of the video segment;

Operation 3.5 In the first period of time, the fitness mirror recognizes the user's actions in the target fitness area, and performs feature extraction on the target fitness area. The feature extraction includes 16 bone key points, and the 16 bone key points correspond to a two-dimensional position Coordinates; 16 bone key points include top of head, bottom of head, neck, right shoulder, right elbow, right hand, left shoulder, left elbow, left hand, right hip, right knee, right foot, left hip, left knee, left foot, patella , to obtain a number of second user gestures corresponding to the frame images one-to-one;

Operation 3.6 compares corresponding several frame images and several second user poses, and obtains the acquaintance score of each second user pose based on the corresponding frame images;

Operation 3.61 train the twin neural network model to get the trained standard pose recognition model

Operation 3.62 Obtain the bone key points of the second standard pose and the second user pose and the position coordinates corresponding to each bone key point, wherein the second standard pose and the second user pose both include 16 bone key points and 16 bone key points The points correspond to a two-dimensional position coordinate; 16 bone key points include the top of the head, the bottom of the head, the neck, the right shoulder, the right elbow, the right hand, the left shoulder, the left elbow, the left hand, the right hip, the right knee, the right foot, the left hip, Left knee, left foot, and patella; input the position coordinates corresponding to each bone key point of the second standard pose and the second user pose into the trained standard pose recognition model, and obtain the output vector V1 of the second standard pose and the second user pose respectively. The output vector V2 of the attitude;

Calculate the Euclidean distance between the output vector V1 of the second standard posture and the output vector V2 of the second user posture;

Whether the second user gesture is of the same type as the second standard gesture is judged by the Euclidean distance between the output vector V1 of the second standard gesture and the output vector V2 of the second user gesture.

Operation 3.7 converts the Euclidean distance into the acquaintance score between the user posture and the standard posture, and obtains the user posture with the highest acquaintance score as the scoring result;

Operation 3.8 Obtain the second user posture with the highest acquaintance score as the scoring result; if the scoring result is greater than or equal to the scoring threshold, the second standard posture and the second user posture are of the same class, and the user performs fitness training; if the scoring result is less than or is equal to the scoring threshold, then the second standard posture and the second user posture are not in the same category, and the user does not perform fitness training. The Euclidean distance threshold T is obtained based on the standard pose recognition model, and the threshold T is used to judge whether the user pose and the standard pose are of the same type; if the Euclidean distance of the scoring result output by the standard pose recognition model is greater than or equal to the threshold T, the user pose and the standard pose are of the same type, if the Euclidean distance of the scoring result output by the standard gesture recognition model is less than or equal to the threshold T, then the user gesture is not of the same type as the standard gesture.

Operation 4 acquires the first user posture according to the first exercise video, scores the first user posture, and feeds back the scoring result to the user;

Operation 4.1 The user follows the first fitness video in the target fitness area of the fitness mirror;

Operation 4.2 Preset the first standard posture in the first fitness video;

Operation 4.3 acquires the second time period when the first standard posture appears in the first fitness video;

Operation 4.4 In the second time period, the fitness mirror recognizes the user's actions in the target fitness area, and performs feature extraction on the target fitness area. The feature extraction includes 16 bone key points, and the 16 bone key points correspond to a two-dimensional position Coordinates; 16 bone key points include top of head, bottom of head, neck, right shoulder, right elbow, right hand, left shoulder, left elbow, left hand, right hip, right knee, right foot, left hip, left knee, left foot, patella , get the first user pose;

Operation 4.5 compares the first standard posture with several first user postures, and obtains the acquaintance score of each first user posture based on the first standard posture;

Operation 4.51; Input the skeleton key points of the first standard posture and the first user posture and the position coordinates corresponding to each skeleton key point into the standard posture recognition model, and output the Euclidean distance between the first standard posture and the first user posture;

Operation 4.52 converts the Euclidean distance into an acquaintance score;

Operation 4.53 Obtain the first user gesture with the highest acquaintance score as the scoring result; if the scoring result is greater than or equal to the scoring threshold, the first standard gesture and the first user gesture are of the same category, and the user action meets the standard; if the scoring result is less than or equal to scoring threshold, the first standard gesture and the first user gesture are not in the same category, and the user action does not meet the standard. The Euclidean distance threshold T is obtained based on the standard pose recognition model, and the threshold T is used to judge whether the user pose and the standard pose are of the same type; if the Euclidean distance of the scoring result output by the standard pose recognition model is greater than or equal to the threshold T, the user pose and the standard pose are of the same type, if the Euclidean distance of the scoring result output by the standard gesture recognition model is less than or equal to the threshold T, then the user gesture is not of the same type as the standard gesture.

Operation 4.54 obtains the first user gesture with the highest acquaintance score as the scoring result, and feeds back the scoring result to the user;

If the scoring result is greater than or equal to the scoring threshold, the first standard gesture and the first user gesture are of the same category, and the user action is up to the standard; if the scoring result is less than or equal to the scoring threshold, the first standard gesture and the first user gesture are not the same class, the user action is not up to standard. In this embodiment, the scoring threshold is 40.

In this embodiment, if the first standard posture appears in the fitness video at 10000 ms. Now the fitness video is played to 10,000 milliseconds, because the user is practicing with the video, and his movements are behind or ahead of the course, so we will set an interval around 10,000ms, such as the first 800ms and the last 200ms, that is Time interval [10000-800, 10000+200] In this interval with a total duration of 1 second, the similarity between the first standard pose and the first user pose is calculated for each frame, and then the similarity in this interval is compared to the high The score is output as the final score.

In this embodiment, if the second standard posture appears in the fitness video at 10000 ms. Now the fitness video is played to 10,000 milliseconds, because the user is practicing with the video, and his movements are behind or ahead of the course, so we will set an interval around 10,000ms, such as the first 800ms and the last 200ms, that is Time interval [10000-800, 10000+200] In this interval with a total duration of 1 second, calculate the similarity between each frame and the first user pose, and then use the highest similarity score in this interval as the final Score output.

In specific calculation, both the first standard posture and the second standard posture are static postures, not a continuous movement. The specific method of comparison is to train a Siamese network structure model based on convolutional neural network, which accepts two poses and maps the two poses to a point in high-dimensional space. In this embodiment, the second exercise video may be played in slow motion from the exercise video played for the first time, or may be an action breakdown of the first exercise video.

Embodiment three

Please refer to FIG. 5. FIG. 5 is a schematic flowchart of a fitness training method based on a fitness device. The present disclosure provides a fitness training method based on a fitness device. The fitness training method is based on the above-mentioned action evaluation method based on fitness teaching and training. The fitness The training method can be implemented as an application method of the above-mentioned action evaluation method based on fitness teaching and training, and the fitness training method includes:

Get fitness first fitness video;

According to the posture of the first user, it is judged whether the user performs fitness training;

Presetting a second standard posture according to the second fitness video;

Contrast the second standard posture and the second user posture, obtain the acquaintance score of the second user posture based on the second standard posture, and judge whether the user follows the second fitness video for fitness training according to the scoring result;

Obtain the first fitness video for fitness, and preset the first standard posture according to the first fitness video;

Obtain the second user pose with the highest acquaintance score as the scoring result. Among them, the twin neural network model is trained to obtain the trained standard gesture recognition model;

Obtain the first user pose with the highest acquaintance score as the scoring result;

Inputting the first user pose and the first standard pose into the standard pose recognition model to obtain an acquaintance score;

If the scoring result is greater than or equal to the scoring threshold, the first standard gesture and the first user gesture are of the same category, and the user action is up to the standard;

If the scoring result is less than the scoring threshold, the second standard gesture and the second user gesture are not of the same category, and the user action does not meet the standard.

The fitness training method based on the fitness device in the present disclosure will be introduced in conjunction with specific examples below:

Operation 2.2 Preset the first standard posture in the first fitness video;

Operation 2.4 During the second period of time, the fitness mirror recognizes the user's actions in the target fitness area, extracts features from the target fitness area, and obtains the first user posture;

Operation 3.5 During the first period of time, the fitness mirror recognizes the user's actions in the target fitness area, performs feature extraction on the target fitness area, and obtains a number of second user postures corresponding to the frame images one-to-one;

Operation 3.61 trains the twin neural network model to obtain the trained standard gesture recognition model;

Operation 3.62 Input the second user pose and the second standard pose into the standard pose recognition model to obtain the acquaintance score;

Operation 3.7 Get the second user posture with the highest acquaintance score as the scoring result; if the scoring result is greater than or equal to the scoring threshold, the second standard posture and the second user posture are of the same class, and the user performs fitness training; if the scoring result is less than the scoring threshold threshold, the second standard posture and the second user posture are not of the same type, and the user does not perform fitness training for the user.

Operation 4.2 Preset the first standard posture in the first fitness video;

Operation 4.4 In the second time period, the fitness mirror recognizes the user's actions in the target fitness area, extracts features from the target fitness area, and obtains the first user posture;

Operation 4.51 Input the first user pose and the first standard pose into the standard pose recognition model to obtain an acquaintance score;

Operation 4.52 acquires the first user gesture with the highest acquaintance score as the scoring result;

If the scoring result is greater than or equal to the scoring threshold, then the first standard gesture and the first user gesture are of the same category, and the user action is up to the standard; if the scoring result is less than the scoring threshold, then the first standard gesture and the first user gesture are not of the same category, The user action is not up to standard.

In specific calculation, both the first standard posture and the second standard posture are static postures, not a continuous movement. The specific method of comparison is to train a Siamese network structure model based on convolutional neural network, which accepts two poses and maps the two poses to a point in high-dimensional space.

In this embodiment, the user posture and standard posture are input into the standard posture recognition model, and the specific method for obtaining the acquaintance score is as follows:

Obtain the bone key points of the standard pose and the user pose and the position coordinates corresponding to each bone key point, wherein the standard pose and the user pose both include 16 bone key points, and the 16 bone key points correspond to a two-dimensional position coordinate; 16 A skeleton key point includes the top of the head, the bottom of the head, the neck, the right shoulder, the right elbow, the right hand, the left shoulder, the left elbow, the left hand, the right hip, the right knee, the right foot, the left hip, the left knee, the left foot, and the patella;

The Euclidean distance threshold T is obtained based on the standard pose recognition model, and the threshold T is used to judge whether the user pose and the standard pose are of the same type; if the Euclidean distance of the scoring result output by the standard pose recognition model is greater than or equal to the threshold T, the user pose and the standard pose For the same type, if the Euclidean distance of the scoring result output by the standard gesture recognition model is less than or equal to the threshold T, then the user gesture and the standard gesture are not of the same type;

The Euclidean distance is converted into the acquaintance score between the user pose and the standard pose, and the user pose with the highest acquaintance score is obtained as the scoring result. Specifically, if the Euclidean distance of the two poses exceeds the threshold T, they are considered not to be of the same type; otherwise, they are considered to be of the same model. For each threshold T, the ROC curve can be drawn, as shown in Figure 4, the area under the ROC curve, called AUC, is a value of 0-1, the larger the AUC, the better the model performance. Find an optimal threshold T-best that maximizes AUC on the test set. If the AUC is the largest, the model will judge as many poses that originally belonged to the same class as the same class as much as possible, and at the same time, it will try to misjudge two poses that do not belong to the same class as the same class as little as possible. After obtaining the optimal distance threshold T-best, we set a critical score according to actual business needs, such as 40 points, which means that at this time, the model believes that the two postures are just at the critical point of similarity and dissimilarity. Then the mapping relationship is as follows: when the actual distance t is in the interval [0, T-best], the similarity score s is [100, 40]; when the actual distance t is in (T-best, infinity), the similarity score s is (40 ,0). The threshold T is 40 in this embodiment.

Both the standard pose and the user pose include 16 bone key points, each of which corresponds to a two-dimensional position coordinate. Among them, 16 bone key points include the top of the head, the bottom of the head, the neck, the right shoulder, the right elbow, the right hand, the left shoulder, the left elbow, the left hand, the right hip, the right knee, the right foot, the left hip, the left knee, the left foot, and the patella .

Embodiment four

Operation 1 is based on the third embodiment, and is further used in somatosensory games. For parkour games, we set a villain or animal that simulates the user. For this game, some villains that simulate the user will be set on the road Or the obstacles that small animals need to avoid, require small people or small animals to jump or lean left/right to avoid. Twisting the waist left/right corresponds to the left/right tilting of the villain or the animal, and the jumping of the user on the spot corresponds to the jumping of the villain or the animal. In addition, some other actions can be set, such as the user’s Raising the legs corresponds to the accelerated running of small people or small animals. The user follows the first fitness video in the target fitness area of the fitness mirror;

In the first fitness video, standing, twisting the waist left and right, jumping in place and raising the legs are the first standard postures;

Obtain the second time period when each first standard posture appears in the first fitness video;

In the second time period, the fitness mirror recognizes the user's actions in the target fitness area, extracts the features of the target fitness area, and obtains the first user posture corresponding to standing, twisting left and right, jumping in situ, and raising legs; The first user gesture acquired within the second time period has different actions, which means that the user is doing follow-up, that is, the user is performing fitness training; if the first user gesture is not acquired within the second time period, or If no action occurs in the first user gesture, the user does not follow suit, that is, the user does not perform fitness training. In the second time period, the user's actions are recognized, but scoring is not performed during this process.

In the second fitness video of operation 2, standing, twisting the waist left and right, jumping in situ and raising the legs are used as the second standard posture;

Obtain the first time period when each second standard posture appears in the second fitness video;

In the first period of time, the fitness mirror recognizes the user's actions in the target fitness area, extracts the features of the target fitness area, and obtains the second user posture corresponding to standing, twisting the waist left and right, jumping in situ, and raising the legs. On the basis of the third embodiment, each second user's posture is scored, and it is judged according to the scoring result whether the user follows the second fitness video for fitness training.

In the first fitness video of operation 3, standing, twisting the waist left and right, jumping in situ and raising the legs are taken as the first standard posture;

In the second time period, the fitness mirror recognizes the user's actions in the target fitness area, extracts the features of the target fitness area, and obtains the first user posture corresponding to standing, twisting left and right, jumping in situ, and raising legs; On the basis of Embodiment 3, each first user's standard posture is scored, and the scoring result is fed back to the user.

In this embodiment, since the fitness video is a game video, the movement of small people or small animals in the game video can be controlled through different standard gestures, so when the present disclosure recognizes the user's actions, the fitness training method of the present disclosure can not only It is used to identify actions and evaluate and score actions. At the same time, based on standard postures, user postures can also control the movement of small people or small animals in fitness videos. Standing is used to control animals to move forward, and twisting left and right is used to control small people or small animals. The animal leans its body to the left or right, jumps in place to control the animal to jump, and raises its legs to control the small person or small animal to run faster. Embodiment five

Please refer to FIG. 6. FIG. 6 may be a schematic diagram of the composition of an action evaluation system based on fitness teaching and training. Embodiment 5 of the present disclosure provides an action evaluation system based on fitness teaching and training. The action evaluation system includes:

In this embodiment, further, the comparison module compares several user postures and standard postures, specifically including training a twin neural network model to obtain a trained standard posture recognition model;

In this embodiment, further, the judging module inputs the user posture and the standard posture into the standard posture recognition model, and judges whether the user posture and the standard posture are of the same type, specifically including:

In this embodiment, further, the judgment module obtains the Euclidean distance threshold T based on the standard posture recognition model, and the threshold T is used to judge whether the user posture and the standard posture are of the same type, wherein, if the Euclidean distance output by the standard posture recognition model is less than or equal to the threshold T, the user pose is of the same type as the standard pose, and if the Euclidean distance output by the standard pose recognition model is greater than the threshold T, then the user pose is of a different type from the standard pose.

In this embodiment, further, the standard posture and the user posture both include 16 skeleton key points, and the 16 skeleton key points correspond to a two-dimensional position coordinate respectively, and the 16 skeleton key points include the top of the head, the bottom of the head, the neck, and the right shoulder , right elbow, right hand, left shoulder, left elbow, left hand, right hip, right knee, right foot, left hip, left knee, left foot, patella.

In this embodiment, further, the judging module inputs the user posture and the standard posture into the standard posture recognition model, outputs the acquaintance score between the user posture and the standard posture, obtains the user posture with the highest acquaintance score as the scoring result, and passes the scoring result Determine whether the user gesture is of the same type as the standard gesture.

Embodiment six

Embodiment 6 of the present disclosure provides a fitness training system based on a fitness device. The fitness training system in Embodiment 6 can be implemented as an application system of the above-mentioned action evaluation system. The fitness training system in Embodiment 6 executes:

Obtain the first user posture according to the first fitness video, score the first user posture, and feedback the scoring result to the user.

Embodiment seven

Please refer to FIG. 6. FIG. 6 can be a schematic diagram of the composition of a fitness training system based on a fitness device. Embodiment 7 of the present disclosure provides a fitness training system based on a fitness device. The fitness training system in Embodiment 7 can be implemented as the above-mentioned action evaluation system. Application system, the fitness training system includes:

The judging module is used to judge whether the user performs fitness training or whether the action meets the standard according to the comparison result.

Embodiment eight

Embodiment 8 of the present disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory and operable on the processor. When the processor executes the computer program, the An action evaluation method based on fitness teaching and training.

Wherein, the processor may be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.

The memory can be used to store the computer programs and/or modules, and the processor can realize various functions of the disclosed exercise evaluation device based on fitness teaching and training by running or executing the data stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, at least one application required by a function (such as a sound playback function, an image playback function, etc.) and the like. In addition, the memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disk, internal memory, plug-in hard disk, smart memory card, secure digital card, flash memory card, at least one magnetic disk storage device, flash memory device, or other volatile solid-state memory devices.

Embodiment nine

Embodiment 9 of the present disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory and operable on the processor. When the processor executes the computer program, the A fitness training method based on a fitness device.

The memory can be used to store the computer programs and/or modules, and the processor can realize various functions of the fitness training device based on the fitness device in the disclosure by running or executing the data stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, at least one application required by a function (such as a sound playback function, an image playback function, etc.) and the like. In addition, the memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disk, internal memory, plug-in hard disk, smart memory card, secure digital card, flash memory card, at least one magnetic disk storage device, flash memory device, or other volatile solid-state memory devices.

Embodiment ten

Embodiment 10 of the present disclosure provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the exercise evaluation method based on fitness teaching and training is realized.

The computer storage medium in the embodiments of the present disclosure may use any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more conductors, portable computer disks, hard disks, random access memory (RAM), read-only memory (ReadOnlyMemory, ROM ), erasable programmable read-only memory ((ErasableProgrammableReadOnlyMemory, EPROM) or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

Embodiment Eleven

Embodiment 11 of the present disclosure provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the fitness training method based on a fitness device is implemented.

While preferred embodiments of the present disclosure have been described, additional changes and modifications can be made to these embodiments by those skilled in the art once the basic inventive concept is appreciated. Therefore, it is intended that the appended claims be construed to cover the preferred embodiment and all changes and modifications which fall within the scope of the present disclosure.

It is obvious that those skilled in the art can make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and equivalent technologies thereof, the present disclosure also intends to include these modifications and variations.

Claims

Action evaluation methods based on fitness teaching and training, including:

Obtain fitness videos, in which standard postures are preset;

Obtain the time period corresponding to the standard posture in the fitness video;

Acquire frame images of continuous moments corresponding to the fitness video during the time period;

Identify several user gestures corresponding to frame images at consecutive moments in the time period;

Compare several user postures with standard postures, and determine whether the user postures and standard postures are of the same type.
The action evaluation method based on fitness teaching and training according to claim 1, wherein comparing several user postures and standard postures to determine whether the user postures and standard postures are of the same type specifically includes:

Train the twin neural network model to get the trained standard gesture recognition model;

Input the user pose and the standard pose into the standard pose recognition model, and judge whether the user pose and the standard pose are of the same type.
The action evaluation method based on fitness teaching and training according to claim 2, wherein the user posture and the standard posture are input into the standard posture recognition model, and it is judged whether the user posture and the standard posture are of the same type, specifically comprising:

Get the bone key points of the standard pose and user pose and the position coordinates corresponding to each bone key point;

Input the position coordinates corresponding to each bone key point of the standard pose and the user pose into the trained standard pose recognition model, and obtain the output vector V1 of the standard pose and the output vector V2 of the user pose respectively;

Calculate the Euclidean distance between the output vector V1 of the standard posture and the output vector V2 of the user posture;

It is judged whether the user posture and the standard posture are of the same type by the Euclidean distance between the output vector V1 of the standard posture and the output vector V2 of the user posture.
The action evaluation method based on fitness teaching training according to claim 3, wherein, based on the standard gesture recognition model, the Euclidean distance threshold T is obtained, and the threshold T is used to judge whether the user gesture and the standard gesture are of the same type, wherein, if the standard gesture recognition If the Euclidean distance output by the model is less than or equal to the threshold T, the user pose is of the same type as the standard pose. If the Euclidean distance output by the standard pose recognition model is greater than the threshold T, the user pose is of a different type from the standard pose.
The action evaluation method based on fitness teaching training according to claim 3, wherein both the standard posture and the user posture include 16 key points of bones, the 16 key points of bones respectively correspond to a two-dimensional position coordinate, and the key points of bones include Top of head, bottom of head, neck, right shoulder, right elbow, right hand, left shoulder, left elbow, left hand, right hip, right knee, right foot, left hip, left knee, left foot, patella.
The action evaluation method based on fitness teaching training according to claim 3, wherein the user posture and standard posture are input into the standard posture recognition model, the acquaintance score of the user posture and the standard posture is output, and the user posture with the highest acquaintance score is obtained as Scoring results, by which it is judged whether the user pose is of the same type as the standard pose.
Based on the fitness training method of the fitness device, the fitness training method is based on the action evaluation method based on the fitness teaching training described in claim 1, and the fitness training method comprises:

Obtaining the first user's posture according to the first fitness video, and judging whether the user is performing fitness training;

Acquiring the second user's posture according to the second fitness video, scoring the second user's posture, and judging whether the user follows the second fitness video for fitness training according to the scoring result;

The first user posture is acquired according to the first fitness video, the first user posture is scored, and the scoring result is fed back to the user.
The fitness training method based on a fitness device according to claim 7, wherein the first user posture is acquired according to the first fitness video, and it is judged whether the user is performing fitness training, specifically comprising:

Get the first fitness video;

Identifying the target fitness area of the first fitness video, performing feature extraction on the target fitness area, and obtaining the first user posture;

According to the first user posture, it is judged whether the user performs fitness training.
The fitness training method based on a fitness device according to claim 7, wherein the second user posture is obtained according to the second fitness video, the second user posture is scored, and whether the user follows the second fitness video for fitness training is judged according to the scoring result , including:

Presetting a second standard posture according to the second fitness video;

Identifying the target fitness area of the second fitness video, performing feature extraction on the target fitness area, and obtaining the second user posture;

Comparing the second standard posture and the second user posture, the acquaintance score of the second user posture based on the second standard posture is obtained, and judging whether the user follows the second fitness video for fitness training according to the scoring result.
The fitness training method based on a fitness device according to claim 9, wherein the target fitness area of the second fitness video is identified, feature extraction is performed on the target fitness area, and the second user posture is obtained, which specifically includes:

Obtain the first time period when the second standard posture appears in the second fitness video;

Obtain the video segment corresponding to the second fitness video in the first time period, and process the video segment into frames to obtain frame images corresponding to several consecutive moments of the video segment;

Identifying the target fitness area of the second fitness video in the first time period, performing feature extraction on the target fitness area, and obtaining a plurality of second user postures corresponding to the frame images one-to-one;

Comparing corresponding several frame images and several second user postures, obtaining the acquaintance score of each second user posture based on the corresponding frame images;

Obtain the second user pose with the highest acquaintance score as the scoring result.
The fitness training method based on a fitness device according to claim 7, wherein the first user posture is obtained according to the first fitness video, and the first user posture is scored, specifically comprising:

Obtain the first fitness video for fitness, and preset the first standard posture according to the first fitness video;

Identifying the target fitness area of the first fitness video, performing feature extraction on the target fitness area, and obtaining the first user posture;

Comparing the first standard posture with the first user posture, obtaining the acquaintance score of the first user posture based on the first standard posture, and feeding back the scoring result to the user.
The fitness training method based on a fitness device according to claim 11, wherein identifying the target fitness area of the first fitness video, performing feature extraction on the target fitness area, and obtaining the first user posture, specifically comprising:

Obtain the second time period when the first standard posture appears in the first fitness video;

Identifying the target fitness area of the first fitness video in the second time period, performing feature extraction on the target fitness area, and obtaining several first user postures;

Comparing the first standard posture and several first user postures, obtaining the acquaintance score of each first user posture based on the first standard posture;

Obtain the first user pose with the highest acquaintance score as the scoring result.
The fitness training method based on a fitness device according to claim 11 or 12, wherein comparing the first standard posture and the first user posture specifically includes:

Train the twin neural network model to get the trained standard gesture recognition model;

Inputting the first user pose and the first standard pose into the standard pose recognition model to obtain an acquaintance score;

If the scoring result is greater than or equal to the scoring threshold, the first standard gesture and the first user gesture are of the same category, and the user action is up to the standard;

If the scoring result is less than the scoring threshold, the first standard gesture and the first user gesture are not of the same category, and the user action does not meet the standard.
Action evaluation system based on fitness teaching and training, including:

The obtaining module is used to obtain the fitness video, and obtain the time period corresponding to the preset standard posture in the fitness video, and obtain the frame images of the continuous moments corresponding to the fitness video in the time period;

An identification module, configured to identify several user gestures corresponding to the frame images at consecutive moments in the above time period;

A comparison module is used to compare several user postures and standard postures to obtain comparison results;

The judging module is used to judge whether the user posture and the standard posture are of the same type according to the comparison result.
The action evaluation system based on fitness teaching training according to claim 14, wherein the comparison module compares several user postures with standard postures, specifically including training a twin neural network model to obtain a trained standard posture recognition model;

The judging module judges whether the user posture and the standard posture are of the same type according to the comparison result, specifically including inputting the user posture and the standard posture into the standard posture recognition model, and judging whether the user posture and the standard posture are of the same type.
The action evaluation system based on fitness teaching and training according to claim 15, wherein the judgment module inputs user posture and standard posture into the standard posture recognition model, and judges whether the user posture and standard posture are of the same type, specifically comprising:

Get the bone key points of the standard pose and user pose and the position coordinates corresponding to each bone key point;

Input the position coordinates corresponding to each bone key point of the standard pose and the user pose into the trained standard pose recognition model, and obtain the output vector V1 of the standard pose and the output vector V2 of the user pose respectively;

Calculate the Euclidean distance between the output vector V1 of the standard posture and the output vector V2 of the user posture;

It is judged whether the user posture and the standard posture are of the same type by the Euclidean distance between the output vector V1 of the standard posture and the output vector V2 of the user posture.
The action evaluation system based on fitness teaching and training according to claim 16, wherein the judging module obtains the Euclidean distance threshold T based on the standard posture recognition model, and the threshold T is used to judge whether the user posture is of the same type as the standard posture, wherein, If the Euclidean distance output by the standard pose recognition model is less than or equal to the threshold T, the user pose is of the same type as the standard pose; if the Euclidean distance output by the standard pose recognition model is greater than the threshold T, then the user pose and the standard pose are of a different type.
The action evaluation system based on fitness teaching training according to claim 16, wherein both the standard posture and the user posture include 16 key points of bones, the 16 key points of bones respectively correspond to a two-dimensional position coordinate, and the key points of bones include Top of head, bottom of head, neck, right shoulder, right elbow, right hand, left shoulder, left elbow, left hand, right hip, right knee, right foot, left hip, left knee, left foot, patella.
The action evaluation system based on fitness teaching and training according to claim 16, wherein the judging module inputs the user posture and the standard posture into the standard posture recognition model, outputs the acquaintance score between the user posture and the standard posture, and obtains the highest acquaintance score The user pose is used as the scoring result, and the scoring result is used to judge whether the user pose is of the same type as the standard pose.
A fitness training system based on a fitness device, the fitness training system is based on the action evaluation system based on fitness teaching and training according to claim 14, and the fitness training system executes:

Obtaining the first user's posture according to the first fitness video, and judging whether the user is performing fitness training;

Acquiring the second user's posture according to the second fitness video, scoring the second user's posture, and judging whether the user follows the second fitness video for fitness training according to the scoring result;

The first user posture is acquired according to the first fitness video, the first user posture is scored, and the scoring result is fed back to the user.