KR102617467B1 - Home training system using artificial intelligence - Google Patents

Abstract

A system that provides home training through analysis of a user's exercise movements using artificial intelligence, comprising: a joint measurement unit that measures at least one joint of the user's body; A motion classification unit that uses an artificial intelligence algorithm to recognize the user's exercise movement from the data generated by the joint measurement unit and classifies it as a preset exercise; And it provides a home training system using artificial intelligence, including; a matching rate calculation unit that calculates the motion matching rate by comparing the unit motion of the exercise counted as 1 time with a standard motion.

Description

Home training system using artificial intelligence {HOME TRAINING SYSTEM USING ARTIFICIAL INTELLIGENCE}

The present invention relates to a home training system using artificial intelligence that recognizes and analyzes the user's exercise movements using a camera, etc., and provides feedback on them.

As use of gyms and sports facilities is restricted due to the impact of COVID-19, the number of 'home workout people' (people who exercise at home) who do personal exercise and home training is on the rise. Initially, people started home training to lose weight due to COVID-19, but now more people are doing home training for their health, and home training is becoming a new exercise culture. Due to this, at the same time, the smart healthcare industry is also in the spotlight, and related content is currently being developed.

However, home training has the problem that repetition of incorrect movements without expert knowledge of exercise can cause strain on the joints and ultimately lead to injury. In addition, the method of imitating this through conventional videos was limited to the cross-sectional appearance of the trainer and had the problem of making it difficult to provide feedback on individual posture.

Korean Patent Publication 102031243 (2019.10.04.) Smart free weight training motion recognition device and method thereof (apparatus for recognizing free weight training motion and method thereof) Korean Patent Publication 1020170119333 (2017.10.27) Apparatus and Method for Processing of Exercise Information Korean Patent Publication 102039616 (2019.10.28) APPARATUS AND METHOD FOR MANAGING EXERCISE INFORMATION

Embodiments of the present invention were designed to solve the above problems, and are intended to provide a home training system using artificial intelligence that can maximize the exercise effect of home training without injuries or safety accidents.

We want to accurately recognize the user's exercise movements and calculate the movement agreement rate compared to standard movements. We aim to visualize and provide exercise data including the user's number of exercise times, exercise time, and motion consistency rate.

In addition, we aim to further improve user convenience for this system by developing a dedicated application that is installed on the user's smart device. In addition, we want to accurately identify the user's body and provide feedback on incorrect posture during exercise.

In addition, it facilitates the connection between each component of the system, making maintenance and management convenient.

An embodiment of the present invention is a system that provides home training through analysis of the user's exercise movements using artificial intelligence to solve the above problems, and is a joint measurement system that measures at least one joint of the user's body. wealth; A motion classification unit that uses an artificial intelligence algorithm to recognize the user's exercise movement from the data generated by the joint measurement unit and classifies it as a preset exercise; And it provides a home training system using artificial intelligence, including; a matching rate calculation unit that calculates the motion matching rate by comparing the unit motion of the exercise counted as 1 time with a standard motion.

The joint measuring unit includes a first measuring unit disposed in front of the user and a second measuring unit disposed on one side of the user, and according to the movement classified by the motion classification unit, the first measuring unit and the first measuring unit It is desirable to vary the weight of the data by the two measurement units.

The joint measurement unit preferably measures the spatial position of the joint.

The coincidence rate calculation unit calculates the motion coincidence rate using a first feature group that is a set of angle data formed between a plurality of preset joints and a second feature group that is a set of distance data formed between the joints. It is desirable to do so.

A standard movement consists of a plurality of standard postures, and the standard posture is preferably determined to be a combination of at least one of the first feature group and at least one of the second feature group.

It is preferable to further include a display unit that outputs a statistical graph about the user's number of exercise times, exercise time, and motion consistency rate.

It further includes a sensor unit that is worn on both wrists of the user and measures the amount of change in the angle of the elbow over time, and the exercise preferably includes at least a push-up.

According to the problem solving means of the present invention as discussed above, various effects including the following can be expected. However, the present invention does not require all of the following effects to be achieved.

A home training system using artificial intelligence according to an embodiment of the present invention can maximize the exercise effect of home training without injuries or safety accidents. In addition, it is possible to accurately recognize the user's exercise motion and calculate the motion matching rate compared to the standard motion. Additionally, exercise data including the user's number of exercises, exercise time, and motion consistency rate can be visualized and provided.

Additionally, user convenience can be further improved through a dedicated application installed on the user's smart device. Additionally, it can accurately identify the user's body and provide feedback on incorrect posture during exercise. In addition, the connection relationship between each component of the system can be facilitated, making maintenance and management convenient.

1 is a block diagram of a home training system using artificial intelligence according to an embodiment.
Figure 2 is a conceptual diagram showing an articulation joint for the body of Figure 1.
Figure 3 is a diagram showing angle data and distance data formed between joints at three points arbitrarily set in Figure 2.
Figure 4 is an image of deep learning being executed using an artificial intelligence algorithm to implement exercise movements in the system of Figure 1.
Figure 5 is an example image from various angles for a data set from the system of Figure 1.
Figure 6 is an example diagram showing the motion coincidence rate for unit movements of exercise through the system of Figure 1.
Figure 7 is an example diagram showing a statistical graph related to the system of Figure 1.

In order to fully understand the configuration and effects of the present disclosure, preferred embodiments of the present disclosure will be described with reference to the attached drawings. However, the present disclosure is not limited to the embodiments disclosed below, and may be implemented in various forms and various changes may be made. Hereinafter, in describing the present invention, if it is determined that related known functions may unnecessarily obscure the gist of the present invention as they are obvious to those skilled in the art, the detailed description thereof will be omitted.

Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The above terms may be used only for the purpose of distinguishing one component from another. For example, a first component may be referred to as a second component, and similarly, the second component may be referred to as a first component without departing from the scope of the present disclosure.

In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

The terms used in this application are merely used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. Unless otherwise defined, terms used in the embodiments of the present disclosure may be interpreted as meanings commonly known to those skilled in the art.

Hereinafter, specific embodiments of the present invention will be described in detail with reference to the drawings. Figure 1 is a block diagram of a home training system using artificial intelligence according to an embodiment, Figure 2 is a conceptual diagram showing a joint joint for the body of Figure 1, and Figure 3 is a joint joint at three points arbitrarily set in Figure 2. This is a diagram showing the angle data and distance data formed between them.

Figure 4 is an image of deep learning being executed using an artificial intelligence algorithm to implement exercise movements in the system of Figure 1, and Figure 5 is an example image from various angles for the data set in the system of Figure 1. 6 is an example diagram showing the motion matching rate for unit movements of exercise through the system of FIG. 1, and FIG. 7 is an example diagram showing a statistical graph related to the system of FIG. 1.

1 to 7, a home training system using artificial intelligence according to an embodiment of the present invention includes a joint measurement unit 100, a motion classification unit 1100, a coincidence rate calculation unit 1200, and a communication module 300. ), a display unit 400, a memory unit 500, a sensor unit 130, an external server 600, etc. In this system, the joint measurement unit 100, motion classification unit 1100, coincidence rate calculation unit 1200, communication module 300, display unit 400, and memory unit 500 are to be placed inside the user's smart device. You can.

This system improves the system that provides home training by analyzing the user's exercise movements using artificial intelligence. For this purpose, this system uses a conventional deep learning artificial intelligence algorithm. Deep learning uses multiple data sets to estimate joints in the body and calculates motion accuracy, or motion matching rate, through motion classification for exercise posture.

To this end, in this system, CNN is first used to predict the positions of joints and learn the correlation between joints. This means recognizing the user first and extracting key features from the body. In addition, KNN analyzes joints predicted by CNN and then classifies movements through a machine learning model.

The joint measurement unit 100 measures at least one joint of the user's body. For example, the joint measurement unit 100 may be a camera including a 3D depth sensor. The joint measurement unit 100 may be an RGB camera in which one frame data consists of RGB values. The joint measuring unit 100 may be a measuring device that includes at least one sensor that measures the spatial position of the joint and generates a signal. The joint measurement unit 100 recognizes joints in the user's body. At this time, it is desirable for the joint measurement unit 100 to set the camera's angle of view to the maximum value in order to improve the accuracy of posture detection.

The joint measurement unit 100 may include a first measurement unit 110 disposed in front of the user and a second measurement unit 120 disposed on one side of the user. The first measurement unit may be a camera module already formed by the manufacturer in the user's smart device. In contrast, the first measurement unit may be a sensor module formed within a smart device. The second measurement unit may be an RGB camera that is separate and independent from the first measurement unit. In contrast, the second measuring unit may be a measuring device including a sensor that measures the spatial position of the joint.

Through this, the joint measurement unit 100 can measure and recognize the user's body more accurately. The joint measuring unit 100 measures the position of the joint through the second measuring unit when it is difficult to determine the posture from the front during exercise, for example, lunge or squat, or when part of the user's body is obscured during the exercise. can be measured.

At this time, work to calibrate the data measured in the first measurement unit and the second measurement unit is first performed. This can use the spatial position of the user's joints measured while the user moves freely for a certain period of time.

Meanwhile, in this system, it is desirable to vary the weight of the data by the first measurement unit and the second measurement unit according to the movement classified by the motion classification unit 1100. Depending on the exercise classification, for example, the proportion of data from the first measurement unit may be set higher than that from the second measurement unit. Depending on the exercise, the usability of data depending on the measurement location may be evaluated differently. Additionally, depending on the measurement location, measurement of some of the joints may be omitted during the exercise process. As a result, depending on the exercise classification, for example, the weight of the data may be changed in the form of K% for the first measurement part and (100-K)% for the second measurement part.

Additionally, the joint measurement unit 100 can measure the spatial position of the joint. Here, the joint measurement unit 100 can measure in the form of three-dimensional space coordinates.

The analysis module 1000 can use artificial intelligence as a statistical model to recognize the user's exercise movements, classify them, and calculate the movement consistency rate for standard movements. Artificial intelligence learns knowledge based on data and outputs results, and the analysis module 1000 can calculate the motion matching rate based on the input user's gender, body information, exercise type, and exercise motion. The analysis module 1000 can convert data measured through the joint measurement unit 100 into digital data through an AD converter. The analysis module 1000 includes an operation classification unit 1100 and a coincidence rate calculation unit 1200, and is electrically connected to each of them.

The motion classification unit 1100 uses an artificial intelligence algorithm to recognize the user's exercise movement from the data generated by the joint measurement unit 100 and classifies it as one of the preset exercises. This system can provide a high recognition rate for exercise movements in any direction by learning exercise movements through data sets from various angles. Here, exercises include push-ups, squats, lunges, etc. included in home training. The classification technology of the motion classification unit 1100 uses a general artificial intelligence algorithm, and detailed descriptions will be omitted below.

The coincidence rate calculation unit 1200 calculates the motion coincidence rate by comparing the unit movement of the exercise counted as 1 time with the standard movement. The coincidence rate calculation unit 1200 calculates the motion coincidence rate using the first feature group, which is a set of angle data formed between a plurality of preset joints, and the second feature group, which is a set of distance data formed between the joints. It is desirable to do so.

Additionally, the system may include at least one or more standard movements depending on the exercise being classified. And, the standard operation may be performed in a plurality of standard postures. Additionally, the standard posture may be determined to be a combination of at least one of the first feature group and at least one of the second feature group.

The motion consistency rate can be calculated using the joint joint measured through the joint measurement unit 100. Depending on the exercise, standard movements and the standard postures that make up them are preset. Meanwhile, the motion matching rate for the standard motion may be, for example, an average value of the motion matching rate for each standard posture.

The standard posture may be composed of the same number of correct posture images and incorrect posture images, which are comprised of at least one pre-input image. Here, the correct posture image and the incorrect posture image are images of the same body part that correspond to each other. As such, the correct posture images and incorrect posture images may be limited to some parts of the body. Meanwhile, the weights of the correct posture image and the incorrect posture image may be set differently in the process of determining whether the image is a standard posture depending on the image.

The motion consistency rate between the user's exercise posture and the standard posture can be calculated using data on the joint joint measured by the joint measurement unit 100. At this time, the probability of matching the static posture can be calculated by comparing the exercise posture and the static posture image. Here, the motion matching rate may be the probability of straight posture matching. Additionally, the probability of incorrect posture matching can be calculated by comparing the same exercise posture with the incorrect posture image. Meanwhile, unlike the correct posture image, the incorrect posture image may be composed of multiple incorrect posture images for the same exercise posture. As a result, multiple incorrect posture matching probabilities can be calculated.

Specifically, the coincidence rate calculation unit 1200 calculates the motion coincidence rate using the first feature group, which is a set of angle data formed between a plurality of preset joints, and the second feature group, which is a set of distance data formed between the joints. can be calculated. For example, the articulation joint may be composed of 25 pieces. At this time, the joints are assigned numbers from 1 to 25 and can be distinguished through these numbers.

Here, the angle data means the angle (a) formed between the three joints (A1, A2, A3) with respect to the middle joint (A2) when they are connected to each other in a straight line. For example, the angle data may be the angle (a) formed by the user's upper and lower arms based on the user's elbow. Here, the joint can be expressed as information in the form of three-dimensional space coordinates (x, y, z). For example, the angle data may be the angle formed between the thigh and the calf based on the left knee. That is, the angle data can be set in advance according to the number of joints. Each of these angle data becomes the first feature group that constitutes the exercise posture.

Meanwhile, distance data refers to the distance (d) formed between joints. For example, in space, it may be a straight line distance between the articulating joint (A1) and the articulating joint (A3). These first feature groups and second feature groups can be specified in advance according to the motor actions to be classified. In other words, standard movements are included according to the exercise movements, and the standard postures that make up the standard movements can be divided into the first feature group and the second feature group.

The motion matching rate can be calculated using the angle matching rate based on the angle data constituting the first feature group related to the standard posture and the distance matching rate based on the distance data constituting the second feature group. In one standard posture, the first feature group may include n angle data corresponding to n features. At this time, the angle coincidence rate can be calculated as the arithmetic average of the first angle coincidence rate to the nth angle coincidence rate.

The angle coincidence rate can be calculated using a, which is a reference range value for angle data, b, a reference value for angle data preset in the present system, and c, which is angle data actually measured by the joint measurement unit 100. For example, the first angle coincidence rate is calculated using a1, which is a reference range value for angle data, b1, a reference value for angle data preset in this system, and c1, which is angle data actually measured by the joint measurement unit 100. It is calculated.

In this system, the angle coincidence rate can be calculated, for example, as follows. The angle coincidence rate can be calculated using k ^{(bc)/a} (where k is a constant involved in the angle coincidence rate, 0.8), that is, using an exponential function. Therefore, the first angle coincidence rate is k ^{(b1-c1)/a1} . Using this, the angle coincidence rate can be calculated as follows: {(first angle coincidence rate + second angle coincidence rate + ... + nth angle coincidence rate) / n}.

The distance coincidence rate can be calculated in the same way. In one standard posture, the second feature group may include m pieces of distance data corresponding to m features. At this time, the distance matching rate can be calculated as the arithmetic average of the first to mth distance matching rates. The distance coincidence rate can be calculated using d, which is a reference range value for distance data, f, a reference value for distance data preset in this system, and e, which is distance data actually measured by the joint measurement unit 100. For example, the first distance coincidence rate is calculated using d1, which is a reference range value for distance data, f1, a reference value for distance data preset in this system, and e1, which is distance data actually measured by the joint measurement unit 100. It is calculated.

The distance coincidence rate can be calculated using g ^{(fe)/d} (where g is a constant involved in the distance coincidence rate, 0.9), that is, using an exponential function. Therefore, the first distance coincidence rate is g ^{(f1-e1)/d1} . Using this, the distance matching rate can be calculated as follows: {(1st distance matching rate + 2nd distance matching rate + ... + mth distance matching rate) / m}.

Meanwhile, the display unit 400 outputs statistical graphs about the user's number of exercise times, exercise time, and motion matching rate. Here, the display unit 400 may be, for example, an output unit of the user's smart device. The display unit 400 may visually provide the number of exercises and exercise times according to the date in the form of a bar graph, for example. Additionally, the motion matching rate for each exercise motion can be expressed as a numerical value. In this system, a dedicated application that works with this system can be installed on the user's smart device. The dedicated application can statistically provide the user’s exercise data through intuitive icon placement.

This system may further include a sensor unit 130. The sensor unit 130 is worn on both wrists of the user and measures the amount of change in the angle of the elbow over time. For example, the sensor unit 130 may be formed in a wearable device that can be worn by a user. This amount of angle change can be used as auxiliary data in calculating the motion consistency rate. In other words, the sensor unit 130 can improve the accuracy of the motion matching rate for standard motion through the amount of change in the angle of the elbow. At this time, it is desirable for the exercise to include at least push-ups.

The sensor unit 130 may be, for example, a three-axis (X-axis, Y-axis, and Z-axis) acceleration sensor provided on a board with a Bluetooth 4.0 communication chip built into it. Additionally, the sensor unit 130 may be a MEMS-based 9-axis acceleration sensor consisting of a 3-axis acceleration sensor, a 3-axis gyro sensor, and a 3-axis geomagnetic sensor. The sensor unit 130 can periodically transmit and receive data with the user's smart device at regular time intervals.

The external server 600 communicates with the smart device. This external server 600 stores data defining standard movements, standard postures, etc. Meanwhile, the external server 600 can communicate with a number of different smart devices. The smart device can transmit and receive data with the external server 600 while the dedicated application is running.

Additionally, if the motion consistency rate for the user's exercise posture is less than 0.5, the system can provide the user with various contents related to standard motion and standard posture. This system can maximize exercise efficiency by providing feedback on the standard posture (e.g., raise your arms, bend your knees more, etc.) when the incorrect posture matching probability is 0.8 or more.

The communication module 300 transmits and receives certain information, data, etc. through at least one preset external server 600 and a wired or wireless network. Additionally, the communication module 300 can receive data measured through the sensor unit 130. The communication module 300 can transmit and receive wireless signals to a base station, etc. on a mobile communication network established according to a technical standard or communication method for mobile communication.

The memory unit 500 can store various data related to algorithms for operating the system, raw data measured through the joint measurement unit 100, user interface-related data, etc. As such, the memory unit 500 may include at least one storage medium among flash memory, card-type memory, and RAM. The memory unit 500 may store data regarding the number of exercises, exercise time, and motion matching rate of each user.

As described above, the present invention is described with reference to the illustrated embodiments, but these are merely illustrative examples, and those of ordinary skill in the art to which the present invention pertains can make various modifications without departing from the gist and scope of the present invention. It will be apparent that variations, modifications, and equivalent other embodiments are possible. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the attached claims.

100: Joint measurement unit 1000: Analysis module
1100: Motion classification unit 1200: Match rate calculation unit
300: Communication module 400: Display unit
500: memory unit 130: sensor unit
600: External server

Claims (7)

A system that provides home training through analysis of the user's exercise movements using artificial intelligence,
A joint measurement unit that measures at least one joint of the user's body;
A motion classification unit that uses an artificial intelligence algorithm to recognize the user's exercise movement from the data generated by the joint measurement unit and classifies it as a preset exercise; and
It includes a coincidence rate calculation unit that calculates the motion coincidence rate by comparing the unit motion of the exercise counted as 1 time with the standard motion,
The coincidence rate calculation unit calculates the motion coincidence rate using a first feature group that is a set of angle data formed between a plurality of preset joints and a second feature group that is a set of distance data formed between the joints. do,
The motion matching rate is calculated using an angle matching rate based on the angle data constituting the first feature group and a distance matching rate based on the distance data constituting the second feature group,
The angle consistency rate is calculated using a, which is a reference range value for angle data, b, a reference value for angle data preset in this system, and c, which is angle data actually measured by the joint measurement unit,
The angle coincidence rate is calculated using k ^{(bc)/a} , (k = 0.8),
The distance coincidence rate is calculated using d, which is a reference range value for distance data, f, a reference value for distance data preset in this system, and e, which is distance data actually measured by the joint measurement unit,
The distance coincidence rate is calculated using g ^{(fe)/d} , (g = 0.9),
The standard movement consists of a plurality of standard postures, and the standard posture is determined to be a combination of at least one of the first feature group and at least one of the second feature group,
The standard posture consists of the same number of correct posture images and incorrect posture images, which are composed of at least one pre-input image,
Home using artificial intelligence, which compares the user's exercise posture with the incorrect posture image to calculate the incorrect posture matching probability, and provides feedback on the standard posture to the user if the incorrect posture matching probability is 0.8 or more. Training system.
According to paragraph 1,
The joint measuring unit includes a first measuring unit disposed in front of the user and a second measuring unit disposed on one side of the user,
A home training system using artificial intelligence that varies the weight of the data by the first measurement unit and the second measurement unit according to the exercise classified by the motion classification unit.
According to paragraph 1,
The joint measurement unit is a home training system using artificial intelligence that measures the spatial position of the joint joint.
delete delete According to paragraph 1,
A home training system using artificial intelligence, further comprising a display unit that outputs a statistical graph about the user's number of exercises, exercise time, and motion consistency rate.
According to paragraph 1,
It further includes a sensor unit that is worn on both wrists of the user and measures the amount of change in the angle of the elbow over time,
The exercise is a home training system using artificial intelligence that includes at least push-ups.