KR20200109199A - Calculation system of momentum using camera for indoor sports - Google Patents

Abstract

The present invention relates to an exercise amount calculation system for indoor sports using a camera and, more specifically, to an exercise amount calculation system for calculating the exercise amount of a user who joins in indoor sports. According to the present invention, the exercise amount calculation system comprises a camera installed in a space for indoor sports in which a user is joined to recognize the user′s motion and an exercise amount calculation module calculating an exercise amount from the user′s motion recognized by the camera. Accordingly, the system includes the camera installed in the space for indoor sports in which the user is joined to recognize the user′s motion and the exercise amount calculation module for calculating the exercise amount from the user′s motion recognized by the camera, such that the exercise amount of the user who is joined in the indoor sports is measured without a separate wearable device, thereby maximizing user convenience and monitoring and verifying exercise effect by the indoor sports of each user. Moreover, the exercise amount is calculated in consideration of physical information such as the user′s height and weight, and the like, and thus the exercise amount in accordance with physical characteristics of a child/teenager can be calculated. The user′s exercise amount is estimated based on deep learning by using a previously trained exercise amount estimation model and the user′s motion recognized by the camera, and thus correctness of the calculated exercise amount can be increased.

Description

A system for calculating momentum using a camera for indoor sports {CALCULATION SYSTEM OF MOMENTUM USING CAMERA FOR INDOOR SPORTS}

The present invention relates to an exercise amount calculation system, and more specifically, to an exercise amount calculation system using a camera for indoor sports.

As interest in health increases, portable biometric information measuring devices have been developed and widely distributed. In particular, techniques for measuring various biometric information, rather than measuring the number of simple steps such as a conventional pedometer, are being developed, and information that can be measured through a small measuring device is gradually becoming diverse.

1 is a diagram illustrating a small measuring device for measuring biometric information as an example. As shown in FIG. 1, a form worn on a wrist in the form of a band is most widely used, and may be a form such as a small disc or square plate inserted into a sports vest. Such a biometric information measuring device is in the form of a wearable device worn on the body, and may collect various information such as heart rate, pulse rate, movement distance, movement trajectory, including a heart rate sensor, an acceleration sensor, and a pulse sensor.

On the other hand, modern people have more time to sit in a chair, and in their leisure time, they often sit when they enjoy leisure such as playing games or watching movies using a mobile terminal or a computer, so there is a problem of lack of physical activity. In particular, children/youths who need active physical activity often do not have enough physical activity in their academic and leisure time. In addition, as outdoor activities are restricted due to rapid weather changes, environmental pollution, and fine dust, there is a need for a technology that enables sufficient physical activity indoors.

As such, sufficient physical activity and monitoring are required for physical development in the child/adolescent period. Recently, adolescent obesity and adult diseases have become problems due to the decline in physical activity of children/adolescents. The demands are growing. However, when children/adolescents are active in school or leisure time, no technology has been proposed for effective measurement and management of anthropometric data.

According to such a request, a biometric information measuring device as shown in FIG. 1 can be used for collecting biometric information of children/adolescents, but children/adolescents are active in physical activity, making it difficult to wear the biometric information measuring device. There are many cases of appeal, and there is a high risk of damage or loss due to vigorous sports activities or carelessness, so it is difficult to use a biometric information measuring device. In addition, in group sports where several children/youths participate together, the process of providing biometric information measuring devices to each participant to wear and recovering them is cumbersome, and that all participants have a personal biometric information measuring device. Because it can't be done, there are limits to measuring and monitoring children's/youth's physical activity smoothly.

On the other hand, as a prior art related to the present invention, Patent Publication No. 10-2011-0031444 (name of the invention: health fitness measurement and evaluation and exercise management system and its method, publication date: March 28, 2011), etc. have.

The present invention has been proposed to solve the above problems of the previously proposed methods, and is installed in a space where indoor sports in which the user participates and recognizes the user's motion, and from the user's motion recognized by the camera. By including an exercise amount calculation module that calculates the amount of exercise, it is possible to monitor and verify the exercise effect of each user's indoor sports while maximizing the user's convenience by measuring the amount of exercise of users participating in indoor sports without wearing a separate wearable device. It is an object of the present invention to provide a system for calculating an amount of exercise using a camera for indoor sports.

In addition, the present invention calculates the amount of exercise according to the physical characteristics of the child/youth by calculating the amount of exercise in consideration of body information such as the user's height and weight, and is recognized by a pretrained exercise amount estimation model and a camera. Another object is to provide an exercise amount calculation system using a camera for indoor sports that can improve the accuracy of the calculated exercise amount by estimating the user's exercise amount based on deep learning using the user's motion.

An exercise amount calculation system using a camera for indoor sports according to a feature of the present invention for achieving the above object,

An exercise amount calculation system for calculating an exercise amount of a user participating in the indoor sport during indoor sports,

A camera installed in a space where indoor sports in which a user participates, and recognizing a user's motion; And

It is characterized in that it comprises an exercise amount calculation module for calculating the amount of exercise from the user's motion recognized by the camera.

Preferably, the camera,

Each user's body part can be tracked to detect each user's motion.

More preferably,

Further comprising a registration module for registering body parts of each user participating in the indoor sports,

The camera may detect a motion of each user by tracking a body part registered in the registration module.

Preferably, the camera,

It can be a 3D camera or a stereo camera.

Preferably, the momentum calculation module,

A motion calculation unit that calculates a moving distance and speed of the user from the motion of the user recognized by the camera; And

It may include a momentum calculating unit for calculating the amount of exercise using the calculated movement distance and speed.

More preferably, the momentum calculation unit,

The amount of exercise may be calculated in consideration of body information including the user's height and weight.

More preferably, the momentum calculation module,

Further comprising a feature point extracting unit for extracting feature points of the user's body part from the user's motion recognized by the camera,

The motion calculation unit calculates a moving distance and a speed of the extracted feature point,

The exercise amount calculator may calculate an exercise amount using a moving distance and a speed of the feature point.

Preferably, the momentum calculation module,

Using a pretrained exercise amount estimation model and a user's motion recognized by the camera, the user's exercise amount may be estimated through deep learning.

According to the system for calculating the amount of exercise using a camera for indoor sports proposed in the present invention, the amount of exercise is calculated from a camera that recognizes the user's motion and the user's motion recognized by the camera installed in a space where indoor sports in which the user participates. By including a calculated exercise amount calculation module, it is possible to monitor and verify the exercise effect of each user's indoor sports while maximizing the user's convenience by measuring the amount of exercise of users participating in indoor sports without wearing a separate wearable device. .

In addition, according to the exercise amount calculation system using a camera for indoor sports proposed in the present invention, by calculating the amount of exercise in consideration of body information such as height and weight of the user, it is possible to calculate the amount of exercise according to the physical characteristics of children/youth. In addition, by estimating the user's exercise amount based on deep learning using the pretrained exercise amount estimation model and the user's motion recognized by the camera, the accuracy of the calculated exercise amount may be improved.

1 is a diagram showing a small measuring device for measuring biometric information as an example.
2 is a view showing an example of applying an exercise amount calculation system using a camera for indoor sports according to an embodiment of the present invention.
3 is a diagram showing the configuration of an exercise amount calculation system using a camera for indoor sports according to an embodiment of the present invention.
4 is a diagram showing a detailed configuration of an exercise amount calculation module in an exercise amount calculation system using a camera for indoor sports according to an embodiment of the present invention.

Hereinafter, preferred embodiments will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art may easily implement the present invention. However, in describing a preferred embodiment of the present invention in detail, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. In addition, the same reference numerals are used throughout the drawings for portions having similar functions and functions.

In addition, throughout the specification, when a part is said to be connected to another part, this includes not only the case that it is directly connected, but also the case that it is indirectly connected with another element interposed therebetween. In addition, the inclusion of certain components means that other components may be further included rather than excluding other components unless specifically stated to the contrary.

2 is a diagram illustrating an example of applying an exercise amount calculation system using a camera 100 for indoor sports according to an embodiment of the present invention. As shown in FIG. 2, the exercise amount calculation system using the camera 100 for indoor sports according to an embodiment of the present invention is an exercise amount calculation system that calculates the amount of exercise of a user participating in indoor sports during indoor sports. , Including a camera 100 that is installed in an indoor sports space where a user participates to recognize the user's motion and an exercise amount calculation module 400 that calculates an exercise amount from the user's motion recognized by the camera 100, It is possible to monitor and verify the exercise effect of each user's indoor sports while maximizing the user's convenience by measuring the amount of exercise of a user participating in indoor sports without wearing a separate wearable device.

3 is a diagram showing the configuration of an exercise amount calculation system 10 using a camera 100 for indoor sports according to an embodiment of the present invention. As shown in FIG. 3, the exercise amount calculation system 10 using the camera 100 for indoor sports according to an embodiment of the present invention includes a camera 100 and an exercise amount calculation module 400. The storage module 200 and the registration module 300 may be further included.

Hereinafter, each component constituting the exercise amount calculation system 10 using the camera 100 for indoor sports according to an embodiment of the present invention will be described in detail with reference to FIGS. 2 and 3.

The camera 100 may be installed in a space where indoor sports in which the user participates, and may recognize a user's motion. More specifically, the camera 100 may be a 3D camera 100 or a stereo camera 100, and may be attached to the ceiling or wall of an indoor sports room, which is a space where indoor sports are possible, and as shown in FIG. They can also be installed on the floor. The camera 100 may be one, but may be composed of a plurality of cameras, and may recognize a motion by tracking users who play indoor sports.

In addition, the camera 100 may detect a motion of each user by tracking a body part of each user. To this end, the body part of each user is registered through the registration module 300 which will be described in detail later, and the camera 100 may track the registered body part. However, without physical registration, the user's motion required for calculating the amount of exercise can be captured and analyzed by the camera 100 to be recognized, and each recognized user can be tracked.

In particular, the present invention installs a camera 100 in an indoor sports room where a plurality of children or teenagers play sports together, and tracks the movements of each of a plurality of children or teenagers using the installed camera 100 to control the amount of exercise of an individual user. Can be calculated. To this end, the camera 100 may be connected to the exercise amount calculation module 400 through a wired or wireless network. That is, the network is a wired network such as a local area network (LAN), a wide area network (WAN), or a value added network (VAN), a mobile radio communication network, a satellite communication network, It can include all types of wireless networks such as Wireless Broadband Internet (WIBRO) and High Speed Downlink Packet Access (HSDPA).

The storage module 200, the registration module 300, and the exercise amount calculation module 400, which will be described in detail below, may be implemented in a server connected to a camera through a network, and implemented through a manager device that operates and manages an indoor sports room. It could be. Depending on the embodiment, some of the storage module 200, the registration module 300, and the exercise amount calculation module 400 may be implemented in a manager device, and some may be implemented in a server, and may be implemented in a cloud or virtual server, a software as a service (SaaS). ), etc. The detailed implementation method of each module may vary as needed.

The storage module 200 may store body information including a user's height and weight. Children or adolescents may have a difference in actual amount of exercise despite the same movement according to their physical development situation, and the amount of exercise required may be different from each other. Accordingly, the storage module 200 may store physical development status such as age, gender, height, and weight of users in the storage module 200 and use this when calculating the amount of exercise.

In addition, the storage module 200 may store body information by estimating the height and weight of the user from the image captured by the camera 100. Therefore, the user can be provided with an accurate amount of exercise calculated according to body information without having to separately store height or weight.

More specifically, from a color image and a depth image captured by the camera 100, the height and weight of the user may be measured using image processing technology and deep learning. That is, the color image and the depth image captured by the camera are pre-processed, the user's region is detected from the image, and then the weight can be calculated based on deep learning using depth information. At this time, an artificial neural network combining a convolutional neural network (CNN) and a long short-term memory (LSTM) recurrent neural network (RNN) may be used. In such a process of measuring body weight using an image, a measurement module (not shown) may be separately provided and the calculation may be processed in a measurement module (not shown), but an image captured by the camera 100 is transmitted through a network. The calculation may be processed on a cloud basis, and the storage module 200 may receive and store the calculation result.

The registration module 300 may register body parts of each user participating in indoor sports. The camera 100 may detect a motion of each user by tracking a body part registered in the registration module 300. That is, before the start of indoor sports, each user inputs a signal by taking a predetermined motion in front of the camera 100, and the camera 100 recognizes the predetermined motion and transmits the registration signal to the registration module 300, so that the registration module ( 300) can register the user's body part. In this case, the user's body part may be a silhouette of the entire body, and specific body parts including a head, an arm, a leg, and a joint may be registered.

The exercise amount calculation module 400 may calculate an exercise amount from a user's motion recognized by the camera 100. More specifically, the exercise amount calculation module 400 may estimate the user's exercise amount through deep learning using a pretrained exercise amount estimation model and a user's motion recognized by the camera 100. A detailed configuration of the exercise amount calculation module 400 will be described in detail below with reference to FIG. 4.

4 is a diagram showing a detailed configuration of an exercise amount calculation module 400 in the exercise amount calculation system 10 using the camera 100 for indoor sports according to an embodiment of the present invention. As shown in Figure 4, the exercise amount calculation module 400 of the exercise amount calculation system 10 using the camera 100 for indoor sports according to an embodiment of the present invention, the motion calculation unit 420 and the exercise amount calculation It may be configured to include the unit 430, and may further include a feature point extraction unit 410 and a deep learning unit 440.

The feature point extracting unit 410 may extract feature points of a user's body part from the user's motion recognized by the camera 100. The extracted feature points may be heads, arms, and legs, and may include all joints of the body. If there are a plurality of users recognized by the camera 100, the feature point extracting unit 410 may extract feature points for each user.

The motion calculation unit 420 may calculate a movement distance and a speed of the user from the user's motion recognized by the camera 100. For example, the motion calculation unit 420 may calculate a moving distance per frame of the camera 100, and may also calculate a speed through this. Depending on the embodiment, the motion calculation unit 420 may further calculate the movement direction so that the amount of motion of the horizontal movement such as running and the vertical movement such as jump is calculated differently.

More specifically, the motion calculation unit 420 may calculate a moving distance and a speed of the extracted feature points. In this case, for example, when the head, arm, and leg are extracted as feature points, the moving distance and speed at which the head, arm, and leg respectively moved may be calculated.

The exercise amount calculation unit 430 may calculate an exercise amount using the movement distance and speed calculated by the motion calculation unit 420. More specifically, the exercise amount calculation unit 430 may calculate the amount of exercise using the moving distance and speed of the feature points, and when a plurality of feature points are extracted for one user, the movement distance and speed of each feature point are integrated. So you can calculate the amount of exercise. The exercise amount calculation unit 430 may store the calculated exercise amount in the storage module 200.

Meanwhile, the exercise amount calculation unit 430 may calculate the amount of exercise in consideration of body information including the user's height and weight. That is, even with the same movement distance and speed, the actual amount of exercise may be different according to the height or weight. Therefore, by using the body information including the height and weight of the user stored in the storage module 200, the physical development situation of a child or adolescent Therefore, you can accurately calculate the amount of exercise.

The deep learning unit 440 may estimate a user's exercise amount through deep learning using a pretrained exercise amount estimation model and a user's motion recognized by the camera 100. In this case, a convolutional neural network (CNN) or a recurrent neural network (RNN) may be used as the momentum estimation model. The deep learning unit 440 may pre-train the exercise amount estimation model, and provide the pre-trained exercise amount estimation model to the exercise amount calculation unit 430 to provide the user with the user's motion recognized by the camera 100 through deep learning. The momentum of can be accurately calculated. In addition, the deep learning unit 440 may improve the performance of the exercise amount estimation model by continuously updating data calculated by the exercise amount calculating unit 430 and stored in the storage module 200.

As described above, according to the exercise amount calculation system 10 using the camera 100 for indoor sports proposed in the present invention, the camera 100 is installed in a space where indoor sports in which the user participates and recognizes the user's motion. By including the exercise amount calculation module 400 that calculates the amount of exercise from the user's motion recognized by the camera 100, the user's convenience is maximized by measuring the amount of exercise of the user participating in indoor sports without wearing a separate wearable device. Yet, it is possible to monitor and verify the exercise effect of each user's indoor sports.

In addition, according to the present invention, by calculating the amount of exercise in consideration of body information such as height and weight of the user, it is possible to calculate the amount of exercise according to the physical characteristics of the child/youth, and the pre-trained exercise amount estimation model and the camera 100 By estimating the user's exercise amount based on deep learning using the user's motion recognized by, it is possible to improve the accuracy of the calculated exercise amount.

The present invention described above can be modified or applied in various ways by those of ordinary skill in the technical field to which the present invention belongs, and the scope of the technical idea according to the present invention should be determined by the following claims.

10: momentum calculation system
100: camera
200: storage module
300: registration module
400: momentum calculation module
410: feature point extraction unit
420: motion calculation unit
430: momentum calculation unit
440: deep learning department

Claims (8)

As an exercise amount calculation system 10 for calculating an exercise amount of a user participating in the indoor sports during indoor sports,
A camera 100 installed in a space where indoor sports in which the user participates to recognize a user's motion; And
An exercise amount calculation module (400) for calculating an exercise amount from a user's motion recognized by the camera (100). The exercise amount calculation system (10) using the camera (100) for indoor sports. The method of claim 1, wherein the camera 100,
An exercise amount calculation system 10 using a camera 100 for indoor sports, characterized in that the motion of each user is detected by tracking each user's body part. The method of claim 2,
Further comprising a registration module 300 for registering body parts of each user participating in the indoor sports,
The camera 100 tracks the body part registered in the registration module 300 and detects the motion of each user. The exercise amount calculation system 10 using the camera 100 for indoor sports. The method of claim 1, wherein the camera 100,
An exercise amount calculation system 10 using a camera 100 for indoor sports, which is a 3D camera 100 or a stereo camera 100. The method of claim 1, wherein the momentum calculation module 400,
A motion calculation unit 420 that calculates a moving distance and speed of the user from the motion of the user recognized by the camera 100; And
An exercise amount calculation unit (10) using a camera (100) for indoor sports, characterized in that it comprises an exercise amount calculation unit (430) for calculating an exercise amount using the calculated moving distance and speed. The method of claim 5, wherein the momentum calculation unit 430,
An exercise amount calculation system (10) using a camera (100) for indoor sports, characterized in that the exercise amount is calculated in consideration of body information including the user's height and weight. The method of claim 5, wherein the momentum calculation module 400,
Further comprising a feature point extracting unit 410 for extracting feature points of the user's body part from the user's motion recognized by the camera 100,
The motion calculation unit 420 calculates a moving distance and a speed of the extracted feature point,
The exercise amount calculation unit 430 is an exercise amount calculation system 10 using a camera 100 for indoor sports, characterized in that the exercise amount is calculated using the moving distance and speed of the feature point. The method of claim 1, wherein the momentum calculation module 400,
Exercise amount calculation using a camera 100 for indoor sports, characterized in that the exercise amount of the user is estimated through deep learning using a pretrained exercise amount estimation model and the user's motion recognized by the camera 100 System (10).

Images