KR102296178B1 - Game Based Artificial Intelligence Fitness System and Method - Google Patents

Abstract

The present embodiments relate to a fitness assistance system for assisting fitness. A display unit that provides fitness-related fitness content to a target object using the fitness assistance system, and a display unit that detects a motion of the target object to collect motion information of the target object Provides a fitness assistance system comprising an object recognition unit and a processor that detects motion characteristics according to the movement of the target object based on the motion information of the target object, compares it with a standard object, and controls the aspect in which fitness content is provided on the display unit do.

Description

Game Based Artificial Intelligence Fitness System and Method

The present invention relates to a fitness assistance system and method, and more particularly, to a game-based artificial intelligence fitness assistance system for inducing continuous fitness motivation and interest in users.

The content described in this section merely provides background information for the present embodiment and does not constitute the prior art.

The current fitness content has a high rate of giving up due to the lack of continuous interest and motivation. Interesting factors of game contents include a sense of victory through competition, cooperation through role factors, expectation through luck factors, and a sense of achievement through goal factors. to give

In the field of smart fitness, artificial intelligence-based image recognition technology is one of the technological research fields that are applied to user image processing and information processing. However, when recognizing an image, various obstacles and factors impeding recognition exist within the camera shooting range, and thus, there is a problem in that the image recognition recognition rate is lowered. In this regard, a technology capable of improving the recognition rate of artificial intelligence-based image recognition technology is required, and there are interlocking with Kinect and designing an optimal artificial intelligence model as a way to improve the recognition rate. However, in Kinect-based image recognition, there is a problem in that it is impossible to identify and recognize surrounding factors that interfere with recognition.

Korean Patent Publication No. 10-2021-0030855

Embodiments of the present invention propose a technique for performing artificial intelligence fitness exercise based on a game. The present invention can be applied in the field of smart fitness, and the user's fitness can be induced through game contents such as a fitness battle between users, a character growth system, and a ranking system.

The present invention can provide a personalized fitness program in consideration of the user's personal characteristics such as gender, age, body type, and exercise purpose, and provide continuous motivation through game character implementation compensation according to fitness achievement after fitness is finished, The main purpose of the invention is to improve the recognition rate in the presence of factors that interfere with recognition through an image recognition technique based on artificial intelligence.

Other objects not specified in the present invention may be additionally considered within the scope that can be easily inferred from the following detailed description and effects thereof.

In order to solve the above problems, the fitness assistance system according to an embodiment of the present invention, in the fitness assistance system for assisting fitness, provides fitness content for the fitness of the target object to the target object using the fitness assistance system. A display unit that provides, an object recognition unit that detects the motion of the target object and collects motion information of the target object, and a standard object by detecting motion characteristics according to the motion of the target object based on the motion information of the target object and a processor that compares the movement characteristics and controls the manner in which the fitness content is provided to the display unit.

Here, the processor includes a game execution unit for executing an interactive game content according to an operation of the target object and another target object connected through a network, and the display unit outputs an exercise scene of the other target object.

Here, the object recognition unit includes a Kinect for detecting a motion of the target object to detect Kinect information of the target object, and an image acquisition unit for collecting image information of the target object by detecting a motion of the target object, The image acquisition unit is used to prevent a decrease in the recognition rate due to factors that interfere when the motion detection of the target object through the Kinect is not smooth.

Here, the processor controls a predetermined area of the display unit to form a predetermined brightness, a display control unit that controls to provide the fitness content related to the fitness to the display unit, (i) body information for the target object, (ii) ) Age, (iii) gender, (iv) body type, and (v) a program selector for selecting a program of the fitness content based on the object information when at least one object information of the purpose of exercise is input, and (i) the key Recognizing the target object based on the nect information to detect a motion characteristic according to the movement, or (ii) applying the input image of the target object to an image recognition model to recognize the target object, and to collect the image information and an image processing unit configured to detect a motion characteristic according to the motion of the target object based on the target object.

Here, the program selection unit includes an information integrator for integrating the object information for the input target object, an object information feature generator for generating object information features by extracting the integrated object information through a feature extraction algorithm, and the object A program of the fitness content suitable for the target object based on the grouped information feature and an object information feature classifying unit for classifying the clustered object information feature into a plurality of groups, wherein the program selection unit is based on the classified plurality of groups recommend

Here, the image recognition model includes a contour extracting unit for extracting contour information of the target object, a contour merging unit for collecting body shape information of the target object by merging the contour information extracted from the contour extracting unit, and the target object and a body feature extracting unit that determines each body structure of the target object based on body shape information of the target object and recombines each body structure, and recognizes the target object based on the recombined body structure.

Here, the image processing unit labels the area of the target object based on the detected motion characteristic according to the motion of the target object, the labeling moves according to the motion of the target object, the motion characteristic of the standard object and the Compare the labeling to determine the concordance rate.

Here, the display unit is divided into left and right to extract a motion image of the standard object and a motion image of the target object, and the display unit (i) the coincidence rate, (ii) the number of motions of the target object, (iii) the target The target number of exercise of the object and (iv) information on the amount of calories consumed by the target object are displayed at the top or bottom.

Here, when the matching rate of the target object falls below a preset value, the display unit displays the body organs of the target object with the falling matching rate on the motion image of the target object, and a motion correction induction image to increase the matching rate , and the processor calculates a fitness achievement index of the target object based on the match rate.

Here, the processor performs character reinforcement for strengthening the character of the target object according to the achievement by using through the fitness achievement index, and performs a ranking system that gives a ranking between the target objects based on the fitness achievement index characterized in that

A fitness assistance method according to another embodiment of the present invention is a fitness assistance method by a fitness assistance system for assisting fitness, wherein fitness-related fitness content is provided to the target object using the fitness assistance system through a display unit detecting the motion of the target object to collect motion information of the target object, and detecting motion characteristics according to the motion of the target object based on the motion information of the target object and comparing it with the motion characteristics of a standard object and controlling an aspect in which the fitness content is provided to the display unit.

Here, the collecting of the motion information of the target object includes detecting the motion of the target object to detect the kinect information of the target object, and collecting image information of the target object by detecting the motion of the target object. The step of collecting the image information of the target object is to prevent a decrease in the recognition rate due to factors that interfere when the motion detection of the target object is not smooth in the step of detecting the kinect information of the target object. carry out

Here, the step of controlling the manner in which the fitness content is provided to the display unit includes controlling a predetermined area of the display unit to form a predetermined brightness, and controlling the display unit to provide the fitness content related to the fitness, the At least one object information of (i) body information, (ii) age, (iii) gender, (iv) body type, and (v) exercise purpose for the target object is input, and based on the object information, selecting a program and (i) recognizing the target object based on the Kinect information to detect motion characteristics according to the motion of the target object, or (ii) image recognition of an input image of the target object and recognizing the target object by applying it to a model, and detecting a motion characteristic according to the motion of the target object based on the image information.

As described above, according to the embodiments of the present invention, the present invention can provide an effect of continuously inducing the use of fitness through content that arouses interest in mobile game elements.

According to embodiments of the present invention, the present invention can provide the most effective fitness program to the user through an artificial intelligence algorithm-based personalized fitness program recommendation system.

According to embodiments of the present invention, the present invention can perform feature extraction of image information of a user even in a situation where there are obstacles in the process of user recognition and feature extraction in artificial intelligence-based image recognition, and Kinect It has the effect of improving the recognition rate by using artificial intelligence techniques complementary to image recognition.

Even if it is an effect not explicitly mentioned herein, the effects described in the following specification expected by the technical features of the present invention and their potential effects are treated as if they were described in the specification of the present invention.

1 is a block diagram illustrating a fitness assistance system according to an embodiment of the present invention.
2 is a block diagram illustrating a processor of a fitness assistance system according to an embodiment of the present invention.
3 is an exemplary diagram illustrating the configuration of a fitness assistance system according to an embodiment of the present invention.
4 is a flowchart illustrating a method of operating a fitness assistance system according to an embodiment of the present invention.
5 is a flowchart illustrating the selection of a personalized fitness program by the fitness assistance system according to an embodiment of the present invention.
6 is a flowchart illustrating a fitness process through a fitness assistance system according to an embodiment of the present invention.
7 is an exemplary diagram illustrating an image recognition model process of a fitness assistance system according to an embodiment of the present invention.
8 is a flowchart illustrating a fitness assistance method according to an embodiment of the present invention.

Hereinafter, in the description of the present invention, if it is determined that the subject matter of the present invention may be unnecessarily obscured as it is obvious to those skilled in the art with respect to related known functions, the detailed description thereof will be omitted, and some embodiments of the present invention will be described. It will be described in detail with reference to exemplary drawings. However, the present invention may be embodied in various different forms, and is not limited to the described embodiments. In addition, in order to clearly describe the present invention, parts irrelevant to the description are omitted, and the same reference numerals in the drawings indicate the same members.

and/or includes a combination of a plurality of related recited claims or any of a plurality of related recited claims.

When a component is referred to as being “connected” or “connected” to another component, it is understood that the other component may be directly connected or connected to the other component, but other components may exist in between. it should be

The suffixes "module" and "part" for the components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have a meaning or role distinct from each other by themselves.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

The present invention relates to a game-based artificial intelligence fitness assistance system and method.

1 is a block diagram illustrating a fitness assistance system according to an embodiment of the present invention, and FIG. 2 is a block diagram illustrating a processor of the fitness assistance system according to an embodiment of the present invention. As shown in FIG. 1 , the fitness assistance system 10 includes a display unit 100 , an object recognition unit 200 , and a processor 300 . The fitness assistance system 10 may omit some of the various components illustrated by way of example in FIG. 1 or may additionally include other components.

The fitness assistance system 10 is a fitness assistance system in which game elements are grafted, and as a supplementary means of giving up fitness due to the absence of continuous motivation, it is possible to add game elements such as fitness battles between multiple users, character growth, and ranking to fitness may continue to motivate. The fitness assistance system 10 may perform a game using unity through interaction between a plurality of users as well as the aforementioned fitness competition between users. In addition, the fitness assistance system 10 is not limited by location.

The fitness assistance system 10 can be applied to the smart fitness field, and provides a personalized fitness program in consideration of personal characteristics such as a user's gender, age, body type, and exercise purpose, and a game according to fitness achievement after completion of fitness Character implementation rewards can provide continuous motivation.

The fitness assistance system 10 may consist of user-to-user fitness competition content, character growth content, ranking content, and the like using mobile game elements. In addition, the fitness assistance system 10 may recommend a personalized fitness program through an artificial intelligence algorithm that processes information such as body information, gender, age, and exercise purpose of the target object.

The target object is a user who uses the fitness assistance system 10 , and is a person who assists fitness using the fitness assistance system 10 .

The fitness assistance system 10 may recommend a fitness program suitable for an individual. In addition, the fitness assistance system 10 may perform image information feature extraction of the target object in the presence of an obstacle in target object recognition and feature extraction.

The fitness assistance system 10 may recognize an image through the Kinect 210 and may improve the recognition rate by using artificial intelligence as a complement.

The fitness assistance system 10 is not limited to the exercise equipment used in the fitness program performed by the target object. As the exercise equipment, not only small-volume exercise equipment such as dumbbells and a gym ball, but also large-volume exercise equipment such as a treadmill and a cycle may be used.

The fitness assistance system 10 may induce continuous motivation for exercise through a game in which competition or cooperation between objects using the fitness assistance system 10 is performed. The continuous motivation is not necessarily limited to a game in which competition or cooperation is performed, and may be content that can cause interest while the target object performs an exercise motion.

The image recognition model of the fitness assistance system 10 may be used based on a convolutional neural network (CNN). This is not necessarily limited.

Convolutional Neural Network (CNN) is a type of Deep Neural Network (DNN). It consists of one or several convolutional layers, a pooling layer, and a fully connected layer. It represents a neural network composed of connected layers. A convolutional neural network (CNN) includes a hidden layer in which multiple layers are networked. A layer may include a parameter, and the parameter of the layer includes a set of learnable filters. A convolution filter can be applied as a filter. The parameters include weights and/or biases between nodes.

The fitness assistance system 10 may be applied to a smart fitness service, content using artificial intelligence-based image recognition, a platform for smart fitness service, various content through image recognition, and interactive content through mobile game application.

The display unit 100 may provide fitness content related to fitness to a target object of the fitness assistance system.

The display unit 100 may be divided left and right to extract a motion image of a standard object and a target object of the fitness assistance system 10 . The standard object represents an image of a fitness assistant trainer, and may be an image representing a shape that a target object using the fitness assistant system 10 can see and do fitness.

The display unit 100 may display (i) coincidence rate, (ii) the number of workouts of the target object, (iii) the target number of workouts of the target object, and (iv) information on calories consumed by the target object at the top or bottom. The information located at the top or bottom of the display unit 100 is not necessarily limited to the above-described information, and may further include information that can help the movement of the target object.

When the motion matching rate of the target object falls below a preset value, the display unit 100 displays the body organs of the target object with the falling matching ratio on the motion image of the target object, and may output a motion correction guidance image to increase the matching ratio. have.

The object recognition unit 200 may collect motion information of the target object by detecting the motion of the target object. As shown in FIG. 1 , the object recognition unit 200 includes a Kinect 210 and an image acquisition unit 220 . The object recognition unit 200 may omit some of the various components exemplarily illustrated in FIG. 1 or may additionally include other components.

The Kinect 210 senses and recognizes the motion of the target object, and may detect Kinect information of the target object. Kinect information may include a movement of a target object. The Kinect 210 may play a fitness content program using gestures, voice, and the entire body without a controller.

Kinect 210 generates Kinect information by detecting the motion of the target object. For example, the Kinect 210 generates link information between key points by inputting an image of a target object. The feature point may indicate a joint node, but is not limited thereto.

According to an embodiment of the present invention, fitness content includes game content, a motion image of a target object, a motion image of a standard object, a matching rate, the number of workouts of the target object, the target number of workouts of the target object, calorie information consumed by the target object, etc. It may include content for assisting fitness, but is not limited thereto.

The Kinect 210 includes an infrared emitter, a color sensor, an infrared distance sensor, a tilt motor, a microphone array, and the like. The Kinect 210 transmits infrared rays to scan the target object of the fitness assistance system 10 , and the infrared rays hitting the body of the target object are reflected and received. The Kinect 210 checks the distance per pixel according to the reflected infrared rays, and the image processing software can recognize the target object in real time. Kinect 210 may determine the operation of the target object by recognizing the bone nodes of the target object.

The image acquisition unit 220 may collect image information of the target object by detecting the motion of the target object. According to an embodiment of the present invention, the image acquisition unit 220 may be a camera, but is not necessarily limited thereto, and may be formed as a device for collecting image information of the target object by detecting the motion of the target object.

According to an embodiment of the present invention, the image acquisition unit 220 may be used to prevent a decrease in the recognition rate due to factors that interfere when the motion and voice detection of the target object through the Kinect 210 are not smooth. .

Therefore, according to an embodiment of the present invention, the fitness assistance system 10 preferentially uses the Kinect 210 to detect motion information of the target object, and when the recognition of the target object is not smooth, the image acquisition unit A motion characteristic may be detected by recognizing a target object using the image recognition model 331 learned based on the image information collected through 220 .

The processor 300 may detect an operation state according to the movement of the target object based on the operation information of the target object, and may control an aspect in which the operation state and fitness content according to the movement of the target object are provided to the display unit 100 . have. 1 , the processor 300 includes a display control unit 310 , a program selection unit 320 , an image processing unit 330 , and a game execution unit 340 . The processor 300 may omit some of the various components exemplarily illustrated in FIG. 1 or may additionally include other components.

The display control unit 310 may control a predetermined area of the display unit 100 to form a predetermined brightness, and may control the display unit 100 to provide fitness-related fitness content.

The program selection unit 320 is based on the object information when at least one of (i) body information, (ii) age, (iii) gender, (iv) body type, and (v) exercise purpose for the target object is input. You can select a program of fitness content.

Referring to FIG. 2 , the program selection unit 320 includes an information integrator 322 , an object information feature generator 324 , and an object information feature classifier 326 . The program selection unit 320 may omit some of the various components exemplarily illustrated in FIG. 2 or may additionally include other components.

The information integrator 322 may integrate the input target object information.

The object information feature generation unit 324 may generate object information features by extracting the integrated object information through a feature extraction algorithm.

The object information feature classifying unit 326 may cluster object information features and classify the clustered object information features into a plurality of groups.

The program selection unit 320 may recommend a fitness content program suitable for the target object of the fitness assistance system 10 based on the plurality of groups classified by the object information characteristic classification unit 326 .

A process performed by the program selection unit 320 will be described later in detail with reference to FIG. 4 .

The image processing unit 330 (i) recognizes a target object based on the Kinect information detected by the Kinect 210 and detects a motion characteristic according to the movement, or (ii) the image acquired by the image acquisition unit 220 Based on the image information, an input image of the target object may be applied to the image recognition model 331 to recognize the target object, and a motion characteristic according to the motion of the target object may be detected based on the image information.

The image processing unit 330 labels a region of the target object based on the extracted motion characteristics of the target object, and the labeling moves according to the movement of the target object. The image processing unit 330 may determine the matching rate by comparing the labeling with the motion characteristics of the standard object.

Labeling is a method of grouping and numbering adjacent pixel values in order to identify each object.

The game execution unit 340 may execute the interactive game content according to the operation of the target object and another target object connected through a network. The display unit 100 may output a motion scene of another target object of the interactive game content executed by the game execution unit 340 overlaid with the motion scene of the target object.

The interactive game content may be formed through an interactive method, but is not limited thereto. Interactive is an interactive, interactive program that can input and output as if a target object were talking to another target object.

Referring to FIG. 2 , the image recognition model 331 of the image processing unit 330 includes a contour extractor 332 , an outline merging unit 334 , and a feature extractor 336 . The image recognition model 331 may extract shape and shape features by extracting and merging contour features of an input image based on a convolutional neural network (CNN).

According to an embodiment of the present invention, the image recognition model 331 is a matrix operation performed to transform an image into a certain pattern, and when the same filter is applied to the entire image, it is possible to detect whether the feature is in the data or not. . The image recognition model 331 may extract a feature from an image, classify it, and output a classification result, and if the result of matching the features of the image is the same, it is determined as the same image.

The contour extractor 332 may extract contour information of the body of the target object. The outline extraction unit 332 may extract an outline such as a line frequently detected in an image, such as a point or an edge.

According to an embodiment of the present invention, the contour extraction unit 332 extracts the contour by applying the contour filter, and in the case of a contour, the result value is high, and when it is not a contour or not similar to the contour, the result value is 0. can be approximated.

The contour merging unit 334 may collect the body shape information of the target object by merging the contour information extracted by the contour extracting unit 332 . The outline merging unit 334 recognizes and merges a model such as a circle or a triangle, and may be merged in the shape of each body part of the target object.

The body feature extractor 336 may determine each body structure of the target object based on the body shape information of the target object, and recombine each body structure. The fitness assistance system 10 may recognize the target object based on each recombined body structure.

When the fitness assistance system 10 fails to extract the movement characteristics of the target object, it attempts recognition again using the Kinect 210 , and when the recognition is not smooth, the image information collected through the image acquisition unit 220 is used. Extraction of contour feature information can be performed again based on

The fitness content of the fitness assistance system 10 may induce interest and motivation in the target object through a fitness competition between objects using the fitness assistance system 10 . The fitness competition may be executed by the game execution unit 340 .

According to an embodiment of the present invention, in a fitness match between objects, a fitness achievement index of a target object may be calculated based on a match rate. According to an embodiment of the present invention, the coincidence rate may mean a degree to which the motion characteristic of the target object matches the motion characteristic of the standard object. The display unit 100 may output a superimposed exercise scene of a competitor in a fitness competition between objects.

The fitness competition between objects may include character reinforcement for reinforcing the character of the target object according to the achievement by using the fitness achievement index, and a ranking system for giving a ranking between objects based on the fitness achievement index.

3 is an exemplary diagram illustrating the configuration of a fitness assistance system according to an embodiment of the present invention.

The components of the fitness assistance system 10 may be located as shown in FIG. 3 , and the positions of the components of the fitness assistance system 10 are not necessarily limited thereto.

The Kinect 210 and the image acquisition unit 220 of the object recognition unit 200 of the fitness assistance system 10 are located at positions capable of detecting the motion of the target object. Also, the display unit 100 of the fitness assistance system 10 may be located where the target object of the fitness assistance system 10 can check content. The processor 300 may be located at a place where image information and feature information can be exchanged with the display unit 100 and the object recognition unit 200 .

Referring to FIG. 3 , the target object, the image acquisition unit 220 , and the image acquisition unit 220 and the processor 300 of the fitness assistance system 10 may be located where image information of the target object can be exchanged. . In addition, the processor 300 and the display unit 100 of the fitness assistance system 10 , and the display unit 100 and the target object may exchange extracted feature information.

The display unit 100 may be formed as a general display device, may be a display device formed for the fitness assistance system 10 , and may be output through a home appliance formed as a display device such as a computer or a television.

The Kinect 210 of the object recognition unit 200 is configured to detect the Kinect information by preferentially detecting the motion information of the target object, and the image acquisition unit 220 can collect the image information of the target object in real time. It can be configured with performance.

The processor 300 may be configured to detect an operation state according to the movement of the target object of the collected image information.

Hereinafter, a flow of a fitness exercise process through the fitness assistance system 10 will be described.

4 is a flowchart illustrating a method of operating a fitness assistance system according to an embodiment of the present invention.

The method of operation of the fitness assistance system 10 includes a log-in step (S100), a game content use step (S200), a fitness content use step (S300), a personalized fitness program selection step (S400), a fitness assistance system (10). It includes a fitness step (S500) through fitness, a game character implementation step (S600) according to the result after fitness ends, and a content use end step (S700).

The target object of the fitness assistance system 10 may log in to the fitness assistance system 10 to perform a fitness exercise (S100). In step S100, when the target object logs in to the fitness assistance system 10, information of the target object is input.

When the login ( S100 ) of the fitness assistance system 10 is completed, the target object of the fitness assistance system 10 may determine whether to use the game content ( S200 ).

In step S300, the fitness assistance system 10 may determine whether to use the fitness content, if the fitness content is used, perform step S400, and if the fitness content is not used, determining whether to use the game content for use (S200) You can go back to step.

The step of selecting a personalized fitness program ( S400 ) will be described in detail with reference to FIG. 5 . 5 is a flowchart illustrating the selection of a personalized fitness program by the fitness assistance system according to an embodiment of the present invention.

According to an embodiment of the present invention, the individually customized fitness program selection step ( S400 ) may be performed through the program selection unit 320 of the fitness assistance system 10 .

The personalized fitness program selection step (S400) is a step of integrating the input information when the object information for the target object is input to the fitness assistance system 10 (S410), the step of performing a feature extraction algorithm (S420), the extracted features It includes a step of clustering according to (S430), a step of performing a classification algorithm for each feature group (S440), and a step of recommending a customized fitness program according to the group into which the feature is classified (S450).

When object information of the target object such as body information, body type, age, gender, and exercise purpose of the target object is input, the fitness assistance system 10 may integrate the input object information of the target object into one ( S410 ). Step S410 is performed by the information integration unit 322 of the program selection unit 320 .

The information of the integrated object goes through a feature extraction algorithm (S420). In step S420 , the program selection unit 320 of the fitness assistance system 10 may extract features of the input object information through a feature extraction algorithm. Step S420 is performed by the object information feature generation unit 324 of the program selection unit 320 .

According to an embodiment of the present invention, step S420 may include feature extraction as to whether the target object should increase or decrease body weight, skeletal muscle mass, or body fat amount through body information of the target object. In addition, in step S420, a part that needs to be exercised centrally of the target object may be extracted through the body shape of the target object, and features of a part to be exercised may be extracted through the purpose of the movement of the target object.

When step S420 is completed through the above-described process, the fitness assistance system 10 performs clustering ( S430 ) according to the extracted features. Clustering is a process of grouping similar or related items together and remembering them. Among the extracted features, similar or related features can be grouped together.

The clustered and extracted features are subjected to a classification algorithm for each feature group ( S440 ). In step S440 , the program selection unit 320 of the fitness assistance system 10 may classify the clustered extracted features through a feature group classification algorithm.

According to an embodiment of the present invention, in step S440, the clustered extracted features may be classified into groups. The group may be classified into a group according to a number of characteristics, such as a teenage male group, a 20's female group, and a teenage male body fat reduction group.

Steps S430 and S440 described above are performed by the object information feature classifying unit 326 of the step program selection unit 320 .

When the group is classified, the fitness assistance system 10 performs a step S450 of recommending a customized fitness program according to the group in which the characteristics are classified.

Step S450 is a step of selecting the group classified through step S440 , and may be selected as a group suitable for the target object of the fitness assistance system 10 .

When the fitness assistance system 10 selects a group suitable for the target object of the fitness assistance system 10 in step S440 , a personalized fitness program suitable for the group is recommended.

The target object of the fitness assistance system 10 may perform fitness by selecting one of the personalized fitness programs recommended through the above-described personalized fitness program selection step S400 .

The fitness assistance system 10 may perform step S500 when a fitness program is selected through step S400 .

The fitness step S500 through the fitness assistance system 10 will be described in detail with reference to FIG. 6 . 6 is a flowchart illustrating a fitness process through a fitness assistance system according to an embodiment of the present invention.

According to an embodiment of the present invention, the fitness step S500 through the fitness assistance system 10 may be performed through the image processing unit 320 of the fitness assistance system 10 .

In the fitness step (S500) through the fitness assistance system (10), when the Kinect information detected through the Kinect (210) is input, checking whether the recognition of the Kinect information is smooth (S510), the learned image recognition model ( 331) applying step (S520), target object recognition step (S530), target object image image collection step (S540), target object motion feature extraction step (S550), and feature extraction failure checking step (S560). .

When the Kinect information detected through the Kinect 210 is input, the fitness assistance system 10 may determine whether the recognition of the Kinect 210 is smooth ( S510 ). If the recognition is smooth in step S510, step S530 is performed. If the recognition is not smooth, step S520 is performed through the image information collected through the image acquisition unit 220 .

In operation S520 , when the image information collected through the image acquisition unit 220 is input, it may be applied to the learned image recognition model 331 to detect an operation state according to the movement of the target object. A process of performing the learned image recognition model 331 will be described in detail with reference to FIG. 7 .

7 is an exemplary diagram illustrating an image recognition model process of a fitness assistance system according to an embodiment of the present invention.

In step S520, a convolutional neural network (CNN)-based sequential feature extraction and merging that extracts and merges contour features of an input image through artificial intelligence-based image recognition to extract shape and shape features. In step S520, the target object is recognized based on the extracted shape and shape information, and image information of the target object is collected to extract motion information of the target object.

When the image information collected through the image acquisition unit 220 is input, the fitness assistance system 10 may generate an input image according to the image information. From the input image, contour features may be extracted through step S522.

Step S522 means extracting contour information of the body structure of the target object.

According to an embodiment of the present invention, the contour feature extraction may be formed in a form in which the outer surface of the body of the target object is divided into lines. For example, the head of the target object may be divided into quadrants and extracted as four pieces of outline information, and the head of the target object may be divided and extracted into a plurality of pieces of outline information, such as one neck and two legs. Step S522 is performed by the outline extraction unit 332 of the image processing unit 330 .

In step S524, outlines are merged based on the outline information of the body structure of the target object extracted in step S522. In operation S524, information on the shape of the body structure of the target object may be collected by merging the contour information of operation S522.

According to an embodiment of the present invention, the outline merging is a process of merging the shape in which the outer surface of the body of the target object is divided by lines. For example, the head of the target object divided into quadrants may be merged into head information by gathering three head outlines. The merged body information of the target object may be head, neck, arm, torso, and leg information indicating each body of the target object, but is not necessarily limited thereto, and may be information obtained by decomposing the body of the target object into a plurality of pieces if necessary. have. Step S524 is performed by the outline merging unit 334 of the image processing unit 330 .

In step S526, the shape and shape features of the body shape information of the target object merged in step S524 may be extracted. In step S526, each body structure may be determined based on the merged body shape information in step S524, and features thereof may be extracted.

According to an embodiment of the present invention, the extraction of form and shape features may label the output information by recognizing the body structure of the target object, and recognizing that the target object is moving so that the labeling may follow the movement of the target object . Step S524 is performed by the body feature extraction unit 336 of the image processing unit 330 .

In the above-described step S520, the information formed through steps S522, S524, and S526 is recombined and then post-processed to be reconstructed into an image.

The fitness assistance system 10 may recognize the shape of the target object through step S510 or step S520 ( S530 ).

In step S540, when the shape of the target object is recognized in step S530, an image image of the target object may be collected.

In step S550 , the motion characteristics of the target object may be extracted from the collected target object video image.

In step S560, it is determined whether to extract the feature, and if the feature extraction fails, the process returns to step S510, and if the feature extraction succeeds, the process returns to step S540. If the feature extraction is successful and returns to step S540, a target object video image may be newly collected.

The fitness assistance system 10 collects the target object image image of the fitness assistance system 10 through the process of step S500 described above, and repeats the process of extracting the motion characteristics of the target object of the standard object representing the fitness assistance trainer. The motion matching rate may be determined by comparing the motion with the motion of the target object.

Through the process of step S500, the target object performs fitness, and when the fitness ends, step S600 may be performed.

Step S600 may implement a game character according to the result after the end of fitness. For example, the character of the target object can be strengthened according to the achievement of the target object through the fitness achievement index performed in step S500, and a ranking between objects is given based on the achievement index, and rewards such as game items or game money are provided accordingly. can be given

The fitness assistance system 10 may change the game item worn by the target object as the game character level of the target object increases through reinforcement of the target object character, and the game item may be purchased through game money . Through strengthening and embellishment of the target object character, the target object of the fitness assistance system 10 may be interested and motivated.

The fitness assistance system 10 may perform step S700 when the reward according to the fitness result is divided through step S600 .

In step S700, it is determined whether the use of fitness content is terminated. If you continue to use the fitness content, the fitness assistance system 10 returns to step S300 and checks whether the fitness content is used.

In Figures 4 to 6, it is interposed as executing each process sequentially, but this is only an exemplary description, and those skilled in the art are shown in Figs. Various modifications and variations may be applied by changing the order described, executing one or more processes in parallel, or adding other processes.

8 is a flowchart illustrating a fitness assistance method according to an embodiment of the present invention. The fitness assistance method may be performed by the fitness assistance system, and a detailed description of an operation performed by the fitness assistance system will be omitted.

The fitness assistance method includes the steps of providing fitness-related fitness content to a target object of the fitness assistance system (S810), collecting motion information of the target object by detecting the motion of the target object (S820), and receiving motion information of the target object based on detecting an operation state according to the movement of the target object, and controlling the manner in which fitness content is provided on the display unit (S830).

The step of collecting motion information of the target object (S820) includes the steps of detecting the motion and voice of the target object to detect Kinect information of the target object, and collecting image information of the target object by detecting the motion of the target object. include

In the step of collecting the image information of the target object, in the step of detecting the kinect information of the target object, when the motion and voice detection of the target object are not smooth, in order to prevent a decrease in the recognition rate due to factors that interfere with the image information of the target object collection can be performed.

Detecting the motion state according to the movement of the target object based on the motion information of the target object, and controlling the aspect in which the fitness content is provided on the display unit (S230) controls a predetermined area of the display unit to form a predetermined brightness, Controlling to provide fitness content related to fitness on the display unit, at least one object of (i) body information, (ii) age, (iii) gender, (iv) body type, (v) exercise purpose for the target object Information is input, selecting a program of fitness content based on object information, extracting characteristics of the movement of a target object based on Kinect information or image information, and applying the image information to the image recognition model 331 and detecting an operation state according to the movement of the target object.

Although it is interposed as sequentially executing each process in FIG. 8, it is only illustratively described, and those skilled in the art change the order described in FIG. Alternatively, various modifications and variations may be applied by executing one or more processes in parallel or adding other processes.

The operations according to the present embodiments may be implemented in the form of program instructions that can be performed through various computer means and recorded in a computer-readable medium. Computer-readable medium represents any medium that participates in providing instructions to a processor for execution. Computer-readable media may include program instructions, data files, data structures, or a combination thereof. For example, there may be a magnetic medium, an optical recording medium, a memory, and the like. A computer program may be distributed over a networked computer system so that computer readable code is stored and executed in a distributed manner. Functional programs, codes, and code segments for implementing the present embodiment may be easily inferred by programmers in the technical field to which the present embodiment pertains.

The present embodiments are for explaining the technical idea of the present embodiment, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The protection scope of this embodiment should be interpreted by the following claims, and all technical ideas within the equivalent range should be interpreted as being included in the scope of the present embodiment.

10: Fitness Assistant System
100: display unit
200: object recognition unit
300: processor

Claims (13)

In the fitness assistance system for assisting fitness,
a display unit providing fitness content for fitness of the target object to the target object using the fitness assistance system;
an object recognition unit detecting a motion of the target object and collecting motion information of the target object; and
A processor for detecting a motion characteristic according to the motion of the target object based on the motion information of the target object, comparing it with the motion characteristic of a standard object, and controlling an aspect in which the fitness content is provided to the display unit,
The object recognition unit may include: a Kinect for detecting a motion of the target object to detect Kinect information of the target object; and an image acquisition unit configured to detect a motion of the target object and collect image information of the target object,
The processor (i) recognizes the target object based on the kinect information and detects a motion characteristic according to the motion of the target object, or (ii) converts the input image of the target object into a convolutional neural network. and an image processing unit for recognizing the target object by applying to an image recognition model formed based on the image recognition model, and detecting a motion characteristic according to the motion of the target object based on the image information,
The image recognition model is a matrix operation performed to transform an image into a predetermined pattern, and a contour extractor for extracting contour information indicating a point or an edge of the target object; a contour merging unit for merging the contour information extracted by the contour extracting unit into a model form to collect body shape information of the target object; and a body feature extraction unit for determining each body structure of the target object based on the body shape information of the target object, and recombining each body structure,
The processor extracts a motion characteristic of the target object through the recombined body structure, and when the motion characteristic is not extracted, detects the motion of the target object using the kinect, and the target object through the kinect The fitness assistance system, characterized in that the image information collected through the image acquisition unit is applied again to the image recognition model when the detection of the motion of the user is not smooth. According to claim 1,
The processor includes a game execution unit for executing interactive game content according to the operation of the target object and another target object connected by a network,
The display unit fitness assistance system, characterized in that for outputting the exercise scene of the other target object. According to claim 1,
The image acquisition unit is a fitness assistance system, characterized in that it is used to prevent a decrease in the recognition rate due to factors that interfere when the motion detection of the target object through the Kinect is not smooth. 4. The method of claim 3,
The processor is
a display control unit controlling a predetermined area of the display unit to form a predetermined brightness, and controlling the display unit to provide fitness content related to the fitness; and
When at least one object information of (i) body information, (ii) age, (iii) gender, (iv) body type, and (v) exercise purpose for the target object is inputted, based on the object information, A fitness assistance system further comprising a program selection unit for selecting a program. 5. The method of claim 4,
The program selection unit,
an information integrator for integrating the object information on the input target object;
an object information feature generator for generating object information features by extracting the integrated object information through a feature extraction algorithm; and
and an object information feature classifying unit that clusters the object information features and classifies the clustered object information features into a plurality of groups,
The program selection unit is a fitness assistance system, characterized in that it recommends a program of the fitness content suitable for the target object based on the plurality of classified groups. delete 5. The method of claim 4,
The image processing unit labels the area of the target object based on the detected motion characteristic according to the motion of the target object, the labeling moves according to the motion of the target object, and performs the labeling with the motion characteristic of the standard object. Comparing the fitness assistance system, characterized in that determining the match rate. 8. The method of claim 7,
The display unit is divided into left and right to extract a motion image of the standard object and a motion image of the target object,
The display unit displays (i) the matching rate, (ii) the number of times of exercise of the target object, (iii) the target number of workouts of the target object, and (iv) information on the amount of calories consumed by the target object at the top or bottom fitness assistance system. 9. The method of claim 8,
When the matching rate of the target object falls below a preset value, the display unit displays the body organs of the target object with the falling matching ratio on the motion image of the target object, and outputs a motion correction guidance image to increase the matching ratio and
The processor is a fitness assistance system, characterized in that for calculating the fitness achievement index of the target object based on the matching rate. 10. The method of claim 9,
The processor is
Perform character reinforcement to strengthen the character of the target object according to the achievement using the fitness achievement index,
A fitness assistance system, characterized in that performing a ranking system for assigning a ranking between the target objects based on the fitness achievement index. In the fitness assistance method by the fitness assistance system for supporting fitness,
providing fitness content for fitness of a target object to the target object using the fitness assistance system through a display unit;
collecting motion information of the target object by detecting the motion of the target object; and
Detecting a motion characteristic according to the motion of the target object based on the motion information of the target object, comparing it with the motion characteristic of a standard object, and controlling an aspect in which the fitness content is provided to the display unit,
The collecting of the motion information of the target object may include: detecting the motion of the target object to detect Kinect information of the target object; and collecting image information of the target object by detecting the motion of the target object,
The step of controlling the manner in which the fitness content is provided to the display unit may include (i) recognizing the target object based on the kinect information to detect a movement characteristic according to the movement of the target object, or (ii) the Recognizing the target object by applying an input image of the target object to an image recognition model formed based on a convolutional neural network network, and detecting motion characteristics according to the motion of the target object based on the image information. do,
The image recognition model is a matrix operation performed to transform an image into a certain pattern, extracts contour information indicating a point or an edge of the target object from the input image, and merges the contour information into a model form to Collecting body shape information of the target object, determining each body structure of the target object based on the body shape information of the target object, and recombining each body structure,
The step of controlling the aspect in which the fitness content is provided to the display unit extracts the movement characteristics of the target object through the recombined body structure and compares them with the movement characteristics of the standard object, and when the movement characteristics are not extracted Detecting the motion of the target object using the Kinect, and re-applying the image information collected through the image acquisition unit to the image recognition model when detection of the motion of the target object through the Kinect is not smooth How to do a fitness assistant. 12. The method of claim 11,
The step of collecting the image information of the target object is performed to prevent a decrease in the recognition rate due to factors that interfere when the motion detection of the target object is not smooth in the step of detecting the kinect information of the target object. How to do a fitness assistant. 13. The method of claim 12,
The step of controlling the aspect in which the fitness content is provided to the display unit,
controlling a predetermined area of the display unit to form a predetermined brightness, and controlling the display unit to provide fitness content related to the fitness; and
At least one object information of (i) body information, (ii) age, (iii) gender, (iv) body type, and (v) exercise purpose for the target object is input, and the fitness content is based on the object information A fitness assistant method further comprising the step of selecting a program of.

Images