KR102493903B1 - Method and apparatus for providing exercise informaion - Google Patents

Abstract

Disclosed are a method for providing exercise information and a device using the same. The method for providing exercise information, which is performed by an electronic device, comprises the steps of: receiving, from a first electronic device, second information generated by de-identifying other users than a current user in first information corresponding to a user exercise video; generating the exercise information of the current user by inputting the second information into a pre-trained artificial neural network; and providing the exercise information to an external entity, wherein the second information can be generated by inputting the first information into a first artificial neural network to recognize the current user and de-identifying other users than the current user.

Description

Exercise information providing method and apparatus {METHOD AND APPARATUS FOR PROVIDING EXERCISE INFORMAION}

The following embodiments relate to a method and apparatus for providing exercise information.

The exercise management application is implemented at a level where members directly input their own exercise-related records and convert them into a database.

Accordingly, in order to record the user's exercise-related information, not only is it cumbersome for the user to directly input the exercise record, but also the accuracy of the count of the number of exercises may be low due to arbitrary judgment.

Existing personal training is not only expensive, but also because the trainer directly supervises the exercise next to the user, some users have antipathy depending on their inclination, and the risk of disease infection can be high, especially in the corona situation.

A method of providing exercise information performed by a second electronic device according to an embodiment includes receiving, from the first electronic device, second information in which a user other than a current user is de-identified from first information corresponding to a user's exercise image. doing; generating exercise information of the current user by inputting the second information to a pre-learned artificial neural network; and providing the exercise information to an external entity, wherein the second information inputs the first information to a first artificial neural network to recognize the current user and de-identify users other than the current user. By doing so, it can be created.

The generating of the motion information may include: acquiring third information about the current motion of the user by inputting the second information to a second artificial neural network previously trained to estimate a pose of the user from an input image; obtaining fourth information about motion of an exercise machine currently used by the user by inputting the second information to a third artificial neural network pretrained to estimate a pose of the exercise machine from an input image; and generating exercise assistance information based on the information on the motion of the current user and the information on the motion of the exercise equipment.

The third artificial neural network may include a 3-1 artificial neural network learned to estimate exercise equipment and weight based on first learning data corresponding to an image including exercise equipment distinguished by weight in different colors and sizes; and a 3-2 artificial neural network trained to estimate a pose of the exercise machine based on second learning data, which is an image in which nodes are labeled on the force point and weight of the exercise machine.

The generating of the exercise information of the current user may include moving distance information of the center of gravity of the current user calculated based on the third information and the center of gravity of the exercise equipment used by the current user calculated based on the fourth information. Momentum may be calculated based on the movement distance information.

An exercise information providing method according to an embodiment includes the steps of overlapping a guide image with an image of the current user based on matching between the current user's node calculated in the third information and a pre-stored instructor's node; may further include.

The exercise information providing method according to an embodiment further includes the step of providing, as recommended music, BPM music having a multiple relationship with the reciprocal of the user's heart rate or BPM music having a multiple relationship with the reciprocal of the period of an individual exercise motion. can include

An exercise information providing method according to an embodiment includes recognizing a start time when the user starts an exercise and an end time when the user ends the exercise from sensing information obtained through a pressure sensor disposed on the floor; obtaining an actual exercise image corresponding to a time when a user actually performs an exercise based on the start time and the end time; and providing the actual motion image to an external entity.

The generating of exercise information of the current user may include generating information about a probability of reaching a target value by inputting characteristic information, exercise amount information, and net calorie consumption information of the current user into a fourth artificial neural network. can

An electronic device providing exercise information according to an embodiment includes a memory; and a processor, wherein the processor receives, from the first electronic device, second information that de-identifies other users other than the current user in the first information corresponding to the user's exercise image, and learns the second information in advance. By inputting the first information to the artificial neural network, the current user's exercise information is generated, the exercise information is provided to an external entity, and the second information inputs the first information to a first artificial neural network to recognize the current user. And, it can be created by de-identifying other users than the current user.

1A is a diagram for explaining an exercise information providing system according to various embodiments.
1B is a diagram for explaining an exercise information providing system according to various embodiments by way of example.
2 is a diagram for explaining a configuration of an electronic device according to various embodiments.
3 is a diagram for explaining a method of providing exercise information performed through a first electronic device and a second electronic device according to various embodiments.
4 is a diagram for explaining operations of a second artificial neural network and a third artificial neural network according to an embodiment.
5 is a diagram illustrating an example of an interface provided in a process of receiving a user's image in a sensing device according to an exemplary embodiment.
6 is a diagram for illustratively explaining a method of providing exercise goal attainment probability information, which is one of fifth information, by a second electronic device according to an embodiment.
7 is a diagram for explaining the operation of a method for providing exercise information according to various embodiments.
8A is a diagram for explaining a deep learning operation method using an artificial neural network.
8B is a diagram for explaining a learning and reasoning method of an artificial neural network model according to an embodiment.

Specific structural or functional descriptions of the embodiments according to the concept of the present invention disclosed in this specification are only illustrated for the purpose of explaining the embodiments according to the concept of the present invention, and the embodiments according to the concept of the present invention These may be embodied in various forms and are not limited to the embodiments described herein.

Embodiments according to the concept of the present invention can apply various changes and can have various forms, so the embodiments are illustrated in the drawings and described in detail herein. However, this is not intended to limit the embodiments according to the concept of the present invention to specific disclosures, and includes modifications, equivalents, or substitutes included in the spirit and scope of the present invention.

Terms such as first or second may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another component, for example, without departing from the scope of rights according to the concept of the present invention, a first component may be named a second component, Similarly, the second component may also be referred to as the first component.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle. Expressions describing the relationship between components, such as "between" and "directly between" or "directly adjacent to" should be interpreted similarly.

Terms used in this specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to designate that the described feature, number, step, operation, component, part, or combination thereof exists, but one or more other features or numbers, It should be understood that the presence or addition of steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in this specification, it should not be interpreted in an ideal or excessively formal meaning. don't

A module in this specification may mean hardware capable of performing functions and operations according to each name described in this specification, or may mean computer program code capable of performing specific functions and operations, , or an electronic recording medium loaded with computer program codes capable of performing specific functions and operations, for example, a processor or a microprocessor.

In other words, a module may mean a functional and/or structural combination of hardware for implementing the technical concept of the present invention and/or software for driving the hardware.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, the scope of the patent application is not limited or limited by these examples. Like reference numerals in each figure indicate like elements.

1A is a diagram for explaining an exercise information providing system according to various embodiments.

Referring to FIG. 1A , an exercise information providing system 100 may include a first electronic device 110 , a second electronic device 120 , a sensing device 200 and a user terminal 300 .

According to various embodiments, the first electronic device 110 may include a smart phone, a mini PC, a tablet PC, a portable multimedia device, and the like, and the second electronic device 120 may include a personal computer, a server, and the like. there is. Examples of the above-described first electronic device 110 and second electronic device 120 are only examples, and the first electronic device 110 and the second electronic device 120 may be any electronic device capable of processing calculations. can be implemented.

The first electronic device 110 is a device that performs a primary operation on the data received from the sensing device 200, and can perform calculations with a smaller load than the second electronic device 120. Accordingly, the first electronic device 110 may be implemented as an electronic device having a smaller computing power than the second electronic device 120 .

According to various embodiments, the first electronic device 110 may receive sensing information from the sensing device 200 .

According to various embodiments, the sensing device 200 may include at least one of an arbitrary sensor such as an image capture device, an RFID tag device, and a footrest pressure sensor disposed in a fitness center. The image capture device may include a camera disposed on an individual exercise machine or in an arbitrary area in the fitness center including the ceiling, and may acquire images of users in the fitness center in real time. According to an implementation example, the image capture device has a structure including a front image capture unit acquiring a front image of the user and a side image acquisition unit acquiring a side image of the user (an additional image capture unit may be included according to an example). It may be implemented as, and may be combined with a display (or kiosk) displaying the acquired image to the user. In addition, the RFID tag device may be disposed on individual exercise equipment to collect information about exercise equipment used by each user in the fitness center and characteristics (type, weight, etc.) related to exercise through tagging of individual users.

According to various embodiments, the sensing device 200 provides sensing information including an image of the user, weight information of the user measured through a pressure sensor disposed on the floor, RFID tagging information, and information collected through an arbitrary sensor. Once obtained, the obtained sensing information may be transmitted to the first electronic device 110 .

According to various embodiments, upon receiving sensing information, the first electronic device 110 may identify a user corresponding to the sensing information and provide exercise information to the identified user in association with the second electronic device 120. there is. For example, in a situation where A, a member of a fitness center, uses a bench press, the sensing device 200 acquires an image of A and transmits the image to the first electronic device 110, and the first electronic device 110 A may be identified based on the image of A, and exercise information may be provided to the identified A. The exercise information may be provided through a display provided in the user terminal 300 of A or the sensing device 200 disposed in the exercise machine. The exercise information may include, but is not limited to, information on the amount of exercise of the user, an exercise method suitable for the user, an exercise routine, information on the probability of achieving a target fat mass and a target muscle mass within a target date, etc. Any exercise information that can be provided may be included.

According to an embodiment, the first electronic device 110 identifies the current user's face from an image in which the user's face appears, and sends sensing information to the first artificial neural network pretrained to de-identify the face of another user in the image. By inputting (images of individual users), the privacy of other users included in the images can be protected. De-identification processing may include arbitrary image processing that prevents other users from being identified through images, and may include, for example, blur processing, mosaic processing, and character replacement processing for other users' faces. there is. According to an embodiment, the first artificial neural network may determine a user recognized as being closest to the camera as the current user and perform de-identification processing on the remaining users. According to another embodiment, registration for the current user may be performed. Through the procedure, it is also possible to de-identify users other than the current registered user. For example, when logging in to the first electronic device 110, the first electronic device 110 recognizes the closest face, tracks the recognized face, and blurs faces other than the recognized face, as described above. de-identification can be performed. The first electronic device 110 may transmit, to the second electronic device 120 , data for which image processing has been completed based on the first artificial neural network. Through this, the second electronic device 120 can only perform analysis based on de-identified data and cannot check the original video, so privacy of users other than the current user can be protected to a high level. According to embodiments, when the computational power of the first electronic device 110 is not sufficient, the de-identification process based on the first artificial neural network may be directly performed through the second electronic device 120 . In this case, the second electronic device 120 may perform a de-identification process based on the first artificial neural network and a series of subsequent processes.

In addition, the second electronic device 120 generates exercise information related to the exercise performed by the current user in real time based on the current user identification result in the previous de-identification process, and provides the generated exercise information to the current user. database can be made.

If the user is wearing a mask or the learning is not advanced, the recognition performance of the first artificial neural network may be degraded. A solution to the above issue can be provided by identifying individual users using information collected through tags.

According to various embodiments, detailed operations of the first electronic device 110 and the second electronic device 120 will be described in more detail through accompanying drawings.

1B is a diagram for explaining an exercise information providing system according to various embodiments by way of example.

Referring to FIG. 1B , the exercise information providing system 100 may include a first electronic device 110 , a second electronic device 120 , a sensing device 200 and a user terminal 300 .

In the presented example, the first electronic device 110 is implemented as a mini PC that has no problem in user recognition and de-identification processing even though its computing power is relatively small, and the second electronic device 120 analyzes the movements of the user and exercise equipment. As a possible cloud server, it can be implemented as a device with greater computing power than the first electronic device 110 . In the presented example, the sensor device 200 may be implemented in a form in which a 2-channel camera and a touch screen displaying a shooting situation are combined. The first electronic device 110 identifies the current user from the image and performs de-identification processing of the input image (first information) through the first artificial neural network that de-identifies the faces of other users to obtain second information. may be generated and provided to the second electronic device 120 . The second electronic device 120 may obtain motion information of the user and exercise equipment based on the second artificial neural network and the third artificial neural network based on the second information, and generate exercise assistance information based on the obtained information. Since the first artificial neural network requires less computation than the second artificial neural network, in the exercise information providing system 100, the first electronic device 110 has the advantage of being implemented as a computing device having relatively little computing power, Privacy issues that may occur in information provided to the second electronic device 120 may be reduced.

The second electronic device 120 may provide exercise-related information described below to the user by inputting the received second information to the second artificial neural network and the third artificial neural network. The user may obtain exercise-related auxiliary information by uploading or downloading an image to the first electronic device 120 through the user terminal 300 .

2 is a diagram for explaining a configuration of an electronic device according to various embodiments.

Referring to FIG. 2 , the first electronic device 110 may include a memory 111 , a communication module 112 and a processor 113 . The second electronic device 120 may include a memory 121 , a communication module 122 and a processor 123 .

According to various embodiments, the memories 111 and 121 may be used by at least one component (eg, the communication module 112 and 122 or the processor 113 and 123) of the electronic device 110 and 120. data can be stored. The data may include, for example, input data or output data for software (eg, a program) and commands related thereto. The memories 111 and 121 may include volatile memory or non-volatile memory.

The communication modules 112 and 122 establish a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 110 and 120 and an external electronic device (eg, the sensing device 200 or the user terminal 300), And it can support communication through the established communication channel. The communication modules 112 and 122 may include one or more communication processors that operate independently of the processors 113 and 123 (eg, application processors) and support direct (eg, wired) communication or wireless communication. According to one embodiment, the communication module 112, 122 is a wireless communication module (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (eg, a local area network (LAN)). area network) communication module or power line communication module). Among these communication modules, the corresponding communication module is a first network (eg, a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (eg, a legacy cellular network, a 5G network, It may communicate with an external electronic device (eg, the sensing device 200 or the user terminal 300) through a next-generation communication network, the Internet, or a long-distance communication network such as a computer network (eg, LAN or WAN). These various types of communication modules may be integrated as one component (eg, a single chip) or implemented as a plurality of separate components (eg, multiple chips).

The processors 113 and 123, for example, execute software (eg, a program) to include at least one other component (eg, hardware or software configuration) of the electronic devices 110 and 120 connected to the processors 113 and 123. element), and can perform various data processing or calculations. According to one embodiment, as at least part of data processing or calculation, the processors 113 and 123 may receive instructions or data from other components (eg, memories 111 and 121 or communication modules 112 and 122). It may store in volatile memory, process commands or data stored in volatile memory, and store resultant data in non-volatile memory. According to an embodiment, the processors 113 and 123 may include a main processor (eg, a central processing unit or an application processor) or a secondary processor (eg, a graphic processing unit, a neural processing unit (NPU) that may operate independently of or together with the main processor) unit), image signal processor, sensor hub processor, or communication processor). For example, when the electronic devices 110 and 120 include a main processor and an auxiliary processor, the auxiliary processor may use less power than the main processor or may be set to be specialized for a designated function. A secondary processor may be implemented separately from, or as part of, the main processor.

3 is a diagram for explaining a method of providing exercise information performed through a first electronic device and a second electronic device according to various embodiments.

Referring to FIG. 3 , in step 310, the first electronic device 110 may receive first information corresponding to sensing information from the sensing device 200. For example, the first information may be an image obtained through an image capture device disposed in a fitness center, and additionally, any sensor such as tagging information obtained through RFID or weight information of a user obtained through a pressure sensor. Information acquired through may be further included. According to the embodiment, the user may perform labeling of the exercise item/weight included in the exercise image through the first electronic device 110, and the labeling result is the advancement of the third artificial neural network and the second information, which will be described below. can be used for creation.

In step 320, the first electronic device 110 may generate second information by performing de-identification processing on other users except the current user in the first information. A specific method of generating the second information through de-identification processing of the first information (sensing information) may be the same as the method described above with reference to FIG. 1 . The first electronic device 110 may transmit the generated second information to the second electronic device 120 .

In step 330, the second electronic device 120 uses the pre-learned artificial neural network to extract third information related to the current user's movement information and fourth information related to the exercise equipment currently used by the user from the second information. can create More specifically, the second electronic device 120 may generate third information including motion information of the current user based on a second artificial neural network pretrained to estimate a pose of an individual user in an input image. The second artificial neural network may be trained to estimate the user's pose by being distinguished according to the type of exercise. The second artificial neural network exemplarily estimates the user's pose by referring to the method of a reference document (Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields, Zhe Cao, Tomas Simon et al. 2017) disclosing a skeleton-based pose estimation method However, the second artificial neural network learning method is not limited thereto and may be implemented through any method of estimating a pose using an artificial neural network.

The second electronic device 120 may generate counting information (fifth information) (eg, information about the number of repetitions of an individual exercise) based on the third information. For example, the second electronic device 120 determines whether the knee angle of the user performing the squat satisfies a predetermined critical condition (eg, whether the knee angle forms an angle within 100 degrees). It is possible to generate counting information based on. The second artificial neural network may be separately learned for each exercise to estimate a pose for any exercise other than squat, and the second electronic device 120 generates counting information based on the second artificial neural network corresponding to each exercise. can do.

In addition, the second electronic device 120 may generate fourth information related to an exercise device currently used by the user from the second information by utilizing a pre-learned artificial neural network. More specifically, the second electronic device 120 identifies exercise equipment used by individual users in the input image, generates weight information of the identified exercise equipment, and prelearns to estimate motion information of the exercise equipment. 3 The fourth information may be generated based on an artificial neural network. The weight information may be information about the weight of an exercise device such as a disk, a barbell, or a bar used for individual exercise, and the motion information may include motion information including a movement trajectory and a movement distance of the exercise device. For example, the third artificial neural network recognizes the exercise equipment from the second information, and the 3-1 artificial neural network generates weight information about the exercise machine and the third artificial neural network generates motion (pose) information of the identified exercise machine. -2 May include artificial neural networks. The 3-1 artificial neural network may be implemented based on a convolution neural network (CNN), and the 3-2 artificial neural network may be exemplarily implemented using a skeleton-based motion estimation scheme. Training data (for example, images containing exercise equipment) for training the 3-1 artificial neural network may be in the form of exercise equipment (eg, discs, barbells, etc.) implemented in different colors and sizes for each weight. In addition, since the difference in features of exercise equipment for each weight increases, the prediction performance of the 3-1 artificial neural network can be improved. In addition, the 2-2 artificial neural network is trained to estimate a pose according to the movement of the exercise machine through a skeleton model using an image in which nodes are labeled on the force point and weight part (eg, disc) of the exercise machine as training data. can For example, learning data for learning the 2-2 artificial neural network may be shown as shown in FIG. 4 to be attached below, and motion of the exercise device may be estimated based on the skeleton model-based pose estimation method described above.

The second electronic device 120 may generate counting information (fifth information) (eg, counting the number of squats according to the movement of the barbell, etc.) based on the generated fourth information. For example, counting information based on the fourth information may be generated based on whether a preset specific pose is reached.

In the previous example, the 3-1 artificial neural network and the 3-2 artificial neural network were distinguished and described in the direction of learning separately, but the embodiment is not limited thereto, and the second artificial neural network is implemented based on a single neural network. It will be appreciated by those skilled in the art.

In step 340, the second electronic device 120 may provide the generated third information and fourth information to an external entity.

The second electronic device 120 may generate and provide fifth information, which is auxiliary information capable of assisting the user's exercise, based on the third information and the fourth information.

Fifth information is information related to exercise and its performance, such as counting information including the number of repetitions of individual exercises, information on exercise amount, information on prediction trend of body composition changes according to exercise/meal changes, and information about probability of reaching a target weight, etc. It may contain any relevant information.

The second electronic device 120 may generate fifth information by further considering characteristic information of the current user. For example, the feature information may include the user's age, gender, height, weight, total muscle mass, muscle mass by part, body fat mass, whether or not injured, target weight, target muscle mass and target body fat mass, target achievement time point, physical disposition (e.g., endoderm , mesoderm, ectoderm, etc.) may include at least one of information.

The second electronic device 120 may generate momentum information of the current user based on the previously obtained third and fourth information. More specifically, the second electronic device 120 calculates the amount of exercise based on counting information calculated on the basis of the previously obtained third information and the weight information of the exercise equipment calculated on the basis of the fourth information. information can be generated.

According to an embodiment, the second electronic device 120 provides actual movement of the current user (eg, movement distance information of the center of gravity) and actual movement of the exercise equipment (eg, movement of the center of gravity), rather than counting information. By calculating the amount of exercise based on distance information), it is possible to provide the user with more accurate information about the amount of exercise.

The movement distance information is information about a movement distance for a movement distance between a center of gravity of an exercise device used for exercise and a center of gravity of a user, and may be calculated based on a movement distance in a direction of gravity according to an exercise type. The second electronic device 120 may obtain movement distance information of the center of gravity of the user and the exercise machine based on the previously acquired third information on the pose of the user and fourth information on the pose of the exercise machine. For example, when a user squats, even if 10 times are counted, the actual movement distance of the exercise equipment (bar, etc.) may be different in individual counting. Since different movement distances may affect the measurement of momentum, the second electronic device 120 may reflect movement distance information in the process of calculating momentum information.

According to an embodiment, the second electronic device 120 may calculate momentum information based on Equation 1.

는 운동량 정보,

는 중량 정보,

는 중력 가속도,

는 이동 거리 정보를 의미할 수 있다. 제2 전자 장치(120)는 앞선 수학식 1을 사용자 및 운동 기구 각각에 적용하여 사용자의 운동량에 대한 I1 및 운동기구의 운동량에 I2 를 개별적으로 산출하여 합산함으로써, 운동량에 산출의 정확도를 향상시킬 수 있다.

is the momentum information,

is the weight information,

is the gravitational acceleration,

may mean movement distance information. The second electronic device 120 applies Equation 1 above to each of the user and the exercise machine to individually calculate I1 for the exercise amount of the user and I2 for the exercise amount of the exercise machine and add them together, thereby improving the accuracy of calculating the exercise amount. can

According to another embodiment, the second electronic device 120 may calculate the center of gravity of the user and the entire exercise equipment, and calculate the amount of exercise based on the movement distance of the center of gravity. For example, the second electronic device 120 may determine the position of the center of gravity of the user and the entire exercise machine based on Equation 2.

는 사용자와 운동기구 전체의 무게 중심,

는 사용자의 체중,

는 사용자의 무게 중심의 위치,

는 운동기구의 중량,

는 운동기구의 무게 중심일 수 있다.

is the center of gravity of the user and the entire exercise equipment,

is the user's weight,

is the position of the user's center of gravity,

is the weight of the exercise equipment,

May be the center of gravity of the exercise equipment.

In the case of the center of gravity of a person and an exercise machine, the sensing information of the pressure sensor disposed on the floor described above, the pose prediction information calculated through the second artificial neural network, and the pose prediction information calculated through the third artificial neural network are combined and calculated. It can be.

In free weight exercises such as squats, it is important for the exercise effect and safety to stand up with the center of gravity behind with the feeling of pushing the ground in the direction of the heel in a situation where you stand up after sitting down. In the case of the second artificial neural network for analysis of squats, ankle In addition to the node, a second artificial neural network to which separate nodes are assigned to the heel (heel) and the forefoot (toe) may be utilized.

According to another embodiment, the momentum information may be calculated in a form in which weight information and counting information are mapped. For example, the second electronic device 120 may determine the exercise amount information by squatting 10 times with a weight of 100 kg for the squatting user.

In the case of stretching or bare-body exercise, exercise amount information may be calculated based on motion information of the current user included in the third information. In the case of aerobic exercise such as running on a treadmill or cycling, momentum information may be calculated based on the user's motion information described above. For example, based on the movement speed, information on the amount of exercise for stretching, bare-body exercise, and aerobic exercise may be calculated.

In addition, exercise amount information may be calculated based on measurement information obtained through a measurement device (eg, a heart rate measurement device) disposed in each exercise device. Measurement information itself may be calculated as momentum information, or momentum information may be calculated through some error correction process.

In addition to the information described above, exercise amount information includes information on the time during which an action was performed, information on rest time for each set, information on the exercise part, information on whether or not it is aerobic exercise, information on calories consumed, and information on exercise intensity. may be further included as additional information.

The second electronic device 120 may obtain the user's calorie intake information, which may be calculated based on analysis of the user's food intake image received from the user terminal 300 . According to an embodiment, the second electronic device 120 inputs the food intake image received from the user terminal 300 to the fourth artificial neural network previously trained to calculate calories based on the food image, thereby providing calorie intake information. can be obtained The intake calorie information may be corrected to an appropriate number based on a user input through the user terminal 300 . Intake calorie information may be calculated in the form of calorie information for each component (carbohydrate, protein, fat) in addition to total calorie information.

According to another embodiment, the calorie intake information may be provided as a numerical value previously determined by a company providing food. For example, when a user consumes a lunch menu of a cafeteria of a specific company, the second electronic device 120 receives predetermined calorie information for the cafeteria lunch menu from an external server, and based on the received information Intake calorie information may be acquired.

Fifth information, which is the auxiliary information described above, may be time-sequentially accumulated and stored in a database. Through the databased information, individual user's characteristic information, exercise amount information, and calorie intake information can be inquired for each time period.

The second electronic device 120 may generate probability information to achieve target body fat and target muscle mass within a period preset by the user based on at least one of user characteristic information, exercise amount information, and calorie intake information. In addition to the probability information described above, the fifth information may further include information on an amount of exercise required for each period to reach a goal, information on a recommended calorie intake per period, and information on a recommended exercise routine. Fifth information may further include information about a body part lacking in exercise in the current state.

For example, the second electronic device 120 takes characteristic information (eg, current weight information), exercise amount information, and calorie net consumption information as inputs and outputs fifth information based on a fifth artificial neural network pretrained. Thus, fifth information may be generated. A method of utilizing the fifth artificial neural network will be described in detail with reference to FIG. 6 to be attached below.

Fifth information including user characteristic information, exercise amount information, calorie intake information, and exercise assistance information described above may be databased in a storage device linked with the electronic device 100, a display disposed on an exercise machine, and a user's It may be provided through the user terminal 300 . The form of provided data can be implemented in any form such as text/voice. It will be appreciated by those of ordinary skill in the art that the provided data may be implemented in any presentation manner that can substitute for personal training. For example, the exercise amount information may be provided in the form of a percentile of the position within the entire user using the exercise information providing system, and the trend of change in the user's characteristic information related to the user's body fat mass and muscle mass is expressed in graphs and charts Also, counting information for real-time exercise and auxiliary information for posture correction may be provided as text/voice.

Also, the second electronic device 120 may provide recommended music information corresponding to the user's exercise pattern or heart rate.

For example, the second electronic device 120 may provide music corresponding to BPM (Beat Per Minute), which has a multiple relationship with 1/(cycle) of an individual exercise motion, as recommended music information. Similarly, the second electronic device 120 may provide music corresponding to a BPM that is a multiple of 1/(heart rate) of the user as recommended music information.

In addition, the second electronic device 120 may provide music corresponding to a faster BPM as recommended music information as the exercise intensity (exercise amount information) of the currently performed exercise is stronger.

The second electronic device 120 may obtain an actual exercise image corresponding to a point in time when the user actually performs an exercise from sensing information and provide the actual exercise image to the user.

Images collected through sensing information may include images of a time when the user performs an exercise as well as a time when the user does not exercise. The second electronic device 120 recognizes the start time when the user starts the actual exercise and the end time when the actual exercise ends in the entire image based on the third information about the user's motion (pose), and the start time and end time. An actual exercise image may be obtained based on a viewpoint, and the actual exercise image may be provided to a server or a user terminal. For example, in the case of an exercise in which a specific motion is repeated, such as squat, the second electronic device 120 edits and provides an image from the time the current user starts the exercise to the time the exercise ends, but takes a break in the middle. It is possible to provide the current user with an edited video from which the video of the part taken is removed.

In another example, when the captured exercise image exceeds a preset threshold (eg, 1 minute), the second electronic device 120 determines whether the first part (the first 1/3 of the video), the middle part (the middle 1/3 of the video), etc. length) and the second half (half of the second half), the length of the video can be adjusted at a preset ratio. More specifically, when the preset threshold is 1 minute, the captured image is 90 seconds long, and the ratio set for the first half/middle/second half is 30%/40%/30%, the second electronic device 120 captures Editing can be performed to reduce the amount of 9 seconds at the beginning of the video, 12 seconds at the middle, and 9 seconds at the second half of the video. Editing may be performed based on an arbitrary method, such as a method of adjusting the video speed or a method of removing an amount of rest.

In the process of implementing the previous example, the second electronic device 120 loads the weight (barbell/ It can recognize weight exercise equipment such as dumbbells), and can recognize weight deloading (put down weight exercise equipment). The second electronic device 120 may recognize the time of loading the weight as the start time of starting the exercise, and the time of deloading the weight as the end time of ending the exercise. The second electronic device 120 may acquire actual exercise images corresponding to the recognized start and end times, and store the actual exercise images in a database or provide them to the user terminal.

According to another embodiment, the second electronic device 120 may detect the start and end of an exercise by interlocking with a pressure sensor disposed on the floor. More specifically, it is determined that the exercise has started when the weight of the user increases by more than a predetermined threshold value (for example, 20 kg) through a pressure footrest sensor disposed on the floor where the exercise equipment is located, and in the same way, it is determined that the exercise has started. When the weight decreases, it can be determined that the exercise is over.

In addition, the second electronic device 120 may further provide exercise image recommendation information suitable for the user. More specifically, the second electronic device 120 provides an exercise image targeting a person having similar characteristics to the current user as recommendation information based on the current user's characteristic information, or providing a similar exercise image based on the current user's exercise amount information. Exercise image recommendation information may be provided based on an exercise image related to an exercise in which an exercise quantity is expected to be calculated.

In addition, the second electronic device 120 may provide a guide image, which is a pre-stored exercise image of the instructor, overlapping the user's exercise image. Utilizing the second artificial neural network and the third artificial neural network described above, it is possible to provide overlapping images of mutual movement progress by matching nodes extracted from a user's exercise image with nodes extracted from a previously stored instructor's exercise image. there is. Through this, it is possible to recognize in real time the difference in the posture of the instructor at each stage of the user's exercise. For example, by overlapping a user's exercise image with an instructor's exercise image (eg, the instructor's image may be overlapped with a predetermined level of transparency) and provided, a specific posture (eg, squat progress) is provided. The angle of bending of the knee, the angle of the hip and the leg, etc.) can be effectively recognized. Depending on the embodiment, a means for allowing the user to identify areas requiring improvement in real time may be provided by providing information on the difference between the user and the tutor of important elements together with an image.

In addition, the second electronic device 120 may provide a notification of a point of improvement in real-time posture in the process of overlapping and providing the exercise image of the instructor described above with the exercise image of the user. For example, in the process of performing a squat, if the extent to which the knee moves in front of the toes is important for exercise effectiveness and safety, the second electronic device 120 provides a notification when the knee moves in front of the toes beyond a predetermined threshold. In addition, the final user may provide a notification about the angle of the knee and the like. In the presented example, it was explained that exercise videos and notifications were provided limited to squat for explanation, but important elements (posture) may be monitored depending on the type of exercise, and notifications may be provided according to real-time situations. A person skilled in the art will understand.

According to various embodiments, the feature information further includes exercise propensity information for each user, and the second electronic device 120 may provide recommended exercise route information to a current user entering a fitness center based on the exercise propensity information. . More specifically, the second electronic device 120 may generate movement propensity information for each user by tracking a movement path of an individual user based on the sensing information.

Exercise propensity information includes, for example, information about the type of exercise equipment (Smith machine, treadmill, cycle, etc.) used by individual users, and information about exercise routines (exercise sequence, time required for individual exercise, etc.) can include

The electronic device 120 may determine current occupancy status information of individual exercise equipment based on the sensing information, and occupancy status information, exercise tendency information of a user currently present in the fitness center, and exercise of the current user entering the fitness center. Based on the tendency information, recommended exercise route information may be provided to the current user. More specifically, when the current user has an exercise tendency to give priority to aerobic exercise using a treadmill, the second electronic device 120 prioritizes the treadmill in consideration of the current occupation of the treadmill in the fitness center. An exercise route may be recommended, but if all treadmills in the fitness center are currently occupied, cycles may be guided to a preferential exercise route or prediction information about when the use of the treadmill ends may be provided. If the current user has an exercise tendency to perform chest exercise using a Smith machine after aerobic exercise, the second electronic device 120 provides current occupancy status information of individual exercise equipment, exercise propensity of other users currently present in the fitness center. Based on the information, it is possible to predict whether the Smith machine can be used at the time when the current user completes aerobic exercise, and to provide recommendation information on a second exercise route based on the prediction result. More specifically, if the current user cannot use the Smith machine at the time the current user finishes aerobic exercise in light of the exercise tendencies of other users, the electronic device 120 predicts the possibility of using a bench press exercise machine other than the Smith machine, Can provide information on available bench press machines.

The second electronic device 120 may provide recommended exercise route information so that users may maintain a maximum distance from each other. For example, in the process of recommending a route for lat pull down exercise to user A, the distance between the lat pull down exercise equipment present in the fitness center and other users is measured, and the measured distance is maximized. Recommended exercise path information may be provided in the direction of recommending the lat pull exercise equipment.

In the foregoing embodiments, operations of the second electronic device 120 have been described as being performed based on second information generated based on the first artificial neural network, but depending on implementation, first information rather than second information is used. A person skilled in the art will understand that a series of operations can be performed even through. More specifically, the second artificial neural network and the third artificial neural network may generate third to fifth information using the first information as an input, through which a series of processes described above may be performed.

4 is a diagram for explaining operations of a second artificial neural network and a third artificial neural network according to an embodiment.

According to an embodiment, the second artificial neural network generates third information about motion (pose) information of a user by taking the above-described second information (an image for which de-identification has been performed by another user other than the current user) as an input. can More specifically, as shown in (a), the second artificial neural network extracts nodes 411 corresponding to each joint from the image of the current user, and divides 17 regions 412 based on the extracted nodes. ), the motion (pose) of the user may be estimated. To this end, the second artificial neural network may be trained to estimate each pose by using the labeled image of each node as learning data.

According to an embodiment, the third artificial neural network receives the above-described second information (an image in which de-identification has been performed by another user other than the current user) as an input, and the fourth information about the weight information and motion (pose) information of the exercise machine. information can be generated. More specifically, as shown in (b), the third artificial neural network identifies the exercise mechanism from the second information (identifies the disc area 421 and the bar area 422 and recognizes the disc and bar), and exercises Nodes corresponding to the power point 423 and the center point 424 of the weight part (disc) are extracted from the 3-1 artificial neural network that calculates the weight information of the equipment and the exercise equipment image, and based on the extracted nodes, the exercise equipment It may include a 3-2 artificial neural network estimating the real-time pose of . The 3-1 artificial neural network can be trained based on learning data labeled with the type and weight of the exercise equipment from each image, and the 3-2 artificial neural network uses the image labeled with each node as learning data. It can be learned to estimate each movement (pose) by doing so.

Exemplarily, when an image of a user performing a dead lift is transmitted to the second electronic device 120, the second electronic device 120 identifies the exercise equipment and each element of the exercise equipment through the third artificial neural network. It predicts and provides the weight of (e.g., discs and bars), and the pose of the exercise equipment is divided into three areas ((1) area, (2) area, (3) area divided through areas divided into respective nodes. ), it is possible to estimate the movement (pose) of the exercise equipment. The second electronic device 120 may extract each node through the second artificial neural network and estimate the real-time movement (pose) of the user through 17 distinct regions. In the presented example, the method of labeling each node and the method of identifying exercise equipment is not limited to the presented example, and the number and location of nodes may be set differently depending on the embodiment, and identification according to the type of exercise equipment Structures of elements and nodes may be implemented differently.

5 is a diagram illustrating an example of an interface provided in a process of receiving a user's image in a sensing device according to an exemplary embodiment.

As described above, according to an example, the sensing device may include an image capture device, and the image capture device may include two channels including a front image capture unit and a side image capture unit for acquiring a side image of the user. The presented drawing relates to a display screen (for example, a screen provided through a kiosk disposed with an exercise device, which is an example of the first electronic device 110) linked with a sensing device, as shown by way of example. Similarly, an interface capable of checking images acquired through the front image acquisition unit and the side image acquisition unit in real time may be provided. The corresponding interface may be provided through a display provided in the first electronic device 110 instead of being provided through an app. During the image acquisition process, the current user may obtain an image by selecting weight information related to an exercise item to be acquired. Examples of the type of exercise may include any fitness exercise such as squat, deadlift, and bench press. Additionally, a waiting time can be set through the presented interface, and through this, an effect of providing time for the user to prepare before starting recording can be provided.

Through the previous operation, the user can perform labeling on the type of exercise and exercise weight on the acquired image, and the labeling result can be used in the process of learning and upgrading the artificial neural network described above. 3 In the process of providing information, labeling results can also be automatically recorded and provided together.

6 is a diagram for illustratively explaining a method of providing exercise goal attainment probability information, which is one of fifth information, by a second electronic device according to an embodiment.

According to an embodiment, the second electronic device 120 provides characteristic information (for example, current weight information), exercise amount information (exercise amount performed within a target period), and net calorie consumption information (calories consumed within a target period). Fifth information, which is a probability of achieving a user's exercise goal, may be provided by using a fifth artificial neural network pre-learned to predict achievement of the goal based on the above information. For example, the exercise amount information may be calculated based on the exercise amount measurement using the video described above or information collection using an app/smart watch, and the net calorie consumption information may be calculated using the video-based method described above or the user's app. It can be calculated based on direct input through More specifically, the net calorie consumption information may be calculated based on the difference between the amount of consumption (A) and the amount of input (calorie intake) (B) calculated based on the sum of the basal metabolic rate and the amount of exercise, and the fifth artificial neural network may be It may be learned to estimate the predicted weight by using, as an input, information on the amount of exercise calculated on a weekly basis and information on net calorie consumption calculated on a weekly basis. For example, the fifth artificial neural network may utilize a time series-specific AI model such as RNN or may be implemented as a multivariate time series model (eg, Latent ODE, etc.).

In the previous example, it was described that the current weight is input as a user characteristic to the fifth artificial neural network, but according to implementation, feature information includes age, gender, height, weight, total muscle mass, muscle mass by part, body fat mass, whether or not injured, target weight, Learning may be performed by inputting a target muscle mass, a target body fat mass, a target achievement time point, and a physical tendency (eg, endoderm, mesodermal, ectodermal, etc.).

Referring to FIG. 6 , consumption and input amount may be recorded daily as shown in table 610 to calculate weekly statistics, and body composition measurements may also be performed and recorded for each week as shown in table 620 . The second electronic device 120 may predict a corresponding probability through the normalization value calculated based on Equation 3 and the normal distribution model 630 .

는 목표 체중,

은 제5 인공 신경망의 예측 체중, z는 정규화값일 수 있다.

is the target weight,

is the predicted weight of the fifth artificial neural network, and z may be a normalization value.

For example, when the target weight is 82kg and the predicted weight of the fifth artificial neural network is 83kg, the z value may be calculated as -0.0122 based on Equation 3, and this is reflected in the normal distribution model 630 to target the target weight. The achievement probability can be calculated as 95%. In the normal distribution model 630, data is accumulated according to the user's characteristics, and its shape may change.

According to an embodiment, the second electronic device 120 may provide a goal achievement probability by using the previously described fifth artificial neural network and also provide a degree of correlation between individual elements by using an additional artificial neural network. there is. For example, the second electronic device 120 may include a correlation between body weight and calorie intake, a correlation between body weight and exercise amount, a correlation between body fat and calorie intake, a correlation between body fat and exercise amount, a correlation between muscle mass and calorie intake, and The correlation between muscle mass and exercise volume can also provide information.

7 is a diagram for explaining the operation of a method for providing exercise information according to various embodiments.

According to various embodiments, referring to FIG. 7 , the sensing device 200 may acquire (701) sensing information through an arbitrary sensor and transmit it to the first electronic device 110 (702). If the sensing information is an image of a user, the first electronic device 110 acquires second information through de-identification of a user other than the user (703) and transmits it to the second electronic device 120. can (704). The second electronic device 120 generates third information related to the movement of the user and fourth information related to the weight and movement of the exercise equipment by utilizing the above-described second artificial neural network and the third artificial neural network (705), and generates them. It can be provided to the user terminal 300 . In addition, the second electronic device 120 may perform a series of operations described above.

The user terminal 300 according to an embodiment is a device that receives a predetermined command from a user and drives a corresponding operation, and may be a digital device that includes an audio output function, a wired/wireless communication function, or other functions. According to one embodiment, the user terminal 240 includes a memory means such as a tablet PC, a smart phone, a personal computer (eg, a notebook computer, etc.), a smart TV, a mobile phone, a navigation device, a web pad, a PDA, a workstation, and the like. It may be a concept that includes all digital devices equipped with a microprocessor and equipped with an arithmetic capability.

The user terminal 300 according to an embodiment may mean any user device capable of installing and executing an exercise information providing application related to the second electronic device 120 . At this time, the user terminal 300 may perform overall service operations such as configuration of a service screen, data input, data transmission/reception, and data storage under the control of the exercise information providing application. The diagnosis application is implemented to be usable in the mobile environment as well as the PC environment, and is implemented in the form of a program that operates independently or is configured in the form of an in-app of a specific application and implemented so that it can operate on the specific application. It can be.

8A illustrates the structure of an artificial neural network (eg, a Convolution Neural Network (CNN)) that receives input data and outputs output data. The artificial neural network may be a deep neural network having two or more layers, and the artificial neural network described above may be implemented in the form illustrated in the above figure.

8B is a diagram for explaining a learning and reasoning method of an artificial neural network model according to an embodiment.

Referring to FIG. 8B , an exercise information providing system according to an embodiment may include a learning device 800 and an exercise information providing device 850. The learning apparatus 800 according to an embodiment performs various processing functions such as functions of generating a neural network, training (or learning) a neural network, or retraining a neural network. It corresponds to a computing device having For example, the learning device 800 may be implemented in various types of devices such as a personal computer (PC), a server device, and a mobile device.

The learning apparatus 800 according to an embodiment may generate one or more trained neural networks 810 by repeatedly training (learning) a given initial neural network. Creating one or more trained neural networks 810 may mean determining neural network parameters. Here, the parameters may include, for example, various types of data input/output to the neural network, such as input/output activations, weights, and biases of the neural network. As the iterative training of the neural network progresses, the parameters of the neural network can be tuned to compute a more accurate output for a given input.

The learning device 800 according to an embodiment may transmit one or more trained neural networks 810 to the exercise information providing device 850 . The exercise information providing device 850 may be included in a mobile device, an embedded device, or the like. The exercise information providing device 850 according to an embodiment may be the first electronic device 110 or the second electronic device 120 described above.

The exercise information providing apparatus 850 according to an embodiment drives one or more trained neural networks 810 as they are or processes (eg, quantized) one or more trained neural networks 860. ) can be driven. The exercise information providing device 850 that drives the processed neural network 860 may be implemented in a separate and independent device from the learning device 800 . However, it is not limited thereto, and the exercise information providing device 850 may be implemented in the same device as the learning device 800 . The artificial neural network described above may correspond to the neural networks 810 and 860 described through this figure.

The devices described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA) , a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. The device can be commanded. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program commands recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (10)

In the exercise information providing method performed by the second electronic device,
generating exercise information of the current user by inputting first information corresponding to the current user's exercise image to a pre-learned artificial neural network; and
providing the athletic information to an external entity;
including,
Generating the exercise information,
obtaining information about the current motion of the user through a second artificial neural network previously trained to estimate a pose of the user from an input image;
obtaining information about movement of an exercise machine currently used by the user through a third artificial neural network pretrained to estimate a pose of the exercise machine from an input image; and
Generating exercise assistance information based on information about the current user's motion and information about the motion of the exercise equipment
including,
Generating the exercise information,
The movement information of the center of gravity of the current user calculated based on the information on the movement of the current user and the center of gravity of the exercise equipment used by the current user calculated based on information on the movement of the exercise equipment used by the current user Calculating the amount of exercise based on the movement information of the, exercise information providing method. According to claim 1,
Generating the exercise information,
Receiving or generating second information by which a user other than the current user is de-identified from a first electronic device;
Generating exercise information of the current user by inputting the second information to a pre-learned artificial neural network.
including,
The second information,
A method for providing exercise information, which is generated by inputting the first information to a first artificial neural network to recognize the current user and de-identifying users other than the current user.
According to claim 2,
Learning the second artificial neural network or the third artificial neural network based on a labeling result for at least one of an exercise event or a weight performed by a user on an exercise image through the first electronic device.
Further comprising, exercise information providing method.
According to claim 1,
The third artificial neural network,
a 3-1 artificial neural network learned to estimate an exercise machine and weight based on first learning data corresponding to an image including exercise machines distinguished by different colors and sizes for each weight; and
The 3-2 artificial neural network learned to estimate the pose of the exercise machine based on the second training data, which is an image in which nodes are labeled on the force point and weight of the exercise machine.
Including, exercise information providing method.
delete According to claim 1,
Overlapping and providing a guide image to the current user's image based on matching between the current user's node calculated in the current user's motion information and the pre-stored instructor's node
Further comprising, exercise information providing method.
According to claim 2,
Providing recommended music with BPM music in a multiple relationship with the reciprocal of the user's heart rate or BPM music in a multiple relationship with the reciprocal of the cycle of an individual exercise motion
Further comprising, exercise information providing method.
According to claim 2,
Recognizing a start time when the user starts the exercise and an end time when the user ends the exercise from sensing information acquired through a pressure sensor disposed on the floor;
obtaining an actual exercise image corresponding to a time when a user actually performs an exercise based on the start time and the end time; and
providing the actual motion image to an external entity;
Including, exercise information providing method.
According to claim 1,
The step of generating the exercise information of the current user,
Generating information about a probability of reaching a target value by inputting the current user's characteristic information, exercise amount information, and net calorie consumption information into a fourth artificial neural network.
Including, exercise information providing method.
In an electronic device providing exercise information,
Memory; and
processor
including,
the processor,
Receiving second information that de-identifies other users other than the current user in the first information corresponding to the user's exercise image from the first electronic device;
By inputting the second information into a pre-learned artificial neural network, generating exercise information of the current user;
providing the athletic information to an external entity;
The second information,
Generated by inputting the first information into a first artificial neural network to recognize the current user and de-identifying users other than the current user,
the processor,
Obtaining information on the current motion of the user through a second artificial neural network pretrained to estimate a pose of the user from an input image;
Obtaining information about motion of an exercise machine currently used by the user through a third artificial neural network pretrained to estimate a pose of the exercise machine from an input image;
Generating exercise assistance information based on the information about the current user's motion and the information about the motion of the exercise equipment;
the processor,
The movement information of the center of gravity of the current user calculated based on the information on the movement of the current user and the center of gravity of the exercise equipment used by the current user calculated based on the information on the movement of the exercise equipment used by the current user An electronic device that calculates momentum based on movement information.

Images