KR102108180B1 - Method and wearable device for providing feedback on exercise - Google Patents

Abstract

A wearable device that provides feedback on exercise includes a sensor data collection unit that collects a plurality of sensor data measured from a plurality of sensors embedded in the wearable device, and sensors the collected plurality of sensor data through data fusion. A sensor fusion data generating unit that generates fusion data, detects a feature section from the sensor fusion data, and recognizes an exercise being performed by a user based on the similarity between the data for the feature section and a plurality of pre-stored movement pattern data And an output unit that outputs feedback on an exercise being performed by the user through a display based on the recognized portion and the recognized exercise.

Description

A wearable device and method for providing feedback on exercise. {METHOD AND WEARABLE DEVICE FOR PROVIDING FEEDBACK ON EXERCISE}

The present invention relates to a wearable device and method for providing feedback on exercise.

A wearable device means a wearable device, and science and technology advances to develop wearable smart devices such as T-shirts, glasses, bracelets, watches, and shoes. A typical wearable device is a smart watch, and the smart watch has an existing clock function that tells the time, and a function to check notifications, e-mails, and messages in real time without checking the smartphone has been added.

In relation to a wearable device providing such a smart function, Korean Patent No. 10-1689368, which is a prior art, discloses a wearable device.

Recently, wearable devices provide health-related services using data measured through various built-in sensors. However, this has limitations in that it is difficult to provide intuitive feedback related to exercise when the user is exercising because it is limited to analysis information on the number of steps, movement distance, and exercise time of the user.

The present invention is to solve the above-described problem, generates a plurality of sensor data measured using a variety of sensors embedded in the wearable device as sensor fusion data through data fusion, and for the feature section detected from the sensor fusion data An object of the present invention is to provide a wearable device and method for providing feedback on an exercise for recognizing an exercise being performed by a user based on the similarity between the data and a plurality of pre-stored exercise pattern data.

When a user is aware of an exercise being performed, an object of the present invention is to provide a wearable device and method for providing feedback about an exercise provided with feedback on the exercise to a wearable device and a user terminal.

However, the technical problems to be achieved by the present embodiment are not limited to the technical problems as described above, and other technical problems may exist.

As a means for achieving the above technical problem, an embodiment of the present invention, a sensor data collection unit for collecting a plurality of sensor data measured from a plurality of sensors embedded in the wearable device, the collected plurality of sensor data A sensor fusion data generator that generates sensor fusion data through data fusion, detects a feature section from the sensor fusion data, and based on the similarity between the data for the feature section and a plurality of pre-stored motion pattern data. It is possible to provide a wearable device including an exercise recognition unit for recognizing an exercise being performed by a user and an output unit for outputting feedback on the exercise being performed by the user through a display based on the recognized exercise.

According to another embodiment of the present invention, collecting a plurality of sensor data measured from a plurality of sensors embedded in a wearable device, and generating sensor fusion data through the collected plurality of sensor data through data fusion. Step, detecting a feature section from the sensor fusion data, and recognizing the exercise being performed by the user based on the similarity between the data for the feature section and a plurality of pre-stored exercise pattern data and based on the recognized exercise It is possible to provide a method for providing exercise feedback, including outputting feedback on a workout being performed by a user through a display.

According to another embodiment of the present invention, receiving exercise data for an exercise performed by a user from a wearable device, storing the received exercise data in comprehensive exercise data for each exercise, based on the stored composite exercise data And providing feedback to the wearable device, wherein the motion data is collected from a plurality of sensor data measured from a plurality of sensors embedded in the wearable device, and the collected plurality of sensor data is used for data fusion. ) Through which sensor fusion data is generated, a feature section is detected from the sensor fusion data, and a movement being performed by a user is recognized based on the similarity between the data for the feature section and a plurality of pre-stored movement pattern data. A method of providing feedback can be provided.

The above-described problem solving means are merely examples, and should not be construed as limiting the present invention. In addition to the exemplary embodiments described above, there may be additional embodiments described in the drawings and detailed description of the invention.

According to any one of the above-described problem solving means of the present invention, a plurality of sensor data measured using various sensors embedded in a wearable device is generated as sensor fusion data through data fusion, and from sensor fusion data It is possible to provide a server and method for providing feedback on an exercise for recognizing an exercise being performed by a user based on the similarity between the detected feature data and a plurality of pre-stored exercise pattern data.

When a user's exercise is recognized, it is possible to provide a server and a method for providing feedback on the exercise to provide feedback on the exercise to the wearable device and the user terminal.

1 is a block diagram of an exercise feedback providing system according to an embodiment of the present invention.
2 is a block diagram of a wearable device according to an embodiment of the present invention.
3 is an exemplary view showing a plurality of sensor data measured by a plurality of sensors according to an embodiment of the present invention.
4 is an exemplary diagram illustrating sensor fusion data generated through data fusion of a plurality of sensor data according to an embodiment of the present invention.
5A and 5C are exemplary views for explaining a process of recognizing an exercise being performed by a user according to an embodiment of the present invention.
6A and 6B are exemplary diagrams illustrating feedback about an exercise being performed by a user in a wearable device and a user terminal according to an embodiment of the present invention.
7 is an exemplary diagram for explaining a process of registering new exercise data according to an embodiment of the present invention.
8 is a flowchart of a method for providing feedback about an exercise in a wearable device according to an embodiment of the present invention.
9 is a configuration diagram of a server according to an embodiment of the present invention.
10 is a flowchart of a method for providing feedback on an exercise in a server according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present invention pertains may easily practice. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. In addition, in order to clearly describe the present invention in the drawings, parts irrelevant to the description are omitted, and like reference numerals are assigned to similar parts throughout the specification.

Throughout the specification, when a part is "connected" to another part, this includes not only "directly connected" but also "electrically connected" with another element in between. . Also, when a part is said to "include" a certain component, it means that the component may further include other components, not exclude other components, unless specifically stated otherwise. However, it should be understood that the existence or addition possibilities of numbers, steps, actions, components, parts or combinations thereof are not excluded in advance.

In the present specification, the term “unit” includes a unit realized by hardware, a unit realized by software, and a unit realized by using both. Further, one unit may be realized by using two or more hardware, and two or more units may be realized by one hardware.

Some of the operations or functions described in this specification as being performed by a terminal or device may be performed instead on a server connected to the corresponding terminal or device. Similarly, some of the operations or functions described as being performed by the server may be performed in a terminal or device connected to the corresponding server.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of an exercise feedback providing system according to an embodiment of the present invention. Referring to FIG. 1, the exercise feedback providing system 1 may include a wearable device 110, a server 120, and a user terminal 130. The wearable device 110, the server 120, and the user terminal 130 exemplarily show components that can be controlled by the exercise feedback providing system 1.

Each component of the motion feedback providing system 1 of FIG. 1 is generally connected through a network. For example, as shown in FIG. 1, the wearable device 110 or the user terminal 130 may be connected to the server 120 simultaneously or at a time interval.

Network refers to a connection structure capable of exchanging information between nodes such as terminals and servers, and a local area network (LAN), a wide area network (WAN), and the Internet (WWW: World) Wide Web), wired and wireless data communication networks, telephone networks, and wired and wireless television communication networks. Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi, Bluetooth communication, infrared communication, ultrasound Communication, Visible Light Communication (VLC), LiFi, and the like are included, but are not limited thereto.

The wearable device 110 may be installed and executed with an app that provides exercise feedback. At this time, the wearable device 110 may run an app for providing exercise feedback based on the location information of the wearable device 110. For example, when a user visits a specific place (eg, a fitness center, etc.), an app for automatically providing exercise feedback in the wearable device 110 may be executed.

The wearable device 110 is worn on the user's wrist and may sense the user's movement through a plurality of sensors. The wearable device 110 may collect a plurality of sensor data measured from a plurality of sensors embedded in the wearable device 110. At this time, the wearable device 110 may remove noise from a plurality of sensor data using at least one of a Kalman filter, a low-pass filter, a high-pass filter, a complementary filter, a moving average filter, and a smoothing filter.

The wearable device 110 may generate the collected plurality of sensor data as sensor fusion data through data fusion.

The wearable device 110 may detect the feature section from the sensor fusion data, and recognize the exercise being performed by the user based on the similarity between the data for the feature section and a plurality of pre-stored motion pattern data.

To this end, the wearable device 110 may detect at least two singularities based on at least one of a peak value, a valley, a zero crossing, an area of a graph, a curvature, a time, an angle, an acceleration, a distance, and an orbit. Here, at least two or more singularities may include a first singularity and a second singularity, and the feature section may mean a section between a time corresponding to the first singularity and a time corresponding to the second singularity.

Thereafter, the wearable device 110 extracts a plurality of sensor data for the feature section from the sensor fusion data, analyzes the similarity between the plurality of sensor data for the feature section and a plurality of pre-stored motion pattern data, and performs a user's exercise Can recognize.

The wearable device 110 may determine whether the exercise posture being performed by the user is the correct posture using the values of the acceleration sensor, the gyro sensor, and the quaternion. At this time, the wearable device 110 may determine whether the user's workout posture is the correct posture through machine learning such as a Hidden Markov Model (HMM), Convolution Neural Nerwork (CNN), or Deep Neural Network (DNN).

The wearable device 110 may output feedback on an exercise being performed by the user through a display based on the recognized exercise. Here, the feedback may include dynamic feedback including at least one of an exercise name, exercise number, exercise time and calories burned in real time, exercise accuracy, and static feedback including at least one of an exercise method, an exercise site, exercise intensity, and ranking. Can be.

The wearable device 110 may store exercise data corresponding to each of a plurality of exercise pattern data in a memory. In addition, when the similarity between the data for the feature section and a plurality of pre-stored exercise pattern data is equal to or less than a preset value, the wearable device 110 extracts exercise pattern data for the exercise being performed by the user, and extracts the extracted exercise pattern data. The new exercise data can be stored in the memory. Here, the new exercise data may be shared by another user's wearable device 110 or the user terminal 130.

The wearable device 110 transmits exercise data for an exercise being performed by the user to the server 120. At this time, based on information about the exercise being performed by the user, the user's comprehensive exercise data for each exercise may be updated.

The wearable device 110 may receive feedback about an exercise being performed by the user from the server 120. At this time, the feedback may consist of a picture and text, and may be output through a display, may consist of voice, and may be output through a speaker, or may be output through vibration. Through this, it is possible to provide visual, audible, and tactile feedback to the user.

The wearable device 110 may be connected to the user terminal 130 through Bluetooth communication. For example, when a user wants to create a new workout through the new workout creation menu of the user terminal 130, when a user wearing the wearable device 110 performs a specific workout a predetermined number of times, the wearable device 110 ) May transmit the number of workouts of the corresponding workout to the user terminal 130 connected via Bluetooth.

The server 120 may receive exercise data for an exercise performed by a user from the wearable device 110.

The server 120 may store the received exercise data in comprehensive exercise data for each exercise. The comprehensive exercise data for each exercise includes an exercise method of a specific exercise, an accumulated exercise time of the exercise, an accumulated exercise record, and an accumulated exercise number.

The server 120 may provide feedback based on comprehensive exercise data for each exercise. For example, when the exercise data of the exercise performed by the user is stored in the comprehensive exercise data, the server 120 may provide feedback generated based on the ranking of the exercise to the wearable device 110. For another example, the server 120 may provide the user terminal 130 with feedback including an exercise method of an exercise the user wants to perform.

The server 120 may transmit dynamic feedback to the wearable device 110 and provide static feedback to the user terminal 130. Alternatively, the server 120 may transmit static feedback to the wearable device 110 after the exercise is finished.

The user terminal 130 is connected to the wearable device 110 via Bluetooth communication, and each exercise information is synchronized through an app that provides exercise feedback installed in each of the user terminal 130 and the wearable device 110 Can be.

The user terminal 130 may receive feedback including an exercise method of an exercise the user wants to perform from the server 120. At this time, the feedback may consist of a picture and text, and may be output through a display, may consist of voice, and may be output through a speaker, or may be output through vibration. Through this, it is possible to provide visual, audible, and tactile feedback to the user.

The user terminal 130 may register new exercise data from the user. The user terminal 130 may provide a new exercise data menu and receive and register exercise data from a user wearing the wearable device 110 through communication with the wearable device 110.

2 is a block diagram of a wearable device according to an embodiment of the present invention. Referring to FIG. 2, the wearable device 110 includes a sensor data collection unit 210, a sensor fusion data generation unit 220, an exercise recognition unit 230, a storage unit 240, an output unit 250, and new exercise It may include a registration unit 260 and the communication unit 270.

The sensor data collection unit 210 may collect a plurality of sensor data measured from a plurality of sensors embedded in the wearable device 110. For example, when the user wearing the wearable device 110 starts exercising, the sensor data collection unit 210 collects a plurality of sensor data measured by a plurality of sensors embedded in the wearable device 110. Can be. Here, the plurality of sensors may include an acceleration sensor, a gyro sensor, a heart rate sensor, a gravity sensor, a magnetic sensor, a proximity sensor, an RGB sensor, a brightness sensor, a barometric pressure sensor, and the like.

The acceleration sensor is composed of a chip including gravity acceleration, and can measure linear acceleration of how much force the wearable device 110 is receiving. The acceleration sensor is composed of three dimensions of x / y / z coordinates, and an acceleration value (unit: g) is measured through the acceleration sensor, and a value of ± 4G or more may be used as the value of the acceleration sensor.

The gyro sensor is a sensor that measures the angular velocity. Through the gyro sensor, an angle (unit: dps) based on roll, pitch, and yaw can be derived. It is possible to grasp the user's movement by calculating the angular velocity through the gyro sensor.

Here, the sensor data collection unit 210 may include a noise removal unit 211.

The noise removing unit 211 may remove noise from a plurality of sensor data using a Kalman filter, a low-pass filter, a high-pass filter, a complementary filter, a moving average filter, and a smoothing filter. For example, the sensor data may include noise, and the noise removing unit 211 may remove noise from the sensor data by applying a filter suitable for the sensor data of each sensor.

Kalman filter (Kalman filter) is a filter that removes noise from the input data that contains noise to enable optimal statistical prediction of the current state, and a low-pass filter (LPF) sets a cutoff frequency to generate a high-frequency noise signal. It is a filter that removes and uses only low-frequency signals. The high-pass filter (HPF) is a filter that uses only high-frequency signals by removing the low-frequency noise signal by setting the cutoff frequency, and the complementary filter has good response characteristics in the low-frequency region and the value measured by the gyro sensor with good response characteristics in the high-frequency region. It is a filter that measures the angle by adding the value measured through the acceleration sensor. A moving average filter means a low-pass or flattening filter based on calculating the average value of the surroundings.

The sensor fusion data generator 220 may generate sensor fusion data through the data fusion of the plurality of received sensor data. The process of generating the sensor fusion data through the data fusion of the plurality of sensor data will be described in detail with reference to FIGS. 3 and 4.

3 is an exemplary view showing a plurality of sensor data measured by a plurality of sensors according to an embodiment of the present invention. Referring to FIG. 3, the plurality of sensor data 300 is sensor data measured by a plurality of sensors, and it can be seen that each sensor data has a different waveform.

4 is an exemplary diagram illustrating sensor fusion data generated through data fusion of a plurality of sensor data according to an embodiment of the present invention. 3 and 4, the sensor fusion data generation unit 220 may generate a plurality of received sensor data 300 as one sensor fusion data 400 through data fusion. In this case, the sensor fusion data 400 may include at least two singularities 410. Here, the singularity may be represented as the starting point and the ending point of the movement. The sensor fusion data 400 may be derived through Equation 1, for example.

The sensor fusion data 400 is generated by a plurality of sensor data as one sensor fusion data 400 through data fusion, and the sensor fusion data generating unit 220 is, for example, six-axis sensor data. Can be generated as sensor fusion data 400 of one axis through data fusion. This is, in order to recognize the motion being performed by the user, six or more axes are used, and by generating sensor fusion data from sensor data of six or more axes, it is possible to easily detect singularities. Here, at least two or more singularities detected through the sensor fusion data 400 may be values for finding a flag of the six-axis sensor data.

Returning to FIG. 2 again, the motion recognition unit 230 detects the feature section from the sensor fusion data, and can recognize the exercise being performed by the user based on the similarity between the data for the feature section and a plurality of pre-stored exercise pattern data. have. Here, the motion recognition unit 230 may include a singularity detection unit 231, a sensor data extraction unit 232, and a similarity analysis unit 233. The process of recognizing the exercise being performed by the user will be described in detail with reference to FIGS. 5A to 5C.

5A is an exemplary diagram for describing a process of extracting sensor data according to a feature section according to an embodiment of the present invention.

Referring to FIG. 5A, the singularity detection unit 231 may detect at least two singularities based on a peak value, a valley, zero crossing, an area of a graph, a curvature, time, speed, acceleration, and distance trajectory. Here, at least two or more singularities include a first singularity 501 and a second singularity 502, and the feature section 503 corresponds to a time corresponding to the first singularity 501 and a second singularity 502. It may be an interval between times.

The sensor data extraction unit 232 may extract a plurality of sensor data 520 from the sensor fusion data 510 for the feature section 503. For example, when the feature section 503 is detected based on the first singularity 501 and the second singularity 502 from the sensor fusion data 500, the sensor data extraction unit 232 detects the feature interval 503. A plurality of sensor data 530 for each axis of each sensor may be extracted from the plurality of sensor data 520 included in the sensor fusion data 510.

5B is an exemplary diagram for explaining a process of comparing sensor data for each axis of each sensor according to an embodiment of the present invention and data for each reference axis of each pre-stored sensor.

Referring to FIG. 5B, the similarity analysis unit 233 may compare (530, 540) for each axis by using the sensor data for each axis of each extracted sensor and the reference data for each axis of the previously stored sensor.

For example, the similarity analysis unit 233 compares the x-axis sensor data of the extracted acceleration sensor with the x-axis reference data of the pre-stored acceleration sensor (531), and the y-axis sensor data and the pre-stored acceleration sensor of the extracted acceleration sensor. The y-axis reference data of 532 may be compared, and the z-axis sensor data of the extracted acceleration sensor and the z-axis reference data of the previously stored acceleration sensor may be compared (533). In addition, the similarity analysis unit 233 compares the roll axis sensor data of the extracted gyro sensor and the reference data of the roll axis of the pre-stored gyro sensor (541), and the pitch axis sensor data of the extracted gyro sensor The pitch axis reference data of the pre-stored gyro sensor may be compared (542), and the yaw axis sensor data of the extracted gyro sensor and the yaw axis reference data of the pre-stored gyro sensor may be compared (543).

Through this process, the similarity analysis unit 233 may calculate an accuracy average between data for each axis.

5C is an exemplary diagram for explaining a process of analyzing the similarity between a plurality of sensor data and a plurality of pre-stored motion pattern data for a feature section according to an embodiment of the present invention.

Referring to FIG. 5C, the similarity analysis unit 233 may analyze the similarity between the plurality of sensor data 560 for the feature section 503 and the plurality of pre-stored motion pattern data 570. At this time, in order to analyze the similarity between the plurality of sensor data 560 for the feature section 503 and the plurality of pre-stored motion pattern data 570, a pattern recognition algorithm, a k-nearest neighbors algorithm, KNN ), Recurrent Neural Network (RNN), Dynamic Time Warping (DTW), Perceptrons, Correlation clustering, HMM, MEMMs, etc. may be used.

When the average accuracy of data for each axis of the plurality of sensor data 560 for the feature section 503 is calculated through FIG. 5B, the similarity analysis unit 233 stores data for each axis of the plurality of sensor data 560 and pre-stored Concordance can be calculated by comparing each axis of the motion pattern data.

For example, when the exercise currently being performed by the user is a curl, the similarity analysis unit 233 compares the motion pattern data of the curl and a pre-stored dumbbell press to calculate a match rate of 70%, and the motion pattern of the curl and the pre-stored front raise. By comparing the data, the degree of agreement is calculated as 50%, and by comparing the motion pattern data of the curl and the pre-stored curl, the degree of agreement is calculated as 95%. Calculate and compare the motion pattern data of the curl and the pre-stored lunch to calculate the degree of agreement as 20%.

Through such a process, the similarity analysis unit 233 can recognize that the exercise currently being performed by the user is a 'curl'. Through this, when a user wearing the wearable device 110 is exercising, by automatically recognizing the exercise, it is possible for the user to control the start and end of a separate exercise in order to count the exercise.

The storage unit 240 may store exercise data corresponding to each of the plurality of exercise pattern data in a memory. The storage unit 240 may store exercise data by recognizing each exercise even when a user wearing the wearable device 110 performs a single exercise in addition to performing a single exercise. For example, when the user wearing the wearable device 110 wants to perform the second exercise after performing the first exercise, the server 120 recognizes the first exercise and when the user performs the second exercise , Since it is different from the data of the first exercise, it can be considered that the new exercise has been input and it can be determined that the first exercise has ended. At this time, the server 120 may store the first exercise record, and may consider that the second exercise has started. Alternatively, if no motion is recognized for a few seconds after performing the first exercise, the server 120 may determine that the first exercise is finished and store it as an exercise record.

Through this, it is possible to provide an advantage of automatically counting and recording for each exercise set.

When the new exercise data is stored, the storage unit 240 may update the exercise data by adding new exercise data to the previously stored exercise data. At this time, the pre-stored exercise data and the user's exercise data are compared and analyzed through machine learning to update the optimum exercise data.

Returning to FIG. 2 again, the output unit 250 may output feedback on the exercise being performed by the user through the display based on the recognized exercise. In addition, the output unit 250 may output feedback about the exercise being performed by the user through a speaker or vibration. At this time, if the user does not want to receive feedback during exercise, the feedback is stored in the memory and may be output when the app is closed or the app is started.

Feedback includes dynamic feedback including at least one of the name of the workout, exercise guide, number of workouts, workout time, workout accuracy and calories burned in real time, and static feedback including at least one of workout methods, workout sites, workout intensity and rankings can do. Ranking allows users to determine the level of exercise they are performing (ie, compared to others) based on their physical conditions (eg, age, height, weight, country, gender, body fat percentage, climate, race, etc.). You can make it grasp.

Movement accuracy may be calculated based on the user's movement using the similarity of the movement pattern. In addition, coaching for the user's posture may be provided by converting the user's posture into 3D coordinates through exercise accuracy. For example, by comparing the coordinate values between reference data for each axis of each sensor and sensor data (real-time data) for a user's characteristic section using exercise accuracy, a user's wrong posture (for example, a wrist bend, a body tilt, etc.) ) In real time.

The calories burned can be calculated according to the displacement amount of movement after the quaternion conversion. At this time, calories can be calculated based on the activity level according to gender, age, weight, heart rate, and time. The activity level can be calculated according to the amount of quaternion displacement and the level of the exercise area. For example, the amount of calories burned may be calculated according to the displacement amount of the motion after the sensor data for the feature section is converted into 3D coordinates or quaternions. The calorie consumption can be derived, for example, through Equation 2.

 (Here, T = 1 time, Activity Level = ((quaternion (movement) displacement)) * (exercise level (distinguished motion name))), and exercise area level is related to the Metabolic Equivalent Task (MET) value)

The quaternion displacement amount corresponds to the user's movement, and the displacement amount means the distance the user's wearable device 110 moves. The calories burned may be calculated by measuring the moving distance of the wearable device 110 worn by the user, and using the quaternion displacement as a variable in the existing calorie calculation formula. Here, the amount of calories burned can be calculated by defining the average as '1' in all exercises, and defining a value of less than 1 for a exercise with a small amount of displacement and 1 for a exercise with a large amount of displacement. At this time, the amount of calories burned is changed according to the number of workouts. For example, in the case of an arm exercise, the activity level is set low because the movement of the muscle is lower than the whole body exercise, and in the case of the whole body exercise, the activity level is set high. Can be.

When the user's exercise amount is grasped in units of week, month, and quarter, the exercise intensity may be provided as feedback according to a ratio of performing the exercise. The exercise guide may calculate the accuracy of the user's exercise posture and provide the guide for the exercise posture to the wearable device 110 or the user terminal 130 as feedback. By analyzing the user's exercise data through machine learning, a user can guide an exercise posture to a user, recommend a suitable exercise to the user, and recommend the exercise intensity of the exercise the user is performing. In addition, each calorie consumption for each exercise can be measured and provided to the user.

Through this, a user wearing the wearable device 110 can intuitively check feedback on an exercise being performed through the display of the wearable device 110.

In addition, by informing the user of the degree of exercise progress compared to other users having similar characteristics (eg, height, age, region, etc.) to the user, motivation using a competitive psychology can be provided. At this time, the server 120 may transmit the user's exercise muscles used in accordance with the exercise to the user terminal 130 through graphics to recommend insufficient exercise to the user as feedback.

For example, the output unit 250 outputs feedback such as 'bench press / 4 times / 02: 30' through a display, and outputs a notification sound through a speaker or increases vibration whenever the number of workouts increases. You may. The process of providing feedback on the exercise being performed by the user will be described in detail with reference to FIGS. 6A and 6B.

6A and 6B are exemplary diagrams illustrating feedback about an exercise being performed by a user provided as a wearable device and a user terminal according to an embodiment of the present invention.

6A is an exemplary diagram illustrating feedback on an exercise being performed by a user displayed on a wearable device according to an embodiment of the present invention. Referring to FIG. 6A, the output unit 250 may output dynamic feedback through a display. Here, the dynamic feedback may include an exercise name, an exercise number, an exercise time, and real-time calorie consumption. For example, if the exercise currently being performed by the user is recognized as a 'bench press' by the exercise recognition unit 230, the output unit 250 may include' exercise name-bench press', 'number of exercises-4 times',' It is possible to provide feedback 610 for exercise time- 00:45 ',' consumed calorie-7kcal ', and the like to the wearable device 110. For another example, when the exercise is completed, the output unit 250 may output a feedback 620 (ie, static feedback) of the exercise history performed by the user through the display.

6B is an exemplary diagram illustrating feedback on an exercise being performed by a user displayed on a user terminal according to an embodiment of the present invention.

Referring to FIG. 6B, the communication unit 270 of the wearable device 110 may transmit exercise data for an exercise being performed by the user to the server 120.

At this time, the user terminal 130 may receive feedback about the exercise being performed by the user from the server 120.

Alternatively, the wearable device 110 is connected to the user terminal 130 through Bluetooth communication, and the communication unit 270 may directly transmit feedback on an exercise being performed by the user to the user terminal 130. Here, the feedback may include an exercise name, an exercise number, an exercise time, real-time calorie consumption, exercise accuracy, an exercise method, an exercise part, an exercise intensity, and ranking.

For example, when the exercise recognition unit 230 recognizes the exercise currently being performed by the user as a 'bench press', the communication unit 270 may transmit exercise data related to the 'bench press' to the server 120. At this time, the server 120 may update the comprehensive exercise data for each exercise of the user based on information about the exercise being performed by the user (ie, update the exercise record, ranking, etc.).

At this time, the server 120 may provide feedback related to the 'bench press' to the user terminal 130. At this time, the user terminal 130 receives feedback 630 about the 'Benz Press' exercise method, feedback about the part to be exercised through 'Bench Press' 640, and the time and number of 'Bench Press' exercise Based on the ranking, feedback 650 and real-time exercise feedback 660 may be output.

When performing an exercise that is not pre-stored, the output unit 250 may output a feedback on whether to store the exercise separately through a display. For example, when an exercise corresponding to an unstored exercise pattern is performed by a user, the exercise recognition unit 230 does not recognize the corresponding exercise name, but only recognizes the number of exercise. Accordingly, when the pre-stored exercise is finished, the output unit 250 may output a feedback on whether to store the exercise separately through a display.

The new exercise registration unit 260 extracts exercise pattern data for an exercise being performed by a user and includes extracted exercise pattern data when the similarity between data for a feature section and a plurality of pre-stored exercise pattern data is equal to or less than a preset value. New exercise data to be stored can be stored in the memory. At this time, new exercise data stored in the database may be synchronized with the user terminal 130.

The new workout register 260 may update new workout data to existing workout data through machine learning. At this time, the new exercise registration unit 260 analyzes the properties of the exercise by comparing it with pre-stored exercise pattern data, and the output unit 250 may output feedback according to the exercise properties through a display. Alternatively, the communication unit 270 may transmit feedback according to the exercise properties to the user terminal 130.

Alternatively, the new exercise registration unit 260 may receive new exercise data through interworking with the user terminal 130. For example, the new exercise registration unit 260 may receive new exercise data from a user by entering a step for a user to register a new exercise through an app that provides exercise feedback of the user terminal 130. The process of the user registering the new exercise using the user terminal 130 will be described in detail with reference to FIG. 7.

7 is an exemplary diagram for explaining a process of registering new exercise data according to an embodiment of the present invention. Referring to FIG. 7, the new exercise registration unit 260 may receive an image, an exercise description, and an exercise part from a user through the user terminal 130 and register as a new exercise.

For example, the user may enter the menu 710 for registering a new exercise of the app that provides exercise feedback through the user terminal 130. The user terminal 130 registers a 'new exercise' through the 'make exercise' menu 710. Repeat the same operation 7 times. ”In this case, when a user who wears the wearable device 110 repeats a specific exercise '7 times,' the user terminal 130 displays the 'registration menu'. When a workout name, an exercise part, an exercise description, an image, a video, etc. are input from a user through '720, or an exercise name to be registered through an app is selected by the user, the new exercise registration unit 260 is a user terminal 130 ), The new workout data can be registered and registered as a new workout In this case, if another workout is input while the user repeats a specific workout '7', registration of the new workout may be stopped. When a new exercise is registered in 120, feedback about the new exercise may be provided to the wearable device 110 or the user terminal 130. For example, even if the user creates a new exercise, through machine learning If a similar exercise exists in the sum exercise data, the server 120 may provide feedback to the wearable device 110 or the user terminal 130. At this time, reference data of the similar exercise may be modified or deleted. have.

For another example, as another method of generating a new exercise, the wearable device 110 may perform an unrecognized exercise after the user executes the cognitive exercise in which the exercise pattern is pre-stored, and the number of exercises for the unrecognized exercise. Can only be recognized. In addition, when the unrecognized workout ends, the wearable device 110 may provide the user with feedback on whether the workout is not stored and register as a new workout. Through this, it is possible to register a personalized exercise for the user only, and to create a fitness program through a combination of a pre-stored exercise and a registered new exercise, and provide the advantage of sharing it with other users.

In addition, it is possible to derive the optimal exercise data through machine learning of the exercise data of all users, and provide the optimum exercise data to other users.

8 is a flowchart of a method for providing feedback about an exercise in a wearable device according to an embodiment of the present invention. The method for providing feedback about an exercise in the wearable device 110 illustrated in FIG. 8 includes steps processed in time series by the exercise feedback providing system 1 according to the embodiment illustrated in FIGS. 1 to 7 . Therefore, even if it is omitted below, it is also applied to a method of providing feedback on exercise in the wearable device 110 according to the embodiment illustrated in FIGS. 1 to 7.

In operation S810, the wearable device 110 may collect a plurality of sensor data measured from a plurality of sensors embedded in the wearable device 110.

In operation S820, the wearable device 110 may generate sensor fusion data through data fusion of the collected plurality of sensor data.

In step S830, the wearable device 110 detects a feature section from the sensor fusion data, and recognizes the exercise being performed by the user based on the similarity between the data for the feature section and a plurality of pre-stored motion pattern data.

In operation S840, the wearable device 110 may output feedback on the exercise being performed by the user on the basis of the recognized exercise through the display.

In the above description, steps S810 to S840 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted if necessary, and the order between the steps may be switched.

9 is a configuration diagram of a server according to an embodiment of the present invention. Referring to FIG. 10, the server 120 may include a receiving unit 910, a storage unit 920 and a feedback providing unit 930.

The receiving unit 910 may receive exercise data for an exercise performed by a user from the wearable device 110.

The storage unit 920 may store the received exercise data in comprehensive exercise data for each exercise. For example, the storage unit 920 may accumulate and store the received exercise data in comprehensive exercise data generated for each exercise. The comprehensive exercise data may include a pre-stored exercise data including an exercise method of a specific exercise, an accumulated exercise time of the exercise, an accumulated exercise record, and an accumulated exercise number.

The feedback providing unit 930 may provide feedback to the wearable device 110 and / or the user terminal 130 based on the comprehensive exercise data for each exercise. For example, when the exercise data of the exercise performed by the user is stored in the comprehensive exercise data, the feedback providing unit 930 may provide the generated feedback based on the ranking of the exercise to the wearable device 110. For another example, the feedback providing unit 930 may provide the user terminal 130 with feedback including an exercise method of an exercise desired by the user.

10 is a flowchart of a method for providing feedback on an exercise in a server according to an embodiment of the present invention. The method for providing feedback on exercise in the server 120 shown in FIG. 10 includes steps processed in time series by the exercise feedback providing system 1 according to the embodiment shown in FIGS. 1 to 9. Therefore, even if it is omitted hereinafter, it is also applied to a method of providing feedback on exercise in the server 120 according to the embodiment illustrated in FIGS. 1 to 9.

In step S1010, the server 120 may recognize a workout being performed by a user from the wearable device 110 and receive the recognized workout data.

In step S1020, the server 120 may store the received exercise data in comprehensive exercise data for each exercise.

In step S1030, the server 120 may generate feedback based on the stored comprehensive exercise data.

In step S1040, the server 120 may transmit the generated feedback to the wearable device 110.

In the above description, steps S1010 to S1040 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted if necessary, and the order between the steps may be switched.

The method of providing feedback on exercise in the wearable device and the server described with reference to FIGS. 1 to 10 is also implemented in the form of a computer program stored in a medium executed by a computer or a recording medium including instructions executable by a computer Can be. In addition, the method for providing feedback on exercise in the wearable device and the server described with reference to FIGS. 1 to 10 may also be implemented in the form of a computer program stored in a medium executed by a computer.

Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer readable media may include computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The above description of the present invention is for illustration only, and those skilled in the art to which the present invention pertains can understand that the present invention can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and it should be interpreted that all changes or modified forms derived from the meaning and scope of the claims and equivalent concepts thereof are included in the scope of the present invention. do.

110: wearable device
120: server
130: user terminal
210: sensor data collection unit
211: noise removing unit
220: sensor fusion data generator
230: exercise awareness
231: singularity detection unit
232: sensor data extraction unit
233: similarity analysis unit
240: storage
250: feedback providing unit
260: new exercise register
270: communication unit
910: receiver
920: storage
930: feedback providing unit

Claims (20)

In a wearable device that provides feedback on exercise,
A sensor data collection unit collecting a plurality of sensor data measured from a plurality of sensors embedded in the wearable device;
A sensor fusion data generator for generating the collected plurality of sensor data as sensor fusion data through data fusion;
An exercise recognition unit detecting a feature section from the sensor fusion data, and recognizing an exercise being performed by a user based on the similarity between the data for the feature section and a plurality of pre-stored exercise pattern data; And
An output unit for outputting feedback on the exercise being performed by the user based on the recognized exercise,
The exercise recognition unit is a sensor data extraction unit for extracting a plurality of sensor data for the feature section from the sensor fusion data
That includes, a wearable device.
According to claim 1,
The exercise recognition unit includes a singularity detection unit that detects at least two singularities based on at least one of a peak value, a valley, a zero crossing, a graph area, a curvature, a time, an angle, an acceleration, a distance, and an orbit. .
According to claim 2,
The at least two singularities include a first singularity and a second singularity,
The feature section is a section between a time corresponding to the first singularity and a time corresponding to the second singularity, a wearable device.
delete According to claim 1,
The motion recognition unit analyzes the similarity between the plurality of sensor data for the feature section and the plurality of previously stored motion pattern data.
The wearable device further comprising a.
The method of claim 5,
The similarity analysis unit analyzes the similarity between each axis of each sensor of a plurality of sensor data for the feature section and reference data for each axis of the previously stored sensor.
According to claim 1,
A storage unit that stores exercise data corresponding to each of the plurality of exercise pattern data in a memory
The wearable device further comprising a.
The method of claim 5,
If the similarity between the data for the feature section and a plurality of pre-stored exercise pattern data is equal to or less than a preset value, the exercise pattern data for the exercise being performed by the user is extracted, and new exercise data including the extracted exercise pattern data Workout register that stores data in memory
The wearable device further comprising a.
The method of claim 8,
The new exercise registration unit, after the first exercise performed by the user is recognized based on the data for the first feature section, based on the similarity between the data for the second feature section and the plurality of previously stored exercise pattern data. When the second exercise being performed by the user is not recognized, the wearable device stores new exercise data including exercise pattern data for the unrecognized second exercise in a memory.
According to claim 1,
The feedback includes dynamic feedback including at least one of an exercise name, an exercise guide, an exercise number, an exercise time, exercise accuracy, and real-time calorie consumption, and static feedback including at least one of an exercise method, an exercise site, an exercise intensity, and a ranking. Wearable device that includes.
The method of claim 10,
In the real-time calorie consumption, the sensor data for the feature section is converted into a quaternion, and the calorie consumption is changed according to the displacement amount of the converted quaternion.
According to claim 1,
Further comprising a communication unit for transmitting the exercise data related to the exercise being performed by the user to a server,
A wearable device in which the comprehensive exercise data for each exercise of the user is updated based on exercise data regarding the exercise being performed by the user.
According to claim 1,
The sensor data collection unit is a noise control unit that removes noise from the plurality of sensor data using at least one of a Kalman filter, a low-pass filter, a high-pass filter, a complementary filter, a moving average filter, and a smoothing filter.
That includes, a wearable device.
delete In the method of providing feedback about the exercise on the wearable device,
Collecting a plurality of sensor data measured from a plurality of sensors embedded in the wearable device;
Generating the collected sensor data as sensor fusion data through data fusion;
Detecting a feature section from the sensor fusion data and recognizing a user's exercise based on the similarity between the data for the feature section and a plurality of pre-stored motion pattern data; And
And outputting feedback on an exercise being performed by the user based on the recognized exercise,
The step of recognizing the exercise,
And extracting a plurality of sensor data for the feature section from the sensor fusion data.
The method of claim 15,
The step of recognizing the motion is detecting at least two singularities based on at least one of a peak value, a valley, a zero crossing, an area of a graph, a curvature, a time, an angle, an acceleration, a distance, and an orbit. .
The method of claim 16,
The at least two singularities include a first singularity and a second singularity,
The feature section is a section between the time corresponding to the first singularity and the time corresponding to the second singularity, a method for providing exercise feedback.
The method of claim 15,
The step of recognizing the exercise,
The method of analyzing the similarity between a plurality of sensor data for the feature section and the plurality of pre-stored motion pattern data is provided.
The method of claim 15,
The feedback includes dynamic feedback including at least one of an exercise name, an exercise guide, an exercise number, an exercise time, exercise accuracy, and real-time calorie consumption, and static feedback including at least one of an exercise method, an exercise site, an exercise intensity, and a ranking. Including, how to provide exercise feedback.
In the method of providing feedback about the exercise on the server,
Receiving exercise data for an exercise performed by a user from a wearable device;
Updating comprehensive exercise data for each exercise based on the received exercise data;
Including the step of providing feedback to the user terminal for the exercise performed by the user,
In the exercise data, a feature section is detected from sensor fusion data, and the exercise performed by the user is recognized based on the similarity between the data for the feature section and a plurality of pre-stored exercise pattern data,
In the sensor fusion data, a plurality of collected sensor data (Raw Data) is generated through data fusion,
The sensor data is measured from a plurality of sensors built into the wearable device,
A method of providing feedback, wherein a plurality of sensor data for the feature section is extracted from the sensor fusion data.

Images