WO2010063884A1

WO2010063884A1 - Physical exercise control

Info

Publication number: WO2010063884A1
Application number: PCT/FI2009/050958
Authority: WO
Inventors: Juha Antero FORSTÉN; Jukka Petri RÖNKKÖ
Original assignee: Kineware Oy
Priority date: 2008-12-02
Filing date: 2009-11-30
Publication date: 2010-06-10
Also published as: US20110311955A1; FI123245B; FI20086152A; FI20086152A0

Abstract

A method and corresponding apparatus for controlling physical exercise, in which an exercise program is stored and presented to a user as a computer animated instructor. The computer animated instructor is produced so that the animation corresponds to movements according to the exercise program. The animation is synchronized with music and presented to the user on a display.

Description

PHYSICAL EXERCISE CONTROL

FIELD OF THE INVENTION

The present invention generally relates to physical exercise control.

BACKGROUND OF THE INVENTION

Physical exercise is essential for every human being generally for good health and overall feeling of well-being. There is a vast resource of different forms of methods and equipment geared up to this end. However, the fast pace of modern life makes it difficult to attend to team sports and generally to any exercise with other people. The scarceness of time also calls for efficiency to physical exercise.

Training should be performed at desired intensity. Personal trainers may optimize the exercise but not everyone can afford their services. Hence, new alternatives are needed for physical exercise.

SUMMARY

According to a first exemplary aspect of the invention there is provided a method for controlling physical exercise, comprising: storing an exercise program; and presenting the exercise program to a user; characterized by: prod ucing a computer an imation of an instructor, the animation corresponding to movements according to the exercise program; synchronizing the animation with music; and presenting the computer animation to a user on a display.

Advantageously, an animated instructor may be presented to a user to control her physical exercise according to the exercise program. Hence, the pace and form of exercise may be controlled by means of a technical apparatus.

The producing of the animation may comprise simulating motion of different body parts of a 3-dimensional character. The character may model a human being. Advantageously, the simulation may enable cost-efficient storing and combining of a wide selection of different movements in an exercise program.

The synchronizing of the animation with music may comprise adapting the pace of the animation according to an external source. The external source may comprise a music player and/or an exercise intensity sensor that may be a heart pulse sensor. The method may further comprise receiving a signal corresponding to an external music source by means of a microphone.

Advantageously, the method may enable adapting the exercise program into music chosen by the user.

Advantageously, by using a microphone for receiving a signal corresponding to the external music source it may be possible to use also music that is only accessible during its playback (e.g. DRM protected music or radio stations).

Advantageously, the method may enable obtaining a feed-back from the user's heart rate or pulse. The feedback may be obtained in form of an ANT signal.

Advantageously, the method may enable adjusting the exercise according to the user's heart rate. Further advantageously, standard ANT compatible equipment may be used.

The method may further comprise increasing the pace of the animating of the exercise program if the pulse is below a given lower limit. The method may further comprise reducing the pace of the animating of the exercise program if the pulse is above a given higher limit.

Advantageously, the method may enable fine tuning the intensity of the exercise by adapting the pace of the animating of the exercise program.

The method may further comprise changing the exercise program in order to adapt the exercise to changes in the pace beyond a given threshold. The method may further comprise maintaining a constant safety limit and stopping the animating if the pulse reaches or exceeds the safety limit.

The method may further comprise analyzing the music to be used in conjunction with the exercise at least partly before presenting the animation and producing the animation according to the preceding analyzing. Advantageously, the music may be obtained from a music source such as a CD-ROM, DVD-ROM, USB stick, Internet storage, or local or network music depository.

The method may comprise determining the beat and tempo of the music in realtime and adjusting the timing of the animation while presenting the animation to the user.

The method may further comprise playing music and presenting the animation in a synchronized manner and adjusting the tempo of the music to maintain the synchronization.

The method may further comprise storing user preferences and modifying or preparing the exercise taking into account the user preferences. The preferences may comprise selection of particular movements to be emphasized or to be avoided.

Advantageously, the program may be tailored to account for individual needs such as avoiding risky movements after a surgical operation or strengthening particularly weak muscles.

The method may further comprise determining based on the user input the exercise program. The determining of the exercise program may comprise including in the program elements suited to a given exercise area such as force, aerobic, balance etc. The method may comprise determining the exercise program based on predetermined general or tailored recommendations by an expert. The exercise program may be determined such that muscles for opposite movements are balanced.

The method may further comprise providing additional computer animation for additional instructors. The method may further comprise controlling individually the additional instructors based on feedback received from different users.

According to a second aspect of the invention, there is provided a controller for controlling physical exercise, comprising: a memory configured to store an exercise program; and a video output connectable to a display configured to present the exercise program to a user via the device; characterized in that the controller further comprises: a processor configured to produce a computer animation of an instructor, the animation corresponding to movements according to the exercise program; and the processor is further configured to provide the computer animation through the video output.

The controller may comprise an electronic device that is configured to provide a further function. The controller may be based on a desktop computer, portable computer, game console, portable game device, personal digital assistant device, or mobile communication device.

According to a third aspect of the invention, there is provided a computer program for controlling physical exercise, comprising computer executable program code configured to cause a computer to perform the method according to the first aspect of the invention when executing the program. Accord ing to a fourth aspect of the invention, there is provided a system comprising a controller according to the second aspect of the invention and user feedback equipment for enabling adjustment of the operation of the controller.

The system may further comprise a display device configured to present the animation under control of the controller.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described, by way of example only, with reference to the accompanying drawings, in which:

Fig. 1 shows a schematic picture of a system according to an embodiment of the invention;

Fig. 2 shows a schematic block diagram of a controller suited for operating in the system shown in Fig. 1 ;

Fig. 3 shows a flow chart of a process according to an embodiment of the invention; Fig. 4 shows an exemplary graph illustrating heart rate of the user and different thresholds and events according to an embodiment of the invention; and Fig . 5 shows a flow chart of a process according to an embodiment of the invention.

DETAILED DESCRIPTION

In the following description, like numbers denote like elements.

Fig. 1 shows a schematic picture of a system 100 according to an embodiment of the invention. The system comprises a controller 1 10, a display device 120, a music player 130, a pulse sensor 140, and a remote station 150 communicatively connectable to the controller 110. Fig. 1 also illustrates a user 160, a view 170 on the display device 120 and a remote control 180 for controlling various features of the controller 1 10. The remote control 180 may be, for instance, a dedicated device or an incorporated functionality in a multipurpose device such as a wrist computer or a mobile phone. The multipurpose device may be capable of communicating with the controller 110 using ANT protocol.

Fig. 2 shows a block diagram of a controller 110 suited for operating in the system shown in Fig. 1. The controller 110 comprises a processor 210, a memory 220 for use by the processor in order to control the operation of the controller 110, a persistent memory 230 for storing long-term data such as software 240 typically comprising an operating system and computer executable applications, a user interface 250 for user interaction such as video and audio output and for receiving commands and/or settings from the user 160, an input/output system 260 for communication with remote entities such as the remote station 150, a music player 270 and a pulse sensor 280. The pulse sensor may be any commercially available pulse sensor, such as a pulse detection belt. The pulse sensor may be configured to wirelessly transmit a pulse signal or heartbeat signal as a proprietary or standardised signal such as an ANT signal that is provided by various modern pulse detectors.

The processor 210 may be e.g. a microprocessor CPU, a digital signal processor DSP, or generally any digital signal processor or combination of different individual processors.

The controller 1 10 may be based on an existing electronic device. For instance, the controller may be based on a desktop computer, portable computer, game console, portable game device, personal digital assistant device, or mobile communication device.

Fig . 3 shows a flow chart of a process according to an embodiment of the invention. The process starts from step 310 in which exercise programs are stored so that they are usable in the following steps. The programs may be stored one at the time or in larger number in the remote station 150 or in the controller 110. The programs may be classified according to their primary use (e.g. balance, strength, aerobic training) and have different alternative movement sequences within each main class. Moreover, the different alternative sequences may be associated with weighing factors which correspond to the intensity of the workout.

In step 320, the user 160 selects the type of exercise program or a particular exercise program. The selection may be made using the Ul 250 of the controller 110.

The controller then renders an animation 330 according to instructions contained by the program. The rendering may be performed by the processor 210 according to the software 240 and presented on the display device as illustrated as the view 170. The animation may also comprise audible instructions by means of voice or sounds such as claps of hands. Audible instructions are particularly advantageous, because during exercise people may not wear their normal eyeglasses or may have to turn on occasions such that watching the display device 120 may be inconvenient or even impossible.

The animation may be synchronised 340 with music that is played during the workout. The synchronising 340 may be performed by means of the controller. The synchronising 340 may be performed using an associated music player in real- time or in advanced. Alternatively, an independent music player may be used to play back the music. In case of an independent music play back, real-time detecting of the tempo and beat may be performed under e.g. by the controller 110 or by associated equipment. In case of independently played music, the controller may obtain an electric signal corresponding to the music e.g . by using a microphone or line-in connection. The electric signal may be normalised by means of an automatic gain control circuitry and filtered by removing normal exercise originated noises that can interfere with detection of the rhythm and beat of the music. The controller may then detect the beat of the music in real-time with analogue circuitry or digital circuitry possibly in conjunction with software. In yet another alternative, the controller may obtain the music to be played in digital form together with beat and tempo data such that the player is aware of the beat and tempo during playback without need to analyse the music. Anyhow, it is appreciated that music may provide timing for accurate workout of a given exercise program and that thus the music has a technical effect in enabling synchronising of the user's or trainer's workout.

As part the synchronising 340, audible instructions may also be added or modified in the animation (step 350) in order to inform the user 160 of a change in the beat and/or tempo of the animation.

The controller presents 360 the animation with the display device 120 on the view 170.

In connection with the presenting 360 of the animation, the controller 1 10 may continually obtain pulse information indicative of the pulse of the user 160 from the pulse sensor 140. The controller may then determine 370 if the intensity of the exercise should be altered. The determination may be based on comparing the pulse with given thresholds. The thresholds may comprise any of the following: an upper fine-tuning limit (th₃) on meeting which the exercise should be slowed down; an upper major limit (th₄) on meeting which the intensity of the exercise should be reduced by changing the exercise program to a lighter one; a safety limit (th₅) meeting which the exercise will be stopped; a lower fine-tuning limit (th₂) on meeting which the exercise should be speeded-up; a lower major limit (thi) on meeting which the intensity of the exercise should be increased by changing the exercise program to a heavier one; a lowest limit (th₀) meeting which the pulse detection is apparently inoperable and the user should be advised to check the sensor. The thresholds, associated actions and pulse are further described with reference to Figs. 4 and 5.

Fig. 4 shows an exemplary graph illustrating heart rate of the user and different thresholds and events according to an embodiment of the invention. The thresholds th₀ to th₅ were explained in the foregoing. At start, on time instant to, the user is starting to exercise and her pulse is at about the lower major limit thi. At this point of time, no adjustment is yet to be carried out as the pulse clearly has not yet reached a stationary. Then, on working out the exercise program, the user experiences an increase in the pulse that meets the lower fine tune limit th₂ at a first instant of time ti which is still within an initial start-up period t_st. However, by a second instant of time t₂ the start-up period has lapsed and the pulse is still between the lower fine-tuning limit th₂ and the upper fine-tuning limit th₃. Hence, the pulse now resides in the target region and the exercise continues. There is then a change in the program such that the pulse starts to climb.

On time instant t₂ the pulse has increased to the upper fine-tuning limit th₂. The pace of the exercise is then reduced to fine-tune the exercise. In case that the exercise is synchronised with music, the fine-tuning may be bound to the beat of the music. In one embodiment, further adjustability is gained by altering the tempo of the music playback (slowing down the music by a relatively small amount). Such a tempo reduction may be performed either using frequency domain computation such that the audible frequencies of the music remain subjectively unchanged (as is known e.g. from the Windows Media Player™ version 1 1 ). The exercise may then continue with slightly lower tempo, but in this example the pulse is still climbing fast so that soon after the pulse exceeds the upper major limit th₄ and the exercise program is adapted. Lighter movements are being made and the reduced intensity is reflected by the pulse evening out and then starting to decline.

The pulse then crosses the upper major limit th₄ and the upper fine-tuning limit th₃ on time instants t₄ and t₅, respectively. In this case, the pulse develops towards the target region so that the exercise program is not changed before the pulse exits the target region between the upper and lower fine-tuning limits th₃ and th₂. On time instant t₆, after a relatively short period after the time the pulse has come to the target region, the pulse goes below the lower fine-tuning limit th₂. The tempo of the exercise is then increased in order to slightly intensify the workout. As a result, the curve illustrating the pulse becomes less declining. The pulse yet keeps on decreasing and at instant t₇, the pulse becomes lower than the lower major limit thi. In response, the exercise program is changes to a heavier one. The pulse may still linger under the lower major limit for some time e.g. if the user has paused the exercise or has to spend some time to learn a new movement or sequence of movements, or if the user has to change orientation and thereby spend some time before the exercise may continue. Here, the pulse exceeds again the lower major limit thi at time t₈, but soon after the pulse detection slopes down almost vertically through the lowest limit th₀,

The lowest limit signifies that the pulse information is no longer available. This may be caused, for instance, by a lost contact between a pulse sensor belt contacts and the skin of the user 160 near her heart, or by some other failure. The decline of the pulse may not be vertical because typically the pulse is averaged over some period of time in order to stabilise the control of the exercise intensity. On detecting that the pulse goes below the lowest limit th₀, the user 160 may be instructed to ensure the attachment and operability of the pulse sensor 140.

Fig . 5 shows a flow chart of a process according to an embodiment of the invention suited for the synchronising step 340 in Fig. 3. Fig. 5 presents some embodiments of the invention, including real-time detecting of rhythm in step 510 and determ in ing of beat tim ing 520 of music that may be played by an independent music player. The process determines 530 the timing of repeated movements in the program and then determines a correlation 540 between the detected movement timing and the beat. It is understood that typical movements start on a first time from a start position and arrive on a second time to an end position. Either or both the first and second times may be used as a basis to determine a pace that is aligned with the rhythm of the music. Basically, the basis may be determined by selecting a pace that is closest to the rhythm of the music in terms of sequential beats. If the correlation requires a major change in the pace of the animation, it is apparent that the current exercise program is poorly suited to the present piece of music and thus either one should be changed . In an embodiment (not shown) in which the music source is controlled by the exercise system, a better suited piece of music may be selected if present in which case the operation would resume to step 510. However, in case that the music is not controlled by the exercise system, a major change need in the pace causes advancing to step 550 in which an alternative program is searched and if found, the program is changed to better suit with the currently played music. If no suitable program is identified, the exercise may be stopped and/or the user may be prompted for changing the type of the exercise in order to identify a suitable program. After step 550, the operation resumes to step 530.

If on the other hand the correlation was good such that the pace was close to the rhythm or even matching with the rhythm, the pace of the animation is only fine- tuned 560 so as to maintain the animation in the rhythm of the music. If minor changes are needed, they may be performed gradually based on proportional, differential and/or integrating control.

In step 570, the animation proceeds. The animation is produced from a given point of view. The point of view can be changed and/or another parallel animation from a different point of view may be simultaneously provided in order to show the animated character from different angles and for facilitating learning of a new movement.

In one embodiment, the system comprises a speech recognition unit for guiding the system. Such a unit can be configured to enable speech-controlled shift of the point of view from which the animation is being presented. Further or alternatively, a remote controller 180 may be provided for controlling the animation and/or music play back, to adjust the speed of the animation, to advance or reverse in the animation, to pause the animation, to change the animation to another, and/or to control other options of the system.

In step 580, a heartbeat signal is compared with the target intensity level of the exercise program. If the heart rate is at its target or within the fine-tuning limits, the operation resumes to step 510 (while the presenting of the animation continues). On the other hand, if there is a minor deviation beyond the fine-tuning limits but within the major limits, the operation proceeds to step 540 with an instruction to correspondingly increase or decrease the pace. In order to find a synchronisation with the music, the operation thus resumes all the way to step 540 rather than returns just to step 560 (which is yet done in an alternative embodiment). By resuming to step 540, the exercise may be suitably adjusted to steer the exercise such that the heartbeat would remain within desired range. However, if the heartbeat is far from the target range i.e. below a lower major limit or above a higher major limit (but below the safety limit), the operation resumes to step 550 with an instruction to change the program (to a lighter or heavier one) depending on the heartbeat signal.

Alternatively or additionally to using heart beat, other measures of the intensity of the exercise may also be used as an input to control the exercise by adapting the animation. Such measures involve, for instance, one or more of acoustic observing of the breathing sounds, direct or indirect measurements of breathing such as measuring muscle action or air flow, sweating, heat production (e.g. measured by an infra-red light sensor), and energy directed to the floor by the user on performing the movements,

The inventors have realized that physical exercise can be guided by a computer operable animated virtual instructor using interactive three-dimensional (3D) animation. The interactive 3D animation further enables providing free interactive viewpoint movement during presentation of the animation.

The 3D animation may be based on a commonly known skeletal animation paradigm. The animation is beneficial over recorded video e.g. by efficient animation storage, the possibility to combine animations from different sources also at runtime, ability to produce a common animation with different characters, and the ability to change the point of view without excessive memory consumption

(without parallel recorded video footage). The point of view of the animation may also be shifted by the user during the presentation of the animation.

It is appreciated that the animation may be based on movements recorded from a real person. The animation may also contain synthetic data such as random noise or deviations which makes repeating sequences appear natural.

In the examples of this description, various embodiments were explained by reference to one user who exercises making use of an animated exercise instructor. It is yet to be understood that more than one exercise instructors can be animated using common or separate circuitry so as to enable different parallel views on the similarly acting or of different instructors. For instance, different animated instructors can be used to present workout of differing difficulty and/or intensity. Thus, different users can simultaneously exercise based on a common presentation of different animated instructors. The different animated instructors may be presented on one or more displays connected to a common controller. It is even possible to make use of more than one controller in parallel and have them each operate according to music that is played and listened to by each individual controller. In this case, a common microphone or audio input can be shared between different controllers.

When multiple animated exercisers are presented, and individual intensity feedback equipment (e.g. pulse detector) is used to control the intensity of the exercise, a user can simply follow such an animated instructor that best suits to the users own preferences. Alternatively, different users may have individual intensity feedback equipment (e.g. pulse detector) in order to adapt the operation of a corresponding animated instructor. Alternatively, if there are more than one users corresponding to a common animated instructor, feedback may be neglected or the exercise may be controlled based on an averaging between different users' feedback such that the intensity is kept on average at an appropriate level. In such a case, individual messages may yet be presented to such users who should either work harder e.g . by making more efficient movements, work lighter e.g. by making less efficient movements, stop the exercise altogether or verify the operation of his or her feedback equipment.

The foregoing description has provided by way of non-limiting examples of particular implementations and embodiments of the invention a full and informative description of the best mode presently contemplated by the inventors for carrying out the invention. It is however clear to a person skilled in the art that the invention is not restricted to details of the embodiments presented above, but that it can be implemented in other embodiments using equivalent means or in different combinations of embodiments without deviating from the characteristics of the invention. Furthermore, some of the features of the above-disclosed embodiments of this invention may be used to advantage without the corresponding use of other features. As such, the foregoing description shall be considered as merely illustrative of the principles of the present invention, and not in limitation thereof. Hence, the scope of the invention is only restricted by the appended patent claims.

Claims

Claims:

1. A method for controlling physical exercise, comprising: storing an exercise program; and presenting the exercise program to a user; characterized by: producing a computer an imation of an instructor, the animation corresponding to movements according to the exercise program; synchronizing the animation with music; and presenting the computer animation to a user on a display.

2. A method according to claim 1 , characterized in that the producing of the animation comprises simulating motion of d ifferent body parts of a 3- dimensional character.

3. A method according to claim 2, characterized in that the character models a human being.

4. A method according to any one of the preceding claims, characterized in that the synchronizing of the animation with music comprises adapting the pace of the animation according to an external source.

5. A method according to claim 4, characterized in that the external source comprises a music player.

6. A method according to claim 4, characterized in that the external source comprises a heart pulse sensor.

7. A method according to any one of the preceding claims, characterized by including, based on the user input, desired program elements in the exercise program suited to a given exercise area such as force, aerobic, or balance exercise.

8. A controller for controlling physical exercise, comprising: a memory configured to store an exercise program; and a video output connectable to a display configured to present the exercise program to a user via the device; characterized in that the controller further comprises: a processor configured to produce a computer animation of an instructor, the animation corresponding to movements according to the exercise program; the processor is further configured to synchronize the animation with music; and the processor is further configured to provide the computer animation through the video output.

9. A controller according to claim 8, characterized in that the producing of the animation comprises simulating motion of d ifferent body parts of a 3- dimensional character.

10. A controller according to claim 9, characterized in that the character models a human being.

11.A controller according to any one of claims 8 to 10, characterized in that the synchronizing of the animation with music comprises adapting the pace of the animation according to an external source.

12.A controller according to claim 11 , characterized in that the external source comprises a music player.

13.A controller according to claim 11 , characterized in that the external source comprises an exercise intensity sensor.

14.A controller according to claim 13, characterized in that the exercise intensity sensor is a heart pulse sensor.

15. A controller according to any one of claims 11 to 14, characterized in that processor is configured to change the exercise program in order to adapt the exercise to changes in the pace beyond a given threshold.

16.A controller according to any one of claims 8 to 15, characterized in that the processor is further configured to determine the beat and tempo of the music in real-time and to synchronize the animation with the music while providing the animation to the user through the video output.

17.A controller according to any one of claims 8 to 16, characterized in that the processor is further configured to include, based on the user input, desired program elements in the exercise program suited to a given exercise area such as force, aerobic, balance etc.

18.A controller according to any one of claims 8 to 17, characterized in that the processor is further configured to providing additional computer animation for additional instructors.

19.A controller according to any one of claims 8 to 18, characterized in that the processor is further configured to individually control the additional instructors based on feedback received from different users.

20. A computer program for controlling physical exercise, comprising computer executable program code configured to cause a computer to perform the method according to any one of claims 1 to 7.

21. A system comprising a controller according to any one of claims 8 to 19 and user feedback equipment for enabling adjustment of the operation of the controller.