WO2009083831A1

WO2009083831A1 - Apparatus, method and computer program of time alignment

Info

Publication number: WO2009083831A1
Application number: PCT/IB2008/055084
Authority: WO
Inventors: Richard D. Willmann; Gerd Lanfermann; Juergen Te Vrugt; Stefan Winter; Privender K. Saini; Annick A. A. Timmermans
Original assignee: Koninklijke Philips Electronics N.V.; Philips Intellectual Property & Standards Gmbh
Priority date: 2007-12-20
Filing date: 2008-12-04
Publication date: 2009-07-09

Abstract

The present invention relates to an apparatus of time alignment for time aligning a first movement and a second movement, wherein the first movement is represented by first motion values assigned to first temporal positions and wherein the second movement is represented by second motion values assigned to second temporal positions, wherein the apparatus comprises a time alignment unit (8) for time aligning the first and second temporal positions globally. Preferentially, the time alignment unit is further adapted for time aligning the first and the second temporal positions non-linearly. It is further preferred that time alignment unit (8) is adapted for using dynamic time warping and/or hidden Markov models for time aligning the first movement and the second movement.

Description

Apparatus, method and computer program of time alignment

FIELD OF THE INVENTION

The present invention relates to an apparatus, a method and a computer program of time alignment for time aligning a first movement and a second movement.

BACKGROUND OF THE INVENTION

US 2003/0054327 Al discloses a repetitive motion feedback system with various sensors and devices for monitoring aspects of a repetitive motion sequence, such as a golf swing. The monitored aspects can include motion properties of an object moved by the user, position properties of the user and motion properties of the user. A data processing system for receiving data of the monitored aspects provides feedback data that is provided to a feedback output device, such as a graphical display device or speaker, so that the user is provided with feedback regarding the repetitive motion sequence. In one particular embodiment, the user's performance is compared to a template of a prior performance, with feedback being provided regarding the differences. The user's performance is synchronized with the template by monitoring the moment of ball impact and providing playback for recorded events that occur a selected period of time prior to and after the sensed impact of club with the ball, which is sufficient to provide playback of the entire swing.

The synchronization of the user's performance with a template of a prior performance using only the temporal position of a single event like the impact of a club with a ball is inaccurate, in particular, if movements, which have to be synchronized, are irregular, i.e., for example, if they have different moving phases, in which the movement behaves differently, or if they contain a shivering movement.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an apparatus, a method and a computer program of time alignment for time aligning a first movement and a second movement wherein the quality of the time alignment can be improved, in particular, if the movements are irregular. In an aspect of the present invention, an apparatus of time alignment for time aligning a first movement with one or several second movements is presented, wherein the first movement is represented by first motion values assigned to first temporal positions and wherein the second movement is represented by second motion values assigned to second temporal positions, wherein the apparatus comprises a time alignment unit for time aligning the first and second temporal positions globally.

Movements are generally irregular. For example, at different temporal positions, in particular, in different moving phases of the movement, the velocity and/or the directions of the movement can be different. If the time-alignment is performed globally, i.e. considering the movement behavior at different, at least two temporal positions, in particular considering a coherent movement, and not only one temporal position like the temporal position of an impact of a club with a ball like in the above mentioned prior art, the quality of the time-alignment and, thus, of the synchronization is improved, in particular, if the first and second movement are irregular. In addition, the time-alignment is unit preferentially adapted to perform a time-alignment non- linearly, i.e. e.g. not a uniform scaling of time dimensions in the first and/or second movement is applied to modify them such that they comprise the same temporal length, but at different temporal positions the time-alignment is adapted to the movement at the different temporal positions, wherein the different temporal positions can also be different temporal regions. If a uniform scaling of the time dimension in the second movement and/or in the first movement is performed to modify them such that they have the same temporal length, the alignment will be often quite poor, because of the differing movement speeds. Global and non- linear alignment of the movements allows to find the optimal match between the movements in spite of variations in movement speeds. The terms "a second movement" and "the second movement" do not exclude a plurality. For example, the time alignment unit can be adapted to time align a first movement with several second movements, in particular, by using hidden Markov models.

Preferentially the time alignment unit is adapted for time aligning the first and second temporal positions depending on differences between the first motion values and the second motion values, which correspond to time aligned first and second temporal positions. It is further preferred that the time alignment unit is adapted for time aligning the first and second temporal positions such that the differences between the corresponding first motion values and second motion values fulfill, for example, a predefined criterion. This criterion is, for example, that the first and second temporal positions are aligned such that the differences between the corresponding first and second motion values are minimized or are below a predefined threshold. Furthermore, if the time alignment is performed iteratively depending on differences between the first motion values and the second motion values which correspond to time aligned first and second temporal positions, the criterion could be that the differences are smaller than a predetermined threshold or minimized or that a predetermined number of iterations has been performed.

Only one or several motion values can be assigned to a single temporal position. The motion values are values, which describe in connection with the temporal positions a movement of an object, for example, a movement of a person or a movement of a part of a person, like a movement of one or several limbs of a person. Motion values define, for example, a one-, two- or three-dimensional position of an object or of a part of the object. The motion values can also describe an angular position of an object or of a part of the object. For example, motion values can be assigned to a temporal position, which form an image of the object or of a part of the object. Motion values can especially be joint angles between limbs, which can be represented as either quaternions (4-dimensional objects describing rotations) or Euler angles.

The difference between first and second motion values, which correspond to time aligned first and second temporal positions, is, for example, a distance between the corresponding first and second motion values. For example, if the motion values describe a spatial position, the difference is preferentially the spatial distance between a first spatial position defined by one or several motion values, which have be assigned to a first temporal position, and a second spatial position, which is defined by one or several second motion values, which are assigned to a second temporal position.

The time alignment unit is preferentially adapted such that first and second temporal positions are time aligned by assigning first and second temporal positions to each other, wherein one or several first temporal positions can be assigned to a single temporal position and wherein one or several second temporal positions can be assigned to a single first temporal position. A first and a second temporal position, which are assigned to each other, preferentially define a pair, wherein preferentially a difference is determined between the first and the second motion values, which correspond to the pair of first and second motion values. The first and second temporal positions are assigned to each other preferentially such that the differences of the first and second motion values of the pairs of the first and second temporal positions are reduced, in particular, minimized. It is further preferred that the time alignment unit is adapted for time aligning the first and second temporal positions such that a sum of differences between the first motion values and the second motion values, which correspond to time aligned first and second temporal positions, is minimized. In particular, preferentially the sum of differences between the first and second motion values of pairs of first and second temporal positions, which are assigned to each other, is minimized. This sum is preferentially a weighted sum, wherein the differences are weighted before summation. The weights are preferentially uniform weights. But, in an embodiment, the weights can also be non-uniform weights of any kind. Preferentially the time alignment unit is adapted for using dynamic time warping and/or hidden Markov models for time aligning the first movement and the second movement. The time alignment unit is adapted for time aligning the first temporal positions and the second temporal positions globally and non-linearly with relative low computational efforts, if the time alignment unit is adapted for using dynamic time warping and/or hidden Markov models. Furthermore, by using the hidden Markov models the best alignment is found based on a probabilistic measure generated from a plurality of second movements.

It is further preferred that the time alignment unit is adapted such that a chronological order of the first movement and/or a chronological order of the second movement is not altered. Since a chronological order is not altered, i.e. since an earlier temporal position with respect to a certain temporal position remains an earlier temporal position with respect to this certain temporal position after time alignment, the quality of the time alignment, i.e. of the synchronization, if further improved. Furthermore, one or several first temporal positions can be assigned to a single second temporal positions and one or several second temporal positions can be assigned to a single first temporal position, but an earlier first temporal position cannot be assigned to a second temporal position, which is later than an earlier second temporal position, which is assigned to an earlier first temporal position.

Preferentially, the apparatus further comprises a movement recognition unit for recognizing a movement, which is similar to the second movement, in the first movement and a separation unit for separating parts of the first movement, which are not similar to the second movement, from the first movement. For recognizing a movement of the first movement, which is similar to the second movement, a similarity measure can be used, wherein a movement is regarded as being similar to the second movement, if the similarity measure is below a predefined threshold. Alternatively or in addition, known recognition algorithms can be used for recognizing a movement of a first movement, which is similar to the second movement. The function of the recognition unit is preferentially to segment the desired motion, i.e. the first movement, that should be time aligned to a reference pattern, i.e. the second movement, from those motions of the first movement that are recorded accidentally and are happening, for example, either before or after the desired motion. Any pattern recognition technique is in principle usable for the recognition step. Preferentially, Neural Networks, Bayes classifiers and/or rule-based classifiers operating on morphological properties of the motion are used.

The recognition of a movement, which is similar to the second movement, in the first movement, and the separation of parts of the first movement, which are not similar to the second movement, from the first movement allow removing unwanted movements from the first movement. For example, if the first movement is a movement of a person, which should be time aligned with a second movement, which is, for example, a template, unwanted movements, like scratching of a head, which could disturb the time alignment, can be removed from the first movement, thereby further improving the quality of the time alignment of the first and second movements.

If parts of the first movement, which are not similar to the second movement, are separated from the first movement, the time alignment unit is adapted such that the first movement, from which these parts have been separated, is used for the time alignment with the second movement.

It is also preferred that the apparatus further comprises a moving phase dividing unit for dividing the first movement into several moving phases and for dividing the second movement into several moving phases, wherein the time alignment unit is adapted for time aligning corresponding moving phases of the first and second movement separately. Preferentially, the apparatus further comprises a sensing unit for sensing at least one of the first movement and second movement. This allows sensing the first movement and/or the second movement and to time align the sensed movement with another sensed movement or with a movement previous stored in a storage unit.

In a preferred embodiment, the movement of a person is sensed, for example, by using sensor elements capturing motion parameters of the patient, such as body- worn inertial sensing elements, which transmit the motion parameters to the sensing unit, or another means for motion capture such as a camera system. The sensed motion of the person is, for example, a first movement, which can be time aligned with a stored model or template movement, which can be regarded as the second movement. It is further preferred that the apparatus further comprises a presentation unit for presenting the time-aligned first and second movements. This allows a user to compare visually the first and the second movements. If, for example, the first movement is a sensed movement of a patient, who has to perform certain movements, and if the second movement is a model movement or a template movement, which should be performed by the patient, the patient can see on the presentation unit, if the movement of the patient follows the model movement or the template movement. Thus, the first movement and the second movement, which are time-aligned, are preferentially simultaneously displayed on the presentation unit. If several second movements are present, preferentially one of the second movements is displayed simultaneously with the first movement, in particular, the second movement having the largest similarity with respect to the first movement.

In a preferred embodiment, the apparatus further comprises a deviation determination unit for determining a deviation between the time-aligned first and second movements. It is further preferred that the apparatus further comprises an information presentation unit for presenting information depending on the determined deviation. Preferentially, information is presented, if the determined deviation is larger as a given threshold. The information is, for example, a marker, which marks the part of the movements, which comprise a deviation larger than a given threshold, on the presentation unit or the information presentation unit. The information can be any signal, which indicates that the time aligned movements show a deviation, in particular above a given threshold. The signal is, for example, an acoustical signal or an optical signal. The information can comprise advices, how to improve the first movement. The advices can be stored in a storage unit and be assigned to certain amounts and/or kinds of deviation between the first movement and the second movement. The advices can be presented to the user by the presentation unit and/or the information presentation unit, for example, acoustically or optically, depending on the amounts and/or kinds of deviation, to which the advices have been assigned.

In a further aspect of the present invention a method of time alignment for time aligning a first movement and a second movement is presented, wherein the first movement is represented by first motion values assigned to first temporal positions and wherein the second movement is represented by second motion values assigned to second temporal positions, wherein the first and second temporal positions are globally and preferentially non-linearly time aligned.

In a further aspect of the present invention a computer program of time alignment for time aligning a first movement and a second movement is presented, wherein the first movement is represented by first motion values assigned to first temporal positions and wherein the second movement is represented by second motion values assigned to second temporal positions, wherein the computer program comprises program code means for causing a computer to carry out the steps of the method as defined in claim 10, when the computer program is run on a computer controlling an apparatus as defined in claim 1.

It shall be understood that the apparatus of claim 1, the method of claim 10 and the computer program of claim 11 have similar and/or identical preferred embodiments, in particular, as defined in the dependent claims.

It shall be understood that preferred embodiments of the invention can also be any combination of the dependent claims with the respective independent claim.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter. In the following drawings: Fig. 1 shows schematically and exemplarily an embodiment of an apparatus of time alignment for time aligning a first movement and a second movement,

Fig. 2 shows schematically and exemplarily another embodiment of an apparatus of time alignment for time aligning a first movement and a second movement,

Fig. 3 shows schematically and exemplarily motion values of a first movement,

Fig. 4 shows schematically and exemplarily motion values of a second movement,

Fig. 5 shows schematically and exemplarily a first movement and a second movement, which are time aligned, and Fig. 6 shows a flow chart illustrating an exemplarily embodiment a method of time alignment.

DETAILED DESCRIPTION OF EMBODIMENTS

Fig. 1 shows schematically and exemplarily an embodiment of an apparatus 1 of time alignment for time aligning a first movement and a second movement. The apparatus 1 comprises a sensing unit 4, which interacts, in this embodiment, with sensor elements 5, which are attached to an arm of a person 6. The sensing unit 4 is adapted for sensing a movement of an arm of the person 6 by interacting with the sensor elements 5. The sensor elements 5 are, in this embodiment, markers, which can be identified by the sensing unit 5, which is, in this embodiment, a camera system. In another embodiment, the sensors elements 5 could also be inertial sensors, in particular, body-worn inertial sensors, which interact with the sensing unit 5 for sensing a movement of an arm of the person 6. The sensed movement of the arm of the person 6 can be regarded as first movement in this embodiment. The apparatus 1 further comprises a storage unit 3 for storing a template movement or a model movement. This template movement or model movement is, in this embodiment, a movement of an arm of a person, which has been performed by, for example, a trainer or physiotherapist and which should be performed by the person 6. This template movement or model movement can be regarded as second movement in this embodiment. The second movement can, for example, be generated by sensing a model movement performed by a person, like a therapist or a trainer, using, for example, the sensor elements 5 and the sensing unit 4 and by storing this model movement in the storage unit 3 as second movement.

The apparatus 1 further comprises a movement recognition unit 7 for recognizing a movement, which is similar to the second movement, in the first movement and a separation unit 20 for separating parts of the first movement, which are not similar to the second movement, from the first movement. For example, if the second movement, which should be followed by the person 6, consists of the phases of reaching out for a cup, grabbing it, lifting it and retracting the arm to bring the cup to the mouth etc. and if the patient 6 does another movement, like scratching the head or the like, before, in between or after imitating the second movement, the movement recognition unit 7 can recognize this movement and the separation unit 20 can separate this movement from the first movement.

The first movement can be represented by first motion values assigned to first temporal positions and the second movement can be represented by second motion values assigned to second temporal positions. An example of a first movement is shown in Fig. 3 and an example of a second movement is shown in Fig. 4.

In Fig. 3 the angular position in degree of the right upper arm 13 and of the right lower arm 14 is schematically shown versus time. As can be seen in Fig. 3, to each temporal position of the first movement two first motion values are assigned, i.e. the angular positions of the right upper arm and the right lower arm. The angular positions shown in Figs. 3 and 4 relate to joint angles as used by physiotherapists. It is the flexion angle of elbow and shoulder in the sagittal plane. The first movement schematically and exemplarily in shown in Fig. 3 can be a movement, which has been sensed by the sensing unit 4 and which can be the output of the separation unit 20.

Fig. 4 shows schematically and exemplarily a second movement, in this embodiment, a model movement or template movement, which should be performed by the person 6. In particular, the second movement shown in Fig. 4 consists of the angular positions in degree of the right lower arm 16 and the right upper arm 15 versus time. Thus, the second movement is represented by second motion values 15, 16, wherein two each temporal position two motion values are assigned, i.e. the angular position of the right upper arm and the angular position of the right lower arm.

The apparatus 1 further comprises a time alignment unit 8 for time aligning the first and second temporal positions of the first and second movements depending on differences between the first motion values and the second motion values, which correspond to time aligned first and second temporal positions. In this embodiment, the time alignment unit 8 time-aligns the first and second movement by using dynamic time warping according to following equation:

wherein

d(c) = d(i,j) = \\a₁ - b_J \\ .(2)

A dynamic time warping using equation (1) is described in more detail in Keogh, E.J. and Pazzani, M. J., Derivative Dynamic Time Warping, in First SIAM

International Conference on Data Mining, Chicago, IL, 2001, which is herewith incorporated by reference.

In equation (2) d denotes the distance between a motion value a_t of the first movement and a motion value b_] of the second movement. In this embodiment, distances between corresponding motion values of the first and second movements are determined, i.e. distances d are determined between motion values 13 of a first movement of the right upper arm and motion values 15 of the second movement of a right upper arm and between motion values 14 of the first movement of the right lower arm and motion values 16 of the second movement of the right lower arm. The determination of the distance can, in another embodiment, be determined according to another equation. The determination of a distance between first and second motion values of first or second movements preferentially depends on the kind of first and second motion values.

The variable w(n) denotes a weight for weighting the determined distances d .The weight is preferentially a uniform weight, but can, in another embodiment, also be an non-uniform weight. It is preferentially chosen such that the quality of the time alignment is further improved, for example, by calibration.

The equation (1) finds the best path (with path index k ) through a search space of all possible distances {d} between the first and second movements, i.e. the path and, thus, the time alignment is given by the minimum cumulative distance score D over all allowable paths. In equation (1), F represents all possible paths through the search space. The expression in the denominator serves to normalize the score D in equation (1) in order to make it independent of the number of points on the paths.

Thus, the first temporal positions of the first movement and the second temporal positions of the second movement are preferentially assigned to each other such that the corresponding first and second motion values define a path in the search space, which minimizes the term in brackets in equation (1), i.e. the time alignment unit 8 is preferentially adapted such that the first and second temporal positions are assigned such that a sum of differences between the first motion values and the second motion values, which correspond to the time aligned first and second temporal positions, is minimized.

A result of a time alignment performed by the time alignment unit 8 is exemplarily and schematically shown in Fig. 5.

In Fig. 5 the first motion values 14 of the first movement and the second motion values 16 of the second movement and the corresponding temporal positions are schematically and exemplarily shown together with assignments 19 between the first motion values 14 of the first movement and the second motion values 16 of the second movement and the corresponding temporal positions. The assignments 19 are denoted by lines, wherein each line connects a first motion value and a second motion value and, thus, the corresponding temporal positions. The distances between the connected first and second motion values minimize equation (1). Thus, the assignments 19 have been chosen such that equation (1) is minimized.

In other embodiments, the first and the second movement can comprise different temporal lengths, because one of these movements, in particular the first movement, can take longer than the other of these movements, wherein by the time alignment the first and/or the second movement are scaled such that they have the same length, wherein, if the first movement is temporally longer, a first movement curve has more motion values than a second movement curve, if the sample frequency of both movements is the same and the first movement took longer than the second movement.

The above described dynamic time warping time-aligns the first and second temporal positions globally and non-linearly.

In another embodiment, the alignment unit 8 can be adapted for performing a global and non-linear time alignment by using hidden Markov models. Hidden Markov Models are especially known in speech recognition to find the best alignment of a first sequence with an abstract statistical representation of the second sequence. For a more detailed description reference is made to, for example, Lawrence R. Rabiner, A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceedings of the IEEE, 77 (2), p. 257-286, February 1989, which is herewith incorporated by reference. In this embodiment, the time alignment unit 8 is adapted such that the time alignment is performed under the side condition that the chronological order of the first movement and the chronological order of the second movement is not altered by the time alignment, i.e. one or several first temporal positions can be assigned to a single second temporal positions and one or several second temporal positions can be assigned to a single first temporal position, but an earlier first temporal position cannot be assigned to a second temporal position, which is later than an earlier second temporal position, which is assigned to an earlier first temporal position, i.e. in Fig. 5 the lines denoting the assignments 19 can meet in a single first or second motion value, but they cannot intersect.

The time aligned first and second movements are transferred to a presentation unit 9 for presenting the time aligned first and second movements. The presentation unit 9 is preferentially a monitor having a display which is separated in at least two regions, wherein in a first region the first movement and in a second region the second movement is shown. Preferentially, the presentation unit 9 shows simultaneously the first motion values in the first region which correspond to a first temporal position and the second motion values in the second region, which correspond to the second temporal position, which has been assigned to the first temporal position, of which the first motion values are shown in the first region. It is further preferred that the first motion values and the second motion values are used for determining a position and preferentially a shape of an avatar, wherein in the first region an avatar is shown, which corresponds to the first motion values, and wherein in the second region an avatar is shown, which corresponds to the second motion values. In this embodiment, the representation unit 9 is adapted for presenting an avatar being a model of an arm of a person in the first region and in the second region, wherein the time aligned first and second movements are shown to a user by showing the corresponding movements of the avatar in the first and second regions. A person can, thus, see both movements and compare the first movement, which has been performed by the person 6, with the second movement, which has been stored in the storage unit 3 as a model movement or a template movement.

The apparatus 1 further comprises a deviation determination unit 10 for determining a deviation between the time-aligned first and second movements. The apparatus further comprises an information presentation unit 11 for presenting information depending on the determined deviation. In this embodiment, the information presentation unit 11 is adapted for giving acoustically advices for improving the first movement, if the determined deviation between the time aligned first and second movements exceeds a predefined threshold. Preferentially, deviations are mapped to three categories, which can be called red, yellow and green. In each category there is a standardized comment that is spoken and written on screen. Typically, stroke patients have trouble stabilizing their trunk while moving the arms. A simple cue is given if this is detected "Keep your trunk steady".

Fig. 2 shows schematically and exemplarily another embodiment of an apparatus 1 ' of time alignment for time aligning a first movement and a second movement. Similar units in Figs. 1 and 2 are denoted by similar reference signs and for a detailed description of these similar units reference is made to the above description.

The apparatus 1 ' shown in Fig. 2 differs from the apparatus 1 ' shown in Fig. 1 in that that the apparatus 1 ' comprises a moving phase dividing unit 12 for dividing the first movement into several moving phases and for dividing the second moment into several moving phases, wherein the time alignment unit 8' is adapted for time aligning corresponding phases of the first and second movements separately. The time alignment unit 8' performs the time alignment of the corresponding moving phase of the first and second movement as described above with reference to the time alignment unit 8 shown in Fig. 1. In the following an embodiment of a method of time alignment for time aligning a first movement and a second movement will exemplarily be described with reference to a flow chart shown in Fig. 6.

In step 101, the sensing unit 4 senses a movement being a first movement of the person 6. Preferentially, also in step 101, the person 6 is advised to perform a certain movement by, for example, acoustical or optical advices, in particular, by a presentation of a corresponding second movement, which is stored in the storage unit 3, on the presentation unit 9. The acoustical or optical advice to the patient can, for example, be given by the information presentation unit 11. In step 102, the movement recognition 7 recognizes a movement, which is similar to a second movement transferred from the storage unit 3 to the movement recognition unit 7, in the first movement and the separation unit 20 separates the parts of the first movement, which are not similar to the second movement, from the first movement. In step 103, the first movement, from which movements, in particular, unwanted movements, which are not similar to the second movement, have been removed, and the second movement stored in the storage unit 3 are time aligned by the time alignment unit 8.

If, in another embodiment, for example, the apparatus 1 ' shown in Fig. 2 is used for a time alignment, before step 103 the moving phase dividing unit 12 preferentially divides the first movement into several moving phases and the second movement into several moving phases, wherein the time alignment unit time aligns corresponding moving phases of the first and second movement separately in step 103.

In step 104, the time aligned first and second movements are shown on the presentation unit 9, in particular, by using avatars. Furthermore, in step 105, a deviation between the first and the second movements, which are time aligned, is determined by the deviation determination unit 10, wherein information is presented by the information presentation unit 11 in step 106 depending on the determined deviation. This allows giving a person a feedback about how the person has performed the first movement, which should be like the second movement. The apparatus 1 and the apparatus 1 ' and the corresponding methods and computer programs are preferentially used for stroke patients. The stroke patients have a variety of deficits, most permanently motor problems and deficits in their internal feedback mechanisms due to damaged brain regions. Due to this, stroke patients often have an inappropriate perception of their own movement and need external feedback on their motions to reacquire motor skills during rehabilitation. In traditional physiotherapy the therapist provides the external feedback. By using the above described apparatus, method and a corresponding computer program, a computer-generated feedback convey the necessary information about the accuracy of an executed exercise to the patient. This is, for example, achieved by showing the patient two avatars next to one another on the presentation unit 9, wherein one avatar represents the second movement, i.e. the model movement or the template movement, and wherein the other avatar represents the first movement being the movement executed by the patient and recorded by the sensing unit 4. Furthermore, an automatic feedback can be given by using the deviation determination unit 10 and the information presentation unit 11 as described above.

The avatar is preferentially a rendered figure, which represents the moving object, of which the movement is sensed.

The sensing unit 4 can, for example, be adapted to capture linear acceleration, angular velocity and/or magnetic fields, for example, by using inertial sensors as sensor elements 5, for capturing a three-dimensional motion, in particular, of limbs, to which the sensor elements are attached.

The apparatus, method and computer program can used with different reference movements, i.e. with different second movement. In particular, not a single event like hitting a ball is used for synchronization, but the full reference movement. By this, the movement itself can easily be changed from a golf swing to e.g. a baseball swing and even to movements of rehab, Thai Chi, Yoga etc, which do not involve hitting or touching an object. No pronounced points are needed for synchronization.

Although in the above described embodiments the object, which performs a first movement, is preferentially a patient or a part of patient, like one or several limbs of a patient, in other embodiments, the first and/or the second movement can be a movement of another object, for example, a technical object. Furthermore, the apparatus, the method and the computer program can, in an embodiment, be adapted for improving movements in sports. For example, a golf swing can be performed by a person, wherein the golf swing is sensed as a first movement and time aligned with a model golf swing of, for example, a trainer.

Although in the above described embodiments sensor elements 5 are used, in another embodiment, a sensing unit can be used, which does not have to interact with sensor elements for sensing a first movement. For example, the sensing unit can be a camera system, which can capture a movement of an object, like a person or like a technical object, without the need of further sensor elements attached to the object.

Although in the above described embodiments the apparatus 1 and the apparatus 1 ' comprise a movement recognition unit 7 and a separation unit 20 for separating parts of the first movement, in other embodiments, one of these units or both units can be omitted. Furthermore, although in the above described embodiment a presentation unit 9, a deviation determination unit 10 and a information presentation unit 11 are presented, in other embodiments, the presentation unit 9 and the information presentation unit 11 can be the same unit. Furthermore, in another embodiment, the presentation unit 9 or the deviation determination unit 10 and the information presentation unit 11 can be omitted.

Although in the above described embodiments, in particular, in equation (1), weighing factors w(n) are presented. In other embodiments, this weighting factor can be one or can be omitted. If the weighting factor is omitted, preferentially w(n) in the numerator is one and the enumerator is also set to one in equation (1). The processing of the different units, in particular, the calculations, the determinations and the assignments, performed by one or several units or devices can be performed by any other number of units or devices, for example, the movement recognition, the separation and the time alignment can be performed by a signal unit or by any number of units. The processing of the units, in particular, the calculations, determinations and assignments, and/or a control of the apparatus of time alignment in accordance with a method of time alignment can be implemented as program code means of a computer program and/or as dedicated hardware.

A computer program may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.

Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality.

A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measured cannot be used to advantage.

Any reference signs in the claims should not be construed as limiting the scope.

Claims

CLAIMS:

1. An apparatus of time alignment for time aligning a first movement and a second movement, wherein the first movement is represented by first motion values assigned to first temporal positions and wherein the second movement is represented by second motion values assigned to second temporal positions, wherein the apparatus comprises a time alignment unit (8) for time aligning the first and second temporal positions globally.

2. The apparatus as defined in claim 1, wherein the time alignment unit is adapted for time aligning the first and the second temporal positions non-linearly.

3. The apparatus as defined in claim 1, wherein the time alignment unit (8) is adapted for using dynamic time warping and/or hidden Markov models for time aligning the first movement and the second movement.

4. The apparatus as defined in claim 1, wherein the time alignment unit (8) is adapted such that a chronological order of the first movement and/or a chronological order of the second movement are not altered.

5. The apparatus as defined in claim 1, wherein the apparatus further comprises a movement recognition unit (7) for recognizing a movement, which is similar to the second movement, in the first movement and a separation unit (20) for separating parts of the first movement, which are not similar to the second movement, from the first movement.

6. The apparatus as defined in claim 1, wherein the apparatus further comprises a sensing unit (4) for sensing at least one of the first movement and second movement.

7. The apparatus as defined in claim 1, wherein the apparatus further comprises a presentation unit (9) for presenting the time-aligned first and second movements.

8. The apparatus as defined in claim 1, wherein the apparatus further comprises a deviation determination unit (10) for determining a deviation between the time-aligned first and second movements.

9. The apparatus as defined in claim 8, wherein the apparatus further comprises an information presentation unit (11) for presenting information depending on the determined deviation.

10. A method of time alignment for time aligning a first movement and a second movement, wherein the first movement is represented by first motion values assigned to first temporal positions and wherein the second movement is represented by second motion values assigned to second temporal positions, wherein the first and second temporal positions are globally time aligned.

11. A computer program of time alignment for time aligning a first movement and a second movement, wherein the first movement is represented by first motion values assigned to first temporal positions and wherein the second movement is represented by second motion values assigned to second temporal positions, wherein the computer program comprises program code means for causing a computer to carry out the steps of the method as defined in claim 10, when the computer program is run on a computer controlling an apparatus as defined in claim 1.