WO2005076225A1 - Posture and motion analysis using quaternions - Google Patents

Abstract

A method of generating a weighted combination of one or more elements each of which may be represented as a point on the surface of a sphere or hyper-sphere. The method includes receiving a plurality of basis elements represented as points on a common sphere or hyper-sphere; calculating an approximate spherical mean point on the common sphere or hyper-sphere based on the received points; re-expressing each of the received points as a difference from the approximate spherical mean point in a linear space selected in dependence upon the calculated approximate spherical mean point; calculating a linear weighted average difference from the approximate spherical mean point in the selected linear space, which weighted average difference is based on a received set of weights in respect of one or more of the received elements; and performing a reverse mapping operation on the linear weighted average difference to generate a spherical weighted average point back on the common sphere or hyper-sphere which thus represents said weighted combination of said received one or more elements.

Description

POSTURE AND MOTION ANALYSIS USING QUATERNIONS

Field of the Invention

The present invention relates to animation techniques for animating, in real time, 3 dimensional virtual characters such as avatars and 3 dimensional real (i.e. non-virtual) articulated characters such as robots, as well as relating to techniques for processing and manipulating other systems having a spherical geometry which require the generation in real-time of combinations of points on a sphere or other elements which are geometrically equivalent to points on a sphere (or a hypersphere).

Background to the Invention

A number of applications involve the manipulation of virtual objects, characters and scenery within a three-dimensional (3D) virtual world; the best known and currently most commercially significant such applications being so called "3D games". In such applications there is provided a "3D virtual world" in which scenery, objects and characters are located. In order to display a view of this virtual world to a user, a viewpoint of this world is selected and the resulting view is then "rendered". Rendering involves determining which elements and which parts of those elements in the world would be visible from the selected viewpoint. In general elements within the world are formed from polygons which have a number of (usually flat, triangular) surfaces each of which has an associated surface pattern or surface "texture", sometimes called a surface texture map. Having determined which surfaces and which parts of those surfaces are to be displayed, the appropriate surface texture maps are used to determine the actual values (ie colour and brightness) to be attributed to each pixel or group of pixels on the display device to generate the rendered view.

In order to render moving (rigid or articulated) objects within the virtual world, it is necessary to first determine the exact position and orientation of the objects (firstly within the virtual world and then with respect to the selected viewpoint) for each frame to be rendered. A large number of objects which represent animate characters such as humans or animals can be considered as articulated objects comprising a number of separate rigid bodies hinged together at hinge points (eg the upper arm connects to the lower arm at the respective elbow joint, etc.). Any movement of a rigid body from a first position and orientation to a second position and orientation can generally be described by a combination of a translation (of a point of the object) and a rotation (about that point). Additionally, with respect to a particular viewpoint, the rigid body may also undergo a scaling transformation (to represent the body or part thereof moving towards or away from the viewpoint), but this aspect is relevant only to the function of rendering a view once the positions and orientations within the virtual 3D world of the virtual objects to be rendered has been established. The currently preferred application of the present invention is concerned with establishing the position and orientation of virtual objects within a virtual 3D world rather than with rendering from a particular viewpoint and thus scaling transformations will not be considered further in the present document.

Complex motions to be animated can conveniently be characterised as a set of "key frames" with a particular translation and rotation between each (temporally adjacent) pair of key frames; thus one may describe a complex motion of a human character such as walking or running or performing a discrete action such as sitting down by means of a series of key frames in which each key frame is described in terms of a rotation and translation of each articulated "limb" with respect to the position and orientation of the limb in the preceding key frame (note however that generally speaking the limbs must remain attached to one another which acts as a constraint on the possible translation of each individual limb; thus provided the limbs do remain attached to one another the positions of the limbs can be completely specified by specifying the individual orientations or rotations of the limbs and the translation of only one single limb or alternatively of the centre of mass of the articulated body as a whole, etc.).

If the key frames occur less frequently in ahy given time period than the rate at which new frames are displayed to the user (the frame rate) then some method of interpolating between the key frames is required. Typically Euler angles are used for handling the rotation aspects of moving from one key frame to another. However it has been recognised that there are disadvantages with using Euler angles and it has therefore been proposed that quaternion numbers could be used instead (a discussion of quaternion numbers, which are numbers having one real and three imaginary components, and their use in representing rotations to assist in interpolating between key frames for rendering purposes is given in more detail below in the main body of this document).

Additionally, an articulated object may assume a number of different (static) postures each of which can be represented by specifying the angle of orientation of each articulated lirrib. Typically, the various limb orientations for a number of different discrete postures (eg standing, sitting, pointing a gun, etc.) will be provided (in a suitable library of postures and sets of key frames) for any given articulated object (typically a "skeleton" on which different "bodies" may be draped) for a given application (e.g. an "engine" for use in generating 3D games).

Similarly, for any given application, a single set of key frames will typically be generated (and stored in a library) for each particular complex movement (e.g. running, walking crouching etc.). In this way, an impressive posture or complex motion may be rendered at run time as required. However, because there is only a single set of key frames (or orientations), the action (or posture) always appears the same to the user each time it is animated which eventually makes the action (or posture) seem less realistic, since in nature each character may run, walk, crouch etc in many (possibly quite subtly) different manners. Having a number of different sets of key frames (or limb-orientations) for each complex movement (e.g. running without a gun, running with a gun, running (without a gun) when tired, etc.) or posture could go some way to mitigating this problem, but a large number of different such sets would be required in order to be convincing, and this would require a great effort on the part of the software developer to generate the large number of different sets.

Another application of computer graphics is in generating and editing video animations e.g. for films, television advertisements, computer game previews or introductory clips, etc. In such applications it may be desirable to take an existing animation and to edit it in order to achieve a specific effect. Editing animations in this way is currently achieved in a largely manual manner and is time consuming and difficult.

Summary of the Invention

According to a first aspect of the present invention, there is provided a method of analysing an acquired sample of a movement or posture of an object, the method comprising: determining an associated base movement or posture corresponding to the acquired sample, determining first and second sets of quaternions which represent the base movement or posture and the acquired sample movement or posture respectively, and analysing the first and second sets with respect to a plurality of reference sets of quaternions representing different known styles or variants of the base movement or posture, and, based on the analysing step, re-expressing the acquired sample movement or posture as a weighted combination of one or more of the different known styles or variants of the base movement or posture.

Preferably, the reference sets represent different moods such as happy, tired proud, etc. and the specified weightings are expressed as simple percentages which need not add to 100% such that a system may specify that a character is to perform a walk with 40% happiness and 20% tiredness. Preferably, the weightings may exceed 100% such that a system may specify that a character is to walk with 120% tiredness etc.

According to a second aspect of the present invention, there is provided a method of generating an animation of an articulated object, the method including: providing a plurality of predetermined specific sets of orientations of component parts of the articulated object, all of the sets representing a single generic posture or action with each set representing a respective different specific manner of adopting or performing the generic posture or action; calculating a mean set representing an approximate average generic posture or action formed from all of the predetermined specific sets and a plurality of difference sets representative of the differences between the orientations of the mean set and the corresponding orientations of each of the predetermined specific sets; determining a new specific manner of adopting or performing the generic posture or action as a combination of one or more of the predetermined specific manners according to specified weightings and generating a set of differences therefor; and generating an animation of the articulated object based on the set of differences for the new specific manner of adopting or performing the generic posture or action.

Preferably, the specific sets of orientations are represented using quaternions. Preferably the respective quaternions are processed to find the mean set which also comprises a plurality of quaternions. Preferably each quaternion in each specific set is then transformed into a three component number representing a point on a hyperplane which is tangential to and coincident with the unit quaternion hypersphere at the point represented by the corresponding quaternion in the mean set. Preferably, the new action or posture is then determined by considering the points on the hyperplane as vectors from the point at which the hyperplane and hypersphere are coincident, performing Euclidean vector addition on the resulting vectors scaled according to the specified weightings in respect of each orientation in the respective set to generate a resultant vector and transforming this vector back to a quaternion which is then used to generate the ultimate animation. In a preferred embodiment, a principal component analysis is performed on the transformed points on the hyperplane to generate orthogonal unit vectors which can be used as component vectors to re-characterise the transformed points. If desirable, the lowest order component vectors may be discarded to save processing for an acceptably small cost in terms of the accuracy of the resulting animations.

According to a third aspect of the present invention, there is provided a method of generating an animation of an articulated object comprising the steps of: acquiring a set of data in the form of a weighted combination of components representative of the difference between a movement or posture to be generated and a corresponding base movement or posture of an articulated object; performing a calculation of the corresponding movement or posture using pre-stored representations of the base movement or posture and pre- stored representations of the difference from the base movement or posture of the individual components with predetermined weightings; and generating the animation of the articulated object in dependence upon the calculated movement or posture.

According to a fourth aspect of the present invention, there is provided a method of transmitting information about the movement or posture of an articulated body from a transmitting device to a receiving device comprising analysing the movement or posture in accordance with the method of the first aspect of the present invention, and transmitting the re-expressed movement or posture to the receiving device. Preferably, this aspect of the invention further includes using the method of either the second or third aspect of the present invention to generate the animation at the receiving device.

Preferably, the weighted combination comprises at least two different components or different known styles or variants of the base movement or posture.

According to a fifth aspect of the present invention, there is provided a method of editing an animation of an articulated object comprising the steps of expressing the animation to be edited in terms of a base movement or posture, and a weighted combination of one or more known styles or variants of the base movement or posture; and varying the one or more weights in the weighted combination to alter the animation. By using styles or variants which can be associated (in an editor's mind) with an internal state of the object being animated (e.g. tired, happy, etc.) an editor has a powerful tool for altering an animation to express a desired internal state of the object.

The present invention is also applicable to animations of simple rigid bodies instead of or in addition to articulated bodies. Additionally, the techniques of the present invention may be adapted for use in analysing and animating the movements of non-rigid bodies by approximating the non-rigid bodies as rigid or articulated bodies.

According to a sixth aspect of the present invention, there is provided a method of generating a weighted combination of one or more elements each of which may be represented as a point on the surface of a sphere or hyper-sphere, the method comprising receiving a plurality of basis elements represented as points on a common sphere or hyper-sphere, calculating an approximate spherical mean point on the common sphere or hyper-sphere based on the received points, re-expressing the received points as points on a plane or hyper-plane selected in dependence upon the calculated approximate spherical mean point, calculating a linear weighted average point on the plane or hyper-plane which is based on a received set of weights in respect of one or more of the received elements, and performing a reverse mapping operation on the linear weighted average point to generate a spherical weighted average point back on the common sphere or hyper-sphere which thus represents said weighted combination of said received one or more elements.

Preferably, the selected plane or hyper-plane is approximately tangential to and coincident with the common sphere or hyper-sphere at the approximate spherical mean point, or mathematically equivalent to such a plane or hyper-plane, and the mapping function is such that each point on the plane or hyper-plane has the same or a proportional linear distance from the mean point as the corresponding spherical distance between the corresponding point on the common sphere and the mean point.

Examples of elements which may be combined in this way include: camera angles in three dimensional Euclidean space; the orientation of rigid or articulated bodies; and features on a spherical or approximately spherical body. The present invention also provides methods analogous to the third, fourth and fifth aspects of the present invention based on the method of the sixth aspect of the present invention.

The present invention also provides apparatus, devices and computer programs for carrying out the above mentioned methods.

Brief description of the Figures

In order that the present invention may be better understood, embodiments thereof will now be described, by way of example only, with reference to the accompanying drawings in which:

Figure 1 is an illustration of a general purpose computer system which may form the operating environment of embodiments of the present invention;

Figure 2 is a system block diagram of the general purpose computer system of Figure 1 ;

Figure 3 is a schematic illustration of a display on the computer system of Figures 1 and 2 of two virtual articulated bodies adopting base postures;

Figure 4 is a schematic illustration similar to Figure 3 but in which the bodies are adopting variants of the base postures shown in Figure 3;

Figure 5 is a flow chart illustrating the procedure for generating an animation of an articulated object according to a preferred embodiment of the present invention;

Figure 6 is a flow chart of the pre-processing subroutine of Figure 5;

Figure 7 is a flow chart of the subroutine of Figure 6 for calculating the approximate mean spherical point; and

Figure 8 is a flow chart illustrating the procedure for analysing the movement of an articulated object according to a preferred embodiment of the present invention.

Detailed description of a First Embodiment A first embodiment of the present invention which enables postures and complex movements of articulated objects to be animated, edited and transmitted efficiently, will now be described.

Description of the User Terminal

In the first embodiment, an application program is implemented on a general purpose computer system such as that illustrated in Figure 1 and described in greater detail below. Some of the data used by the application program of the first embodiment, together with other computer programs operating to process (or pre-process) that data may be stored on other computer devices, especially computer servers and databases connected to the general computer system of Figure 1 via a computer network. As with the general computer system of Figure 1 , such devices are well known in the art and are in essence very similar to the computer system illustrated in Figure 1 with different emphases (e.g. a computer server is unlikely to contain a graphics card or a sound card but is likely to contain multiple Central Processor Unit (CPU) processors, etc.).

Figure 1 illustrates a general purpose computer system which provides the operating environment of the embodiments of the present invention. Later, the operation of the invention will be described in the general context of computer executable instructions, such as program modules, being executed by a computer. Such program modules may include processes, programs, objects, components, data structures, data variables, or the like that perform tasks or implement particular abstract data types. Moreover, it should be understood by the intended reader that the invention may be embodied within other computer systems other than those shown in Figure 1 , and in particular hand held devices, notebook computers, main frame computers, mini computers, multi processor systems, distributed systems, etc. Within a distributed computing environment, multiple computer systems may be connected to a communications network and individual program modules of the invention may be distributed amongst the computer systems.

With specific reference to Figure 1 , a general purpose computer system 1 which forms the operating environment of the embodiments of the invention, and which is generally known in the art, comprises a desk-top chassis base unit 100 within which is contained the computer power unit, mother board, hard disk drive or drives, system memory, graphics and sound cards, as well as various input and output interfaces. Furthermore, the chassis also provides a housing for an optical disk drive 110 which is capable of reading from and/or writing to a removable optical disk such as a CD, CDR, CDRW, DVD, or the like. Furthermore, the chassis unit 100 also houses a magnetic floppy disk drive 112 capable of accepting and reading from and/or writing to magnetic floppy disks. The base chassis unit 100 also has provided on the back thereof numerous input and output ports for peripherals such as a monitor 102 having a display screen 3 (see figures 3 and 4) used to provide a visual display to the user, a printer 108 which may be used to provide paper copies of computer output, and speakers 114 for producing an audio output. A user may input data and commands to the computer system via a keyboard 104, or a pointing device such as the mouse 106.

It will be appreciated that Figure 1 illustrates an exemplary embodiment only, and that other configurations of computer systems are possible which can be used with the present invention. In particular, the base chassis unit 100 may be in a tower configuration, or alternatively the computer system 1 may be portable in that it is embodied in a lap-top or note-book configuration. Other configurations such as personal digital assistants or even mobile phones may also be possible.

Figure 2 illustrates a system block diagram of the system components of the computer system 1. Those system components located within the dotted lines are those which would normally be found within the chassis unit 100.

With reference to Figure 2, the internal components of the computer system 1 include a mother board upon which is mounted system memory 118 which itself comprises random access memory 120, and read only memory 130. In addition, a system bus 140 is provided which couples various system components including the system memory 118 with a processing unit 152. Also coupled to the system bus 140 are a graphics card 150 for providing a video output to the monitor 102; a parallel port interface 154 which provides an input and output interface to the system and in this embodiment provides a control output to the printer 108; and a floppy disk drive interface 156 which controls the floppy disk drive 112 so as to read data from any floppy disk inserted therein, or to write data thereto. In addition, also coupled to the system bus 140 are a sound card 158 which provides an audio output signal to the speakers 114; an optical drive interface 160 which controls the optical disk drive 110 so as to read data from and write data to a removable optical disk inserted therein; and a serial port interface 164, which, similar to the parallel port interface 154, provides an input and output interface to and from the system. In this case, the serial port interface provides an input port for the keyboard 104, and the pointing device 106, which may be a track ball, mouse, or the like.

Additionally coupled to the system bus 140 is a network interface 162 in the form of a network card or the like arranged to allow the computer system 1 to communicate with other computer systems over a network 190. The network 190 may be a local area network, wide area network, local wireless network, or the like. In particular, IEEE 802.11 wireless LAN networks may be of particular use to allow for mobility of the computer system. The network interface 162 allows the computer system 1 to form logical connections over the network 190 with other computer systems such as servers, routers, or peer-level computers, for the exchange of programs or data.

In addition, there is also provided a hard disk drive interface 166 which is coupled to the system bus 140, and which controls the reading from and writing to of data or programs from or to a hard disk drive 168. All of the hard disk drive 168, optical disks used with the optical drive 110, or floppy disks used with the floppy disk 112 provide non-volatile storage of computer readable instructions, data structures, program modules, and other data for the computer system 1. Although these three specific types of computer readable storage media have been described here, it will be understood by the intended reader that other types of computer readable media which can store data may be used, and in particular magnetic cassettes, flash memory cards, tape storage drives, digital versatile disks, or the like.

Each of the computer readable storage media such as the hard disk drive 168, or any floppy disks or optical disks, may store a variety of programs, program modules, or data. In particular, the hard disk drive 168 in the embodiment particularly stores a number of application programs 175, application program data 174, other programs required by the computer system 1 or the user 173, a computer system operating system 172 such as Microsoft® Windows®, Linux™, Unix™, or the like, as well as user data in the form of files, data structures, or other data 171. The hard disk drive 168 provides non volatile storage of the aforementioned programs and data such that the programs and data can be permanently stored without power.

In order for the computer system 1 to make use of the application programs or data stored on the hard disk drive 168, or other computer readable storage media, the system memory 118 provides the random access memory 120, which provides memory storage for the application programs, program data, other programs, operating systems, and user data, when required by the computer system 1. When these programs and data are loaded in the random access memory 120, a specific portion of the memory 125 will hold the application programs, another portion 124 may hold the program data, a third portion 123 the other programs, a fourth portion 122 the operating system, and a fifth portion 121 may hold the user data. It will be understood by the intended reader that the various programs and data may be moved in and out of the random access memory 120 by the computer system as required. More particularly, where a program or data is not being used by the computer system, then it is likely that it will not be stored in the random access memory 120, but instead will be returned to non-volatile storage on the hard disk 168. ,

The system memory 118 also provides read only memory 130, which provides memory storage for the basic input and output system (BIOS) containing the basic information and commands to transfer information between the system elements within the computer system 1. The BIOS is essential at system start-up, in order to provide basic information as to how the various system elements communicate with each other and allow for the system to boot-up.

Whilst Figure 2 illustrates one embodiment of the invention, it will be understood by the skilled man that other peripheral devices may be attached to the computer system, such as, for example, microphones, joysticks, game pads, scanners, digital cameras, or the like. In addition, with respect to the network interface 162, we have previously described how this is preferably a wireless LAN network card, although equally it should also be understood that the computer system 1 may be provided with a modem attached to either of the serial port interface 164 or the parallel port interface 154, and which is arranged to form logical connections from the computer system 1 to other computers via the public switched telephone network (PSTN).

Where the computer system 1 is used in a network environment it should further be understood that the application programs, other programs, and other data which may be stored locally in the computer system may also be stored, either alternatively or additionally, on remote computers, and accessed by the computer system 1 by logical connections formed over the network 190. Quaternion Representation of Rotation

Rotations in 3-space are vital to the generation of human motion, which is represented as a set of joint rotations. However, representing them can be difficult. The most fundamental representation is an axis of rotation and an angle about that axis. However, this representation is difficult to manipulate. It is difficult to smoothly interpolate animations in this form. Another approach is to use Euler angles, these are three angles, one about each of the co-ordinate axes. This is a popular method but problematic, as it treats the rotation as three independent rotations about each axis. However, the rotations are not independent. One problem is "gimbal lock", a rotation of 90 degrees about one axis can rotate the other two axes so that they are co-linear and so a degree of freedom is lost, making the rotations about the other two axes have incorrect results. There are also problems with interpolation and therefore weighted sums. There is more than one way to decompose a rotation into Euler angles, and the different ways give very different results for interpolation. Also transforming the co-ordinate axes changes the result. This makes them unsuitable for interpolating animations and for weighted sum methods. To get over these problems a representation is needed that allows smooth interpolation of rotations. Quaternions provide this sort of representation.

Quaternions were first introduced by William Hamilton in the 1830s as a generalisation of complex numbers to 3 dimensions. They use three rather than one imaginary component: q = w+xi + yj + zk Shoemake (1985) (see bibliography at end of description for full reference details of this and other papers mentioned in this specification) suggested them as a representation of rotations in R3. A rotation:

I" » \^axis ' Saxis ' ^axts Al

Is represented as a quaternion with:

x = x_axis Φ) y = y_axis sin{θ/2) z = z_aχi_s sin(0/2) where _a is, y _xi_s and z_aχi_s are the x, y and z components of the axis of rotation, which can be represented as a vector u_= {x_axis, y_axfe, z_axis). Quaternion Rotations

A vector y: v = {x, y, z)

is rotated by a quaternion q by first converting it into a quaternion: v = 0 + xi + yj + zk

and then performing the multiplication:

(see Shoemake 1985 for details of the quaternion multiplication and inverse operations).

All quaternions that represent a valid rotation are unit length. Thus a useful geometric model for quaternions is to consider them lying on the unit hyper-sphere in R4. A non-unit length quaternion will not represent a valid rotation and so needs to be normalised. This will be important for our further discussion.

Quaternion Weighted Sums The spherical geometry of quaternions means that ordinary weighted sums do not work for them. In general the simple weighted sum of a set of quaternions will no longer lie on the sphere. This means that it will have to be projected back onto the sphere. However, a weighted sum should minimise the weighted mean square error from the points (Buss and Fillmore 2001). The projected point is not the error minimising point in terms of spherical distance (i.e. angle).

Shoemake (1985) presented a method that solves this problem for the case of linear interpolation. It performs an interpolation based on spherical distance (i.e. angle) that is an accurate, smooth linear interpolation between two angles. This method, Spherical Linear Interpolation (or "slerp") does not generalise to more than two quaternions, however. Buss and Fillmore (2001) have presented a method for calculating averages of points on a sphere that minimise weighted mean square spherical distance. They use an exponential map, which is a mapping between the surface of the (hyper) sphere and a tangent (hyper) plane at a point p which maintains spherical distance from p. The point to be mapped is first mapped to a co-ordinate system where p points along one axis and the other axes are the axes of the tangent hyper-plane. A point pi on the sphere is then mapped onto the tangent hyperplane (the logarithmic map) by: θ ^{1 J} smθ

Where p,y is the component of p, along the /^h axis (for axes 1..n where the dimensionality is n+1 , axis 0 being the axis pointing along p normal to the tangent plane) and θ is the angle between p, and p. The inverse mapping (the exponential map) is:

Pm = ^cos(^r)

Where r is the distance in the tangent plane between p, and p, the mapping ensures that r= θ.

A weighted average can be computed in the tangent hyper-plane using linear distance and then mapped back to the hyper-sphere. As distances to the origin of the tangent point are preserved this weighted average will be the exact spherical weighted average when the tangent is taken at the spherical weighted average point. Buss and Fillmore use an iterative technique that uses the tangent plane at successive approximation to the average to compute the next approximation. This method will produce an arbitrarily close approximation. It is fairly efficient and sufficient for many real time applications but the iterative process means that it is does provide a moderate overhead which renders it insufficiently quick for many computationally intensive applications such as modern 3D games. Johnson (1999) presented a similar method that did not, however, attempt to find the correct tangent hyper-plane in which to perform the interpolation. Various methods related to this technique are presented below after a brief discussion of 3D computer animation. Firstly a fast approximation to the spherical weighted average is presented. Then a method for interpolating differences from a mean quaternion using the tangent plane is described. Finally an exact method for calculating weights that correspond to a particular example quaternion is described.

3D Animation and Motion Capture Motion capture is becoming the industry standard way to produce realistic animation for computer generated characters. An actor performs an action while having their motion tracked by special sensors. The motion is then mapped onto the animated character. This is a very useful technique for off-line film animation but it has a number of drawbacks for real-time interactive animation, such as computer games. The motion that the character makes must be known in advance so that it can be captured. With real time animation the characters actions are only known at run time so they cannot be captured in advance. Currently this problem is solved but capturing (or hand animating) a limited set of actions and playing the closest one that fits a situation. However, this will only produce a roughly matching motion so the quality of the animation is not that great. Recently, there has been a lot of research on how to reuse an existing motion in a new context so as to be able to synthesise new appropriate motion from a limited set of pre-captured motions, see Gleicher (2001) for an overview.

One promising approach (Rose, Cohen and Bodenheimer 1998) is to use a set of motions of the same type (e.g. walking, sitting down) but with different styles (e.g. "happy", "angry"). Combining these motions will create a new motion with a different style. One way of doing this combination is to use a weighted sum (Rose et al. in fact use a combination of a linear weighted average and radial basis functions). One difficulty with this approach is that it uses the Euler angle representation of joint rotations, this representation has a number of problems as described below, and the quaternion representation is generally more accurate. However, a technique for quaternion weighted sums has only recently been introduced and may not be efficient enough for real time applications (ie Buss and Fillmore 2001).

The present embodiment provides an efficient method for calculating approximate quaternion weighted sums and enables the synthesis of a new motion from sets of existing motions. The present embodiment also provides an inverse procedure which can be used to extract stylistic data from motion for editing and motion information compression and transmission purposes. Note that the methods described herein are applicable to any system which is mathematically equivalent to a set of points on a sphere, and thus the present invention in its most general aspects also applies to such systems. These include systems that actually are points on sphere, e.g. points on the earth's surface. Also, there are other systems that are roughly spherical for example points on a human head. Also, systems which involve directions from a point, including applications such as camera direction, light sources, star fields and heat sources are included. Weights in these systems could represent another factor, such as distance for stars or intensity of a light or heat source.

Fast Approximate Weighted Sums

As will be made clear below, the method of the present embodiment requires determining a hyperplane onto which quaternions representing a posture or movement of an articulated body may be projected using the transformations described above. The "fast approximate weighted sums" method described hereinbelow represents one possible way of identifying an appropriate hyperplane for this purpose. It may additionally be used in the alternative applications discussed briefly above in order to identify an appropriate plane in 3-Space applications (eg for manipulating points on a sphere or approximate sphere such as a human head). The aim of the "fast approximate weighted sums" method in the present embodiment is to find a plane which is tangential to and coincident with the relevant hypersphere or sphere, on which a set of points are located, at a point on the sphere which is, approximately the "mean" of the set of points being considered as determined by the "fast approximate weighted sum" method.

A significant benefit of the "fast approximate weighted sum" method described below is the speed with which it can be calculated compared with the more accurate method described in the paper by Buss and Fillmore (for details of which see the section at the end of the description sub-titled "references").

In real time motion synthesis applications such as computer games often there are very strong constraints on performance, so algorithms need to be very efficient. However, the results need only be visually accurate not exact. The "fast approximate weighted sum" method is therefore a fast method to compute approximate weighted sums on quaternions (or in general on the sphere). The method is based on an analogy to linear weighted sums, which are of the form: p = w₀p₀ + w_lp_l + — + w_np_n

Where the w, are the weights and the p_/ are the points. This formula cannot be evaluated directly on the sphere however it is possible to perform linear interpolation on the sphere (slerp). Additionally, it is possible to redefine linear weighted sums in terms of a sequence of linear interpolations:

By analogy the present "fast approximate weighted sum" method uses a weighted average in terms of "slerp": wAv({w₀ ,p₀)---{w_n, p_n )) =

As an example, consider the following three unit (to 3 decimal places) quaternions:

qi = [0.5, 0.5, 0.5, 0.5] q₂= [0.717, 0.0, 0.717, 0.0] q₃= [0.717, 0.5, 0.0, 0.5]

As mentioned above, this method relies on the slerp (spherical linear interpolation) on quaternions. The symbol Θ instead of + to signify that the weighted sum does not consist of normal linear addition but rather the fast approximate weighted sum method. Thus a weighted sum becomes something like:

1/3 q^ θ ₂ q₂ θ 1/6 q₃

The first step is to re-bracket the sum to make it into a linear interpolation between two terms of the form:

w q_! θ (1-w) q

Where q is a composite term. This is done by separating the first term from the rest, thus: 1/3q₁ώ(%q₂Φ1/6q₃)

A coefficient of (1-1/3) = 2/3 is taken out of the second term:

1/3 q, θ 2/3(3/4 q₂ θ 1/4q₃)

The method then recourses onto the second term converting it into a linear interpolation. As it is already in this form in the case where there are only 3 quaternions to be "summed" no more work needs to be done in the present example. The linear interpolations can now be calculated by slerp:

Slerp(1/3, q.,, slerp(3/4, q₂, q₃))

= slerp(1/3, q_1f [0.797, 0.407, 0.213, 0.407])

= [0.611,0.478,0.411,0.478]

A second example has all equal weightings (i.e. equal weighting coefficients) of 1/3, 1/3, 1/3 but the same three quaternions are used as before:

=1/3q₁θ(1/3q₂θ1/3q₃)

=1/3 q_! θ 2/3(1/2 q₂ θ 1/2q₃)

=Slerp(1/3,q₁,slerp(1/2,q₂,q₃))

= slerp(1/3, q_1f [0.824, 0.287, 0.412, 0.287])

= [0.622, 0.438, 0.481, 0.438] This is a non-iterative technique that in tests was found to be approximately an order of magnitude faster than Buss and Fillmore's techniques. To be more accurate it was approximately 15 times faster than their linear convergence rate technique. They also describe a quadratic convergence method that was not implemented by the present inventors but according to tests reported by Buss and Fillmore took slightly more than half the time of the linear method (such that the "fast approximate weighted sums" method would be about 8 times faster).

Unfortunately the "fast approximate weighted sum" method is no longer an exact method in the sense that because multiple "slerps" are used it no longer produces an exact spherical weighted average as defined in the paper by Buss and Fillmore. However, in tests the errors produced relative to the exact method were low, about 0.5% on average and less 2% over all of the tests carried out by the inventors. This is small enough not to produce a visible difference in graphical output. This means it is usable in computer animation applications, as long as the weighted averages are not performed repeatedly, in a way that would cause errors to build up.

A table showing the results of such tests is set out below:

TABLE

Buss & Fillmore timing fast average difference in res 620 40 0.00900742 371. 20 0.00544449 330 21 0.0049507 861 60 0.00673873 471 30 0.0064882 761 50 0.00485738 330 20 0.00309238 271 20 0.00582934 641 40 0.0128557 731 40 0.00701619 310 20 0.00718465 611 40 0.0151757 271 20 0.00182419 110 10 0.000801456 400 30 0.0137869 421 20 0.00865561 150 10 0.0038378 681 30 0.00420297 781 40 0.0176745 281 10 0.00192402 751 50 0.00782766 330 21 0.00308639 480 20 0.0040702 331 20 0,.00576211 440 20 0, .00401207 391 10 0, .012595 551 20 0, .00485274 671 40 0, .0125601 701 50 0, .0120304 360 10 0, .00827823 mean 480.3 27.73333333 0. .007214108 variance 39391.21 190.2622222 1. ■77454E-05 standard deviation 198.4721895 13.79355727 0. .004212526

The above timings are in milliseconds and clearly show the increased speed of the fast approximate method. The differences are fractions and clearly show the differences to be generally of the order of 1 %.

Interpolated differences from a mean

For a given set of quaternions representing the orientations of a rigid body (in different variations of a generic posture or movement of a rigid body or articulated object of which the rigid body is a part), the "fast approximate weighted sum" method or an alternative method such as that described by Buss and Fillmore can therefore be used to find an approximate mean point of the quaternions (or in corresponding 3D spherical applications, the approximate mean spherical point). In one embodiment of the present invention, this mean point is then used for setting the point on the hypersphere (or sphere) at which to "locate" the corresponding hyperplane (or plane) for use in the method of the present embodiment in which the various points on the hypersphere represented by the quaternions (or the points on the sphere in 3D spherical cases) are mapped onto the corresponding hyperplane (or plane). The origin (0,0,0) of this hyper plane (or origin (0,0) of a plane) is then set to be the approximate mean point calculated in this way such that the points of the quaternions when mapped onto this hyperplane represent the difference from the approximate mean of each of these quaternions.

Having specified these points in this way, it is then possible to generate a new quaternion q_n (or point on a sphere) by specifying "weights" for the known quaternions (i.e. if three quaternions q1 , q2 and q3 are used, an approximate mean quaternion q_m is found; thereafter a new quaternion can be generated which is say 10% q1 and 20% q2 and 0% q3 to generate a new quaternion q_n) and generating a standard linear weighted average of the mapped points on the hyperplane (or plane) to generate a new point on the hyperplane (or plane) which is then reverse mapped back onto the hypersphere (or sphere) to generate the new quaternion q_n (or point on the sphere). Since the tangent plane is known (and can in many applications be pre-calculated in advance) no iteration is required as in the Buss and Fillmore method and new quaternions can be generated very quickly indeed.

One significant benefit of this method is that there is no requirement that the "weights" need to add to 1. It is often desirable to have the freedom to use weights which do not add to i, for example to enable extrapolations (eg "I want a 120% happy walk). Doing this with a simple weighted sum is undesirable as it would scale the entire motion rather than just increasing the "happy" components. To perform extrapolations, a mean of the set of motions is taken instead and then each motion has this mean subtracted from it. A weighted sum of these differences can then be added to the mean to generate a new motion as described above. Giving weights that do not sum to 100% will no longer scale the entire motion, only the components.

Application of method of present embodiment to avatar postures Referring now to Figure 3, a pair of virtual articulated objects in the form of computer avatars 5, 6 (i.e. virtual humanoid figures) are shown displayed within a window 4 on the display 3 of the computer system 1 illustrated in Figures 1 and 2. Also shown on the display 3, but in a separate window 7 is a user interface comprising a plurality of slider bars by which a user may cause the posture of the avatars 5 and 6 to be altered by varying one or more of a set of weightings. For example, in Figure 3, the avatars 5 and 6 are both in their "base" or neutral postures in which the figures are erect, with hands by their sides and feet together (note the base posture here plays the same role as a mean posture in which each quaternion in the set of quaternions used to represent this base posture corresponds to the "mean" point at which a tangent hyperplane is formed for subsequent generation of new quaternions to generate variations of a standing posture). Note that in some cases it will be preferable to choose a non-mean base posture (or movement) rather than the mean of a set of variations on the theme, because a more natural base posture (or movement) can be intuitively selected; this is the case in the example chosen in Figure 3 where the base posture is a basic upright standing posture as is commonly used in 3D graphics for initially defining the innate characteristics of a particular humanoid figure.

However, in Figure 4 avatar 6 (referred to as Bill in the user interface in window 7) is in a variant posture formed by setting the "machismo" variant weighting to 14 (which in this example corresponds to a weighting of 140%). As can be seen in Figure 4, with this weighting the posture is varied such that the avatar's feet are apart somewhat and its arms are held forwards away from the body and slightly apart, but with the torso and head held straight up as in the neutral posture.

Similarly, avatar 5 (Bob) is also in a non-neutral position which, in this example (although it cannot be clearly seen in Figure 4 because the relevant part of the user interface in window 7 has been obscured by window 4), has been formed by a weighting of -8 (i.e. - 80%) for the characteristic "superiorityToOther" and a weighting of 5 (i.e. 50%) for the "friendliness" characteristic. In this example, this has resulted in a posture in which the arms are held parallel but slightly forward, feet together (as in the neutral posture) and head and torso bowed forwards.

In order to enable the display of an avatar 5, 6 in Figures 3 and 4 to be generated, the animation system requires a set of orientations of each of the rigid bodies used to make up the displayed figures 5, 6. In the present embodiment, these orientations are expressed as quaternions representing the rotations of the rigid bodies making up the figures from predefined initial positions (which in the present embodiment are, in fact, the positions of the rigid bodies as illustrated in the base postures shown in Figure 3). For ease of understanding, the rigid bodies may be referred to as "limbs" but of course they include vertebrae (or groups of vertebrae) as well as jaws and even possibly things like face muscles, etc (basically anything which can move) depending on the sophistication of the object/model.

In addition, the system is provided with further sets of quaternions representing the orientations of each of the rigid bodies forming one of the figures when standing in an alternative posture. In the present example, 8 different postures which represent different variations or species of the generic standing pose have been captured. The 8 different postures are called "flirtatiousness" "friendliness" "machismo" "otherLiking" "sad" "selflmportance" "shyness" "superiorityToOther" (each of these postures is reasonably self-explanatory, although naturally somewhat subjective) and are used as "attributes" in generating new postures (note that in fact in the present example, the pose used for the "superiorityToOther" actually represents a weighting of -100% rather than the more normal +100%). Each of these different postures has a corresponding set of quaternions representing the orientations of individual component rigid bodies (eg the left and right fore-arm, upper-arm, upper-leg, etc.).

In order to generate a new posture, weightings for each of the "attributes" are set and then the "interpolation from a difference" method described above is used to calculate a new set of quaternions for the new posture and this information is fed back into the animation engine to generate and display the figure in the new posture. For example, Figure 4 shows Bill 6 adopting a new posture in which the attribute "machismo" has been given a weighting of 14 (out of 10, i.e. 140%) and the other attributes have been set to zero.

The method of generating such variant postures according to the present embodiment is now described with reference to Figures 5, 6 and 7. Thus, referring to Figure 5, the method commences at subroutine S100 in which pre-processing is carried out to determine the appropriate hyper-plane to use for mapping the points in a base set of postures (each of which is represented by a set of quaternions). The details of this pre- processing subroutine are described below with reference to Figure 6.

Having calculated the relevant hyper-plane to use in respect of a base set of variations of a basic posture, the method proceeds to step S20 in which the desired weightings for the new posture to be generated are received; this can be done in any number of ways such as, for example, by a user selecting the weightings using a suitable user interface such as a set of slider bars, or alternatively by an automatic selection means (eg to enable a 3D games engine to select suitable weightings based on the state of a particular character, etc.).

At step S30 the weightings are used together with the points on the hyper-plane corresponding to the respective variant postures to generate a new point on the hyperplane (ie by considering each point as a vector, changing its magnitude according to its respective weighting and then performing vector addition of all surviving vectors to generate the new point) in respect of each quaternion needed to assemble a complete posture. At step S40 the resulting point on the hyper-plane is then mapped back onto the corresponding hyper-sphere to generate a corresponding new quaternion in respect of each quaternion needed to form the new posture.

The resulting set of quaternions is then output in step S50. The output set of quaternions is then fed to an appropriate graphics engine to generate the resulting image of the avatar adopting the new posture.

It should be noted that all of steps S20 to S50 can be carried out very quickly on a modern processor and, since the pre-processing step can be carried out in advance,' this method of posture generation is well suited to use in real-time applications, as well as providing an excellent method to enable editors to fine-tune the postures of animated characters in animated film, etc.

Referring now to Figure 6, the pre-processing subroutine S100, in the present embodiment, at step S110 receives a set of basis "spherical points" which in this example comprise a set of quaternions. Note that for a posture (i.e. such as avatar 6 standing) there will actually be a set of sets of quaternions initially entered, with each set corresponding to a different variant posture and comprising the quaternions needed to describe the orientation of each "limb" in respect of a single posture. These sets must then be reformed into new sets of corresponding quaternions, i.e. into a set of all of the different quaternions representing the orientation of the left upper arm limb, a set of all of the different quaternions representing the orientation of the left lower arm limb, etc. Each of these reformed sets must then be pre-processed by subroutine S100 to calculate the appropriate corresponding hyper-plane. Figure 6 illustrates the steps involved in calculating one such hyper-plane only, but it is to be understood that, for postures, this is to be repeated for each "limb" (that is for each rigid element of the articulated body). For actions (as is discussed below), this subroutine will need to be repeated for each limb in each key frame!

In step S120, it is determined if the set includes a preferred neutral point. (For example, the neutral position adopted by the avatars in Figure 3 is one which is chosen by design as a neutral standing position. Similarly it might be preferred to choose an a priori neutral sitting position. In other cases however, it may be preferred to try to find the mean of a set of postures. Thus, for an unusual posture such as typing it might be difficult to choose an appropriate neutral posture and in such a case it may be advantageous to calculate a neutral posture as the approximate mean posture of an input basis set of variant postures. This is particularly likely to be the case for actions rather than postures. If there is a pre- specified preferred neutral posture, then a corresponding preferred neutral point will appear in each of the reformed sets being processed by the subroutine 100.) If the set being processed includes a preferred neutral point, the method proceeds directly to step S130. Otherwise, the method proceeds to step S130 via subroutine S200 in which a neutral point is calculated. Subroutine S200 is described in greater detail below with reference to Figure 7.

In step S130, the appropriate hyper-plane for the particular set of quaternions being processed is calculated, and in particular the appropriate mapping rules for mapping each quaternion in the set of quaternions onto the corresponding hyper-plane are determined. Also calculated are the inverse mapping rules for enabling a point on the hyper-plane to be mapped back to a corresponding quaternion. The actual mapping and inverse mapping rules used are discussed above in the section headed "Quaternion Weighted Sums".

The method then proceeds to step S135 in which the mapping rules determined in step S130 are employed to generate a new set of points on the hyper-plane each of which corresponds to one of the quaternions in the input set of quaternions.

In step S140 the results of the calculations performed in step S130 are output to the next stage in the method. Exactly what is output may vary depending on what will subsequently be required by the remainder of the method employing the output of the preprocessing subroutine. In the case of the generation method of Figure 5, what is required is the vectors representing the points on the hyper-sphere together with the reverse mapping rules so that the new point generated on the hyperplane can be converted back to a quaternion.

Figure 7 illustrates the sub-steps of subroutine S200 in which a neutral point is calculated in respect of a set of spherical points (eg quaternions representing valid orientations) which does not contain a preferred neutral point. In the present embodiment this is performed using a fast approximate method based on performing "Multiple Slerp" as described above in the section entitled "Fast Approximate Weighted Sums".

Thus, at step S210 the set of spherical points is received for further processing. At step S220 the "Multiple Slerp" method is used with equal weightings on each of the input set of spherical points to generate the approximate mean spherical point of the input set. Finally, in step S230, the approximate mean spherical point calculated in this way is output (to whichever method has called the subroutine - i.e. in this case back to step S130 in the pre-processing subroutine S100).

Inverse Weighted Sums

Given a set of quaternion basis values and a new example it can be useful to calculate what set of weights over the basis values will produce the example. For example, given a new piece of motion it could be used to calculate the style of this motion, or given a new posture, to calculate the style of that posture, etc.

For the simple weighted sum case we need to calculate a set of weights which when applied to the basis values produce the example. In this case the result of the weighted sum is already known (it is simply the example) so the calculations can be done in (exactly) the correct tangent hyper-plane. The logarithmic map is used to map the basis values into the tangent plane at the example point. The values are then in a linear system and it is therefore possible to form a set of linear equations for the weights:

The equation has two components. Q is a matrix of the mapped basis values which when summed with the weights w sum to the zero vector. The second line is a further equation which states that the weights themselves sum to 1. This (not necessarily square) linear set of equations can be solved by linear algebra techniques such as by using the pseudo- inverse Cf (also sometimes referred to as the Moore-Penrose pseudo-inverse):

For the case of the mean and differences the weights can also be solved for by linear algebra. This time however, it is done in the tangent hyper-plane at the mean and the constraint that the weights should add to one no longer applies.

Principle Component Analysis

Principle component analysis (PCA) is a powerful tool for analysing a set of data. It extracts a set of orthogonal vectors that characterise the data. The data is initially represented as a set of N-dimensional vectors, with the mean of the data set removed. The process of PCA is then performed to extract the component vectors that characterise the set (they are the eigen-vectors of the covariance matrix of the data set). Each vector in the set can then be re-represented as a weighted sum over these components. These components are ordered with the initial components amounting for more variation in the data set than later components. This means that the data set can be represented in a more compact way by ignoring later components that do not add a significant contribution. PCA can be used as a method of analysis of new data items, as the components are orthogonal a new data vector can be projected onto the co-ordinate system defined by the principle components. This produces a set of weights across the orthogonal components which can be used to analyse the new vector in terms of the original set. The principle components often correspond to meaningful variations in the set and they can be tagged with information about their meaning. Projecting a new vector onto the components can then give information by extracting the meaning of the various components. For example, a principle component might correspond to leaning forward during a motion and the weight of that component in a new piece of motion would correspond to the degree of leaning forward.

Principle component analysis has not been applicable to motion up to now as it relies on a linear vector representation of the data whereas motion is represented as a set of rotations which form a spherical space. However, the above described method of interpolated difference from a mean quaternion converts a data set of motions as a linear set of differences from a mean quaternion, which is precisely the format required for PCA. Thus it is possible using this method to perform PCA over a set of motions by firstly calculating the mean of the set. Then each motion is expressed in the tangent plane of the mean. PCA can then be performed on the set and components can then just be reconstructed by projecting back onto the hypersphere (or sphere) (as the mean on the components is the same as the mean of the original data). Application to motion animation of articulated objects (Motion Synthesis)

The simplest method of synthesising motion using weighted sums would be to start with a set of example motions and perform a weighted sum on them with varying weights to produce different results. This would be able to create a range of good quality motions that are similar to the original examples but whose style varied. This could be done efficiently in real time with the above described fast approximate weighted sum method.

However, a more flexible approach is to use an adapted version of the difference from the mean approach described above in which the mean of the set of example motions is determined by using, for example, either the weighted sum technique described above or the Buss and Fillmore technique. This mean can then be used as a basic "neutral" motion. The examples themselves could then be re-expressed as differences from the mean in the tangent hyper-plane as described above. These differences could then be added to the mean with a set of weights to synthesise a new motion. The differences themselves could represent different styles of performing a motion. This method has the advantage that there is a clearly defined base motion that can be used normally. More importantly the weights no longer have to sum to 1 to be well defined so the motions can be extrapolated.

The motion differences can be merely the differences of the original motions from the mean. However, it is also possible to use techniques such as Principle Component Analysis (PCA) to extract a more meaningful set of differences. PCA can also be used to reduce the number of differences needed and to make the differences orthogonal to each other.

A final possibility is that some of the differences that represent style of motion might transfer from one basis motion to another. Thus applying the difference between a happy walk and a neutral walk to a sitting motion would produce a happy sitting motion. This has yet to be tested. It is believed that some, but not all differences can be transferred to other motions. Thus a limited set of transferable differences or styles could be created which can be used over a large set of different motions. This would greatly reduce the amount of motion that needs to be captured or animated.

The Pipeline for creating a set of styles would be this: i) First a set of example motions would be created. These would all be the same type of motion (e.g. walking) but with different styles. Before they are used they must be scaled temporally as in Rose et al. (1998) so that there are clear correspondences between the motions. Key times in the motion are identified where each motion must correspond to the others, for example in a walking motion this might be a footfall or the highest point. in the motion. The times between all these key points are normalised to 1. Each actual timing for each example motion is stored as a separate parameter. Thus the example motion now has two components: Q t) M =

Where the Qj(t) are the time varying, quaternion orientation of the i^th joint, where the timings are equivalent across all example motions (note that in order to be able to calculate the quaternion orientation of each joint at any arbitrary time, t, in the motion (cycle) a suitable spline interpolation technique such as that described in the Shoemake 1985 paper is used). The Tj are the actual timings of the j^th key time (ie the actual time taken in the example motion to transition from the j^th key time to the G+1)^th key time - note that the keytimes are not the same as the underlying keyframes of the underlying motions which can be at different times in different motions).

The next step is to take the mean of all the motions on a component by component basis. The joint values are evaluated at a set of "mean" keyframes (which are chosen by some suitable method, such as, for example, by choosing a desired number of keyframes for the mean motion and distributing these evenly through the motion) and the mean is taken using, for example, Buss and Fillmore's quaternion weighted average method or the above described fast approximate weighted average method of all of the different motions expressed at each of the mean keyframes. The means of the timings are taken as simple scalar means. This will then result in a new "mean" motion:

M M — Q^μt(t) _Tμ . The mean is then subtracted from each of the examples to extract a set of differences. In the case of the joint angles this is done by projecting into the tangent hyper-plane at the mean so that the differences are expressed in a linear space despite the quaternions themselves being in a spherical space. One variant would be to choose a particular value from which to express the differences rather than the mean. For example, in some cases it might be desirable to have differences from a neutral pose (in a particular key frame) rather than the mean pose.

The last operation gave a vector of linear differences for each example, motion. The next step in the method of the present embodiment is to perform Principle Component Analysis on these vectors to obtain the principle components of the example motion. These are a set of orthogonal vectors that express all the variation of the original example motions. The principle components are ordered in terms of how significant they are to the original data. The least significant component(s) can sometimes be discarded if there is very little variation between the samples along this (these) component(s). Finally, in this embodiment, the components are manually inspected and given meaningful names. They are tested with other types of motion to see if they are transferable and if so they are added to the set of transferable motion styles.

To generate a new motion a base motion is chosen (e.g. standing up, or running) and a set of weights are chosen across the styles (e.g. 70% sad and 40% angry). The differences are added to the base motion with the given weights. For the quaternion components this means performing the addition in the tangent plane at the new base quaternion (ie the one to be modified) and then mapping back to the sphere. For the timing components all that is required is a simple scalar addition.

Analysis of an example motion using PCA

A final useful ability would be to take a new piece of motion and extract the styles. This could be useful for a number of reasons. Firstly the set of style parameters is a more compact representation for transmission. Secondly it allows the style of the motion to be edited. Finally it would allow the style of the new motion to be transferred to other types of motion, thus capturing a particular person's style of motion as a whole. Extracting the style would simply be a matter of performing the inverse weighted sum methods described above on the new motion. Referring now to Figure 8, a method according to a preferred embodiment of the present invention for analysing an input example motion to extract the style of the motion (for transfer to other motions or for compact transmission or other purposes discussed above) commences at subroutine S100 in which (as has been described above) an appropriate hyper plane and corresponding mapping rules are generated based on a set of input variant-on-a-theme motions (e.g. a number of different variations of standing up or running).

The method then proceeds to step S310 in which Principal component analysis is carried out on the points on the hyper-plane in respect of each hyper-plane generated in subroutine S100 (i.e. in respect of each "limb" in each key frame). The result of the principal component analysis will be a set of three orthogonal component vectors in the hyper-plane which can then be used to enable any other point on the hyper-plane to be expressed as a sum of the corresponding principal components.

The method is now ready to receive an example motion to be analysed at step S320.

In step S340 the principal components of each quaternion within the received example motions are determined. This is done by mapping each quaternion in the received example motion onto the corresponding hyper-plane and then calculating the components in the linear hyper-plane in the normal manner.

Finally, at step S340 the thus calculated principal components of each quaternion are output for use in whatever manner they are desired (e.g. for transferring the style of the movement onto different base movements, for transmission over a limited bandwidth, or for editing purposes).

Applications to other areas The techniques described above could be used for any application that requires weighted sums on quaternions or on any type of data that is equivalent to points on a sphere. In particular, the fast approximate weighted sums could be used for creating spherical splines as described in Buss and Fillmore (2001). One use of linear weighted sums is in direct-manipulation. A spline is directly manipulated by the user and linear algebra is used to calculate the control points needed to implement the manipulation (see Hsu, Hughes and Kaufman 1992 for an application to free-form deformations). Similarly, the above described inverse weighted sum could be used for Direct-manipulation quaternion splines, or other quaternion direct manipulation applications. This would work by the user directly manipulating a quaternion object that is the result of a weighted sum. The system would then be able to calculate the weights which are needed to reproduce the user's edits. The new object can be displayed at interactive speeds while the user edits giving the user the impression they are directly interacting with the object. For example, a piece of motion could be edited using a standard motion editing interface but the editing process would result in a set of weights over a set of basis motions (as described above). As the motion is updated at interactive speeds the user has the impression they are directly interacting with the motion.

As for other application areas there are various applications to systems that involve points on a sphere. For example this is a good approximate representation of points on the earth's surface so the above-described techniques could be used to find averages of geographical data. An application might be to find epicentres of earthquakes from sensor readings. The magnitude of the tremor at each sensor gives a weight and the weighted sum might give a good approximation to the epicentre. A less spherical system is a human head but a spherical representation would still be good for many applications. For example, spherical weighted averages could be used when manipulating 3D scan data or 3D mesh graphical representations of heads. These representations consist of sets of points, sample points in the case of scan data and vertices in the case of meshes. Spherical averages or their inverse could be used for manipulating points. For example, when reducing the number of points a spherical average of a set of points could be used to replace that set of points.

Systems that involve directions from a point are also suitable for applying these techniques. These systems include a set of light sources illuminating a scene, where a weight might be the intensity of the light divided by the square of the distance. An application could be to replace a large number of light sources by a single average light source. This could lead to a fast solution for approximating global illumination in computer graphics. Similarly heat diffusion could be modelled with points representing sampled directions and the weights representing the conductance in that direction. Inverse weighted sums could be used to calculate the conductances of a large number of directions from a limited number of samples. The samples could be represented as a weighted sum of the large number of directions and this sum could be solved for the weights. Viewpoints are also well represented as a set of directions for a point, for example a telescope in astronomy or a camera in films (or video games). A weighted average of the set of points that the user wants to view (e.g. stars (ie suns) in astronomy, stars (ie actors/actresses) in a film), could produce a good direction for the camera to point in. Another application to cameras is to virtual cameras, where a limited number of real cameras are used to simulate any new viewpoint. (The classic example is used in the film "the Matrix" where a large number of static cameras are used to simulate a moving camera to produce a shot that appears to pan around a character instantaneously). If a good way of interpolating between the cameras can be found the number of cameras can be reduced. If the cameras are all pointing at a single point (as is likely) the direction of a new viewpoint from that point can be expressed as a weighted sum of the direction of the real cameras. The inverse weighted sum can be used to find the weights corresponding to a new viewpoint. These new weights can be used to interpolate the images from the camera.

References

Buss S. R. and Fillmore J. P. 2001 Spherical Averages and Applications to Spherical Splines and Interpolation in ACM Transactions on Graphics : 20(2) pp95-126

Gleicher, M. 2001 Comparing Constraint-Based Motion Editing Methods in Graphical Models : 63 pp107-134

Hsu, W. M., Hughes J. F. and Kaufman, H. 1992 Direct manipulation of free-form deformations in Proceedings ACM SIGGRAPH '92 pp177-184 ACM Press

Johnson, M, P. 1999 Multi-Dimensional Quaternion Interpolation in SIGGRAPH '99 technical sketches (url: http://aries.www.media.mit.edu/people/aries/quats/index.htm)

Rose, C, Cohen, M. F. and Bodenheimer, B 1998 Verbs and Adverbs: Multidimensional Motion Interpolation in IEEE Computer Graphics and Animation : 18(5) pp 32-40

Shoemake, K 1985 Animating Rotation with Quaternion Curves in Proceedings of SIGGRAPH, Computer Graphics : 19(3) pp 245-254

Claims

1. A method of analysing an acquired sample of a movement or posture of an object, the method comprising: determining an associated base movement or posture corresponding to the acquired sample; determining a set of quaternions which represents the acquired sample movement or posture; mapping each quaternion in the determined set to a point in a linear space which represents the corresponding difference in the linear space to the spherical difference in a spherical quaternion space between the quaternion in the determined set and a corresponding quaternion in a quaternion representation of the associated base movement or posture; analysing each point generated in this way with respect to a plurality of reference sets of points in the linear space, each such set representing a corresponding different known style or variant of the base movement or posture; and re-expressing the acquired sample movement or posture as a weighted combination of one or more of the different known styles or variants of the base movement or posture.

2. A method according to claim 1 wherein the mapping is a logarithmic mapping which preserves distance from the origin in both spaces.

3. A method of transmitting an animation of a movement or posture comprising: analysing the movement or posture to be transmitted according to the method of claim 1 or 2; and transmitting the weightings of the different known styles or variants of the base movement or posture.

4. A method of editing an animation of a movement or posture comprising: analysing the animation of a movement or posture to be edited according to the method of claim 1 or 2; and editing the weightings of the different known styles or variants of the base movement or posture.

5. Apparatus for analysing an acquired sample of a movement or posture of an object comprising: input means for acquiring a sample of a movement or posture of an object; determination means for determining an associated base movement or posture corresponding to the acquired sample; calculation means for calculating a set of quaternions which represents the acquired sample movement or posture; mapping means for mapping each quaternion in the determined set to a point in a linear space which represents the corresponding difference in the linear space to the spherical difference in a spherical quaternion space between the quaternion in the determined set and a corresponding quaternion in a quaternion representation of the associated base movement or posture; analysis means for analysing each point generated in this way with respect to a plurality of reference sets of points in the linear space, each such set representing a corresponding different known style or variant of the base movement or posture; and output means for re-expressing the acquired sample movement or posture as a weighted combination of one or more of the different known styles or variants of the base movement or posture and outputting a signal indicating the respective weights.

6. Processor implementable instructions for causing a processor to carry out the method of any one of claims 1 to 4 during execution of the instructions.

7. Carrier means carrying the processor implementable instructions of claim 6.