WO2001015088A1

WO2001015088A1 - Image morphing by using principal component analysis

Info

Publication number: WO2001015088A1
Application number: PCT/GB2000/003207
Authority: WO
Inventors: Paul O'higgins; Nicholas Martin Jones
Original assignee: University College London
Priority date: 1999-08-20
Filing date: 2000-08-17
Publication date: 2001-03-01
Also published as: GB2353451A; WO2001015088A8; GB9919819D0; GB2353451B; AU6708400A

Abstract

A method of morphing data defining an image of a first object using the morphological behaviour of a second object comprises processing a plurality of sets of data using the reference shape. Each data set defines a form of at least part of the second object and is indexed by a form reference. The data sets are processed to remove effects of size, location and rotation to define a corresponding plurality of shapes of the second object. Principal component analysis is carried out on the data sets to determine principal components and principal component scores. A form reference is received and used to look up principal components and principal component scores. A data set defining a reference shape of the first object is modified using the determined principal components and principal component scores to generate a modified data set defining a morph of the image.

Description

MORPHING OF AN OBJECT USING THE

MORPHOLOGICAL BEHAVIOUR OF ANOTHER OBJECT

The present invention generally relates to the

morphing of an image and in particular relates to the

morphing of an image of an object using a model of

morphological behaviour obtained from another object.

Computer-based morphing of objects, in particular

faces, is being actively pursued in a number of academic and industrial centres. A particular area of activity is the morphing of faces in order to provide animation. In

essence, many prior art methods operate by displacing a

series of control points representing parts or the whole

of the face. This displacement is so defined that the

desired facial morphing, expression or phoneme production

is achieved.

Many different ways of achieving this have been used

in the prior art.

In the field of animation of talking faces, use of principal component analysis has been frequently used in the prior art. For example, in a paper by Kuratate et al. ("Kinematics based synthesis of realistic talking faces" Proceedings of "Auditory-visual Speech Processing"

Conference, Terrigal, NSW, Australia, pp 185-190) a

method is described for animating talking f ces in which

each point on the face is represented in terms of x, y

and z coordinates. Samples of the form of the face are taken for a number of phonemes . A generic mesh is

applied to each face scan and the meshes are lined up

along feature contours. Certain nodes are adjusted to

match eighteen facial control points. The remainder of

the nodes are matched via a field morphing technique.

The mean mesh in this particular registration is

estimated and the principal components of the covariance

matrix of the node x, y coordinates with respect to the

mean are extracted as eigenvectors. This results in a

set of principal component values for each phoneme and thus facial movement can be driven by inputting phonemes

to derive principal components. Back projection can be

applied to the principal components in order to derive

the displacements in the space of the original

coordinates. Thus movement can be achieved by the input of phonemes to cause the synchronous animation of the face with the speech from which phonemes are derived. In this technique separate analyses are used to drive the

face and lips.

A similar technique has been applied to the analysis

of lip movements by Reveret and Benoit (Proceedings of

"Auditory-visual Speech Processing" Conference, Terrigal,

NSW, Australia 1998). In the technique disclosed in this

document lips in various phoneme positions are digitised

and thirty landmarks are positioned through a mixed

manual/polynomial interpolation approach. The

coordinates of these landmarks are subjected to principal component analysis after alignment to a reference

position and the first three principal components which

account for most of the differences between positions are

used to drive an animation through back projection using

eigenvectors and the mean.

Similar techniques are used by L. M. Arslan and D.

Talkin ("3-D face point trajectory synthesis using an

automatically derived visual phoneme similarity matrix"

Proceedings of "Auditory-visual Speech Processing" Conference, Terrigal, NSW, Australia 1998). In their technique they take 54 samples in three dimensions and use principal component analysis to reduce the number of dimensions to 20.

A similar technique is also disclosed in the work by

Galanes et al. ("Generation of lip-synched synthetic

faces from phonetically clustered face movement data"

proceedings of "Auditory-visual Speech Processing"

Conference, Terrigal, NSW, Australia 1998). In this work

58 three-dimensional samples are taken non-uniformly

distributed around the face. Principal component

analysis is then carried out to reduce this to 20

dimensions. By back projection using phonemes, a synchronous animated face movement is provided.

The inventors have realised that these techniques of

the prior art only allow the morphing or animation of an

object, e.g. a face, based on training data obtained from

that object. The inventors have realised that this is a

limitation since it requires the object such as the face

to be animated to be available to carry out the training

in order to obtain the principal component data required

for the morphing necessary for animation by back projection.

The present invention provides a solution to this problem by providing a technique in which samples are taken defining different forms of an object e.g. due to different facial expressions or due to form changes

during pronunciation of phonemes . The measurements taken

are not restricted to measurements from a single

reference object e.g. measurements may be taken for the

form of the faces of a number of subjects during the

pronunciation of a phoneme. The sets of measurements

(the training set) are subjected to registration in order

to remove the effects of differences in size, location

and rotation. This has the effect of providing data in a lower dimensional shape space. Each shape defined by

a set of data comprises a point in the shape space. The

shape space is then subject to statistical processing in

order to identify the directions in the shape space of

the most correlated changes in shape. This results in a

determined coordinate system and coordinates defining the

data sets within the coordinate system. Each coordinate

defining a data set is indexed by a form reference e.g.

a phoneme or a facial expression where the object is a

face. Thus these data comprise a model of the morphological behaviour of an object with the effects of size, translation and rotation removed. The model is purely a model of the change in shape of an object as defined from a reference shape.

The inventors have realised that because of the

removal of size, location and rotation effects, this

model can be applied to the back projection and animation

of another object. A reference form for the target

object to be morphed using the model is obtained and when

the coordinates are determined using input form

references such as phonemes, instead of morphing the

reference form of the reference object, the reference form of the desired object is morphed by the amount determined from the coordinates.

Thus in this way a desired object can be made to

have the morphological behaviour of a reference object.

The present invention is applicable to the morphing

of the whole of an object or to any one or more parts of

the object. If more than one part of the object is to be

morphed, the parts can be treated separately.

Preferably, the present invention employs data which

comprise vector data defining the form of the object.

Thus , the data can comprise landmark data providing three-dimensional coordinates of morphologically important features of the object, or data comprising

control points of splines which define the form of the

object.

In an embodiment of the present invention the

reference shape is determined from the mean of the

reference data.

In an embodiment the processing to remove the

effects of size, location and rotation comprises scaling

the sets of data according to centroid size, translating

the sets of data to make the centroids of the sets of data coincident, and rotating the data sets about the

centroids to minimise the sum of the squares of the

differences between equivalent members of the data sets.

This technique is, in one embodiment, performed as a

generalised Procrustes analysis.

Generally the shape space defined by the sets of

samples in two or three dimensions defines a non-

Euclidean shape space. Although it is possible to

perform the statistical analysis, e.g. principal component analysis, in the non-Euclidean shape space, it is convenient to transform the data into a Euclidean shape space to make the computation simpler. Embodiments of the present invention will now be described with reference to the accompanying drawings in

which:

Figure 1 is a flow diagram of the operation of a

general embodiment of the present invention;

Figure 2 is a schematic diagram of the general

embodiment of the present invention;

Figure 3 is a schematic diagram of a computer system

implementing the technique to build the model in accordance with a first embodiment of the present invention;

Figure 4 is a flow diagram illustrating the

operation of the embodiment of Figure 3 ;

Figure 5 is a diagram explaining the concepts of

principal component analysis in the tangent to the shape space;

Figures 6a and 6b illustrate two example facial

postures ;

Figure 7a is a diagram of superimposed sets of data points for the forms of the training face;

Figure 7b illustrates the mean coordinates; Figure 7c is a wire frame drawn between the mean coordinates;

Figure 7d illustrates the rendering of the polygons

between the coordinates;

Figure 8 is a graph of the first two principal

components in an embodiment showing the effects of

changes in principal component scores on the training

face;

Figure 9 is a plot of the third and fourth principal

components showing the effects of the principal component scores on the shape of the training face;

Figure 10 is a diagram of a computer system for

inputting and registering the reference form of the desired object to be morphed or animated;

Figure 11 is a flow diagram illustrating the

operation of the embodiment of Figure 10;

Figure 12 is a plot of the first two principal

components illustrating the score of the target face to

be morphed using the model of the training face;

Figure 13 is a diagram of a computer system for

morphing or animating the target face using the training face in accordance with an embodiment of the present invention; Figure 14 is a flow diagram illustrating the

operation of the embodiment of Figure 13;

Figure 15 is a diagram illustrating how the

principal component score is used together with the

values for the eigenvectors in order to back project and

provide coordinates in shape space;

Figure 16 is a diagram of an example of grid warping

using a triplet of thin plate splines;

Figure 17a is a diagram of a NURBS curve; and

Figure 17b is a diagram of the NURBS curve after deformation produced in translating control point B₇.

The general embodiment of the present invention will

now be described with reference to Figures 1 and 2.

A schematic diagram of the general embodiment is

illustrated in Figure 2. This embodiment can be

implemented on a general purpose computer using software

or in a special purpose hardware machine.

In step SI of Figure 1 input reference data sets for

a plurality of forms of a reference object indexed by a

form reference are input into the reference object data store 1 shown in Figure 2. Such data can comprise spline control points or landmarks in two or three-dimensional coordinates for a number of forms of the object. Each

form is characterised by a form reference which

identifies the form. For example, for a face, the facial

expressions can be indexed by an expression identifier,

e.g. happy, sad, or crying, or by a phoneme identifier

which identifies the phoneme being pronounced resulting

in the facial expression. Thus the indexing of the data

sets by the form reference enables the data to be

referenced for back projection as will be described in

more detail hereinafter.

In order to remove the effects of size, translation

and rotation from the data sets, in step S2 the data sets

are registered against a reference data set for a

reference form to thus define a point in shape space for

each form. This is achieved using the reference object

data registrar 2 illustrated in Figure 2. Thus by

reference to the reference data set the effects of size,

location and rotation are removed thus converting the

data from data on the form of the object to data on the

shape of the object (where form = shape + size).

The sets of data then undergo statistical analysis using the statistical analyser 3 to determine a space of reduced dimensionality based on directions in the shape space of the most correlated changes in shape. The

technique used for the statistical analysis can be

principal component analysis or a regression technique

for example. Having determined the coordinate system,

the coordinates for each form with respect to the

reference form are then determined in step S4 and this

data is then stored in the data store 4 illustrated in

Figure 2.

So far the steps have resulted in the formation of a model of the morphological behaviour of the reference

object with reference to the indexed form reference.

Thus if the form reference comprises phonemes, and the

reference data sets comprise data sets for a face, the

model comprises a model of the relative movement of the face when pronouncing phonemes, i.e. it defines what

features of the face move during the pronunciation of a

phoneme. This model can thus be used for morphing or

animating.

In order for this model to be applied to another object, e.g. another face, a reference data set for a reference form of the target object is input in step S5 and this is stored in the target object reference data store 5 of Figure 2. In order to remove the effects of

size, location and rotation from the input data set, it

is registered in step S6 against the reference data for

the reference object by the target object registrar 6 in

Figure 2.

The system now has all the data necessary to perform

a morphing or animation operation on the reference data

for the target object simply from input form reference data. Thus in step S7 the process awaits the receipt of form reference data from the form reference receiver 7.

Once this is received, in step S8 coordinates in the

space of reduced dimensionality are looked up in the data

store 4 in Figure 2 and in step S9 these are applied to

the registered data set for the reference form of the

target object by the target object reference data

transformer 8. In step S10 the transformed data set and

the morphed target object are then output using the

transformed target object data set output device 9. It can thus be seen that because the effects of size, translation and rotation are removed, the target object reference data can be used to replace the reference object reference data for the back projection

process thus enabling the morphing or animation of the

target object instead of the reference object.

A specific embodiment of the present invention will

now be described of the formation of the model of the

morphological behaviour of an object with reference to

Figures 3 to 9.

Figure 3 is a schematic diagram of a computer system

embodying the invention in which an input device 11 allows the input of n sets of coordinate data ( landmark

matrices) in accordance with step S20 of the flow diagram

of Figure 4. A display unit 12 is provided for the

display of the coordinate data sets. Processor 13 is

provided for the implementation of computer program

modules provided to the computer system by a storage

medium (floppy disc) 16 and stored in the program storage

15. Within the program storage 15 there is stored

program code for carrying out generalised Procrustes

analysis which when loaded in the processor 13 results in the implementation of the generalised Procrustes analysis module 13a. Also, the program storage 15 stores program code for calculating the mean of the reference data which

when loaded in the processor 13 causes the implementation of the mean calculator 13b. Program code for

implementing the tangent projection is also provided in

the program storage 15 and when loaded in the processor

13 implements the tangent projection module 13c. Program

code for implementing a principal component analysis is

also provided in the program storage 15 and which when

used by the processor 13 implements the principal component analysis module 13d.

The program storage 15 in this embodiment can be

provided by a conventional mass-storage device such as a

hard disc of a general purpose computer. The processor

13 comprises the microprocessor of a general purpose computer.

The computer system also comprises a working memory

14 in which data is stored. The processor 13 uses the

working memory 14 to store and retrieve data used by the

various modules. The working memory 14 is provided by

the random access memory (RAM) of a general purpose computer.

All of the components of the computer system are linked together by the data and control bus 10.

The operation of the embodiment of Figure 3 will now

be described in more detail with reference to Figures 4

and 9.

In step S20 a plurality (n) of coordinate data sets

are input to form a plurality n of landmark matrices . In

this embodiment of the present invention the data

comprises x, y, z coordinates for facial landmarks from

a reference face, k landmarks are taken in m dimensions. Thus there are n x k x m data points. Thus where m = 3

(i.e. measurements are taken in three dimensions) k = 31

(i.e. 31 landmarks are used) and n = 15 (i.e. there are

15 specimens (forms) taken), the form space has a

dimensionality of 93 with 15 coordinates.

The coordinates for each specimen identify important

morphological features which will move.

In order to remove the effects of size, location and

rotation and thus define the data sets as points in shape

space, the data sets are registered with respect to each other. Differences in size, translation and rotation of specimens can occur, e.g. due to movement of the person to put the face in a different position or rotation of the camera used to obtain the coordinate data.

This is achieved by minimising the sum of squared

distances between the equivalent landmarks of forms.

This is termed "generalised Procrustes analysis" (GPA).

Where there are n specimens, each represented by a k x m

matrix of landmark coordinates, X_i where i = l,...n

results in registered training faces denoted, X'i, for

which the sum of squared differences, d_P ², between them

is minimised using:

Scaling is thus according to centroid size (the

square root of the sum of squared Euclidean distances

from each landmark to the centroid which is the mean of

landmark coordinates).

Having carried out registration, the mean shape of

the specimens is determined as the mean of the landmark

coordinates in step S22. This is used as the reference in shape space for the tangent projection and for the principal component analysis as will be described in more detail hereinafter.

Once registration is carried out, each shape

(specimen) can be represented as a point in a "shape

space". Because of the removal of the effects of size,

location and rotation, the shape space has a

dimensionality of km-m-(m(m-l ) /2 )-l , since rotations

provide m(m-l)/2 degrees of freedom, locations provide m

degrees of freedom and scaling provides just one degree

of freedom. The shape space with a distance resulting from the generalised Procrustes analysis is termed "Kendall's

shape space" (Kendall DG (1984) "Shape manifolds,

Procrustean metrics and complex projective spaces"

Bulletin of the London Mathematical Society, volume 16

pages 81-121). Kendall's shape space has the desirable

property that independent isotropic distribution of

landmarks results in isotropic distribution of points

representing specimens in the shape space. This means

that if landmarks vary in location according to an isotropic model we can expect to find an isotropic

distribution of specimens in the shape space. Conversely, deviations from independent isotropic distribution landmark variations will lead to a non- isotropic distribution of specimens in the shape space.

This also means that shapes separated by a particular

Procrustes distance anywhere in the shape space will have

the same net difference in landmark coordinates in the

space of the original object (i.e. the real space in

which the object exists: termed "figure space"). The

principal directions of variations of shape in the shape

space are often of interest since they indicate correlated landmark differences.

Kendall's shape space is non-Euclidean (i.e. it is

curved) . Thus although it is possible to perform

analysis of the principal directions of variations of

shape in the non-Euclidean shape space, it is not

straightforward. For example, for triangles the space is

equivalent to the surface of a sphere of unit diameter as

illustrated in Figure 5. As can be seen in Figure 5,

equilateral triangles lie at the poles: the southern

hemisphere being a reflection of the northern hemisphere.

The sphere is divided into twelve equal half lunes (six in each hemisphere). If the apices of the triangles are unlabelled and reflections are ignored all the triangles lie in one half lune. Isosceles triangles lie along the lines dividing the lunes and flat triangles lie at the

equator.

For more than three landmarks (k landmarks in m

dimensions) the space is high dimensional and more

complex. Because of this, great care is needed in

carrying out analyses and modelling of movements. For

this reason, in this embodiment the analysis of the

principal directions of variation of shape are carried out in the tangent space to Kendall's shape space. As can be seen in Figure 5 for triangles the scattered

points on the spherical shape space representing

variation within the range of samples is projected into

a Euclidean tangent plane at a tangent to the mean of the

sets of samples. Thus the coordinates of the points

representing specimens are no longer given in terms of

the sphere, but rather as coordinates in the plane. As

long as the projection has not resulted in excess

distortion (as might occur if the projection encompasses

a large proportion of the sphere) useful analyses can be carried out in this plane. For higher dimensions, the tangent plane to the shape space can be imagined as a space of km-m-m(m-l ) /2-1 dimensions.

Figure 5 illustrates three steps (as numbered). The

first step is the generalised Procrustes analysis to

obtain Kendall's shape space from the landmark matrices.

The second step is the transformation of the data from

the non-Euclidean Kendall's shape space to the Euclidean

tangent space in which principal component analysis takes

place as will be described in more detail hereinafter.

A visualisation of data can then be achieved by back

projection using the principal component scores as will be described in more detail hereinafter.

The Procrustes tangent coordinates are estimated in

step S23 using a technique which would be apparent to a

skilled person in the art such as the technique used in

the paper by Dryden and Mardia ( "Multivariate shape

analysis" Sankhya volume 55(A) pages 460-480 1993) the

content of which is hereby incorporated by reference.

This projection results in a (k-l)m vector of tangent

space shape coordinates with respect to the mean for each specimen. The vectors of the tangent space are of rank km-m-m(m-l)/2-l. Principal component analysis is then carried out in step S24 using tangent space coordinates to extract km-m-m(m-l)/2-l eigenvectors which are the principal components of variation of shape. Principal

component scores for the coordinates of the tangent

matrices are then determined in step S25 so that the

points in tangent space can be identified by principal

components and their scores . Thus in t s way a model of

the morphological behaviour with respect to form

references is provided as principal components and

principal component scores. The principal components are

all orthogonal and pass through the origin which comprises the mean of the specimens.

The principal components comprise eigenvectors

defined in the tangent space. Each eigenvector has an

eigenvalue which gives the statistical significance of

the eigenvector. Where there exists significant

correlations between landmark displacements amongst the

sample of shapes, it is reasonable to expect that the

first few principal components will serve as an adequate

representation of shape differences among that sample.

The method of generation of the model as principal components and principal component scores will now be described in more detail with reference to specimens as illustrated in Figures 6 to 9.

A set of training faces is used to develop models of

facial posture deformation such as between phonemes and

expressions. These training faces represent the

movements of one particular face classified in terms of

phonemes and standard expressions for example so that

they encompass the range of all possible facial

movements. Alternatively, the training set could consist

of several different faces in different postures.

Each face is defined in terms of landmarks hereinafter termed "control points".

In the example illustrated in Figures 6a and 6b, two

example facial postures are shown each defined by 107

anatomical landmarks in three dimensions. A mesh is

drawn between the landmarks to deliver a visual impression.

Figure 7a illustrates superimposed coordinates of

control points of the set of training faces after

generalised Procrustes analysis. Figure 7b illustrates the mean coordinates for the mean facial posture and

Figure 7c illustrates a wire frame drawn between the mean coordinates. Figure 7d illustrates the polygons rendered between the control points .

The next phase involves the extraction of principal

components of the covariance matrix between registered

control point coordinates. The principal components

represent, within the training faces, aspects of

correlated variation amongst the control points. Since

the face is only capable of a limited range of movements

(many of which are correlated through physical,

neurological and anatomical constraints ) , it is expected

that many fewer principal components than control point coordinates (in this example 107 landmarks x three

dimensions = 321) will encompass the whole range of

movements. It is found in this example that 9 principal

components account for 91% of the total variance in face

shapes within the training set.

The principal components in this training set

clearly represent aspects of facial movement which make

sense intuitively as well as numerically. In effect they

are control parameters for the set of possible integrated movements of the training faces.

Figure 8 is a plot of the first and second principal components. As can be seen, the linear scattering of points passing diagonally along the plot represents variability amongst the training faces in mouth opening

and its correlated movements. In Figure 8 these

movements are visualised by back projection from the

principal component scores to warp the mean training

face. The upper left diagonal arrow indicates the

direction in which the warped mean representations were

computed. The second diagonal arrow, lower right, shows

the direction in which the warped mean face was computed to indicate coordinated eye closure and lip movement.

Figure 9 is a plot of the third principal component versus the fourth principal component. It can be seen

that these principal components represent asymmetric

aspects of facial shape variability. The four faces have

been obtained by back projecting from the principal

component scores in their respective positions in the plot.

Thus the embodiment described with reference to

Figures 3 to 9 result in the generation of a model for

the training faces as principal components and principal component scores . These are defined in relation to the shape of an object and not the form of an object. An embodiment of the present invention will now be described with reference to Figures 10 to 12 in which a

reference set of data for a target object is entered and

stored. This process can be carried out as illustrated

on a separate computer system to the computer system used

for the generation of the model. Alternatively, it can

be carried out on the same computer system and thus the

systems illustrated in Figures 3 and 10 would be

combined. Figure 10 illustrates a computer system for generating the reference shape data. An input device 110

is provided for the input of a set of reference

coordinates which define a reference shape to be warped

or animated. This should be matched as closely as

possible to the reference shape, i.e. the mean shape, of

the training data for the best results. A display unit

120 is provided for display of the coordinate data. A

processor 130 is provided for implementing program code

stored in the program storage 150. The program storage

150 stores program code for carrying out the generalised

Procrustes analysis and when loaded in the processor 130 implements the generalised Procrustes analysis module 130a. Also, the program storage 150 stores program code for implementing the tangent projection and when this is

loaded in the processor 130 it implements the tangent

projection module 130b.

The program code can be provided to the computer

system via a storage medium such as a floppy disc 160 and

loaded into the program storage 150 for the

implementation of the process.

A working memory 140 is provided for use by the

processor 130. The working memory stores the data necessary for processing by the processor 130.

The components of the computer system are all linked

by a control and data bus 100.

The computer system of Figure 10 is implemented by

a general purpose computer under software control. The

processor 130 comprises the microprocessor of a general

purpose computer. The program storage 150 comprises the

mass-storage medium, e.g. hard disc, of the general purpose computer. The working memory 140 comprises the

random access memory (RAM) of the general purpose computer. The input device 110 can comprise a keyboard, disc drive, or modem for example as any means of inputting the reference data for the target object to be

warped or animated.

The operation of the system will now be described

with reference to Figures 11 and 12.

In step S30 the target object landmarks are input as

a set of reference coordinates. In step S31 these are

registered against the mean matrix for the training data

stored in the working memory 140 in order to remove the

effects of size, translation and rotation. This is performed by the generalised Procrustes analysis module

130a.

In order to transform the data matrix from non-

Euclidean Kendall's shape space into the Euclidean

tangent space, the tangent projection module 130b carries

out the tangent projection process using the mean matrix

to form the tangent matrix for the object i.e. to provide

a point in tangent shape space. This can be visualised

in Figure 12 where for illustrative purposes a principal

component analysis has been carried out for the training

faces and for the target object, i.e. the frog. The first principal component PCI shows the change between the mean shape of the training face to the shape of the front face. Principal component 2 (PC2) shows the change in shape from the face having an open mouth and open eyes

to a closed mouth and closed eyes. The faces illustrated

in the diagram are shown in relation to their relative

principal component positions on the plot. It can thus

be seen that by changing principal component 2 starting

from the principal component value for the frog, the

principal values can be obtained which when projected

back result in an image of a frog which has the animated properties of the training faces.

The process of generating data for the morphed or

animated target object will now be described with reference to Figures 13 to 15.

Figure 13 is a schematic diagram of a computer

system for generating a morphed image or an animated

image from a reference data set for the image and using

a model of morphological behaviour of a reference object.

Since the system is provided with the principal

components, principal component scores, and mean matrix provided by the system of Figure 3, and the tangent matrix for the object provided by the system of Figure 10, the system for animating or warping can be separate to the systems of Figures 3 and 10. Alternatively, the systems of Figures 3, 10 and 13 could be combined in

general purpose computer to provide a computer system

having the functionality of all three separate systems .

The computer system of Figure 13 includes an input

device for inputting a form reference such as a phoneme

or facial expression which has been used to index the

principal component scores for the training faces . A

display unit 1200 is provided for displaying the warped or animated image. A processor 1300 is provided for implementing program code stored in the program storage

1500. In the program storage 1500 there is provided

program code for implementing the model look-up module

1300a in order to look up the principal components and

principal component scores using the input form

reference. Also within the program storage 1500 there is

provided program code for implementing the matrix warp

module 1300b in order to use the principal component

scores and principal components to generate a warped

matrix. Further within the program storage 1500 there is provided program code for implementing the inverse tangent projection module 1300c within the processor 1300 in order to transform the warped matrix into Kendall's shape space. Further within the program storage 1500

there is provided program code for implementing the wire

frame and render processing module 1300d in order to wire

frame and render the landmark matrix for the object for

display on the display unit 1200.

A working memory 1400 is provided for storing each

of the data matrices and PC scores used by the processor

1300.

The program code for the implementation of the process in the computer system can be provided on a

storage medium such as floppy disc 1600 for loading into

the program storage 1500.

The computer system is implemented using a general

purpose computer and an appropriate program. Thus the

processor 1300 comprises the microprocessor of the

general purpose computer. The program storage 1500

comprises a mass-storage device e.g. hard disc drive of

the general purpose computer. The working memory 1400 comprises the random access memory (RAM) of the general purpose computer.

The components of the computer system are interconnected via the data and control bus 1000.

The method of operation of the system of Figure 13

will now be described with reference to Figures 14 and

15. In step S40 the shape space reference is input.

This can comprise a phoneme or facial expression which

has been used as an index for the principal components

stored in the model. In step S41 the principal component

scores and vector matrix for each principal component are

looked up using the input shape space reference. Thus what is now obtained is the change in shape from the

reference for the training faces. In order to morph or

animate the target face, it is however necessary to apply

this change to the reference face for the target object

rather than to the reference face for the training faces.

This is achieved in step S42 by taking, for each

principal component the eigenvectors multiplied by the

desired principal component score adding this to the

tangent matrix for the object. This is illustrated for

principal components 1 and 2 in Figure 15. Principal components 1 and 2 comprise eigenvectors in the tangent shape space and thus the principal component scores identify coordinates in the tangent shape space relative

to the mean for the training faces. Thus this change can

simply be applied to the tangent matrix for the target

object. For example, in Figure 12, the change

illustrated in Figure 15 to the point illustrated for the

frog face would cause a change in the face vertically

towards an open mouth posture in view of the change in

PC2.

In order to obtain coordinates in shape space it is

then necessary in step S43 to inverse tangent project the tangent matrix to obtain the landmark matrix for the

target object. In step S44 the landmarks are then wire

framed and rendered in order to generate an image for

display on the display unit 1200.

In the embodiments described hereinabove, so far

only how controlled points might be animated has been

described. In practical applications, however, it is

likely that the rendering achieved just using polygons

drawn between control points will be inadequate for high

quality animation and graphics. In these cases it will be necessary to develop a polygon mesh for rendering in concert with the control points . This can be achieved by use of a splining function such as the thin plate spline. A triplet of splines will work all 3D mesh coordinates

from frame to frame. This warping is such that good

points whose coordinates are identical to control points

in the reference will map directly into the equivalent

(matching) control point coordinates in the target. In

between control points the mapping of grid points between

reference and target shapes will be such that the grid is

minimally bent. Figure 16 illustrates the use of triplet splines to warp a regular grid between the reference and target

images as described hereinabove.

Although in the specific embodiment described

hereinabove, landmark data is used, the present invention

is also applicable to the use of splines.

The thin plate spline method provides a very

effective means of calculating the vector of motion of

the vertices of a polygon mesh representing a face based

on the displacement of control points . Thus a complex rendering model can be animated from only the analysis of

a few coordinates. Polygon mesh itself can be derived from connecting x, y and z coordinate information gathered from a 3D camera, laser scanner etc. in a simple

linear fashion usually to form a network of planar

triangles or quadrilateral facets (generally these facets

are an approximation to a curved surface). A typical

face might require over 1000 polygon facets before using

a crude appearance. Therefore the ability to summarise

and analyse facial movement using the motion of key

control points common to all faces and then reapplying

those movements to different, final high polygon-count,

render objects is greatly desirable. This can be achieved by representing the face or any object in the

form of connected spline patches. Any curve can be

represented mathematically as a parametric function of

polynomials of order three. The class of such curves is

known as B-splines and the most powerful are non-uniform

rational B-splines (NURBS) which are in fact

generalisations of all B-splines. B-splines consist of

curve segments whose polynomial coefficients depend on

tangent vectors to the segments . The end points of these

vectors define the shape of the splines and are therefore called control points. Figure 17a illustrates a NURBS curve with control points and Figure 17b illustrates the same curve after deformation produced in translating

control point B₇.

Thus instead of generating polygon facets from the

sampled vertices of a 3D digitizer, it is well known in

the prior art to produce splines which fit to these

vertices producing a spline mesh that has minimum bending

energy.

There are a number of advantages to working with

NURBS. They can represent virtually any desired shape,

from points, straight lines, and polylines to conic sections (circles, ellipses, parabolas and hyperbolas ) to

freeform curves with arbitrary shapes. They provide a

great deal of control over the shape of the curve. A set

of control points and knots (points where one spline

joins another) which guide the curves shape can be

directly manipulated to control its smoothness and

curvature. They can represent very complex shapes with

remarkably little data. Thus the use of NURBS greatly

reduces the magnitude of information needed to preserve

accuracy. Thus the present invention encompasses the use of spline control points as the data sets instead of landmark data. Although in the specific embodiment described hereinabove, the whole image of the object was morphed or

animated, the present invention is applicable to any

parts of the image of an object. For example, the lips

and eyes of a face can be animated independently using

the same technique. The separate animations of adjacent

or overlapping anatomical features can thus be

reconstructed into whole faces through least squares

registration of shared landmarks against a neutral face

with shared landmarks. This approach allows more easy control of the individual movements of these features but

will lose information about correlation of movements in

between features .

In the embodiment described hereinabove only one

face in different postures is used in the training set

and thus a set of principal components is determined in

which each posture is defined in terms of the coordinates

of a single point. If several faces are used in the

training set then each posture will be represented by several points in the shape space. The difference between any two points representing the same posture would in part be due to differences in face and in part to differences between faces in the way the facial posture is adopted. The differences due to face should

be factored out by regression or by disregarding

principal components that contained only information

relating to face differences. Residual variation on

principal components could then be used as facial control

parameters as for the case of a single training set.

Alternatively, and computationally more simple, would be

the straightforward taking of the Procrustes means of all

training faces in each posture. These averages would then be subjected to principal component analysis as

above and used to generate principal component control

parameters .

Since the method for generating movement through

principal component analysis of tangent coordinates

returns displacements in x, y and z which are applied to

the mean or alternative face, it matters only a little if

these are very different in shape e.g. human mean

training face and a frog alternative face) since the

displacements still apply. If, however, very large discrepancies between relative facial proportions leads to movements which are relatively too small or too large for the alternative face these can be corrected by differentially multiplying movements applied to sets of

control points, in particular anatomical regions.

Alternatively, it could be dealt with by use of separate

principal component analyses if possibly overlapping

anatomical regions with subsequent registration of these

regions to each other.

In order to get the movements of alternative faces

to look acceptable, it is preferable that these faces are presented in a posture that is like that of the mean training face. This is because the movements are

calculated as displacements of the mean coordinates.

This posture can be achieved by careful control at the

time of digitization of the alternatives or by taking

control point coordinates from the alternatives in

numerous poses using back projection of principal

component analyses from shape analyses of these postures

to generate ones which are most satisfactory.

Although in the specific embodiments described hereinabove the object to be morphed or animated is

unrelated to the training object, this need not be the case. It is possible for the object to be animated to be a caricature of the training object. For example, the principal components can be weighted to increase the

emphasis of certain features. This modified or warped

object can then be used as the reference data for the

target object. The weighting can be achieved by taking

the weighted average of more than one object. For

example, as shown in Figure 12, the reference face could

be an average of the frog face and the training face.

Another method of achieving a caricature is to

weight the principal components which are applied to the reference data for the target object. In this way the

movements applied as a morphing the object can be

exaggerated or reduced.

It can thus be seen from the embodiments hereinabove

described of the present invention that the present

intention is widely applicable to the morphing and

animation of an object using the referenced movement of

another object. The object can comprise any moveable

object e.g. the body of a person or animal or a

mechanical object.

The present invention is ideally suited to implementation by a computer program implemented on a general purpose computer. Thus the present invention can be embodied as a carrier such as a storage medium e.g.

floppy disc, a signal e.g. a signal carrying a software

over network such as Internet, or an electronic device

e.g. a programmable read only memory.

Since in the embodiments described hereinabove the

Procrustes analysis eliminates scaling, the variations

quantified by the principal component analysis are shape

rather than form variations . In order to restore size

for the training set, regressions of principal component scores verses centroid size for the significant principal

components can be carried out. Alternatively, the

distances between two standard points in the object e.g.

inter pupillary distance for a face, could be used to

quickly rescale the generated morph of the target object

once the shape has been reconstructed through back

projection from principal component scores to landmark

(control point) configurations. Where morphing or animation of only part of an object has taken place a

third point can be taken and used to allow not only rescaling but also re-registering.

In the present invention the model of the morphological behaviour of a group of objects, e.g.

faces, or parts of objects, e.g. the mouth of one or more

faces, can be generated in one computer as illustrated in

Figure 3 and stored for later use on the same computer or

transmitted to another computer, e.g. electronically over

a network such as the Internet, or via any other storage

medium such as a floppy disc. The recipient of the model

can use this model to change the morphological behaviour

of another object. For example, a person could generate a model of their own facial morphological behaviour and transmit this together with a message, e.g. text or

speech, to a recipient. The data transmitted using this

technique is far less than the data which would be

required to be transmitted to provide a moving image of

the face, e.g. video. The recipient of the model and the

message can then use the model and the message to

generate a morph or animation of another object, e.g. a

cartoon character. Thus using this technique an audio

message can be replayed either as a direct recording of the speech or as a result of speech generation from the text, in synchronism with an animated morph of an object such as a cartoon character having the morphological behaviour of the originator of the message.

Although the present invention has been described

hereinabove with reference to specific embodiments, the

present invention is not limited to such embodiments and

it would be apparent to a skilled person in the art that

modifications are possible within the spirit and scope of

the present invention.

Claims

1. A method of morphing data defining at least a part of an image of a first object using the morphological behaviour of at least a part of at least one second object, the method comprising: processing a plurality of data sets using a reference shape, each data set defining a form of at least a part of a second object and being indexed by a form reference, to remove effects of size, location and rotation to define a corresponding plurality of shapes of the at least a part of the or each second object; statistically processing said data sets defining shapes of said at least a part of the or each second object to determine a coordinate system based on directions of the most correlated changes in shape and to determine coordinates for each set of data in said coordinate system referenced to said reference shape; receiving a form reference; determining coordinates in said coordinate system for a said set of data using the received form reference; and modifying a data set defining a reference shape of at least a part of the first object using the determined coordinates to generate a modified data set defining a morph of at least a part of the image.

2. A method according to claim 1, wherein said data sets comprise vector data.

3. A method according to claim 1 or claim 2, wherein said statistical processing step determines a coordinate system of reduced dimensionality by determining the most statistically significant dimensions.

4. A method according to any preceding claim, wherein said statistical processing step comprises principal component analysis to determine said coordinate system as principal components defined as eigenvectors , wherein said coordinates comprise principal component scores for each set of data for each eigenvector.

5. A method according to claim 4, wherein each principal component has an eigenvalue giving the statistical significance of the principal components.

6. A method according to claim 4 or claim 5, wherein the step of modifying the data set defining the reference shape of at least a part of the first object comprises using the eigenvectors and principal component scores to modify said data set.

7. A method according to any preceding claim, wherein said reference shape of said at least a part of a said second object is determined from a mean form of said forms of said at least a part of the or each second object.

8. A method according to any preceding claim, wherein said processing steps to remove the effects of size, location and rotation comprise scaling said sets of data according to centroid size, translating the sets of data to make the centroids of the sets of data coincident, and rotating the data sets about the centroids to minimise the sum of the squares of the differences between equivalent members of the data sets.

9. A method according to any preceding claim, wherein said processing step to remove the effects of size, location and rotation comprises a generalised Procrustes analysis.

10. A method according to any preceding claim, wherein said processing step to remove the effects of size, location and rotation includes a transformation step of transforming said data sets defining shapes in non Euclidean shape space to respective data sets defining shapes in a Euclidean tangent space, and said step of modifying the data set defining the reference shape includes using the determined coordinates to determine a data set defining a modified shape of said at least a part of said first object in said Euclidean tangent space and transforming said determined data set to said non Euclidean shape space to obtain said modified data set defining said morph of at least a part of the image.

11. A method according to any preceding claim, wherein said data sets comprise landmarks comprising coordinates in two or three dimensions defining important features of form.

12. A method according to any one of claims 1 to 10, wherein said data sets comprise control points for splines defining forms of at least a part of the or each second object.

13. A method according to any preceding claim wherein said form reference comprises a phoneme and the or each second object comprises a face.

14. A method according to claim 13, including the step of extracting phonemes from text or speech.

15. A method according to any one of claims 1 to 12, wherein the or each second object comprises a face and said form reference comprises a facial expression.

16. A method according to any preceding claim including the step of reapplying size information removed in the processing step to the modified data set.

17. A method according to any preceding claim, including processing a data set for a reference form of at least a part of the first object with reference to the data set for said reference shape of said at least a part of a second object to remove effects of size, location and rotation to provide said data set defining said reference shape of said at least a part of the first object.

18. A method according to any one of claims 1 to 16, wherein said first object comprises a modified form of said second object, and said data set defining a reference shape of at least a part of the first object is generated to define a morph of said reference shape of said at least a part of the second object.

19. A method according to claim 18, wherein said morph comprises a caricature of said second object.

20. A method according to any preceding claim, wherein said step of modifying the data set defining the reference shape includes weighting the determined coordinates .

21. A method according to any preceding claim, wherein said plurality of data sets define a form of at least a part of a plurality of second objects, and including taking an average of any of said data sets having the same form reference and using said average as the data set for a form reference.

22. A method according to any preceding claim wherein the method is applied to a plurality of parts of the image of the first object independently and a modified data set for the whole image of the first object is generated by combining the modified data sets generated for the plurality of parts of the image of the first object.

23. A method according to any preceding claim, wherein the steps of receiving a form reference, determining coordinates, and modifying the data set defining said reference shape of said at least a part of the first object are repeated to generate data defining an animated morph of said at least a part of the image of the first object.

24. Image processing apparatus for morphing data defining at least a part of an image of a first object using the morphological behaviour of at least a part of at least one second object, the apparatus comprising: first processing means for processing a plurality of data sets using a reference shape, each data set defining a form of at least a part of a second object and being indexed by a form reference, to remove effects of size, location and rotation to define a corresponding plurality of shapes of the at least a part of the or each second object; statistical processing means for statistically processing said data sets defining shapes of said at least a part of the or each second object to determine a coordinate system based on directions of the most correlated changes in shape and to determine coordinates for each set of data in said coordinate system referenced to said reference shape; receiving means for receiving a form reference; determining means for determining coordinates in said coordinate system for a said set of data using the received form reference; and modifying means for modifying a data set defining a reference shape of at least a part of the first object using the determined coordinates to generate a modified data set of defining a morph of at least a part of the image.

25. Image processing apparatus according to claim 24, wherein said data sets comprise vector data.

26. Image processing apparatus according to claim 24 or claim 25, wherein said statistical processing means is adapted to determine a coordinate system of reduced dimensionality by determining the most statistically significant dimensions.

27. Image processing apparatus according to any one of claims 24 to 26, wherein said statistical processing means is adapted to carry out a principal component analysis to determine said coordinate system as principal components defined as eigenvectors, wherein said coordinates comprise principal component scores for each set of data for each eigenvector.

28. Image processing apparatus according to claim 27, wherein said statistical processing means is adapted to determine an eigenvalue for each principal component giving the statistical significance of the principal components .

29. Image processing apparatus according to claim 27 or claim 28, wherein said modifying means is adapted to use the eigenvectors and principal component scores to modify said data set.

30. Image processing apparatus according to any one of claims 24 to 29, including means for determining said reference shape of said at least a part of said second object from a mean form of said forms of said at least a part of the or each second object.

31. Image processing apparatus according to any one of claims 24 to 30, wherein said first processing means is adapted to scale said sets of data according to centroid size, to translate the sets of data to make the centroids of the sets of data coincident, and to rotate the data sets about the centroids to minimise the sum of the squares of the differences between equivalent members of the data sets.

32. Image processing apparatus according to any one of claims 24 to 31, wherein said first processing means is adapted to perform a generalised Procrustes analysis.

33. Image processing apparatus according to any one of claims 24 to 32, wherein said first processing means includes transformation means for transforming said data sets defining shapes in non-Euclidean shape space to respective data sets defining shapes in Euclidean space and said modifying means is adapted to use the determined coordinates to determine a data set defining a modified shape of said at least a part of said first object in said Euclidean space and includes back-transformation means for back-transforming said modified data set to said non-Euclidean shape space to obtain said modified data set defining said morph of at least a part of the image .

34. Image processing apparatus according to any one of claims 24 to 33, wherein said data sets comprise landmarks comprising coordinates in two or three dimensions defining important features of form.

35. Image processing apparatus according to any one of claims 24 to 33, wherein said data sets comprise control points for splines defining forms of at least a part of the or each second object.

36. Image processing apparatus according to any one of claims 24 to 35, wherein said receiving means is adapted to receive a phoneme and said data sets define an image of a face.

37. Image processing apparatus according to claim 36, including means for extracting phonemes from text or speech.

38. Image processing apparatus according to any one of claims 24 to 36, wherein the or each second object comprises a face and said receiving means is adapted to receive indicators of facial expressions.

39. Image processing apparatus according to any one of claims 24 to 38 including means for reapplying size information removed in the first processing means to the modified data set.

40. Image processing apparatus according to any one of claims 24 to 39, including second processing means for processing a data set for a reference form of at least a part of the first object with reference to the data set for said reference shape of said at least a part of a said second object to remove effects of size, location and rotation to provide said data set defining said reference shape of said at least a part of the first object.

41. Image processing apparatus according to any one of claims 24 to 39, wherein said first object comprises a modified form of said second object, the apparatus including means to generate said data set defining a reference shape of at least a part of said first object to define a morph of said reference shape of said at least a part of the second object.

42. Image processing apparatus according to claim 41, wherein said morph comprises a caricature of said second object.

43. Image processing apparatus according to any one of claims 24 to 42, wherein said modifying means is adapted to weight the determined coordinates .

44. Image processing apparatus according to any one of claims 24 to 43, wherein said plurality of data sets define a form of at least a part of a plurality of second objects, the apparatus including means for taking an average of any said data sets having the same form reference and using said average as the data set for a form reference.

45. Image processing apparatus according to any one of claims 24 to 44, wherein said first processing means and said statistical processing means are adapted to independently process data sets defining a form of a plurality of parts of the or each second object, said determining means is adapted to determine said coordinate in the coordinate systems determined in the statistical processing means, and said modifying means is adapted to independently modify a plurality of said reference shapes to generate modified data sets, the apparatus including means for combining the modified data sets to generate a data set for the whole image of the first object.

46. Image processing apparatus according to any one of claims 24 to 45, wherein said receiving means, said determining means and said modifying means is adapted to operate repeatedly to generate data defining an animated morph of said at least a part of the image of the first object.

47. A method of generating data defining an animated morph of at least a part of an image of a first object using a stored model of morphological behaviour of at least a part of at least one second object, the model comprising a plurality of coordinates indexed by form references for a coordinate system based on directions of the most correlated changes in shape in a shape space defined by a plurality of data sets defining forms of at least a part of at least one second object processed to remove the effects of size, location and rotation and indexed by form references, the method comprising the repeated steps of: receiving a said form reference; determining coordinates in said coordinate system for a said set of data using the received form reference; and modifying a data set defining a reference shape of at least a part of the first object using the determined coordinates to generate a modified data set defining a morph of at least a part of the image.

48. A method according to claim 47, wherein said stored model comprises eigenvectors defining principal components of variation of shape and a principal component score for each data set indexed by form reference, the determining step comprises determining the principal component scores , and the modifying step comprises modifying the data set using the principal component scores.

49. A method according to claim 47 or claim 48, wherein said coordinates of the model are derived from directions of the most correlated changes in a Euclidean shape space obtained from said plurality of data sets, and the step of modifying the data set defining the reference shape includes using the determined coordinates to determine a data set defining a modified shape of said at least a part of said first object in said Euclidean space and transforming said determined data set to non Euclidean shape space to obtain modification data set defining said morph of at least a part of the image.

50. A method according to any one of claims 47 to 49, wherein said form reference comprises a phoneme and the or each second object comprises a face.

51. A method according to claim 50 including extracting phonemes from text or speech.

52. A method according to any one of claims 47 to 51, wherein said step of modifying the data set defining the reference shape includes weighting the determined coordinates.

53. A method according to any one of claims 47 to 52, wherein the method is applied to a plurality of parts of the image of the first object independently and a modified data set for the whole image of the first object is generated by combining the modified data sets generated for the plurality of parts of the image of the first object.

54. Apparatus for generating data defining an animated morph at least the part of an image of a first object using a stored model of morphological behaviour of at least part of at least one second object, the model comprising a plurality of coordinates indexed by form references for a coordinate system based on directions of the most correlated changes in a shape space defined by a plurality of data sets defining forms of at least a part of at least one second object processed to remove the effects of size, location and rotation and indexed by form references, the apparatus comprising: receiving means for receiving a stream of form references; determining means for repeatedly determining coordinates in said coordinate system for sets of said data using the received form references; and modifying means for repeatedly modifying a data set defining a reference shape at least a part of the first objects using the determined coordinates to generate modified data sets defining an animated morph of at least a part of the image.

55. Apparatus according to claim 54, wherein said stored model comprises eigenvectors defining principal components of variation of shape and principal component scores for each data set indexed by form reference; said determining means being adapted to determine the principal component scores, and said modifying means being adapted to modify the data set using the principal component scores .

56. Apparatus according to claim 54 or claim 55, wherein said coordinates of the model are derived from directions of the most correlated changes in a Euclidean shape space obtained from said plurality of data sets; and said modifying means is adapted to modify the data set defining the reference shape by determining a data set defining a modified shape of said at least a part of said first object in said Euclidean space and transforming said determined data set to non Euclidean shape space to obtain said modification data set defining said morph of at least a part of said image.

57. Apparatus according to any one of claims 54 to 56 wherein said form reference comprises a phoneme and the or each second object comprises a face.

58. Apparatus according to claim 57 including means for extracting phonemes from text or speech.

59. Apparatus according to any one of claims 54 to 58, wherein said modifying means is adapted to weight the determined coordinates.

60. Apparatus according to any of claims 54 to 59, wherein said model stores a plurality of sets of coordinates indexed by form references for a corresponding plurality of parts of the or each second object, said determining means is adapted to determine a plurality of sets of coordinates or the plurality of parts of the image from the model, and the modifying means is adapted to independently modify a plurality of said reference shapes to generate a plurality of modified data sets, the apparatus including means for combining the modified data sets to generate a data set for the whole image of the first object.

61. A computer system for morphing data defining at least a part of an image of a first object, the computer system comprising: a memory for storing processor implementable instructions; a processor for implementing the instructions stored in the memory; the processor implementable instructions comprising instructions for: controlling a processor to process a plurality of data sets using a reference shape, each data set defining a form of at least part of a second object and being indexed by a form reference, to remove effects of size, location and rotation to define a corresponding plurality of shapes of the at least a part of the or each second object; controlling a processor to statistically process the data sets defining shapes of said at least a part of the or each second object to determine a coordinate system based on directions of the most correlated changes in shape and to determine coordinates for each set of data in said coordinate system referenced to said reference shape; controlling the processor to receive a form reference; controlling the processor to determined coordinates in said coordinate system for a said set of data using the received form reference; and controlling the processor to modify a data set defining a reference shape of at least a part of the first object using the determined coordinates to generate a modified data set defining a morph of at least a part of the image.

62. The computer system according to claim 61, wherein the said processor implementable instructions include instructions for controlling the processor to carry out the statistical processing step by determining a coordinate system of reduced dimensionality by determining the most statistically significant dimensions.

63. The computer system according to claim 61 or claim 62, wherein the processor implementable instructions include instructions for controlling the processor to carry out the statistical processing by carrying out principal component analysis to determine said coordinate system as principal components defined as eigenvectors , wherein said coordinates comprise principal component scores for each set of data for each eigenvector.

64. The computer system according to claim 63, wherein the processor implementable instructions include instructions for controlling the processor to modify the data set defining the reference shape of at least part of the first object by using the eigenvectors and principal component scores to modify the data sets.

65. The computer system according to any one of claims 61 to 64, wherein the processor implementable instructions include instructions for controlling the processor to process the data sets to remove the effects of size, location and rotation by scaling said sets of data according to centroid size, translating the sets of data to make the centroids of the sets of data coincident, and rotating the data sets about the centroids to minimise the sum of the squares of the difference between the equivalents of the data sets.

66. The computer system according to any one of claims 61 to 65, wherein the processor implementable instructions include instructions for controlling the processor to process the data sets to remove the effects of size, location and rotation by performing a generalised Procrustes analysis.

67. The computer system according to any one of claims

61 to 66, wherein the processor implementable instructions include instructions for controlling the processor to process the data sets after removal of the effects of size, location and rotation to transform said data sets defining shapes in non Euclidean shape space to respective data sets defining shapes in Euclidean space, and the processor is controlled to modify the data set defining the reference shape to initially determine a data set defining a modified shape of said at least part of said first object in said Euclidean space and to transform the determined data set to said non Euclidean shape space to obtain said modified data set defining said morph of at least a part of the image.

68. A computer system for generating an animated morph of at least part of an image of the first object, the computer system comprising: a memory for storing processor implementable instructions, and a model of the morphological behaviour of at least part of at least one second object, the model comprising a plurality of co-ordinates indexed by form references for a coordinate system based on directions of the most correlated change in shape in a shape space defined by a plurality of data sets defining forms of at least a part of at least one second object processed to remove the effects of size, location and rotation and indexed by form references; a processor for implementing the instruction stored in the memory; the processor implementable instructions comprising instructions for: controlling a processor to repeatedly receive a said form reference; controlling the processor to determine coordinates in said coordinate system for sets of said data using the received form references; and controlling the processor to modify a data set defining a reference shape of at least a part of the first object using the determined coordinates to generate modified data sets defining an animated morph at least a part of the image.

69. A method of generating a model of the morphological behaviour of at least a part of at least one object, the method comprising: processing a plurality of data sets using a reference shape, each data set defining a form of at least a part of a said object and being indexed by a form reference, to remove effects of size, location and rotation to define a corresponding plurality of shapes of the at least a part of the or each object; and statistically processing said data sets defining shapes of said at least a part of the or each object to determine a coordinate system based on directions of the most correlated changes in shape and to determine coordinates for each set of data in said coordinate system referenced to said reference shape.

70. A method according to claim 69, wherein said data sets comprise vector data.

71. A method according to claim 69 or 70, wherein said statistical processing step determines a coordinate system of reduced dimensionality by determining the most statistically significant dimensions.

72. A method according to any one of claims 69 to 71, wherein said statistical processing step comprises principal component analysis to determine said coordinate system as principal components defined as eigenvectors, wherein said coordinates comprise principal component scores for each set of data for each eigenvector.

73. A method according to any one of claims 69 to 72, wherein said reference shape of said at least a part of a said object is determined from a mean form of said forms of said at least a part of the or each second object.

74. A method according to any one of claims 66 to 73, wherein said processing steps to remove the effects of size, location and rotation comprise scaling said sets of data according to centroid size, translating the sets of data to make the centroids of the sets of data coincident, and rotating the data sets about the centroids to minimise the sum of the squares of the differences between equivalent members of the data sets.

75. A method according to any one of claims 69 to 74, wherein said processing step to remove the effects of size, location and rotation comprises a generalised Procrustes analysis.

76. A method according to any one of claims 69 to 75, wherein said processing step to remove the effects of size, location and rotation includes a transformation step of transforming said data sets defining shapes in non Euclidean shape space to respective data sets defining shapes in a Euclidean tangent space.

77. A method according to any one of claims 69 to 76, wherein said form reference comprises a phoneme and the or each object comprises a face.

78. A method according to claim 77, including the step of extracting phonemes from text or speech.

79. A method according to any one of claims 69 to 76, wherein the or each object comprises a face and said form reference comprises a facial expression.

80. Apparatus for generating a model of the morphological behaviour of at least a part of at least one object, the apparatus comprising: processing means for processing a plurality of data sets using a reference shape, each data set defining a form of at least a part of a said object and being indexed by a form reference, to remove effects of size, location and rotation to define a corresponding plurality of shapes of the at least a part of the or each object; and statistical processing means for statistically processing said data sets defining shapes of said at least a part of the or each object to determine a coordinate system based on directions of the most correlated changes in shape and to determine coordinates for each set of data in said coordinate system referenced to said reference shape.

81. Apparatus according to claim 80, wherein said data sets comprise vector data.

82. Apparatus according to claim 80 or claim 81, wherein said statistical processing means is adapted to determine a coordinate system of reduced dimensionality by determining the most statistically significant dimensions.

83. Apparatus according to any one of claims 80 to 82, wherein said statistical processing means is adapted to carry out a principal component analysis to determine said coordinate system as principal components defined as eigenvectors, wherein said coordinates comprise principal component scores for each set of data for each eigenvector.

84. Apparatus according to claim 83, wherein said statistical processing means is adapted to determine an eigenvalue for each principal component giving the statistical significance of the principal components.

85. Apparatus according to any one of claims 80 to 84, including means for determining said reference shape of said at least a part of said second object from a mean form of said forms of said at least a part of the or each object.

86. Apparatus according to any one of claims 80 to 85, wherein said processing means is adapted to scale said sets of data according to centroid size, to translate the sets of data to make the centroids of the sets of data coincident, and to rotate the data sets about the centroids to minimise the sum of the squares of the differences between equivalent members of the data sets.

87. Apparatus according to any one of claims 80 to 86, wherein said processing means is adapted to perform a generalised Procrustes analysis.

88. Apparatus according to any one of claims 80 to 87, wherein said processing means includes transformation means for transforming said data sets defining shapes in non-Euclidean shape space to respective data sets defining shapes in Euclidean space.

89. Apparatus according to any one of claims 80 to 88, including means for extracting phonemes from text or speech.

90. A computer system for generating a model of the morphological behaviour of an object, the computer system comprising: a memory for storing processor implementable instructions; a processor for implementing the instructions stored in the memory; the processor implementable instructions comprising instructions for: controlling a processor to process a plurality of data sets using a reference shape, each data set defining a form of at least part of a said object and being indexed by a form reference, to remove effects of size, location and rotation to define a corresponding plurality of shapes of the at least a part of the or each object; and controlling a processor to statistically process the data sets defining shapes of said at least a part of the or each object to determine a coordinate system based on directions of the most correlated changes in shape and to determine coordinates for each set of data in said coordinate system referenced to said reference shape.

91. The computer system according to claim 90, wherein the said processor implementable instructions include instructions for controlling the processor to carry out the statistical processing step by determining a coordinate system of reduced dimensionality by determining the most statistically significant dimensions.

92. The computer system according to claim 90 or claim 91, wherein the processor implementable instructions include instructions for controlling the processor to carry out the statistical processing by carrying out principal component analysis to determine said coordinate system as principal components defined as eigenvectors, wherein said coordinates comprise principal component scores for each set of data for each eigenvector.

93. The computer system according to any one of claims 90 to 92, wherein the processor implementable instructions include instructions for controlling the processor to process the data sets to remove the effects of size, location and rotation by scaling said sets of data according to centroid size, translating the sets of data to make the centroids of the sets of data coincident, and rotating the data sets about the centroids to minimise the sum of the squares of the difference between the equivalents of the data sets.

94. The computer system according to any one of claims 90 to 93, wherein the processor implementable instructions include instructions for controlling the processor to process the data sets to remove the effects of size, location and rotation by performing a generalised Procrustes analysis.

95. The computer system according to any one of claims 90 to 94, wherein the processor implementable instructions include instructions for controlling the processor to process the data sets after removal of the effects of size, location and rotation to transform said data sets defining shapes in non Euclidean shape space to respective data sets defining shapes in Euclidean space.

96. A carrier carrying processor implementable instructions for controlling a processor to carry out the method of any one of claims 1 to 23 or 69 to 79.