WO2009043927A1

WO2009043927A1 - Apparatus for acquiring and processing information relating to human eye movements

Info

Publication number: WO2009043927A1
Application number: PCT/EP2008/063277
Authority: WO
Inventors: Fiora Pirri Ardizzone; Anna Belardinelli; Andrea Carbone
Original assignee: Universita' Degli Studi Di Roma 'la Sapienza'
Priority date: 2007-10-05
Filing date: 2008-10-03
Publication date: 2009-04-09
Also published as: ITRM20070526A1

Abstract

An apparatus for acquiring and processing information about human eye movements, comprised of a harness (1) to which at least one video-camera (8) is connected to film the scene, a micro-camera (6) for recording the eyes of a subject wearing the apparatus, at least one led (7) for illuminating each eye, an inertial platform (9), a means of processing and transmitting the data generated by the apparatus and methods for processing data for recognizing the pupils and for transforming the coordinates of the device into the coordi- nates of a learning robotic platform suitable for receiving said transmitted data.

Description

APPARATUS FOR ACQUIRING AND PROCESSING INFORMATION RELATING TO HUMAN EYE MOVEMENTS Field of the invention

The present invention relates to the field of apparatuses for tracking human attentive ac- tivities, for the purpose of training robots. State of the art

Several systems exist for the study of visual behaviour in real environments and for tracking eye movements. Some of these systems use depth data, while others use rotation angles of the head, and yet others limit their activity to tracking eye movements, usually for limited movements and with subjects sitting and/or positioned in front of screens, thereby information about distance is not required.

The scene is normally recorded by one or more video-cameras, integral with the head of the observer. It is still always necessary, however, to include head movements to take account of attentive mechanisms originating from factors not included in the view field of the observer and/or of the possible video-camera, such as possible acoustic stimuli. Some known systems place the video-camera as close as possible to the eye, i.e. to the eye's view field, so that the point recorded by the camera coincides with the view point of the observer.

There is known an apparatus provided with a video-camera pointing at one eye only (illuminated by one or more infrared (IR) LED), with a portable computer for processing the image data, and with an external display, fixed or not to the helmet carrying the sensors. However, tracking head movements lacks, as well as a video-camera for aquiring a view of the observed scene and tracking the second eye. Such an apparatus provides for a calibration procedure based on "image processing" techniques by which, once the pupil region has been extracted, the centre thereof is calculated and then re-projected on a display placed in front of one of the user's eyes. Such a calibration procedure, which is the basis for mapping the image coordinates of the eye frames and the coordinates on the screen, is based on the calculation of the coefficients of two third-order polynomial functions, setting N points at fixed locations on a screen.

Another known apparatus consists of two video-cameras turned towards the scene, therefore it may effectively determine the point the observer is looking at. This is achieved, however, by means of a complex motorised system formed of two cardan joints (gimbals) that rotate the video-cameras next to the eyes so that they are kept parallel to the optical axis of each eye. The direction of the axis of the eyes, illuminated by IR LED, is reflected onto dichroic mirrors recorded by two more video-cameras serving the video-cameras recording the scene. The complexity of such an apparatus has negative consequences in terms of weight, making it bulky and uncomfortable for the observer. Yet another apparatus is also based on a portable system for tracking the eye movement. It inlcudes a CCD micro-camera for recording images of the eye while another camera looks at the scene. The coordinates of the centre of the pupil are extracted by means of an image recognition program. The method used involves calibration based on predetermined eye regard points having a number between 25 and 100, and automatically determines the mapping function between the two cameras as a correlation function suited using quadratic regression. A form of compression and classification of the sequences is conducted by classifying the eye regard points.

Hence, the result is a laborious calibration process combined with a poor precision in determining the point looked at, even if the eye is not staring, also called the "point of re- gard".

Yet other apparatuses record movements of the head by means of fixed magnetic transponders, or by recording the observer with a video-camera, which may limit the scope of operation of the apparatus itself. Therefore, the apparatuses provided by the known art are not able to simultaneously track observer eye and head movements, and employ architectures that are often bulky and complex.

Summary of the invention

The object of the present invention is to solve the above-mentioned problems by providing an apparatus, for brevity hereafter referred to as "gaze machine", for studying the dynam- ics of eye movements, which automatically acquires, processes and stores data relating to movements of the eyes and head of an observer, while at the same time being light, precise and reliable.

Therefore, the present invention suggests to achieve the above examined objects by providing an apparatus for aquiring and processing information relating to human eye activi- ties, according to claim 1 , and including:

- one or more video-cameras for the stereoscopic recording of the same scene observed by an observer;

- one or more micro-cameras for recording the eyes of the observer;

- at least one IR led illuminating each eye; - an inertial platform; - a harness that may be firmly fit on the head of the observer, and on which said one or more video-cameras, said one or more micro-cameras and said inertial platform are firmly mounted;

- data processing means that may produce data structures containing the information generated by said one or more micro-cameras, by said one or more video-cameras and by said inertial platform;

- means for the reception-transmission of said data.

According to another aspect of the invention, said apparatus may also be applied when the data obtained therefrom is used by a robotic platform to mimic the observing activities of an observer.

Advantageously, the apparatus according to the present invention allows to record both the speed of the eye as it looks at the scene, and the depth of the observed point of regard, and to track the movements made by the eyes while the observer performs other sensory-motor activities, for example walking, manipulating objects, and in dynamic envi- ronments in general.

In particular, tracking head movements is performed by inertial sensor devices that, given their light weight, allow freedom of movement and precision of data, thus avoiding the observer from being recorded with an external micro-camera or to have a fixed magnetic transponder to estimate the orientation of the head. The dependent claims describe preferred embodiments of the invention. Brief description of Drawings

Other features and advantages of the invention will become more apparent from the following detailed, though not exclusive description of a preferred embodiment of an apparatus for acquiring and processing information relating to human eye activities, which is illustrated by way of non-limitative example in the accompanying drawings, in which: Figure 1 shows a side view of a gaze machine according to the invention; Figure 2 shows a front view related to the above;

Figure 3 shows an illuminated pupil with the projection of a Purkinje image; Figure 4a shows Le Grand's simplified representation of the eye, and Figure 4b shows possible cases of reflections of Purkinje images;

Figure 5 shows the sequence of processing steps carried out on the image of the eye in order to identify the pupil thereof;

Figure 6 illustrates the calculation of the centre of the CPT pupil, in a sequence of two frames. In the sequence, the bottom figure illustrates the filtered image of the eye, while the top image indicates the saturated image. The top image, in addition to the centre of the pupil, also shows the vectors corresponding to the line of sight and the combination of the Purkinje image and the line of sight;

Figure 7 shows a calibration model that may be used to identify a correspondence function between the coordinates of the centres of the pupils, as seen in several frames by a micro-camera focusing on the eyes, and the coordinates of the corresponding image- scene recorded by a pair of video-cameras;

Figure 8 shows the transformations that need to be carried out to obtain the projection of a point of the scene onto the eye and onto the video-camera pointing at the eye; Figure 9 diagrammatically shows a point being viewed by an observer, and shared by a robotic platform;

Figure 10 shows a preliminary vector composition diagram for the transformations between the apparatus reference system and the reference system of a learning robotic platform; Figure 1 1 shows a flow diagram referred to a method for processing information acquired by the gaze machine;

Figure 12 shows a preferred embodiment of the device, provided with a micro-camera for each eye.

Detailed description of a preferred embodiment of the invention

A preferred embodiment of the apparatus includes a gaze machine and a robotic platform for receiving the data transmitted by said gaze machine, for the purpose of aligning the platform with the same observation point as the observer, with the aim of training said robotic platform.

In particular, a preferred embodiment of the gaze machine is comprised of a helmet provided with an inertial platform 9, two video-cameras 8, at least one micro-camera 6, two infra-red leds 7, one for each eye, and means for processing and transmitting data.

In addition, said helmet defining the harness for the observer's head is provided with adjusting means to ensure a better fitting thereto.

In particular, such means include a nut 4 and bearings 5 positioned at the nape of the neck. Connected to the helmet 1 there is an arm 2 that may be oriented by means of hinges 3, and adapted to support the micro-camera(s). This arm should preferably be made of carbon, to be stiff and light at the same time.

Said at least one micro-camera 6 comes with CMOS technology and is mounted to a rigid support 5. Since CMOS technology is infra-red sensitive, to the sides of the micro-cameras, and held by further supports 5, there is at least one pair of infra-red leds 7, adapted to illuminate the observer's eyes.

Using said infra-red leds 7 allows to increase the contrast between the pupil, the central zone of the iris, and the sclera. In particular, the optical axis of the micro-camera and the emission axis of the infra-red leds 7 are deliberately not aligned in order to encourage manifestation of the so-called 'back-pupil' effect, by which the pupil appears to have an intense black colour, increasing the contrast with respect to the region around the iris. In a preferred embodiment of the invention, a micro-camera 6 and an infra-red led 7 are used for each eye. In another preferred embodiment, a single micro-camera is placed in a central position so that it may record both eyes simultaneously. This solution, with one micro-camera, allows to finely determine the vergence of the pupils in a single image, and the exact point at which the gaze of each eye meets. This is because the point of observation of the video- cameras on the scene may never exactly match that of the user, and so the projection of a single eye gives a line of possible points of regard, as illustrated in more detail below.

The inertial platform 9 is a so-called motion tracker, which allows to record absolute angles of rotation of the head, with at least 3 degrees of freedom. Said inertial platform 9 is integral with said helmet 1 by means of a support 10. Using a pair of video-cameras 8 for stereoscopic recording the same scene being looked at by the observer is preferred.

The method of aquiring and processing information is explained with the aid of Figure 1 1 , including the following steps: aquiring the image of the scene and the image of the eyes;

- identifying the pupil and identifying the centre thereof; - projecting the point of regard of the eyes on the scene image;

- storing in a single structure the image data gathered at the previous step, together with information about the position of the head;

- transmitting and receiving said data and transforming the point of regard into the coordinates of a learning robotic platform; - converging the head defining said robotic platform onto the point of regard of the observer.

In particular, the video-cameras 8 of the gaze machine, which are turned towards the scene, allow to aquire RGB images and range, by calculating the disparity between the two optics. For every pixel of every frame, therefore, intensity values on three colour channels and the global coordinates [X Y Z]^τ with respect to the current observation point are available.

At the end of every recording, the apparatus, in virtue of said means of data processing, generates data structure vectors, labelled to the moment of acquisition, and including:

- RGB frames related to the scene recorded by the video-cameras; - the position of the observed point, projected onto the frame expressed in image coordinates (pixels);

- the depth image corresponding to the colour image;

- the orientation of the head, expressed in Euler angles.

When the gaze machine is used by a stationary (e.g. sitting) observer, the positions of the gaze along the frames of the sequence refer to the same view point, thus they may be re- composed in a single global reference system, coinciding with the reference system integral with the position of the observer. In this case, it is possible to determine, by means of known procedures, the direction and speed of the saccades made by the observer with respect to the global coordinates referred to the reference system with the position of the observer as the origin.

The data provided by the gaze machine are transmitted to a per se known robotic platform, which further processes and uses said data.

A preferred robotic platform is provided with a stereoscopic camera and a pan-tilt unit that may mimic the posture of the head in terms of pitch and yaw angles, acquiring the data provided by the gaze machine through its own means of data reception-transmission, and processing this data using its own processing means, which may conduct roto-translating transformations, from the observer reference system, defined as global, to the reference system of the robotic platform, so that the platform may mimic in real time the acquisition process used by the observer/tutor, and sharing the same observed point. The transmission of data from the aquiring apparatus to the robotic head is achieved by means of a wireless connection, made in virtue of said communicating means connected to the computer, defining said processing means.

The data are made available by the gaze machine in real time so that they may be used even by several robotic platforms and/or further processing systems. A method is described below, which is also an object of the present invention, for identifying the pupil.

The pupil, in dynamic environments with variable lighting conditions, does not display the classic distinction between the iris and the sclera that occurs in indoor environments with stable lighting conditions, as seen from Figure 3, in which the pupil is blurred by the re- flection of the sky. To solve the problem of identifying the pupil, the reflection of the led on the eye and the Purkinje images thereby generated are used.

Of the four Purkinje images, as seen from Figure 4, only the first has the maximum intensity, i.e. values between 245 and 255 in any image, both indoor and outdoor. The infra-red leds are positioned at about 15 cm from the eye in the micro-camera reference system, and are placed along the X and Y axes, to ensure that the Purkinje image is positioned within the limbal ring of the iris, by means of the rotations of the optical axis up to 30 degrees. The term Lc refers to the point on the image plane that represents the position of each in- fra-red led, i.e. where it keeps the distinction characteristics, as seen in Figure 3.

For the identification of the centre of the pupil, the Purkinje I image is mainly determined, after which the appropriate transformations are calculated as described below. There is no need for calibration of the led, in the image of the eye, centred with respect to the micro- camera, i.e. looking in the direction of the objective, the Purkinje I image needs to appear inside the pupil, as seen in the first type of reflection of the Purkinje image, see Figure 4. The reflection of the at least 2 leds through the Purkinje images depends on the position of the CMOS micro-camera with respect to the eyes (which changes every time the gaze machine is put on), on the curvature of both the cornea and the limbal zone typical of each individual. To identify a correspondence function between the coordinates of the centres of the pupils, as seen by a micro-camera focusing on the eyes, and the coordinates of the corresponding image-scene, a calibration procedure is used which requires a number of images of each eye in a similar number of specific positions. At the end of the procedure, the preferred method for identifying the pupil involves the following steps.

Macro-step 1 : The image of the eye captured by the micro-camera, measuring 130x220, is cropped to an image measuring 130x130 pixels. The image is then saturated (see Figure 5a) and filtered (see Figure 5b), using a Gaussian filter derivative measuring 3 x 3 with a variance of 0,28. If Ix represents the transformed image, then the Purkinje I image is identified by the upper-limit region of Ix containing minimal points. Let Pk be the identified region.

Macro-step 2: given M images of the eye, identifying extremal rotations along the axis of rotation as indicated in Figure 5c, with M ≥ 8, we have the equation x = [A^TA]^"1A^Tb where matrix A is an Mχ8 matrix whose standardised elements are: - the position of Pk, in the image at frame t and the position Pk_t+i in the image at frame t + 1 ,

- the position of the centre of the pupil Cp, in the image at frame t;

- the position of the intersection point with the straight line L1 , which passes through Lc and Pk,;

- the normal straight line x of the image, which passes through the centre of the pupil Cp,, called the nodal point N.

The vector b, measuring M x 2, consists of the centres of the pupils Cp₁₊₁ at frames t+1 . The bi-dimensional vector x, measuring 8 x 2, is the coefficient vector, i.e. the mse (mean squared error) obtained by estimated regression is less than 20.

Macro-step 3: given the coefficient vector x, at each computational step the values indicated by an observation of matrix A are calculated for the current image, and the vector T is obtained, whose elements are those deriving from an observation of matrix A. The internal product [T] [x] gives an approximation of the centre of the pupil, as seen from Figure 6, in which the pupil is in extremal conditions and in saccadic phase.

The current positions Cp₁₊₁, calculated with Tx, and the values N₁₊₁ (nodal point), and Pk₁₊₁ (Purkinje image of the subsequent frame), calculated as indicated at step 2, are assigned to the corresponding positions for the next computational step. For every cycle, defined by said three computational steps, the algorithm provides the po- sition of the centre of the pupil, and the direction and amplitude of the movement with respect to the previous image.

A calibration procedure is illustrated below which is used for the pupil identification procedure. The calibration process is repeated each time an observer puts the gaze machine on, be- cause the relative position of the gaze-machine with respect to the observer's eyes influences the outcome of the procedure.

The observer wearing the gaze machine must observe a sequence of distinct points in space, such as those shown in Figure 7. Each points needs to be visible to the video- camera pointing at the scene. To this purpose, a specific reference pattern needs to be prepared to carry out the calibration. The pattern consists of a grid containing a minimum of 7 planar and non-planar references. The instances corresponding to each regard are directly selected by the observer by means of any type of man/processor interface, e.g. by means of the keys on a keyboard. For every regard thus carried out, the following data are saved: a) the images of the video-cameras pointing at the scene; b) the images of the two eyes that are looking at the single reference point. Once the reference/eye pairs are collected, they are subjected to a series of operations as listed below, to perform the transposition of the observation point onto the observed scene: a) extraction of the coordinates of the centre of the pupil for each eye and reference coordinates in the corresponding image of the scene; b) estimate of the mapping function for the point of regard by calculating the trifocal tensor.

Thus, summarizing, the preferred calibration procedure includes the following steps: - a reference model is prepared with a number > 7 of planar and non-planar references;

- in a number n of instances, the observer looks at a reference point and records said point by means of an interface;

- the coordinates of the centre of the pupil for each eye and the reference coordinates in the corresponding image of the scene are extracted; - the mapping function of the point of regard is estimated by calculating a matrix called the trifocal tensor T.

Roto-translation procedure from the global reference system to the micro-camera reference system. Let us consider two reference systems: the eye reference system, which it is assumed to be coincident with the global reference system, and the reference system of the micro- camera looking at the eye. These systems are linked by a rigid transformation (R, t) in which R is a 3x3 rotation matrix and t is a translation vector (see Figure 8 for calibration between the two reference systems). The projection equations of a point P = (x, y, z) on the eye are given by:

wherein it is assumed that the point Pe, corresponding to the projection of P, is in the Nm- bal region of the eye, which is assumed to be planar, and coincides with the centre of the pupil.

The coordinates of the point Pc, corresponding to the projection of Pe on the micro- camera, are determined as follows:

There is therefore a projective matrix directly linking the points in the scene with the points on the image from the micro-camera looking at the eye.

By performing a similar procedure for the second eye, we obtain three image planes, those for the two eyes and the one for the micro-camera recording the scene, in which the coordinates of the projections of a point of the scene, are reciprocally linked by a matrix T, called the trifocal tensor. A preferred method for calculating said trifocal tensor, and then how to calculate the projection of the point onto the third image starting from T and from the projection of a point on two images, is disclosed below. Method for identifying the trifocal tensor.

Given n > 7 points at the three image planes, the algorithm for calculating the trifocal tensor involves the following steps:

- there are three transformation matrices, H, H' and H" to be applied to the three images;

- each point x¹ of the i-th image is transformed into the point x = H^x¹ for standardisation purposes;

- T is linearly calculated in terms of the transformed points, solving the series of equations of the form At = 0, using a pseudo-inverse calculation and the minimum square method. Matrix A is a matrix of 27 lines such that for each equation of the type x^ιx^ιlx^Λε_jqsε_krtT?r = 0_st there are nine equations, one for each choice of s and t.

In the above-mentioned equation, ε_jqs and ε_qr1 are tensors defined by r, s, t = 1 , 2, 3 as follows:

0 unless r, s, t are not distinct ε _rst ⁼ 1 if rst is an even permutation of 123 - 1 if rst is an odd permutation of 123

and 0_st is also a tensor.

- The trifocal tensor is calculated by applying the inverse de-standardisation transformation to obtain the original tensor: τ/^k = H; (H'-¹ y_s (H"-¹ )';f_r ^st Procedure for correspondence transfer from a pair of views to a third view.

The projection of the point of regard of the eyes onto the image of the scene, for the purposes of enabling the convergence of said robotic head with the observer's observation point, is achieved by means of a roto-translation that first converts the reference system associated with said micro-camera 6 into the reference system for the eyes and then into the reference system for said video-camera 8 recording the scene observed by the subject.

Therefore, let x -> x' be an exact correspondence between the first and the second views calculated by using the usual mode. Let F₂i be the basic matrix linking the first two views, and determined from the trifocal tensor T. To obtain the correspondence in the third view, simply follow the following steps:

- the line /' , which perpendicularly passes through x', is calculated by means of the following equation V _e = F_2lx = (IJ₂I₃)T ,

- the coordinates of the transferred point x"^k = x'V _} T are determined. The present apparatus therefore allows to create a man-robot interface that may be used for various automatic learning tasks applied to artificial vision, simplifying the acquisition of data and of scanning patterns. These may be used not only in the domain of cognitive robotics, but also in surveillance systems, for remote-operated systems for disabled people, or for computer graphics and virtual reality applications. In other fields, it may also be used for marketing surveys, usability analysis for systems and interfaces, various studies about human attention, for therapeutic purposes for subjects affected by cognitive dysfunctions linked to viewing, and in general for any applications in which to know what an observer is looking at is of interest. The specific embodiments described herein are not limiting the content of this application, which covers all the variants of the invention defined by the claims.

Claims

1 . An apparatus for aquiring and processing information relating to human eye movements made by an observer, including:

- one or more video-cameras (8) adapted for stereoscopic recording a scene observed by an observer;

- one or more micro-cameras (6) adapted for recording the eyes of the observer;

- at least one IR led (7) illuminating each said eyes;

- an inertial platform (9);

- a harness (1 ) that may be fit firmly on the head of the observer, to which said one or more video-cameras, said one or more micro-cameras and said inertial platform are firmly mounted;

- data processing means adapted to produce data structures containing the information generated by said one or more micro-cameras, by said one or more video-cameras and by said inertial platform; - means for the reception-transmission of said data; being said data received by a robotic platform which is adapted to extract the points of a scene observed by the observer therefrom, so as to train the robotic platform.

2. An apparatus according to claim 1 , wherein said harness defines a helmet (1 ) provided with adjusting means to better fit to the head of the observer.

3. An apparatus according to claim 2, wherein said helmet is provided with an arm (2) which may be oriented by means of hinges (3), used to support said one or more micro- camera^) (6) and a led (7) for each eye.

4. An apparatus according to claim 1 , wherein said one or more micro-cameras (6) have a CMOS technology.

5. An apparatus according to claim 1 , wherein said inertial platform (9) is a motion tracker, adapted for detecting absolute angles of rotation of the head of the observer, with at least 3 degrees of freedom.

6. An apparatus according to claim 1 , wherein said video-cameras (8) are adapted for acquiring RGB images and range by calculating the disparity between two optics of said video-cameras.

7. An acquiring apparatus according to claim 1 , wherein said means of data processing, at the end of every recording, are adapted for generating data structure vectors labelled to the moment of acquisition, and including:

- RGB frames of the scene recorded by the video-cameras; - the position of the observed point, projected onto the RGB frames, expressed in image coordinates (pixels);

- the depth image corresponding to the colour image;

- the orientation of the head, expressed in Euler angles.

8. A method of acquiring and processing information, by means of the apparatus claimed in claim 1 , including the following steps: acquiring an image of a scene and an image of the eyes;

- identifiying a pupil of said eyes and identifying the centre thereof; projecting a point of regard of the eyes on said image of the scene;

- storing in a single structure the image data gathered at the previous step, together with information about the position of the observer's head;

- transmitting and receiving said data and transforming the point of regard into the coordinates of a learning robotic platform; converging the head defining said robotic platform onto the point of regard of the observer.

9. A method according to claim 8, wherein said step of identifying the pupil involves the following macro-steps: Macro-step 1 :

- capturing an image of the eye by means of a micro-camera, measuring 130x220;

- cropping said image to give another image measuring 130x130 pixels; - saturating and filtering the latter image using a Gaussian filter derivative measuring 3 x 3 with a variance of 0,28, making it possible to obtain the Purkinje I image by identifying an upper-limit region of the image obtained, containing minimal points;

- labelling the identified region with the symbol Pk; Macro-step 2: - making the equation x = [A^TA]^"1A^Tb, given a number M ≥ 8 of images of the eye, identifying extremal rotations along the axes of rotation, wherein matrix A is an M x 8 matrix whose standardised elements are:

- the position of the region identified as Pk, in the image at frame t and a position Pk₁₊₁ identified in the image at frame t + 1 , - the position of the centre of the pupil Cp, in the image at frame t;

- the normal straight line x of the image, which passes through the centre of the pupil Cp,, called the nodal point N; wherein the vector b, measuring M x 2, consists of the centres of the pupils Cp,₊i at frames t+1 ; the bi-dimensional vector x, measuring 8 x 2, being the coefficient vector, such that the mse, i.e. the so-called "mean squared error", obtained by estimated regression, is less than 20; Macro-step 3: - given the coefficient vector x, at each computational step calculating the values indicated by an observation of matrix A for the current image, and obtaining the vector T, whose elements are those deriving from an observation of matrix A; since the internal product [T] ^*[x] gives an approximation of the centre of the pupil, in extremal conditions and in saccadic phase; - assigning the current positions Cp₁₊₁, calculated with Tx, and the values N₁₊₁, and Pk₁₊₁ , calculated as indicated at macro-step 2, to the corresponding positions for the subsequent computational step; whereby, for every cycle defined by said three computational macro-steps, the position of the centre of the pupil, the direction and amplitude of the movement with respect to the previous image are provided.

10. A method according to claim 8, including a further, preventive calibration procedure involving the following steps:

- preparing a reference model with a number n > 7 of planar and non-planar references;

- in a number n of instances, looking by the observer at a reference point and recording said point by means of an interface;

- extracting the coordinates of the centre of the pupil for each eye and the reference coordinates in the corresponding image of the scene;

- estimating the mapping function of the point of regard by calculating a matrix called the trifocal tensor T.

1 1. A method according to claim 10, wherein the calculation of said trifocal tensor in the calibration procedure involves the following steps:

- linking a reference system of a first eye and a reference system of a micro-camera looking at the eye by a rigid transformation (R, t) in which R is a 3 x 3 rotation matrix and t is a translation vector, so that the projection equations of a point P = (x, y, z) on the eye are given by:

u_e = — x

wherein it is assumed that a point Pe, corresponding to the projection of P, is in the limbal region of the eye, which is assumed to be planar, and coinciding with the centre of the pupil;

- determining the coordinates of a point Pc, corresponding to the projection of Pe on the micro-camera, by means of the following matrix equation:

- performing a similar procedure for the second eye;

- obtaining three image planes, those for the first and second eyes and the one for the micro-camera pointing at the scene, in which the coordinates of the projections of a point of the scene are linked by said matrix T.

12. A method according to claim 1 1 , wherein said transformation of the reference system of said first eye into the reference system of the micro-camera, involves the following steps:

- identifying a number n > 7 of points at said three image planes;

- defining three transformation matrices, H, H' and H", to be applied to the three image planes;

- each point x' of an i-th image is transformed into the point x = H)X¹ for standardisation purposes; - calculating said trifocal tensor T linearly in terms of the transformed points, solving the series of equations of the form At = 0, by using a pseudo-inverse calculation and the minimum square method; being A a matrix of 27 lines such that for each equation of the type x x^ιlx^nkε_HSε_krtT?r = 0_st there are nine equations, one for each choice of s and t, where ε_jqs and ε_qr1 are the tensors defined by r, s, t = 1 , 2, 3 as follows:

0 unless r, s, t are not distinct

'-'rst 1 if rst is an even permutation of 123 - 1 if rst is an odd permutation of 123 and 0_st is also a tensor.

- calculating the trifocal tensor T by applying the inverse de-standardisation transformation to obtain the original tensor, by means of the following equation:

13. A method according to claim 8, wherein said projection of the point of regard of the eyes onto the image of the scene, in order to allow the convergence of said robotic head with the observer's observation point, is achieved by means of a roto-translation that first converts a reference system associated with a means used to acquire said images of the eyes and then converts it into a reference system associated with a means adapted to acquire said images of the scene.

14. A method according to claims 9 and 13, wherein said roto-translation is achieved by means of a function of the type x -> x', which is an exact correspondence between a point in a first reference system and in a second reference system, then, given F₂i, a basic matrix linking the two reference systems which may be determined from said trifocal tensor T, in order to obtain the transformation into a third reference system, the following steps are performed:

- a line /' , which perpendicularly passes through x', is calculated by means of the follow- ing equation l\ = F_2lx = (I₁I₂I₃ )T

- the coordinates of a transferred point are calculated by means of the following equation x"^k = _x'r, τ