WO2006097722A2

WO2006097722A2 - Interface control

Info

Publication number: WO2006097722A2
Application number: PCT/GB2006/000925
Authority: WO
Inventors: David Cumming; Stefan Veeser
Original assignee: Intelligent Earth Limited
Priority date: 2005-03-15
Filing date: 2006-03-15
Publication date: 2006-09-21
Also published as: WO2006097722A3; GB0505362D0

Abstract

An apparatus and a method are disclosed which control the viewable contents of a graphical user interface (GUI) on a display, or the position of a mechanical or electrical device. Images of an object, for example a user’s face, in the field of view of an imaging means are recorded and compared to provide a measure of the difference in the images of the object. A change in the position of the object with respect to the imaging means is calculated based upon the difference in the images of the object and the viewable contents of the GUI are changed, or the device is moved, in response to the change in position of the object.

Description

Interface Control

The present invention relates to an interface control that allows a user of computing means with a graphical user interface (GUI) to control the viewable content of the graphical user interface or to control a mechanical or electronic device. A graphical user interface is any graphical interface displayed on a computer screen, a television or the like which is connected to computing means' such as a computer games console, personal computer or a handheld device such as a mobile phone, PDA or the like.

In the field of computer games, for example, there are problems with character control for multiple simultaneous operations where a character has to be controlled to for example, run, look or fire a gun. Joysticks with buttons, switches and levers provide an interface that can be difficult to use for complex tasks. In business, software manipulation of data objects like large spreadsheets or images using a mouse and keyboard can be awkward. As well as being non-intuitive and time consuming, the use of keyboard and mouse can cause repetitive strain injury and can result in a slow work rate and operator boredom.

There have been some attempts to improve the extent to which the interaction between the user and the interface is intuitive. For example, immersive body suits and tactile gloves have been developed to allow a user's natural movement to control the GUI. Another example is the Nouse (nose mouse) . This device consists of a computer and software plus a camera to estimate where the user's nose is pointing. Nouse then uses this estimate to guide a pointer on the screen.

The Eye Toy uses a Sony Playstation®, a USB camera and a television screen as a medium for playing simple games. The video image of a person is put on screen and motion detection is used to allow the position of the image of a person on screen to be altered by movement of the person. This allows the person to control the movement of his image on screen and to have the image interact with other on-screen images.

A particular problem with handheld devices, such as PDAs and mobile phones, is that the generally small screen size limits the amount of information that can be displayed at any one time. For example, a large image either requires to be scaled down to a size to display on the screen, or it can be displayed full size with scroll bars present to manually scan over the image. Scaling images to display on the screen of a mobile device will generally cause some loss of detail as there may be a significant difference between the resolution of the screen and the resolution of the image. Furthermore, it is inconvenient to have to use a stylus or thumb pad to scroll back and forth in order to view an image that has been rendered full size.

It is an object of the present invention to provide an improved graphical user interface and in particular to provide a graphical user interface that allows a user to control movement of graphical objects/on screen functionality in an intuitive manner.

In accordance with a first aspect of the invention there is provided an apparatus for controlling the viewable contents of a graphical user interface (GUI) on a display, the apparatus comprising: imaging means for obtaining at least one image of an object in the field of view of the imaging means; first analysis means for determining one or more parameters of the at least one image relative to a reference; second analysis means for calculating the position of the object with respect to the imaging means based upon the value of the one or more parameters of the at least one image; and computing means for changing the viewable contents of the GUI in response to the position of the object.

In accordance with a second aspect of the invention there is provided an apparatus for controlling a device in response to the movement of an object, the apparatus comprising: imaging means for obtaining at least one image of the object in the field of view of the imaging means; first analysis means for determining one or more parameters of the at least one image relative to a reference; second analysis means for calculating the position of the object with respect to the imaging means based upon the value of the one or more parameters of the at least one image; and computing means for providing a change in the device in response to the movement of the object.

Preferably, the reference is an image of the object.

Optionally, the reference is the centre point of the at least one image.

Alternatively, the reference is a zero position relative to the imaging means.

Preferably, the one or more parameters of the at least one image of the object include the position of the object.

Preferably, the one or more parameters of the at least one image of the object include the size of the object.

Optionally, the one or more parameters of the at least one image of the object include the distance of a feature of the object from the centre point of the at least one image.

Preferably, the imaging means is mounted in a predetermined position with respect to a reference point. Preferably, the imaging means is mounted in a fixed position with respect to the reference point.

Alternatively, the imaging means is movably mounted with respect to the reference point.

Preferably, the imaging means captures successive images at a rate of at least 5 frames a second.

More preferably, the imaging means captures successive images at a rate of at least 10 frames a second.

Preferably, the reference point is on or near the display.

Preferably, the first analysis means detects the position of the object image in the imaging means field of view.

Preferably, the first analysis means detects features of the object.

Preferably, the first analysis means detects shape features of the object.

Preferably, the first analysis means detects the boundary of the object image and defines the position of the object image in terms of its boundary.

Preferably, the second analysis means calculates the change in position of the object and provides a 2 dimensional (2D) to 3 dimensional (3D) transformation. Preferably, the second analysis means uses one or more imaging means parameters .

Preferably the imaging means parameters are the angle of view of the imaging means and/or the focal length of the imaging means and/or the resolution of the imaging means and/or the distance between the imaging means and the object.

Preferably, the second analysis means calculates the change in the number of pixels in successive object images to determine a 3D position of the object with respect to the imaging means.

Preferably, the second analysis means of the first aspect calculates changes in orientation of the object to determine changes in the orientation of the object with respect to the GUI on the display.

Preferably, the change in the viewable contents of the GUI of the first aspect provides a virtual viewpoint which is a view of the GUI which is alterable in response to changes in successive object images.

Preferably, the computing means of the first aspect changes the viewable contents in order to enlarge the viewable contents when the object moves towards the display.

Alternatively, the computing means of the first aspect changes the viewable contents in order to enlarge the viewable contents when the display moves towards the object. Preferably, the computing means of the first aspect changes the viewable contents in order to reduce the size of the viewable contents when the object moves away from the display.

Alternatively, the computing means of the first aspect changes the viewable contents in order to reduce the size of the viewable contents when the display moves away from the object.

Preferably, the computing means of the first aspect changes the viewable contents in order to pan the viewable contents to the right when the object moves to the left relative to the display.

Alternatively, the computing means of the first aspect changes the viewable contents in order to pan the viewable contents to the right when the screen moves to the right relative to the object.

Preferably, the computing means of the first aspect changes the viewable contents in order to pan the viewable contents to the left when the object moves to the right relative to the display.

Preferably, the computing means of the first aspect changes the viewable contents in order to pan the viewable contents to the left when the display moves to the left relative to the object.

The present invention allows the user to control the viewable contents of a display by moving the object, typically, the user's head with respect to the display and imaging means in order to change the contents of the display. Additionally, or alternatively, the display can be moved with respect to the user's head to change the contents of the display.

The computing means of the first aspect can change the viewable contents such that the changes in the viewable contents mimic the changes that would occur where the displayed screen contents was a real 3D scene.

Preferably the device of the second aspect is a mechanical device.

Preferably, the computing means of the second aspect moves the device when the object moves relative to the imaging means.

The present invention allows the user to control the position of a device by moving the object, typically, the user's head with respect to the imaging means in order to move the device.

In accordance with a third aspect of the invention there is provided a method for controlling the viewable contents of a graphical user interface (GUI) on a display, the method comprising the steps of: obtaining at least one image of an object in the field of view of an imaging means; determining one or more parameters of the at least one image relative to a reference; calculating the position of the object with respect to the imaging means based upon the value of the one or more parameters of the at least one image; and changing the viewable contents of the GUI in response to the position of the object,

In accordance with a fourth aspect of the invention there is provided a method for controlling a device in response to the movement of an object, the method comprising the steps of: obtaining at least one image of the object in the field of view of an imaging means; determining one or more parameters of the at least one image relative to a reference; calculating the position of the object with respect to the imaging means based upon the value of the one or more parameters of the at least one image; and changing the device in response to the position of the object.

Preferably, the step of determining one or more parameters of the at least one image relative to the reference comprises comparing the at least one image with an image of the object.

Alternatively, the step of determining one or more parameters of the at least one image relative to a reference comprises comparing the at least one image with a zero position relative to the imaging means.

Preferably, the one or more parameters of the at least one image of the object include the position of the object. Preferably, the one or more parameters of the at least one image of the object include the size of the object.

Optionally, the one or more parameters of the at least one image of the object include the distance of a feature of the object from the centre point of the at least one image .

Preferably, the step of obtaining at least one image of the object includes positioning the imaging means in a predetermined position with respect to a reference point.

Preferably, the imaging means is positioned at a fixed position with respect to the reference point.

Alternatively, the imaging means is movable with respect to the reference point.

Preferably, the step of obtaining images of the object captures successive images at a rate of at least 5 frames a second.

More preferably, the step of obtaining images of the object captures successive images at a rate of at least 10 frames a second.

Preferably, the step of determining one or more parameters of the at least one image includes detecting the position of the object in the imaging means field of view. Preferably, the step of determining one or more parameters of the at least one image includes detecting features of the object.

Preferably, the step of determining one or more parameters of the at least one image includes detecting shape features of the object.

Preferably, the step of comparing determining one or more parameters of the at least one image includes detecting the boundary of the object image and defining the position of the object image in terms of its boundary.

Preferably, the step of calculating the position of the object includes calculating the change in position of the object and providing a 2D to 3D transformation.

Preferably, calculating the position of the object uses one or more imaging means parameters .

Preferably, the one or more imaging means parameters are the angle of view of the imaging means and/or the focal length of the imaging means and/or the resolution of the imaging means and/or the distance between the imaging means and the object.

Preferably, calculating the position of the object includes calculating the change in the number of pixels in successive object images and determining a 3 dimensional position of the object with respect to the imaging means. Preferably, calculating the position of the object includes calculating changes in orientation of the object to determine changes in the orientation of the object with respect to the GUI on the display of the third aspect.

Preferably, changing the viewable contents of the GUI includes providing a virtual viewpoint which is a view of the GUI of the third aspect that is alterable in response to changes in successive object images.

Preferably, changing the viewable contents of the GUI of the third aspect includes enlarging the viewable contents when the object moves towards the display.

Alternatively, changing the viewable contents of the GUI of the third aspect includes enlarging the viewable contents when the display moves towards the object.

Preferably, changing the viewable contents of the GUI of the third aspect includes reducing the size of the viewable contents when the object moves away from the display.

Preferably, changing the viewable contents of the GUI of the third aspect includes reducing the size of the viewable contents when the display moves away from the object.

Preferably, changing the viewable contents of the GUI of the third aspect includes panning the viewable contents to the right when the object moves to the left relative to the display. Alternatively, changing the viewable contents of the GUI of the third aspect includes panning the viewable contents to the right when the display moves to the right relative to the object.

Preferably, changing the viewable contents of the GUI of the third aspect includes panning the viewable contents to the left when the object moves to the right relative to the display.

Alternatively, changing the viewable contents of the GUI of the third aspect includes panning the viewable contents to the left when the display moves to the left relative to the object.

Preferably, the device of the fourth aspect is a mechanical device.

Preferably, changing the device in response to the position of the object includes moving the device when the object moves relative to the imaging means.

The present invention will now be described by way of example only with reference to the accompanying drawings in which:

Figure 1 is a schematic representation of an apparatus in accordance with the present invention; Figure 2 is a block diagram showing an example of the apparatus of the present invention and its functionality; Figure 3 is a flow diagram showing an example of a method of operation of the present invention; and Figure 4 is a block diagram showing a method of operation of an alternative embodiment of the present invention.

Figure 1 shows a schematic representation of an apparatus in accordance with the present invention. The apparatus 1 comprises imaging means in the form of a digital camera 5 that is mounted on top of a display screen 7. The camera 5 is connected to a personal computer or the like which contains analysis means 13, second analysis means 15 and computing means 17. The computing means 17 provides modifications to the graphical user interface contained on display screen 7 in response to changes in the position of object 3.

In a preferred embodiment of the present invention the object 3 is a person's head. Changing the position of the head by moving the head up and down or from side to side or backwards and forwards with respect to the screen in three dimensions as indicted by arrows 9 provides a change in the image viewed by the camera 5 which is analysed and used to change the view of the graphical user interface seen by the user. In addition movement of a persons head from side to side will be detected as a change in the direction of gaze of the person which will change the view of the graphical user interface seen by the person on screen.

In this way the system can be configured to give the user the illusion that they are looking through a window into a three dimensional world. This is achieved by using movement of the persons head or object to move the graphical scene of the graphical user interface in a way that corresponds to the changes a person would experience in the view that they were seeing were they looking through a window.

Figure 2 is a block diagram showing an example of the apparatus of the present invention and its functionality.

The arrangement shown generally by reference numeral 21 comprises a camera 23 which takes an image 25 of a scene containing an object which will typically be the user's head. The head position detector 27 comprises first and second analysis means which contain a face detector 29 which detects a human face within the image and locates the boundary values of the face, that is, the positions in the image where the face ends and the rest of the image begins.

When these values have been obtained for a plurality of images, software 31 transforms these images and in particular the changes in shape, size and position of the image face within the image to provide a measure of the change in three dimensional position of the face within the overall image. A number of camera parameters such as angle of view, focal length and resolution can be used to provide this three dimensional position information.

Alternatively, the change in size and position of the image face within the image can be determined from a single image, by referring to either a reference image to which all subsequent images are compared, or to a reference or "zero" position with associated reference or "zero" properties.

In a particular case envisaged by the applicant, the imaging means consists of a single camera and the object which is viewed is the face of a person, who looks at the camera. The first analysis means, which is a piece of software, determines the location and size of the face viewed in the camera image .

The parameters as determined by the first analysis means are given with respect to a reference. In one example this reference is the centre point of the camera image and the position of the left eye may be given as distances (number of pixels) in x and y directions from this centre point.

The second analysis means may then transform the information given by the first analysis means into an actual real world location of the face in front of the camera.

The output from the head detector module is information 33 on the real world position and real world gaze direction of the person. Once this has been calculated user set linkage values are used to map 35 these calculated real viewpoints on to a virtual world viewpoint with respect to the contents of the graphical user interface. In effect changes in these real world viewpoints are used to provide a change in the screen content of the graphical user interface. The virtual position and user virtual gaze direction information is then fed into a 3D world rendering module 37 which produces a 2D projection 39 which is then displayed on the screen 41.

Accordingly, in this example of the present invention the information displayed on the screen is a change in the viewable contents of the graphical user interface which is controlled by the change in position of the persons head as measured by the camera and image detection and analysis software.

Details of the process of the present invention will now be described with respect to figure 3.

Acquire Image from camera (s) 51

The present invention uses any camera which can input images to a computer while leaving enough computer resources for the modules outlined below. In this example, there should be sufficient computer resources to allow the computer and camera system process to capture at least 10 frames per second.

Detect face pattern in image 53

The image is processed by a face detector to detect the position and size of the biggest frontal face as it appears in the image. In normal circumstances this will also be the nearest face i.e. the user. The quality at which the system works will depend much on the accuracy at which the detector recovers the size and position of the face. The face detector is able to distinguish between face patterns and background features in a cluttered multicoloured environment. The detector must also be largely resistant to being affected by changes in the light, both of brightness and in the position of light sources.

Calculate the 3D position of the head in relation to the screen 55

Where the camera is placed on top of the screen the distance of the head from the camera can be calculated using the size of the face in the camera image.

In an ideal camera with no distortions the size of the face in the camera image measured in pixels is linearly proportional to the inverse of the distance of the face from the camera. The proportionality factor connecting these two quantities is defined by the focal length and the angle of view of the camera and the resolution of the camera image. The same factor defines the linear relationships between the position of the face in the camera image and the real position of the head in front of the camera. In a strongly distorting camera or in the case where the image is made up using several cameras these relationships are not linear anymore but can still be worked out given good enough estimates of the camera (s) parameters and the nature of the distortion.

The range of detection depends on the angle of view of the camera and the number of cameras used. An angle of view of around 50 degrees , if a single camera is used, produces satisfactory results .

Calculate real world gaze direction 57

The 2D impression given when a person looks at a 3D scene does not change when their head orientation and position is kept fixed and only the eyes are moved. This means a screen which is simulating a window into a 3D world does not need to change with the user's eye movement. Rather than the eye gaze direction, it is the user's head position that orients GUI view. The overall gaze direction is given as the vector from head position to the middle of the screen. Both positions are known with relation to the camera, as long as the camera is put in a defined place. Ideally just on top of the screen, equal distance from both sides.

Update Virtual Observer position and Virtual observer gaze direction in the Virtual world using the real world head position and interpolated real world head pose 59

In order to achieve the impression that the screen is a window into a 3D world, the virtual orientation and the virtual distance between the hypothesised virtual subject viewpoint and the virtual terrain must exactly coincide with real user viewpoint and onscreen image. So when the real user moves in relation to the computer screen image, his real position and hypothesised gaze direction must be mirrored exactly in the virtual world. This adjustment is 1:1 (if scaling calculations necessary to convert real world spatial units into virtual spatial units are not considered) . Project virtual 3D world onto screen using new virtual observer position and virtual observer gaze direction 61

Normally the projection is symmetrical in the sense that the observer's position is fixed in front of the middle of the screen at a defined distance, with the observer's gaze direction perpendicular to the screen surface. In the present invention, the user's position and angle between the gaze line and the screen surface varies. The parameters of a projection can be adjusted accordingly using 3D world rendering software packages allow such more complicated projections.

Using this projection the 3D world is rendered into the 2D virtual window. This image is then displayed on the real screen and the cycle shown in figure 3 starts again from the beginning.

Figure 4 shows a block diagram describing the method of operation of the present invention as incorporated in a hand held device. The hand held device may be, for example, a personal digital assistant (PDA) , a mobile phone or a hybrid device. The camera, display and computing elements are small and can be integrated into such a single hand held device.

This embodiment may take advantage of the trend in current mobile phones to provide a camera facing, and intended for recording images of, a user. Generally such cameras are located near the screen and so have a view of the user that is similar to that which would be provided at the display. This embodiment differs from the above in that in use, the user changes the position and orientation of the device rather than his own head position in order to change the content of the display. However, the user's head remains the anchor for the display contents.

A hand held device displaying a document comprises a camera 71 which captures images 73 containing the user's face, ideally at a rate of over 10 frames per second. The images 73 are passed to a fast face detector 75 which extracts the location and the size 77 of the user's face within the images 73.

Image calculations 79 are carried out using the location and size 77 of the user's face within the images 73 to calculate the position and orientation 81 of the hand held device relative to the user's face. This information may then be used to calculate 83 the position and size of a rectangle inside the area of the document. This "display frame" 85 defines the portion of the document that is visible on the hand held device display.

The display frame 85 moves left, right, up and down (or any combination of these movements) within the document responsive to the image calculations 79 detecting tilt left and right or tilting up and down (respectively) of the hand held device with respect to the user's face. The display frame 85 may also increase in size to include more of the document, if the distance between the user's face and the camera is increased (i.e. zooming out as the hand held device is moved away from the user) . Conversely the display frame 85 may reduce in size to include less of the document id the distance between the user's face and the camera is decreased (i.e. zooming in as the hand held device is moved toward the user's face) .

The portion of the document within the display frame 85 is then adjusted 87 such that it fits within the display size and resolution of the display. Typical adjustments include cropping, downsampling or upsampling, and aligning. The resulting image content 89 is then passed to the display which is refreshed 91 to show the image content 89. This procedure repeats 93 for as long as the document is being viewed on the screen or until some user interaction brings it to a halt.

The system described can thus be configured so as to appear to be a window into a virtual 3D world as described in relation to Figure 3 above. Any application which the above system would be suitable for could also be implemented in the hand held system.

Examples of 2D content that would also be suitable for this hand held implementation could be a web page, a page of a comic book, or a photograph. This would result in improved internet browsing, enhanced enjoyment of electronic comics on the move and improved image viewing facilities respectively. In these examples, the present invention solves the problem of how to present large amounts of information or large image content on the small screens of mobile devices.

This embodiment may also have applications in navigation, for example in route finding utilities or guided museum tours, or as windows into other virtual worlds e.g. interactive computer games. The hand held embodiment allows for a much improved degree of freedom insofar as it is more convenient to rotate or move a handheld device than for the user to displace his head. It is envisaged however that there may be incorporated a scaling factor such that minor movements of the user's head are translated into more significant movements of the content of the display.

In either of the abovementioned embodiments, it is also envisaged that movement of the user's head and movement of the display or imaging device can be processed in combination. For example, the hand held device may be moved to the left to bring part of an image outwith the display into the viewable area, and the user can simultaneously move his head toward the display in order to zoom into the image being brought into the display. Similarly the hand held device could be moved away from the user while he moves his head from side to side, up and down, to survey a wider area of the virtual 3D world.

Relative movements of the camera and/or the user's face may be used to provide a further enhanced interaction with a virtual 3D world. For example, movement of the user's head relative to the camera or display may cause appropriate movement of parts of the on-screen imagery. With a fixed viewpoint, movements or gestures may be translated into changes in direction, accelerations and control of other characteristics of objects within the virtual 3D world. One application in which the applicant envisages this mode of operation is in a 3D tennis simulation where movement of the user's face relative to the device (or of the device relative to the user's face) results in manipulation of an on-screen character such- ^• that it returns a serve from a simulated opponent .

A further application within the scope of the present invention is to control attributes of a mechanical or electrical device in response to movement of, for example, the user's face or hand. By way of example, movement back and forward, side to side, of the user's face or hand may be translated into movement back and forward, side to side, of a mechanical device. This may have applications in medicine where operations can be carried out remotely within patients, or in bomb disposal applications where it is necessary for the operator to be distant from a potentially explosive item.

Improvements and modifications may be incorporated herein without deviating from the scope of the invention. For example, the hand held device may be any device having a display and an imaging device. Additionally, the imaging device may be any such device capable of recording an image of an object, e.g. an infrared camera suitable for use in the dark.

Claims

CLAIMS :

1. An apparatus for controlling the viewable contents of a graphical user interface (GUI) on a display, the apparatus comprising: imaging means for obtaining at least one image of an object in the field of view of the imaging means; first analysis means for determining one or more parameters of the at least one image relative to a reference; second analysis means for calculating the position of the object with respect to the imaging means based upon the value of the one or more parameters of the at least one image; and computing means for changing the viewable contents of the GUI in response to the position of the object.

2. An apparatus for controlling a device in response to the movement of an object, the apparatus comprising: imaging means for obtaining at least one image of an object in the field of view of the imaging means; first analysis means for_ determining one or more parameters of the at least one image relative to a reference; second analysis means for calculating the position of the object with respect to the imaging means based upon the value of the one or more parameters of the at least one image; and computing means for providing a change in the device in response to the movement of the object.

3. The apparatus as claimed in Claim 1 or Claim 2, wherein the reference is an image of the object.

4. The apparatus as claimed in Claim 1 or Claim 2 , wherein the reference is a zero position relative to the imaging means .

5. The apparatus as claimed in any one of the preceding Claims, wherein the one or more parameters of the at least one image of the object include the position of the object.

6. The apparatus as claimed in any one of the preceding Claims, wherein the one or more parameters of the at least one image of the object include the size of the object.

7. The apparatus as claimed in Claim 1, wherein a reference point is defined on or near the display.

8. The apparatus as claimed in Claim 2, wherein a reference point is defined on or near the device.

9. The apparatus as claimed in Claim 7 or Claim 8, wherein the imaging means is mounted in a predetermined position with respect to the reference point.

10. The apparatus as claimed in Claim 9, wherein the imaging means is mounted in a fixed position with respect to the reference point.

11. The apparatus as claimed in Claim 9, wherein the imaging means is mounted in a movable position with respect to the reference point.

12. The apparatus as claimed in any one of the preceding Claims, wherein the imaging means captures successive images at a rate of at least 5 frames a second.

13. The apparatus as claimed in any of Claims 1 to 11, wherein the imaging means captures successive images at a rate of at least 10 frames a second.

14. The apparatus as claimed in any one of the preceding Claims, wherein the first analysis means detects the position of the object image in the imaging means field of view.

15. The apparatus as claimed in any one of the preceding Claims, wherein the first analysis means detects features of the object.

16. The apparatus as claimed in Claim 15, wherein the first analysis means detects shape features of the object.

17. The apparatus as claimed in any one of the preceding Claims, wherein the first analysis means detects the boundary of the object image and defines the position of the object image in terms of its boundary.

18. The apparatus as claimed in Claim 1, wherein the second analysis means calculates changes in orientation of the object to determine changes in the orientation of the object with respect to the GUI on the display.

19. The apparatus as claimed in Claim 1 or Claim 18, wherein the change in the viewable contents of the GUI provides a virtual viewpoint which is a view of the GUI which is alterable in response to changes in successive object images.

20. The apparatus as claimed in any one of Claim 1 and Claims 18 and 19, wherein the computing means changes the viewable contents in order to enlarge the viewable contents when the object moves towards the display.

21. The apparatus as claimed in any one of Claim 1 and Claims 18 to 20, wherein the computing means changes the viewable contents in order to enlarge the viewable contents when the display moves towards the object.

22. The apparatus as claimed in any one of Claim 1 and Claims 18 to 21, wherein the computing means changes the viewable contents in order to reduce the size of the viewable contents when the object moves away from the display.

23. The apparatus as claimed in any one of Claim 1 and Claims 18 to 22, wherein the computing means changes the viewable contents in order to reduce the size of the viewable contents when the display moves away from the object.

24. The apparatus as claimed in any one of Claim 1 and Claims 18 to 23, wherein the computing means changes the viewable contents in order to pan the viewable contents to the right when the object moves to the left relative to the display.

25. The apparatus as claimed in any one of Claim 1 and Claims 18 to 24, wherein the computing means changes the viewable contents in order to pan the viewable contents to the right when the screen moves to the right relative to the object.

26. The apparatus as claimed in any one of Claim 1 and Claims 18 to 25, wherein the computing means changes the viewable contents in order to pan the viewable contents to the left when the object moves to the right relative to the display.

27. The apparatus as claimed in any one of Claim 1 and Claims 18 to 26, wherein the computing means changes the viewable contents in order to pan the viewable contents to the left when the display moves to the left relative to the object.

28. The apparatus as claimed in Claim 2, wherein the device is a mechanical device.

29. The apparatus as claimed in Claim 2 or Claim 28, wherein the computing means moves the device when the object moves relative to the imaging means.

30. The apparatus as claimed in any one of the preceding Claims, wherein the second analysis means calculates a change in position of the object and provides a 2 dimensional (2D) to 3 dimensional (3D) transformation.

31. The apparatus as claimed in Claim 28, wherein the second analysis means uses one or more imaging means parameters .

32. The apparatus as claimed in Claim 31, wherein the one or more imaging means parameters are selected from the group comprising; the angle of view of the imaging means, the focal length of the imaging means, the resolution of the imaging means, and the distance between the imaging means and the object.

33. The apparatus as claimed in any one of the preceding Claims, wherein the second analysis means calculates the change in the number of pixels in successive object images to determine a 3D position of the object with respect to the imaging means.

34. A method for controlling the viewable contents of a graphical user interface (GUI) on a display, the method comprising the steps of: (a) obtaining at least one image of an object in the field of view of an imaging means; (b) determining one or more parameters of the at least one image relative to a reference; (c) calculating the position of the object with respect to the imaging means based upon the value of the one or more parameters of the at least one image; and (d) changing the viewable contents of the GUI in response to the position of the object.

35. A method for controlling a device in response to the movement of an object, the method comprising the steps of: (a) obtaining at least one image of the object in the field of view of an imaging means; (b) determining one or more parameters of the at least one image relative to a reference; (c) calculating the position of the object with respect to the imaging means based upon the value of the one or more parameters of the at least one image; and (d) changing the device in response to the position of the object.

36. The method as claimed in Claim 34 or Claim 35, wherein the step of determining one or more parameters of the at least one image comprises comparing the at least one image with an image of the object.

37. The method as claimed in Claim 34 or Claim 35, wherein the step of determining one or more parameters of the at least one image comprises comparing the at least one image with a zero position relative to the imaging means .

38. The method as claimed in any one of Claims 34 to 37, wherein the one or more parameters of the at least one image of the object include the position of the object.

39. The method as claimed in Claim any one of Claims 34 to 38, wherein the one or more parameters of the at least one image of the object include the size of the object.

40. The method as claimed in Claim 34, wherein a reference point is defined on or near the display.

41. The method as claimed in Claim 35, wherein a reference point is defined on or near the device.

42. The method as claimed in Claim 40 or Claim 41, wherein the step of obtaining at least one image of the object includes positioning the imaging means in a predetermined position with respect to the reference point.

43. The method as claimed in any one of Claims 40 to 42, wherein the imaging means is positioned at a fixed position with respect to the reference point.

44. The method as claimed in any one of Claims 40 to 42, wherein the imaging means is positioned at a moveable position with respect to the reference point.

45. The method as claimed in any one of Claims 34 to 44, wherein the step of obtaining at least one image of the object captures successive images at a rate of at least 5 frames a second.

46. The method as claimed in any one of Claims 34 to 45, wherein the step of obtaining at least one image of the object captures successive images at a rate of at least 10 frames a second.

47. The method as claimed in any one of Claims 34 to 46, wherein the step of determining one or more parameters of the at least one image includes detecting the position of the object in the imaging means field of view.

48. The method as claimed in any one of Claims 34 to 47, wherein the step of determining one or more parameters of the at least one image includes detecting features of the object.

49. The method as claimed in Claim 48, wherein the step of determining one or more parameters of the at least one image includes detecting shape features of the object.

50. The method as claimed in any one of Claims 34 to 49, wherein the step of determining one or more parameters of the at least one image includes detecting the boundary of the object image and definining the position of the object image in terms of its boundary.

51. The method as claimed in Claim 34, wherein calculating the position of the object includes calculating changes in orientation of the object to determine changes in the orientation of the object with respect to the GUI on the display.

52. The method as claimed in Claim 34 or Claim 51, wherein changing the viewable contents of the GUI includes providing a virtual viewpoint which is a view of the GUI that is alterable in response to changes in successive object images.

53. The method as claimed in any one of Claim 34 and Claims 51 and 52, wherein changing the viewable contents of the GUI includes enlarging the viewable contents when the object moves towards the display.

54. The method as claimed in any one of Claim 34 and Claims 51 to 53, wherein changing the viewable contents of the GUI includes enlarging the viewable contents when the display moves towards the object.

55. The method as claimed in any one of Claim 34 and Claims 51 to 54, wherein changing the viewable contents of the GUI includes reducing the size of the viewable contents when the object moves away from the display.

56. The method as claimed in any one of Claim 34 and Claims 51 to 55, wherein changing the viewable contents of the GUI includes reducing the size of the viewable contents when the display moves away from the object.

57. The method as claimed in any one of Claim 34 and Claims 51 to 56, wherein changing the viewable contents of the GUI includes panning the viewable contents to the right when the object moves to the left relative to the display.

58. The method as claimed in any one of Claim 34 and Claims 51 to 57, wherein changing the viewable contents of the GUI includes panning the viewable contents to the right when the display moves to the right relative to the object.

59. The method as claimed in any one of Claim 34 and Claims 51 to 58, wherein changing the viewable contents of the GUI includes panning the viewable contents to the left when the object moves to the right relative to the display.

60. The method as claimed in any one of Claim 34 and Claims 51 to 59, wherein changing the viewable contents of the GUI includes panning the viewable contents to the left when the display moves to the left relative to the object.

61. The method as claimed in Claim 35 wherein the device is a mechanical device.

62. The method as claimed in Claim 35 or Claim 61 wherein changing the device in response to the position of the object includes moving the device when the object moves relative to the imaging means.

63. The method as claimed in any one of Claims 34 to 62, wherein the step of calculating the position of the object includes calculating the change in position of the object and providing a 2D to 3D transformation.

64. The method as claimed in any one of Claims 34 to 63, wherein calculating the position of the object uses one or more imaging means parameters.

65. The method as claimed in Claim 64, wherein the one or more imaging means parameters are selected from the group comprising; the angle of view of the imaging means, the focal length of the imaging means, the resolution of the imaging means, and the distance between the camera and the object.

66. The method as claimed in any one of Claims 34 to 65, wherein calculating the position of the object includes calculating the change in the number of pixels in successive object images and determining a 3 dimensional position of the object with respect to the imaging means.