JP2003510864A - Method and system for time / motion compensation for a head mounted display - Google Patents

Abstract

SUMMARY A method and system for time and motion compensation for use in a head mounted display is disclosed. According to this method, a remote camera captures an image to be displayed on a head mounted display (HMD) 1 including a view window. The image and camera position data are transmitted to a system including the HMD and displayed to a person wearing the HMD. The HMD position is determined. The offset between the HMD position and the known position of the HMD is determined to be the same as the offset between the camera position and the known camera position. The image is offset with respect to the image view window based on the difference between the two determined offsets.

Description

Detailed Description of the Invention

FIELD OF THE INVENTION The present invention relates generally to telepresence systems, and more specifically to motion compensation in telepresence systems.

BACKGROUND OF THE INVENTION The field of remote control has advanced much since the days of seeing model airplanes flying under the control of portable controllers. Robotics and remote control of robots are creating a strong and pressing need for better remote control systems from a greater distance. Obviously, the ideal form of remote control would need to provide the operator with all the sensation of operating a remote robot without inherent danger or movement. A telepresence system is used to achieve this.

A telepresence system is a sensory feedback system that allows sensing and monitoring of remote systems. Typical telepresence sensors are cameras and head-mounted displays. This system provides the operator with visual feedback from a remote location. For example, a telepresence system for an automobile is equipped with a camera on the windshield. Control of the vehicle is provided by an actuator that automatically operates the vehicle. The operator is provided with a copy of the car cabin. The windshield is replaced by a display and control is linked via communication with actuators in the vehicle. When you turn the steering wheel inside the car's cabin, the steering wheel turns the vehicle. Similarly, the camera captures an image of the front of the car, which is displayed on the display in the car's cabin.

Currently, there is a tendency to use head-mounted displays (HMDs) to provide visual feedback. A head-worn display (head-mounted display) is one or two small displays worn by the user on the head. Advantageously, an HMD with two displays provides a stereo image, allowing the user to perceive the depth of field. Alternatively, such an HMD provides two identical images on each display. Unfortunately, head-mounted displays only provide the user with information that is approximately in front of the user. Therefore, when the user rotates his / her head, the image seen and the expected image are different. Therefore, the camera is mounted on a mechanism that moves according to the detected movement of the HMD. Thus, the image in front of the user follows the position of the user's head.

In general, the purpose of a telepresence system is to provide a visual sensation at the location of the robot and at the same time to provide a control system for controlling the robot. Thus, the telepresence system aims to provide feedback suitable for different situations.

Unfortunately, the camera does not move in perfect synchronization with the HMD, so the image does not exactly match the user's expectations while the head is moving. This discrepancy can result in disorientation of the operator and nausea.

US Pat. No. 5,579, issued on Nov. 26, 1996 in the name of Tabata.
The disclosure of US Pat. No. 026 and the disclosure of US Pat. No. 5,917,460, issued June 29, 1999 in the name of Kodama, focus on, for example, virtual reality and image displays used in games. Described herein is a head-worn display in which the position of the projected image can be displaced in response to a control unit or in response to a rotating movement of the operator's head. From the user's perspective, the essence of the head-tracking run is that by manipulating it in the opposite direction of the head's movement, the image can remain at a nearly stationary position in space while the head is moving. is there. Importantly, these patents do not address visual telepresence using slave cameras. In a slave camera implementation, the camera should follow the HMD's motion, so the image is always headed, and compensation for the HMD's motion is unnecessary.

Further, US Pat. No. 5,579,026 relates to displaying a simulated planar image, such as a simulation of a television screen in a virtual space in front of a user, which wears an HMD. Providing a fixed frame for those who are Thus, an image in any direction is simulated and formed as needed. Unfortunately, telepresence systems often do not have video data available for a particular direction of view. This adds significant complexity to the system, so the prior art for displaying video data is not really applicable and the person skilled in the art does not refer to such.

US Pat. No. 5,917,4 issued Jun. 29, 1999 in the name of Kodama
In No. 60, a system that deals with the movement of the HMD in three axes (up / down, left / right, front / rear) is provided. Movement is linear in appearance and is coordinated via a mechanical mechanism. Since the display is moved in response to the detected head movement, the subject remains in a substantially stationary position from the user's field of view.

This is not suitable for use in telepresence where the camera tracks HMD movement. One of ordinary skill in the art would not be able to draw a telepresence system in which the camera rotates in response to head movement without the aid of wisdom to maintain a visual reference when the head rotates. Let's do it. Of course, the harder the problem, the harder the solution.

In telepresence systems, for example, the delay between capturing an image of the camera and head movement is often indeterminate. Trying to solve this problem by running the system of the above patent does not work. The solution is not trivial as there are unknown delays resulting from camera response times and communication delays.

US Pat. No. 5,933,125 discloses the use of head motion estimation to pre-compensate for expected delays in the generation of virtual images in a nominally simulated environment. There is. By this means, the time delay in the generation of the image is compensated by shifting the scene to provide a stable reference visual frame. This method is applicable for short delays and small movements where the head tracking information can be used to predict the next head position with reasonable accuracy. This patent discloses 100 msec as a normal value. Effective prediction of head movements is aided by comprehensive information about head movements, including angular velocity and acceleration of the head. For slight head movements, the error introduced is small. These typically occur in short time periods. This disclosed example relies on knowledge of the time delay, which is considered to be nominally constant. Unfortunately, when the time delay is large and the head can move a lot, the error in the prediction algorithm becomes unknown and the system fails.

Further, US Pat. No. 5,933,125 cannot compensate for unexpected image movements, only image movements that occur in response to correct operator head movements. Nor does this patent mention a visual telepresence system using remote slave cameras.

It would be highly advantageous to provide a system that does not rely on any form of compensation prediction and that will work with varying delays between image capture and image display.

OBJECTS OF THE INVENTION The present invention seeks to overcome these and other drawbacks of the prior art by providing a method of compensating for the time delay between head movement and camera movement in a telepresence system. To aim.

SUMMARY OF THE INVENTION The present invention is a method and method for providing a person wearing a HMD with a stable visual reference frame, even in the presence of time delays or undesired motion in a visual capture / visual display system. Regarding the device.

According to the invention, in order to partially eliminate the loss of directional sense caused by the time delay in the movement of the camera when the head is moved, the head is re-synchronized with the position of the HMD until the position of the camera is resynchronized. The image displayed on the display of the component mounted display (HMD) is offset with respect to the visual field of the HMD. Offsetting the image results in areas of the display where image information is not available. Areas of these displays are provided with padding data in the form of a set of predetermined features that provide plain shading or visual cues. When the altered image again overlaps the display, no vegetation is needed.

According to the present invention, there is provided a method of motion compensation for a head mounted display (head mounted display). The method includes the following steps. Providing an image from an image capture device to a head mounted display including a monitor with a field of view; providing camera position data associated with the image; providing head position data; Adjusting the location of the image relative to the field of view of the monitor according to the camera position data and the head position data;
Displaying the portion of the image that remains in the field of view at the adjusted location.

The position data typically includes at least one of orientation data and location data. Place data is also referred to as displacement data. Typically, parts of the field of view without image data are filled with a predetermined fill. If there is no image data displayed in the field of view, then the entire field of view is filled with a predetermined fill.

For example, the image is adjusted in the next step. These steps determine the offset between the position of the head-mounted display and the position of the camera; the image is adjusted to be an amount equal to the offset between the position of the head-mounted display and the position of the camera. This is the step of offsetting.

Advantageously, such a system is not limited by the accuracy of the prediction process, nor by the time delay between image capture and image display. Such systems are responsive and use the sensed information about the position of the HMD and the position of the camera to formulate changes in the captured image. The present invention allows the required head position information and camera position information to be sensed at different times to compensate for the delay between one and another sensing. There is no limit on the time delay for compensation to be possible.

Further advantageously, the present invention does not require knowledge of the time delay in the system and will work correctly with non-constant time delays. It is not necessary to measure the time delay and it is not used in determining the change in the image.

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a telepresence system operating over a long distance such that there is a significant time delay between the movement of the head and the display of an image of the current head orientation. explain. However, the present invention is equally applicable when the camera drive provides a poor response rate and does not provide a comfortable view of the image with normal head movements. It is likewise applicable in situations where the camera may have undesired, unmodeled movement, such as when the camera is mounted on a moving platform.

Referring to FIG. 1, a simple block diagram of a telepresence system is shown. An operator 5 wears a head mounted display (HMD) 1 including a triaxial head tracking device 3. The HMD 1 is connected to the first computer 7, and as shown in FIG. 2, the HMD 1 has a pitch angle 21, a yaw angle 22,
And the HMD value for the position of the HMD in the form of a roll angle 23. Of course, since the HMD 1 is worn by the operator 5, these HMD values are related to the position of the operator 5's head. These values are preferably provided to the first computer 7 at more than 100 intervals per second, but other intervals are also acceptable. The HMD value is converted by the first computer 7 into a control value for controlling the position of the camera 11. The control value is transmitted to the mechanism 13 and the camera is oriented so as to influence the direction of the camera. As seen in FIG. 2, this mechanism controls pitch 24, yaw 25, and roll 26 of camera 11. In response to the control value, the position of the camera moves according to the movement of HMD1.

When the camera pointing mechanism 13 is physically coupled to the first computer 7, the camera 11 begins to move when a motion of the HMD is detected. Camera movement and HMD
The delay between movements of is determined by very small communication delay, minimal processing delay, and varying pointing mechanism behavior. These delays often result in an image provided by the camera 11 being stationary or moving much slower than the HMD's movements when the HMD 1 is moving. Of course, the operator's mind expects movement in a visual framework, so still or slow motion in the image is confusing, often resulting in nausea and loss of direction. become.

This problem arises when used to control a system in space from the ground, such as
It becomes more noticeable when the communication delay time is large. If the delay is on the order of seconds, H
The loss of sense of direction for the operator during movement of the MD increases. Significantly, loss of orientation causes operator fatigue, resulting in limited operator system use or limited system use during the day.

Referring back to FIG. 1, the camera 11 is constantly acquiring images at the video data capture rate. Each image is transmitted to the first computer, processed if necessary, and supplied to the HMD 1. According to the invention, the remote system also provides the location information of the camera and each associated image to the first computer 7. Thus, each frame received by the first computer 7 has camera position information associated with the frame. The camera position information is preferably position relative to a known direction. Alternatively, the camera position information is stored in the first computer 7
Is transformed into position information for the known camera direction and placed in a coordinate space similar to the coordinate space of the HMD 1.

The HMD position value is used to determine the current HMD orientation in a coordinate space similar to that of the camera 11. Thus, the offset between the camera direction and the HMD direction can be determined. Since the operator 5 is wearing the HMD 1, the HMD 1
The direction of is directly correlated to the position of the operator's 5 head. Of course, the direct correlation is related to the sensed position data and, in use, is generally a near direct correlation due to the refresh rate of the HMD position sensor. The offset between the camera direction and the HMD direction is related to the delay between the local system and the remote system.

Therefore, when a non-zero offset is determined, the first computer offsets the image provided by the camera with respect to the field of view of the HMD and compensates for the determined offset. Referring to FIGS. 3a, 3b, 3c, 3d, 3e, some examples of image displays are shown. In Figure 3a, an image is shown for zero offset between the camera direction and the HMD direction. This is a steady state of the feedback system, as the HMD 1 is facing the same direction as the camera 11 and the image displayed on the display in the HMD 1 is the same as the image captured by the camera. When the direction of the HMD is pointing below the position of the camera, the image is offset in the vertical direction as shown in Figure 3b. When the camera direction is horizontally offset, the image is horizontally offset as shown in Figure 3c. In Figures 3d and 3e, the image is rotated and offset with respect to the field of view because the camera orientation is rotated and offset with respect to the HMD orientation.

While the immediate corrections as illustrated in FIGS. 4a, 4b, 4c, 4d appear simple, the steady state nature of the system is that the field of view of the image and the field of view of the display are constantly changing. Need. Therefore, it is necessary to compare between the two dynamic sets of sensed position data.

Referring to FIGS. 4a, 4b, 4c and 4d, there is shown a field of view for the HMD 1 when the operator turns his head to the left. In the first steady state (before turning the head), the exact image captured by the camera 11 is shown in the display of the HMD 1 of Figure 4a. When the head is rotated, the operator 5 "expects" to move the image to the right because the image is in the field of view, not part of the operator 5.
To do. This expectation may be conscious or unconscious. Imagine that the image is still when the HMD moves, but it is clear that an individual loses the sense of direction as he or she grabs clues from the field of view while the head is moving.

In order to provide the operator 5 with the movement of the object within the “expected” image, the image is offset approximately at the same directional difference between the HMD 1 and the camera 11. For example, in the image of FIG. 4b, the rotation of the head causes the lighthouse to shift out of view and disappear. When the HMD is rotated by α degrees, the operator expects a stationary object in the field of view, such as a lighthouse, to shift by α degrees in the field of view. This is important to keep the operators cozy in their personal vision system (their eyes and mind). At the same time, the camera begins to move so that the orientation of the camera itself matches the orientation of the HMD. Therefore, more of the image in the operator's field of view is available from the camera 11, as shown in the image of FIG. 4c. When the direction of the camera “catch up” with the direction of the HMD, the view of the camera and the view of the HMD further overlap. When the camera 11 "has finished catching up" with the HMD's field of view,
Again, as shown in the image of FIG. 4d, it shows the full image captured by the camera.

Referring to FIG. 5, another embodiment of the invention for use on a network is shown. Here, for example, two computers 7 and 15 communicate via a network (or a plurality of networks) 17. The first computer 7 is an HMD
1 is included as a peripheral device. The second computer 15 includes a camera 11 and a camera pointing mechanism 13 as peripheral devices. The processing is now performed in either the first computer 7 or the second computer 15, but there may be network delays that cause image offsets and result in the operator 5 being disoriented. Therefore, the image processing is preferably performed in the first computer 7. Of course, it is important to perform image processing on the first computer when network latency is known to be large.

Referring to FIG. 6, a simple flow diagram of a method of practicing the present invention is shown.
The image is captured by the camera 11. The sensor captures position data in the form of camera orientation values for pitch, roll, and yaw. The position data is preferably captured simultaneously with the image. Alternatively, the position data is captured at about the same time, but offset by a limited amount, either immediately after or just prior to image capture. The position data is then associated with the image data. A simple way to associate the data is to encode the position data with the image data either as header information or trailer information. Of course, the image data and the position data can also be identified by associated identifiers such as image frame numbers. Alternatively, the two data are sent in parallel in a synchronous environment.

The image data and the position data are then transmitted to the first computer 7. When the image data and the position data are received, the first computer 7 prepares for processing. The position data of the HMD 1 is then acquired by the first computer 7 and used to change the image according to the invention. The changed image is provided to the display and displayed to the operator 5 on the display.
Since the HMD position data is collected just before it is needed, the delay between capturing the HMD position data and displaying the modified image is very small and the operator has little or no loss of orientation. .

At the same time, position data is provided at intervals to the mechanism 13, which moves the camera 11 according to the received position data and the current direction of the camera 11.

Typically, the steps of modifying the image data include the following steps, some of which are performed in advance. The correlation between angular movement and display or image pixels is determined so that the offset of α degrees results in an image movement of N pixels in a first direction and M pixels in a second alternative direction. Changes in image rotation based on rotation are also determined. Preferably, the changes are simple enough to be able to provide fast image processing. In other words, a small image processing delay is almost the same as the delay in displaying the data, and is therefore acceptable.

When the image data is received, the image data is stored in memory for high speed processing. The HMD position data is acquired and compared with the camera position data. The difference is used to perform the change, and if there is an HMD motion that mechanism 13 has not yet taken into account, the image position is corrected for that HMD motion. Also, when the position sensor of the camera, such as an inertial position sensor, is independent of the mechanism 13, this method also corrects unconscious movement of the camera 11.

In the above embodiments, a general purpose processor is used to transform the image. Alternate embodiments use hardware implemented changes. Hardware implementation is hard to modify, but has a great effect on performance. A parallel hardware change processor can change an image in a fraction of the time needed to perform a software change of the image.

Referring to FIG. 7, a satellite-based telepresence system is shown.
Here, a delay of the order of seconds between head movement and image display results. Moreover, delays are not always predictable. Here, the HMD 101 is shown attached to the head of the operator 5. The HMD comprises a head tracking device 103 that senses position data for the HMD. The HMD also provides the display data to the HMD,
Coupled to a computer 107 that provides HMD location data on communication link 108. The communication link 108 uplinks the HMD position data to the satellite 117,
The D position data is transmitted from satellite 117 to transceiver 116. Computer 115 in communication with the transceiver provides the data to gimbal 113 and corrects the position of camera 111 according to the data. The camera 111 has the opposite communication path, namely computer 115, transceiver 116, satellite 117, communication link 108.
Captures an image provided by the communication path of the computer 107. Optionally, a different return path is used. Here, the image data is HMD1
Processed for display in 01. Along with the image, camera position data sensed by sensor 114 is also provided. The camera position data is associated with the image (s) captured at about the same time the camera 111 was at the sensed position.

Computer 107 uses the camera position data and the image with the data received from head tracking device 103 to transform the image in accordance with the present invention. As can be seen, the delay between the HMD movement and the camera movement can be measured in seconds. The delay between capturing the camera image and receiving the image at computer 107 can also be measured in seconds. Therefore, without the application of the present invention, a significant loss of orientation for the user would result.

Referring to FIG. 8, an image captured by the camera 111 is shown. The image is displayed as captured when the direction of the HMD and the direction of the camera are aligned, and when the direction of the camera when capturing the image and the direction of the HMD when displaying are aligned. When the directions are offset from one to another, the image is shifted in the operator's field of view, as shown in FIG. Since there is no image data beyond the field of view of the camera, the remaining display area is shaded with an intermediate color such as gray. Alternatively, the remaining display area is shaded to provide a reference point, further limiting the loss of orientation of the operator 105. As a further alternative, the part of the field of view for which no image data is available remains blank. The blank areas are typically black so as not to distract the operator. Referring to FIG.
When the orientation of the camera 111 "catch up" in the direction of the HMD, the HMD's field of view again displays the full image captured by the camera.

Alternatively, the camera captures a larger image area than can be displayed and only a portion of the image is displayed. This method increases bandwidth requirements and is often considered less desirable as additional data is often not displayed for no reason.

Advantageously, the inaccuracy of the mechanism 13, the delay caused by the communication, the mechanism when carried out with a position indicator independent of each of the HMD 1 and the camera 11 and independent of the mechanism 13 for moving the camera. All types of motion are compensated, including delays caused by 13, processing delays, fine operator motion, and so on.

When processing is done locally on the HMD or on a computer at the same location as the HMD with minimal delay between them, a time delay error associated only with the processing and the delay reading the HMD sensor data. Within each image is exactly aligned on the display.

Therefore, discontinuous scene changes are transformed into smooth transitions according to the expected visual result.

It is also within the scope of the invention to process the image data before displaying it to determine features or locations in the image data that are highlighted or shown in the displayed image. is there. For example, the contrast can be improved for generally bright or dark images. Also, features can be identified and labeled or highlighted. Alternatively, the image or another image is superimposed on the displayed image without processing the image.

Alternatively, the control value is determined in a pointing mechanism that points the camera rather than the first computer. In such an embodiment, the HMD position data is transmitted to the remote system to determine and initiate camera movements related to HMD movements.

The embodiments described above compensate for directional movement about any of the three axes of rotation. Alternatively, the invention compensates for linear movement of movement along the axis. As a further alternative, the present invention compensates for linear motion and motion about any axis of rotation. Movement and direction are both forms of position, and data relating to one or both is referred to herein as position data.

The embodiments described above do not correct the image for field distortion. Compensating for field distortions is possible within the concept of time / motion compensation according to the present invention, but due to the varying depth of field of view of the scene being observed, it is generally not suitable for use with a single camera. Not suitable. Compensating for field distortions would require capturing depth data using range sensors or a three-dimensional vision system.

Although the above embodiments have been described with reference to physical or wireless communication links between different components, both are clearly useful within the invention, as far as practicable. Although the HMD has been described as a peripheral device for a computer, the HMD may include an internal processor and function as a stand-alone device.

According to another embodiment of the invention, regions in the field of view that do not correspond to the locations of the displayed images are filled with current image data associated with previously captured images for those locations. Preferably, any previously captured image is not highlighted in the field of view to avoid operator confusion with "old" image data. For example, each image received from the camera is buffered with associated position data. When some area in the field of view is not occupied by image data, the processor determines another image with image data for those locations in the field of view, which location is associated with the previously captured image. It is determined according to the change performed based on the obtained camera position data and the current HMD position data.
The image data is then displayed as "transparent" at the defined location (s). For example, image data can be displayed with low contrast, which almost looks like ghost. Alternatively, lighten the colors to make the image data look more like a ghost. Still alternatively, the image is displayed as if it were current image data.

[0053]   The descriptions above are intended to be illustrative, not limiting.

[Brief description of drawings]

  The present invention will be described with reference to the following drawings.

FIG. 1 is a simplified block diagram of a system incorporating an HMD coupled to a first computer and in communication with a remote camera.

[Fig. 2]   FIG. 6 is a simplified diagram showing axes of motion of the associated system.

FIG. 3a   FIG. 6 is a simplified diagram showing a simulated scene of an image appearing within the field of view of an HMD.

3b is a simplified diagram showing a simulated scene of a portion of the image of FIG. 3a vertically offset within the field of view of the HMD in response to the user's head moving downwards.

FIG. 3c is a simplified diagram showing a simulated scene of a portion of the image of FIG. 3a horizontally offset within the field of view of the HMD in response to the user's head moving sideways.

FIG. 3d is a simplified diagram showing a simulated scene of a portion of the image of FIG. 3a tilted within the field of view of the HMD in response to the user tilting his or her head.

FIG. 3e shows the HM in response to a combination of sideways horizontal and tilted movements of the user's head.
3b is a simplified diagram showing a scene simulating a portion of the image of FIG. 3a, vertically and horizontally offset and tilted within the field of view of D. FIG.

Figure 4a   FIG. 6 is a simplified diagram showing a simulated scene of an image appearing in the field of view of an HMD.

4b is a simplified diagram showing a simulated scene of a portion of the image of FIG. 4a horizontally offset in the field of view of the HMD in response to a lateral movement of the user's head.

FIG. 4c: When the camera movement is partially caught up with the sideways movement of the user's head,
An image seen in the field of view of the HMD, including a portion of the image of FIG.
FIG. 6 is a simplified diagram showing a scene simulating additional image data displayed within the field of view of the MD.

4d after the movement of the camera has completely caught up with the sideways movement of the user's head, FIG.
FIG. 3 is a simplified diagram showing a scene simulating an image visible in the field of view of the HMD including a portion of the image of FIG. 1 and additional image data captured and displayed in the field of view of the HMD.

FIG. 5 is a simplified block diagram of a system incorporating an HMD that is coupled to a first computer and communicates over a network with a second computer coupled to a remote camera.

[Figure 6]   3 is a simple flow diagram of the method according to the invention.

FIG. 7 is a simplified block diagram of a telepresence system communicating via a satellite communication link.

[Figure 8]   The image is as captured by the remote camera.

[Figure 9]   9 is the same image as that of FIG. 8 changed in the endless image display space.

FIG. 10 is the same image as that of FIG. 9 displayed on a display having a finite image display space.

Claims (22)

[Claims] 1. Providing an image captured from an image capture device to a head mounted display including a display with a field of view; camera position data related to the position of the camera and associated with the captured image. Providing HMD position data related to the position of the head-mounted display; according to the camera position data and the HMD position data, a display position of a stationary object in an image with respect to a field of view. Changing the image so as to change it; and displaying a portion of the image that remains in the field of view at the display location, the method for compensating motion for a head-worn display. 2. The HMD position data is data related to the current position of the head mounted display and the camera position data is data related to the position of the camera when the associated image was captured. Changing the image, determining an offset between the HMD position and the camera position; offsetting the image to an amount proportional to the offset between the HMD position and the camera position The method of claim 1, further comprising: 3. The method according to claim 2, wherein a portion of the visual field for which image data is not available is filled with a predetermined vegetation. 4. The method of claim 3, wherein the predetermined vegetation has the characteristic of maintaining the orientation of the person wearing the head mounted display. 5. The method of claim 1, wherein the captured image is larger than the image required to fill the field of view of the head mounted display, and only a portion of the captured image is displayed. 6. The method of claim 1, wherein the system comprises two head mounted displays and two cameras. 7. The method of claim 1, wherein the system comprises an independent position sensor for sensing camera position and providing camera position data. 8. The system senses the position of the head-mounted display and determines its H
8. The method of claim 7, comprising an independent position sensor that provides MD position data. 9. The system senses the position of a head-mounted display and determines its H
The method of claim 1, comprising an independent position sensor that provides MD position data. 10. The system comprises a mechanism for moving a camera; means for transmitting HMD position data to the system in communication with the mechanism for moving the camera, the mechanism for moving the camera responsive to changes in the HMD position data. The method of claim 1, wherein the camera is moved by moving the camera. 11. The method of claim 1, wherein the HMD position data comprises orientation data. 12. The method of claim 11, wherein the camera position data comprises orientation data. 13. The method of claim 12, wherein the camera position data comprises displacement movement data. 14. The method of claim 1, wherein the HMD position data comprises displacement movement data. 15. The method of claim 14, wherein the camera position data comprises displacement movement data. 16. The method of claim 15, wherein the camera position data comprises orientation data. 17. A step of altering the image to reduce distortion of the field of view,
The method of claim 1, wherein the camera comprises a range sensor. 18. A head-mounted display that displays the portion of the image that remains in the field of view at the displayed position; an image capture device that provides the captured image to a head-mounted display that includes a monitor with a field of view. A sensor for providing camera position data related to the position of the camera and associated with the captured image; a sensor for providing HMD position data related to the position of the head mounted display; A processor for changing the image to change the displayed position of a stationary object in the image with respect to the field of view according to the HMD position data; 19. The image capture device comprises a range sensor.
A motion compensation device for the head mounted display described. 20. The motion compensating apparatus for a head mounted display of claim 19, wherein the range sensor comprises a stereo vision system. 21. When a part of the field of view for which image data is not available is detected, when the part is changed depending on the associated camera position data and HMD position data, it is displayed in that part. 3. The method of claim 2 including the steps of determining previously captured images; and displaying the altered previously captured image data in those portions of the field of view. 22. The method of claim 21, wherein the step of displaying the modified previously captured image data comprises the step of not enhancing the modified previously captured image data.