JPH08116553A - Stereoscopic vision system - Google Patents

Abstract

PURPOSE: To obtain a natural stereoscopic image by setting convergence of a 3-dimension camera and an HMD depending on a distance up to an image pickup object so as to extend a range of the image pickup distance possible for stereoscopic vision in the stereoscopic vision system comprising the 3-dimension camera and the HMD. CONSTITUTION: A convergence setting section 154 changing the direction of the left/right cameras 101L, 101R is provided in a 3-dimension camera 1, and a convergence setting section 241 changing the direction of left/right virtual image projectors 201L, 201R is provided in an HMD 2, a distance up to an object is detected based on the distance between left and right images of the 3-dimension camera and the convergence setting section of the 3-dimension camera and the HMD is controlled based on a detected object distance. Furthermore, the convergence setting section 154 of the 3-dimension camera sets focus linked with the convergence setting and a diopter is set linked with the convergence setting by the convergence setting section 241 of the HMD.

Description

Detailed Description of the Invention

[0001]

FIELD OF THE INVENTION The present invention relates to a three-dimensional camera and an HMD.
The present invention relates to a stereoscopic vision system consisting of, and particularly to setting of congestion of the stereoscopic vision system.

[0002]

2. Description of the Related Art Conventionally, there is known a stereoscopic system for observing an image taken by a three-dimensional camera with a three-dimensional observation device. The three-dimensional camera captures an object with two left and right cameras and provides left and right images with parallax. The three-dimensional observation device presents a stereoscopic image of the target by displaying the provided left and right images and guiding each to the left and right eyes.

In recent years, as a three-dimensional observation device, an HMD (Head Mounted Display) mounted on the head for use has been remarkably popular. The HMD has a pair of left and right virtual image projection devices including a display device and an eyepiece lens, and is configured to guide the image light of the display device to the eye through the eyepiece lens to give a virtual image of the display image to the eye. Since the HMD wearer is presented with a parallax image in front of his / her eyes, he / she can easily observe a stereoscopic image.

In order to stereoscopically observe an object with such a stereoscopic system, it is premised that the object is photographed by both the left and right cameras. However, in the three-dimensional camera with the left and right cameras fixed, the shooting range commonly taken by the left and right cameras is fixed, so that the imageable range is limited depending on the distance to the observation target. When the target is sufficiently far from the 3D camera, it is possible to shoot with both left and right cameras. However, as the target gets closer, the range that can be captured with both cameras decreases, and when it comes to a close range, one camera A situation occurs in which the other camera cannot shoot even if it can.

As a method of expanding the distance range in which stereoscopic viewing is possible, it is known to variably set the mutual distance between the left and right cameras. For example, in Japanese Examined Patent Publication No. 5-52116, in a three-dimensional camera having a taking lens and an image sensor on the left and right, the left and right image sensors are arranged so as to be displaceable in the left and right directions. In the three-dimensional camera with this configuration, when the distance to the image capturing target is short, the interval between the image capturing devices is set wide so that light from the target is imaged on the left and right image capturing devices by the left and right image capturing lenses.

Further, the above-mentioned technique discloses a configuration in which a video display element is arranged so as to be displaceable in the left-right direction in a three-dimensional observation apparatus having a video display element and an eyepiece optical system on the left and right. When the image pickup device is widened and the image is taken, the distance between the left and right image display devices is set to be narrow, so that the congestion of the observation device can be set corresponding to the congestion of the three-dimensional camera. In the three-dimensional camera and the three-dimensional observation device having such a configuration, the distance to the observation object capable of capturing the stereoscopically visible left and right images is increased, and the congestion at the time of photographing and the congestion at the time of observation are set to substantially match. Is possible.

However, Japanese Patent Publication No. 5-52116 does not disclose a specific mechanism for setting the congestion of the three-dimensional camera and the three-dimensional observation device, or a specific method for making the congestion correspond to the distance to the observation target. . Further, there is no mention of a method for detecting the deviation between the convergence of the camera and the convergence of the observing device and for making the both coincide with each other.

[0008]

In order to set the congestion of the three-dimensional camera according to the distance to the object to be photographed, the distance information to the object to be photographed or the congestion of the camera set at that time is the photographing distance. We need information on how far we are from congestion that is suitable for.

Further, in a stereoscopic system for observing an image taken by a three-dimensional camera with a three-dimensional observation device, an observer usually observes an image at a place away from a shooting site where the three-dimensional camera is installed. It will be. Therefore, in order to set the congestion of the observation device, the information about the congestion setting of the three-dimensional camera or the distance information to the imaging target is required.

Further, it is desirable that the congestion setting of the three-dimensional camera and the congestion setting of the three-dimensional observation apparatus be performed in conjunction with each other so that the congestion at the time of photographing and the congestion at the time of observation always match. With the stereoscopic system configured as described above, it is possible to always observe a stereoscopic image having a natural stereoscopic effect.

In order to perform sharp photography with a three-dimensional camera, it is necessary for the left and right cameras to be properly focused according to the distance to the subject. It is operationally preferable that this focus setting is made in conjunction with the congestion setting of the camera. On the other hand, in an observation device using a virtual image projection device, it is important for natural stereoscopic image observation that the distance from the eye to the virtual image is appropriately set according to the shooting distance. It is desirable that the setting of the distance from the eye to the virtual image, that is, the diopter setting should be performed consistently with the convergence setting of the observation device.

The present invention, in a stereoscopic system including a three-dimensional camera and an HMD, has a wide range of a stereoscopically viewable photographing distance with a simple structure and is suitable for both the three-dimensional camera and the HMD depending on the distance to the photographing object. It is an object of the present invention to provide a stereoscopic system in which the congestion setting is performed. Further, another object of the present invention is to provide a stereoscopic system that appropriately sets the focus of the three-dimensional camera and the diopter of the HMD according to the shooting distance, and reproduces a sharp and natural stereoscopic image.

[0013]

In order to achieve the above object, according to the present invention, a three-dimensional camera having a pair of left and right cameras equipped with a photographing lens and an image pickup device for receiving light transmitted through the photographing lens, and an image display. The left and right images of the HMD, which are the left and right images of the object captured by the left and right cameras of the three-dimensional camera, are composed of an HMD having a pair of left and right virtual image projection devices equipped with an eyepiece optical system that guides the image light of the image display element In a stereoscopic system that displays a virtual image of left and right display images on the left and right eyes by displaying on a display element, changing the direction of the left camera and the right camera
Camera congestion setting means for setting the congestion of the three-dimensional camera, HMD congestion setting means for changing the orientation of the left virtual image projection device and the right virtual image projection device to perform the congestion setting of the HMD, and camera congestion for giving a driving force to the camera congestion setting means The driving means, the HMD convergence driving means for giving a driving force to the HMD convergence setting means, the distance detecting means for detecting the distance from the left and right images of the photographing object received by the left and right imaging elements to the photographing object, and the distance detecting means. Based on the detected distance to the observed object,
Control means for controlling the camera convergence drive means so that light from the object to be imaged is imaged substantially at the center of the image sensor, and for controlling the HMD convergence drive means so that the congestion of the HMD matches the congestion of the three-dimensional camera. It is configured to have.

Furthermore, the focus setting of the three-dimensional camera is performed by changing the distance between the photographing lenses of the left and right cameras and the image pickup device in conjunction with the convergence setting by the camera convergence setting means.

Further, the diopter setting of the HMD is performed by changing the distance between the image display elements of the left and right virtual image projection devices and the eyepiece optical system in conjunction with the convergence setting by the HMD convergence setting means.

[0016]

As described above, the three-dimensional camera having the left and right cameras provided with the taking lens and the image pickup device and the HMD having the left and right virtual image projection device provided with the image display device and the eyepiece optical system are provided. A stereoscopic system for displaying virtual images on the left and right image display elements and giving respective virtual images to the left and right eyes includes camera convergence setting means, HMD convergence setting means, camera convergence driving means, HMD congestion driving means, distance detection means and control means. With such a configuration, images with parallax on the left and right are captured by the three-dimensional camera, and the left and right images are given to the left and right eyes by the HMD. Further, at the time of shooting by the three-dimensional camera, the distance to the shooting target is detected by the distance detecting means.

In the three-dimensional camera, the camera convergence drive means is controlled based on the detected distance to set the congestion. As a result, the object to be imaged forms an image at substantially the center of each of the left and right imaging elements. As a result, the object to be photographed is displayed substantially at the center of each of the left and right display elements of the HMD. Also, 3
The HMD congestion drive means is also controlled along with the control of the congestion setting of the three-dimensional camera, and the congestion of the HMD matches the congestion of the three-dimensional camera.

In the configuration in which the left and right cameras are set in focus in conjunction with the congestion setting of the three-dimensional camera, in addition to the image formation in the approximate center of the image sensor due to the congestion setting, the light from the object is sharpened to the image sensor. It is possible to set the focus according to the shooting distance so that the image is formed at.

Further, in the configuration in which the diopter setting of the left and right virtual image display devices is performed in conjunction with the congestion setting of the HMD, it is possible to set the diopter consistent with the congestion. As a result, left and right images that give a natural sense of distance are displayed on the HMD.

[0020]

EXAMPLE FIG. 23 shows an outline of a stereoscopic system which is a combination of a three-dimensional camera and an HMD (head mounted display). The three-dimensional camera 1 has two left and right cameras, and is installed near the subject O. The HMD 2 has a built-in left and right virtual image projection device that projects a virtual image on each of the left and right eyes. The subject image captured by the left camera of the three-dimensional camera 1 is used as the left eye, and the subject image captured by the right camera is used as the right eye. By projecting each virtual image, the stereoscopic image SV is provided to the observer wearing the HMD 2. HMD2 is 3D camera 1
The HMD wearer can observe an image at a place distant from the shooting site without having to place it near the. A controller 3 controls the three-dimensional camera 1 and the HMD 2. The controller 3 allows the HMD wearer to operate the HMD.
Placed near the.

In order to capture the left and right images of the observation object with the three-dimensional camera 1 and display these left and right images on the HMD 2 to reproduce a three-dimensional image having a natural three-dimensional effect, the following human vision is used. It is necessary to provide the features to the 3D camera 1 and the HMD 2.

That is, human vision recognizes perspective and stereoscopic effect by adjusting the distance between the left and right eyes and the refractive power of the crystalline lens of each eye. Since the left and right eyes are apart, so-called parallax occurs in the images formed on the left and right retinas when observing the object. Due to this parallax, the relative perspective of the object is perceived. Further, when observing the object, the visual axes of both eyes point toward the object, and so-called convergence is set. The angle formed by both visual axes, that is, the vergence angle, becomes larger as the object is closer, and becomes smaller as the object is farther away, and the two visual axes become parallel at infinity.

Further, in order to clearly form an image of the object on the retina, the refractive power of the crystalline lens is changed according to the distance to the object to adjust the focus, that is, the diopter. The eye strain required for diopter adjustment is greater as the object is closer, and smaller as the object is farther away. From this tension, the distance to the object is recognized to some extent.

A three-dimensional camera for picking up left and right images must be able to correctly set the convergence on an object, and also be set to be correctly focused for picking up a clear image. Three
The congestion setting of the two-dimensional camera will be described with reference to FIG. In FIG. 24, A1, B1, and C1 indicate that the left camera 101L and the right camera 101R of the three-dimensional camera have constant congestion,
It shows a state in which the distance from the three-dimensional camera to the observation object O is different. The images captured by the left camera 101L and the right camera 101R in each state are A2, B2, and C2, respectively.
As shown in.

In the state of B1 in which the optical axes of the left and right cameras 101L and 101R intersect on the object O and the convergence is properly set, the left and right images are displayed at the centers of the left and right image frames FL and FR as shown in B2. To position. When these left and right images are observed to be fused during stereoscopic viewing, the left and right image frames FL and FR also coincide with each other, as in B3. Therefore, an image that matches the left and right sides including the background in the image frame is obtained, and a natural stereoscopic effect is reproduced.

On the other hand, in the state of A1 where the congestion is too large, the left and right images are left and right image frames F as shown by A2.
The image is taken outside the center of L and FR. On the contrary, when the image is captured in the state of C1 where the congestion is too small, the left and right images are located inside the center of the image frame like C2. A2 and C
When the left and right images of 2 are stereoscopically fused, a mismatch occurs between the left and right image frames FL and FR as shown in A3 and C3. The disagreement between the left and right image frames gives the observer a feeling of strangeness, but the sense of discomfort increases as the deviation of the vergence increases, and eventually the image cannot be viewed stereoscopically. On the other hand, when the congestion setting is not appropriate, when observing with the image frames FL and FR aligned, A2 and C2
As is clear from the above, the images of the object do not match on the left and right, and it is also difficult to stereoscopically view.

As described above, in order to capture left and right images that allow a natural stereoscopic image to be observed, the convergence of the three-dimensional camera is appropriately set according to the distance to the object to be observed (hereinafter referred to as the subject distance). Need to When the distance between the left and right cameras is constant, the convergence is uniquely determined according to the subject distance.
Further, the focus setting is uniquely determined corresponding to the congestion setting.

In order to observe the left and right images taken by appropriately setting the convergence and focus of the three-dimensional camera as a natural stereoscopic image by the HMD, the HMD also needs to appropriately set the convergence and the diopter.

The HMD 2 is provided with a display device and a pair of left and right eyepieces each including a convex lens as a virtual image projection device, and guides the image displayed on the display device to the observer's eyes through the eyepieces. FIG. 21 schematically shows a state in which an image on the display device is observed through an eyepiece lens. Display device 202
When the eyepiece lens 203 is set so that is positioned between the eyepiece lens 203 and the front focus position thereof, the image light of the display device 202 is emitted from the surface PV farther than the display device 202, as shown in FIG. Perceived as
A virtual image is obtained on V. The position of the virtual image plane PV moves back and forth by moving the display device 202 back and forth with the eyepiece lens 203 fixed, or by moving the eyepiece lens 203 back and forth with the display device 202 fixed. . The diopter of the observer's eye E is adjusted so as to be in focus according to the distance to the virtual image plane PV.

FIG. 22 shows the left and right display devices 202L and 202R.
Shows a state in which the image of is observed with both eyes EL and ER. By inclining the left eye display device 202L and the eyepiece lens 203L, and the right eye display device 202R and the eyepiece lens 203L, respectively, it is perceived that an image exists at a position indicated by OP in the drawing. In the figure, β is the angle of convergence. If this vergence angle does not match the vergence angle of the three-dimensional camera at the time of shooting, the subject distance is not correctly reproduced and a natural stereoscopic effect cannot be obtained.

For example, consider the situation of FIG. In the figure, A represents a state in which the first observation target object O1 is photographed by the three-dimensional camera, and the second target object O2 exists farther than the target object O1. Left camera 101L and right camera 10
The optical axis of 1R intersects with the convergence angle θ on the first object O1 as shown by the solid line. A state in which the left and right images thus photographed are observed with the HMD in which the convergence is set to an infinite distance is shown in B. The left and right images of the first object O1 are located at the centers of the left and right display devices, respectively, as in B2 of FIG. 24, and the left and right optical axes 206L and 206R of the HMD are parallel to each other. At this time, the left and right images of the second object O2 are displayed outside the optical axes 206L and 206R. In order to stereoscopically view the image of the object O2 in this state, the visual axes EXL and E of the left and right eyeballs EL and ER are used.
Each XR must face outward. It is extremely difficult for an ordinary person to direct the visual axes of both eyes to the outside at the same time, and the observer cannot stereoscopically view the image.

The vergence angle is θ
The state observed with the HMD set to is shown in C. In this case, the left and right visual axes EXL, EXR of the observer with respect to the left and right images of the second object O2 are directed inward, the convergence is appropriately reproduced, and the second object O2 is generated together with the first object O1. It is correctly observed as a stereoscopic image SV. In this state, the subject distance is accurately reproduced and a natural perspective is obtained.

As described above, it is necessary for the HMD to set the congestion in accordance with the congestion of the three-dimensional camera. In addition to this convergence setting, the distance to the virtual image plane PV is set to match the distance from the eye to the intersection of the left and right visual axes, and the left and right virtual images are positioned at the intersection of the visual axes, resulting in a more natural three-dimensional image. The image can be reproduced. Below, set the distance from the eye to the virtual image plane to H
The diopter setting of MD is different from the diopter adjustment for correcting myopia and hyperopia described below.

Normally, the human eye can clearly observe an object to be observed from a close range to infinity, but in myopia and hyperopia, the function of adjusting the refractive power of the crystalline lens deteriorates, and the long range or near range is affected. The focus cannot be adjusted for the object. A method of correcting the HMD will be described with reference to FIG. In the figure, A is an example showing a state of image formation of an emmetropic eye, and the display device 202 is at the front focus position of the eyepiece lens 203. The image light passes through the eyepiece lens 203 to become parallel light, and enters the eye E as light from infinity. In an emmetropic eye, the refracting power is appropriately adjusted for this parallel light, and an image is formed on the retina.

B shows the state of image observation with myopia when the display device 202 and the eyepiece lens 203 are set in the same state. In myopia, the refractive power cannot be adjusted for parallel light or light that is nearly parallel, and the image light intersects in front of the retina. Therefore, it does not form a clear image on the retina. in this case,
As shown in C, when the eyepiece lens 203 is moved slightly away from the eye, the image light intersects on the retina. This correction enables clear image observation. On the contrary, in the case of hyperopia, since the image light intersects behind the retina, the image light is corrected so as to intersect on the retina by bringing the eyepiece lens close to the eye.

The moving distance ΔMD of the eyepiece lens 203 necessary for this correction differs depending on the degree of myopia and hyperopia, and even the same observer has a difference between the left and right eyes. Therefore, HM
It is desirable that D is capable of right and left diopter correction. The diopter can be corrected by changing the position of the eyepiece lens 203 and moving the display device 202 forward or backward.

In the stereoscopic system of the present embodiment shown in FIG. 23, the three-dimensional camera 1 and the HMD 2 are constructed in consideration of the above human visual characteristics, and the three-dimensional camera 1 is not congested. The focus setting is variable, and the HMD 2 also has variable convergence and diopter settings, and can also perform diopter correction. Furthermore, the movement of the head of the observer wearing the HMD2 is detected by the HMD2, the orientation of the three-dimensional camera 1 is set to follow the orientation of the HMD2, and the observation target that the observer is trying to pay attention to is photographed. Further, the subject distance is detected based on the left and right images captured by the three-dimensional camera 1, and the convergence and focus of the three-dimensional camera 1 and the convergence and diopter of the HMD 2 are set according to the subject distance.

As described above, in the stereoscopic vision system of the present embodiment, the operation of the three-dimensional camera for photographing and the operation of the HMD for display are interlocked with each other, so that the observer can obtain a desired subject in real time and can clearly see it. It provides a realistic 3D image.

A three-dimensional camera will be described with reference to FIGS. As shown in FIG. 1, two cameras 101L and 101R are installed on a base plate 131. Left camera 1
01L has a taking lens 102L, a CCD (charge coupled device) sensor 103L, a lens barrel 104L, and a lens holding lens barrel 105L. The CCD sensor 103L receives light transmitted through the taking lens 102L and converts the light into an electric signal, and is installed so as to be perpendicular to the optical axis 106L of the taking lens 102L and pass through the center of the optical axis 106L. The CCD sensor 103L is fixedly held by the lens barrel 104L, and the taking lens 102L is the lens holding lens barrel 10.
It is fixedly held at 5L.

The lens holding barrel 105L is a taking lens 10
The lens barrel 104 is movable so as to be movable along the optical axis 106L of 2L.
It is loaded into L, and it is possible to set the focus on subjects from the closest distance to infinity. FIG. 1 shows a state in which an object at infinity is in focus.
Distance L from D sensor 103L to taking lens 102L
₀ is almost the shortest within the movable range of the lens holding barrel 105L.

A pin 107L is provided on the outer surface of the lens barrel 104L located below the center of the light receiving surface of the CCD sensor 103L, and a hole 108L having a diameter slightly larger than the diameter of the pin 107L is formed in the base plate 131. ing. The lens barrel 104L is held by the base plate 131 by inserting the pin 107L into the hole 108L, and is rotatably arranged around the pin 107L. The pin 10 is also provided below the outer surface of the lens holding barrel 105L.
9L is provided, and the pin 109L is engaged with the cam groove 110L formed in the base plate 131. Left camera 101L
The pin 109L moves along the cam groove 110L and the lens holding barrel 105L moves along the optical axis 106L according to the rotation of the. Therefore, the amount of extension of the taking lens 102L also changes according to the rotation of the camera 101L.

Like the left camera 101L, the right camera 101R is also provided on the taking lens 102R, the CCD sensor 103R, the lens barrel 104R, the lens holding lens barrel 105R, the pin 107R provided on the lens barrel 104R and the lens holding lens barrel 105R. Has a pin 109R and a base plate 131 has a hole 1
08R and a cam groove 110R are formed. Right camera 10
In 1R, the pin 107R and the pin 109R engage with the hole 108R and the cam groove 110R, respectively, and rotate with respect to the base plate 131 about the pin 107R as an axis. In the three-dimensional camera 1, the distance between the pins 107L and 107R, that is, the distance between the centers of the light receiving surfaces of the left and right CCD sensors 103L and 103R is the baseline length. Optical axis 106 of left and right cameras 101L and 101R
Although L and 106R are parallel to each other in FIG. 1, the left and right cameras 101L and 1L are correspondingly arranged as the subject distance becomes shorter.
01R rotates inward, and the optical axis 106L,
106R intersects.

FIG. 2 shows the states of the left and right cameras 101L and 101R when photographing a subject located at a short distance. The left and right cameras 101L and 101R face inward so that the subject is located on the intersection of the optical axes 106L and 106R. The angle formed by the left and right optical axes 106L and 106R is the convergence angle of the three-dimensional camera 1.

Pins 109 of the left and right cameras 101L and 101R
Since L and 109R are engaged with the cam grooves 110L and 110R, the lens holding barrels 105L and 105R are the camera 101L,
The position is set according to the orientation of 101R. The distance between the CCD sensors 103L and 103R and the taking lenses 102L and 102R for an infinite subject was L ₀ , but for short-distance subjects, the lenses 102L and 102R were extended, and the distances between the sensors 103L and 103R and the lenses 102L and 102R were It becomes L ₁ . That is, a difference of L ₁ -L ₀ occurs in the focus position. In the present three-dimensional camera 1, the cam groove 1 is designed so that appropriate focusing is always performed according to the subject distance.
10L and 110R are formed in consideration of the rotation angles of the left and right cameras 101L and 101R and the moving amounts of the taking lenses 102L and 102R.

Since the pins 107L and 107R, which serve as the rotation axes, are provided below the center of the light receiving surfaces of the CCD sensors 103L and 103R as described above, the left and right sensors 103L and
The center-to-center distance of 103R, that is, the base line length, is always constant, and does not change during infinite subject shooting or short-distance subject shooting.

FIG. 3 shows the congestion setting mechanism of the left and right cameras 101L and 101R. In this figure, the left and right cameras 101L,
101R is a state in which an object at infinity is being photographed as in FIG. A gear 111L partially having teeth is fixedly attached to the pin 107L of the left camera 101L.
Gear 111R with teeth on part of pin 107R of 01R
Is fixedly installed. The gear 111L and the gear 111R mesh with each other, and the rotation of the gear 111L causes the left camera 10 to rotate.
1L rotates around the pin 107L, and the right camera 101
The R rotates about the pin 107R in the opposite direction to the left camera 101L by the same angle. The gear 111L also meshes with a gear 133 connected to a step motor 132 (not shown) described later. The rotation of the step motor 132 is performed by the gear 133,
111L to the left camera 101L, and the gear 111
It is transmitted to the right camera 101R via R, and the left and right cameras 10
1L and 101R rotate at the same angle in opposite directions at the same time.

A biasing spring 112L is provided between the gear 111L and the base plate 131, and biases the gear 111L counterclockwise. A biasing spring 112R is also provided between the gear 111R and the base plate 131, and the gear 111R is biased in the clockwise direction. Since the backlash between the gears 111L and 111R is moved in one direction by the biasing springs 112L and 112R, the left and right cameras 101L and
There is no error in the direction setting of 101R.

The base plate 131 is provided with an electric switch 134, and opens and closes by contacting and separating with an eccentric pin 135 provided on the gear 111L. The switch 134 is the left camera 101.
It is closed when L is facing the direction to shoot a subject at infinity, and is open otherwise. The open / close position can be adjusted by setting the left and right cameras 101L and 101R to the optical axes 106L and 1L with both terminals of the switch 134 opened.
06R is set to be parallel, and the eccentric pin 13
This is done precisely by rotating the eccentric pin 135 so that one terminal of the switch 134 pushed by 5 contacts the other terminal. The closing signal of the switch 134 is the left / right camera 1.
It is used as a signal indicating the reference position of rotation of 01L and 101R.

Further, in order to regulate the rotation range of the gear 111L, a cutout portion 136 is formed in the gear 111L, and a pin 137 is provided on the upper surface of the base plate 131 at a position corresponding to the cutout portion 136. . This prevents the inconvenience that the left and right cameras 101L and 101R are contacted and damaged due to the erroneous congestion setting.

FIG. 4 shows a state in which the convergence is set for a short-distance subject. The left and right cameras 10 are arranged so that the subject exists at the intersection of the extension lines of the optical axes 106L and 106R.
1L, 101R gear 13 by step motor 132
It is rotated symmetrically via 3, 111L, 111R.
As described above, the focus position setting is automatically performed along with the congestion setting. At this time, the switch 134 is open.

FIG. 5 shows the connection of the mechanical parts of the three-dimensional camera 1. The gear 133 is connected to the step motor 132 and meshes with the gear 111L. The pin 107L, which serves as the rotation axis of the left camera 101L and the gear 111L, engages with the hole 108L formed in the base plate 131. The base plate 131 is also formed with a hole 108R that engages with the pin 107R that is the rotating shaft of the right camera 101R and the gear 111R, but this is not shown in the figure. The pin 109L provided on the lower portion of the lens holding barrel 105L of the left camera 101L is inserted into the cam groove 110L, and the pin 109R of the right camera 101R is inserted into the cam groove 110R.

The step motor 132 is rotated by the gear 13
Left and right cameras 101L, 1 through 3 111L, 111R
01R, the left and right cameras 101L and 101R rotate, and the congestion is set. Left and right cameras 101L, 101R
The left and right pins 109L and 109R move along the cam grooves 110L and 110R, respectively, according to the rotation of the camera lens 1.
The feeding adjustment of 02L and 102R is performed. Therefore, the focus setting is also performed in conjunction with the congestion setting.

The play of the gears 111L and 111R is the biasing spring 1.
The left and right cameras 101L and 101R are rotated in one direction by the 12L and 112R, and the rotation range of the left and right cameras 101L and 101R is restricted by the pin 137 formed on the base plate 131 and the notch 136 of the gear 111L. In addition, the left and right cameras 101
It is determined whether L and 101R face infinity, that is, whether the left and right optical axes 106L and 106R are parallel.

In the three-dimensional camera 1 constructed as described above, the optical axes 106L, 1L of the left and right cameras 101L, 101R are
The convergence setting is performed by rotating the step motor 132 so that the subject is located at the intersection on the extension line of 06R. Left and right cameras 101L and 101R are pins 109
Since L and 109R are engaged with the cam grooves 110L and 110R, the feeding amount is adjusted in conjunction with the convergence setting and the focus setting is automatically performed. Also, the left and right cameras 101L,
Since the baseline length of 101R is fixed, it is possible to capture left and right images that can always reproduce the correct sense of distance.

FIG. 6 shows the appearance of the three-dimensional camera 1. The three-dimensional camera 1 has a fixed base plate 138, and the fixed base plate 138 is installed in a horizontal place or set horizontally using a tripod or the like for use. On the fixed base plate 138, a rotary base plate 139 that can rotate horizontally 180 degrees each horizontally.
Is installed. The left and right cameras 101L and 101R and the base plate 131 are housed in the camera case 140 and installed on the rotary base plate 139.

The camera case 140 has a rotary shaft 141 on both left and right sides, and a rotary base plate 139 is provided with a pair of rotary shaft support arms 142 for supporting the rotary shaft 141. The camera case 140 has a rotary shaft 141 parallel to the rotary base plate 139 and is rotatably supported by the rotary support arm 1.
Supported by 42. As a result, the camera case 140 is held on the rotary base plate 139 and is vertically rotatable. The rotating shaft 141 is parallel to a straight line connecting the centers of the light receiving surfaces of the left and right CCD sensors 103L and 103R,
Moreover, it is set so as to be orthogonal to the central axis of rotation of the rotary base plate 139. The left and right cameras 101L and 101R are C
The centers of the light receiving surfaces of the CD sensors 103L and 103R are installed so as to be equidistant from the center of rotation of the rotary base plate 139.

Signals for controlling the movements of the left and right cameras 101L and 101R, video signals from the left and right cameras 101L and 101R, etc. are input and output via a signal cable 143. The camera case 140 is provided with a horizontal sensor 144 composed of a mercury switch, and detects the horizontal of the three-dimensional camera 1. The horizontal sensor 144 is closed when the camera is set horizontally, that is, when the vertical rotation about the rotation axis 141 is not performed. Δ in FIG.
BC indicates the distance between the centers of the light receiving surfaces of the left and right CCD sensors 103L and 103R, and is the baseline length of the above-described three-dimensional camera.

FIG. 7 shows a side view of a three-dimensional camera partially transparently. A gear 145 is provided at the end of the rotary shaft 141 of the camera case 140. The gear 145 meshes with the gear 147 connected to the step motor 146 in the rotation support arm 142, and the rotation of the step motor 146 is transmitted to the rotation shaft 141 to rotate the camera case 140 in the vertical direction. The left and right cameras 101L and 101R, the base plate 131 and the gears 111L and 111R are shown in perspective. Driver circuits 113L and 11 of CCD sensors 103L and 103R are provided behind the left and right cameras 101L and 101R.
3Rs are provided, and each has a flexible substrate 11
CCD sensor 103L, 103R by 4L, 114R
Is connected to

FIG. 8 shows a horizontal direction setting mechanism of the three-dimensional camera 1. The step motor 14 is attached to the fixed base plate 138.
8 is installed, and the step motor 148 has a gear 1
49 are connected. On the other hand, a rotary shaft 150 is provided at the center of the rotary base plate 139, and a gear 151 is coupled to the rotary shaft 150. The gear 149 and the gear 151 mesh with each other, and the rotation base plate 139 rotates with respect to the fixed base plate 138 by the rotation of the step motor 148.

A small conductive pattern 152 is formed on the surface of the gear 151, and an electric switch 153 is provided on the fixed base plate 138 so as to come into contact with the conductive pattern 152. The switch 153 is brought into conduction by contacting both terminals with the conductive pattern 152 at the same time. The conductive pattern 152 is arranged so as to conduct the switch 153 when the rotary base plate 139 faces a predetermined direction with respect to the fixed base plate 138, and is used for detecting the reference direction of the three-dimensional camera 1. The rotary base plate 139 rotates 180 ° in both left and right directions from the position where the switch 153 is closed.

As shown in FIG. 7, when the three-dimensional camera 1 is installed with the fixed base plate 138 horizontal, the rotary base plate 13
The rotation center of 9 is located ΔXC behind the light-receiving surfaces of the CCD sensors 103L and 103R, and the vertical rotation center of the camera case 140 is located ΔYC below the center of the light-receiving surfaces of the CCD sensors 103L and 103R.

Three-dimensional camera 1 constructed as described above
Can make a rotational movement similar to the rotation of a human head. That is, the human head is, as shown in FIG.
In the vertical direction, the point RP located at the back of the neck at the center of left and right
And rotates about an axis RV which is perpendicular to the plane of the drawing. The position of the rotation axis RV is ΔXH behind and ΔYH below the retina position of the eyeball E. In the horizontal direction, it rotates about an axis RH passing through the point RP.

FIG. 31 shows how the head is rotated in the vertical direction for image observation. The observation object O is located near the face of the observer and below the horizontal visual axis EX (A). In order to observe the object O, the head rotates downward around the point RP on the back of the neck (B). This is a natural movement of a human, and at this time, the visual axis EX forms an angle of α with the horizontal,
The distance between the eyeball E and the object O is Db. When the object O is observed by rotating the head downward about the eye E (C), the visual axis EX at that time forms an angle of β with the horizontal, and the distance between the eye E and the object O is Dc. Become. Further, even if the object O is observed by rotating the eyeball E downward without moving the head, the positional relationship between the object O and the eye E is the same as the motion of C, and the line of sight is between the horizontal angle and β. None, the distance between the eye E and the object O is Dc.

Here, β is larger than α and Dc is larger than Db. The motion of C that rotates the head about the eyeball E is extremely unrealistic, and the motion of moving only the eyeball E without moving the head is an unnatural motion that is intentionally performed. The difference between α and β and Db and Dc is small when the distance between the observer and the observation object is large, but becomes more remarkable as the distance becomes smaller.

Therefore, also in the three-dimensional camera, the setting position of the rotation axis is important for taking a natural image.
As shown in FIG. 7, the horizontal rotation center of the three-dimensional camera is ΔX from the light receiving surfaces of the CCD sensors 103L and 103R.
It is located at the rear of C, and the center of rotation in the vertical direction is located ΔYC below the center of the light receiving surface of the CCD sensors 103L and 103R. This is the binocular and rotation axis Δ of the above human head.
This is the same as the relationship between XH and ΔYH, and reproduces the natural human motion shown in FIG. 31B.

The distance ΔBH between both eyes shown in FIG. 30 is the basis of distance recognition by human vision, and corresponds to the base line length ΔBC in a three-dimensional camera. In order to take a stereoscopic image naturally, in addition to the appropriate setting of the vertical and horizontal rotation center positions of the three-dimensional camera, the baseline length Δ
BC also needs to be set correctly in consideration of the distance from the light receiving surfaces of the CCD sensors 103L and 103R to the center of rotation. In the present embodiment, the three-dimensional camera and the rotation of the human head have a similar relationship, and a natural three-dimensional image is captured, so that the image on the HMD is more realistic. Specifically, the above-mentioned ratios of ΔXC and ΔXH, ΔYC and ΔYH, and ΔBC and ΔBH are set to be constant.
A three-dimensional camera is similar to a human head.

FIG. 27 shows the second structure of the three-dimensional camera. Left and right cameras 171L and 171R are symmetrically installed on the base plate 191. The CCD sensor 173L of the left camera 171L is fixedly installed vertically to the base plate 191. In front of the CCD sensor 173L, a left photographing lens 172L fixedly installed in the lens holding barrel 175L is arranged. A pair of pins 177L and 178L are provided below the outer surface of the lens holding barrel 175L and engage with cam grooves 179L and 180L formed in the base plate 191 respectively. Further, the lens holding barrel 175L is provided with an arm 182L via a pin 181L. The arm 182L is rotatable around the pin 181L.

The right camera 171R is also constructed similarly to the left camera 171L, and the left and right CCD sensors 173L, 1
73R is arranged so that the light receiving surface is located on the same plane. A step motor 192 is fixedly installed on the base plate 191 between the left and right CCD sensors 173L and 173R.
It is coupled to a rotary shaft 194 supported by a shaft support frame 193. A mover 195 that is movable in the axial direction is provided in contact with the rotating shaft 194. A male screw is cut on the outer periphery of the rotary shaft 194, and a female screw that meshes with the mover 195 is cut. The mover 195 moves back and forth along the rotary shaft 194 by the rotation of the step motor 192.
An arm 196 extending forward is fixed to the mover 195, and a front end of the arm 196 has left and right lens holding lens barrels 1.
The ends of the arms 182L and 182R connected to 75L and 175R are connected via a pin 197. Left and right arm 1
82L and 182R rotate about the pin 197 with respect to the arm 196. With this configuration, the step motor 192
Is rotated by a moving element 195, arms 196, 182L, 18
It is transmitted to the left and right lens holding barrels 175L and 175R via 2R.

In FIG. 27, the left and right cameras 171L, 171R
Is a state in which convergence is set to infinity, and the optical axis 17
6L and 176R are parallel. Step motor 19
The left and right lens holding lens barrels 175L and 175R are driven by the rotation of 2 to set the convergence on the near object.
8 shows. Even at this time, the left and right photographing lenses 172L, 172R
The optical axes 176L and 176R of are kept parallel. Left CC
The subject is located on the intersection of a line 183L connecting the center of the light receiving surface of the D sensor 173L and the center of the left lens 172L and a line 183R connecting the center of the light receiving surface of the right CCD sensor 173R and the center of the right lens 172R.

The distance from the taking lens 172L, 172R to the light receiving surface of the CCD sensor 173L, 173R was L ₀ when the convergence was set for an infinite subject, but became L ₁ when the convergence was set for a short-distance subject, which was long. ing. By properly performing this distance adjustment, the correct focus setting according to the subject distance is performed. Cam groove 17 of base plate 191
The shapes of 9L, 180L, 179R, and 180R are set so that the focus is properly set according to the change in congestion. Furthermore, so that the left and right optical axes 176L, 176R are always kept parallel, the cam grooves 179L, 180L, 179R,
180R is formed.

The detection of the subject distance in the present stereoscopic system will be described. FIG. 19 shows a state in which the congestion is set large (A), properly (B), and small (C) with respect to an object located at a constant distance from the three-dimensional camera, and the left and right CCD sensors 103L in each state. The image captured by 103R is shown. Left and right cameras 101L, 101R
Is directed to the subject and the convergence is properly set (B), the subject is imaged in the left-right center in both the left CCD sensor 103L and the right CCD sensor 103R. The distance P ₀ between the left and right subject images at this time is
The left and right cameras 101 like the three-dimensional camera of this embodiment
L and 101R are CCD sensors 103L and 10 respectively
When rotating around the center of the light receiving surface of 3R, the value becomes a constant value peculiar to the camera regardless of the distance from the camera to the subject. The distance P ₀ between the left and right images is called the reference image distance. When the vergence angle is too large (A), the subject image is closer to the outside than the center of the light receiving surface, and when the vergence angle is too small (C), the subject image is closer to the inside than the center. The subject distance can be obtained from the distance P between the left and right subject images and the congestion of the three-dimensional camera set at that time.

The interval P between the left and right subject images is detected as follows. Now, a horizontally long rectangular distance measuring area MA is provided in the central portion of the light receiving surfaces of the left and right CCD sensors 103L and 103R in the vertical direction.
Consider L and MAR. Left and right distance measurement areas MA of A in FIG.
FIG. 20 is an enlarged view of L and MAR. It is assumed that each ranging area includes n pixels in the horizontal direction and m pixels in the vertical direction, and the data included in each pixel in the left ranging area MAL is XL (i,
j), the data contained in each pixel of the right distance measuring area MAR is XR
It is represented by (i, j) (i = 1, n, j = 1, m). First, the pixel data of the left and right distance measurement areas MAL and MAR are added in the vertical direction and converted into one-dimensional data strings YL (i) and YR (i) in the horizontal direction, respectively.

[0073]

[Equation 1]

[0074]

[Equation 2]

Subsequently, detection blocks B1, B2 and B3 are set in the left distance measuring area MAL with the block size bn. B1 = YL (i) (i = k1, k1 + bn-1) B2 = YL (i) (i = k2, k2 + bn-1) B3 = YL (i) (i = k3, k3 + bn-1) )

For each of these detection blocks B1, B2, B3, a one-dimensional data string YR in the right distance measuring area MAL
Perform correlation calculation with (i).

[0077]

(Equation 3)

[0078]

[Equation 4]

[0079]

(Equation 5)

By performing the above calculation, Z1 (ks), Z2 (ks), Z
For 3 (ks), P is obtained by finding the minimum ks. FIG. 20 shows a state where P is obtained in the detection block B1. When the distance P between the left and right subject images is obtained in this way, the distance from the three-dimensional camera to the subject is calculated based on the baseline length of the three-dimensional camera and the convergence angle of the left and right cameras at that time.

Although the number of detected blocks is 3 here, the number of blocks is set so that all the elements of the one-dimensional data string YL (i) are included in any of the blocks. Further, the block size bn is also set according to the number of blocks, but adjacent blocks may be partially overlapped so that the elements of the one-dimensional data string YL (i) belong to two or more blocks, Each element of the data string YL (i) may be set so as to belong to only a single block.

The structure of the HMD of this embodiment is shown in FIG. H
Inside the MD2, a pair of left and right virtual image projection devices 201L and 201R are installed on a base plate 231. The left virtual image projection device 201L includes a video display device 202L and a video display device 202.
Eyepiece 2 consisting of a convex lens that guides the L image to the left eye EL
03L, a display device holding lens barrel 204L that holds the image display device 202L, a lens holding lens barrel 205L that internally houses and holds the eyepiece lens 203L, and a diopter correction lens barrel 207L that holds the lens holding lens barrel 205L movably back and forth. Have. An LCD (liquid crystal display device) is used as the image display device 202L, and is set so as to be perpendicular to the optical axis 206L of the eyepiece lens 203L and the optical axis 206L passes through the center of the display surface.

The inner dimension of the display device holding lens barrel 204L is formed slightly larger than the outer dimension of the diopter correction lens barrel 207L, and the inner surface of the rear portion of the display device holding lens barrel 204L and the front of the diopter correction lens barrel 207L. It is set to be in sliding contact with the outer surface.

The lens holding barrel 205L has a cylindrical shape and has an external thread formed on the outer periphery thereof, and an internal thread which meshes with this is formed on the inner surface of the diopter correction barrel 207L. By rotating the lens holding barrel 205L around the optical axis 206L, the lens holding barrel 205L moves back and forth along the optical axis 206L together with the eyepiece lens 203L. As a result, the distance between the eyepiece lens 203L and the image display device 202L changes, and the diopter correction can be performed according to the individual difference of the observer. The right virtual image projection device 201R is also the left virtual image projection device 2
The configuration is similar to that of 01L, and the left and right diopter corrections are performed independently.

Pins 208L and 208R are provided on the lower portions of the outer surfaces of the diopter correction lens barrels 207L and 207R, respectively.
It is inserted in the cam grooves 209L and 209R formed in No. 1. Further, pins 210L and 210R are formed on the lower portion of the outer surface of the display device holding lens barrels 204L and 204R, respectively.
The cam grooves 211L and 211R formed in 31 are respectively inserted. A step motor 232 is fixedly installed on a base plate 231 between the left and right eyepieces 203L and 203R, and is connected to a rotary shaft 234 supported by a shaft support frame 233. A male screw is cut on the outer periphery of the rotary shaft 234, and the male screw 235 meshes with the mover 235.
The mover 235 moves back and forth along the rotation shaft 234 by the rotation of the step motor 232.

The mover 235 is provided with a pin 236,
Two arms 212 that are rotatable around a pin 236
L and 212R are attached. A pin 213L is provided on the lower portion of the outer surface of the left display device holding lens barrel 204L,
It is rotatably connected to the tip of the arm 212L.
Similarly, the right display device holding lens barrel 201R is also connected to the arm 212R via the pin 213R. Arm 212L, 2
Pins 215L and 215R are provided in the middle of 12R, respectively, and cam grooves 216L and 2L provided on the base plate 231 are provided.
It is inserted in each 16R. An electric switch 237 is installed on the base plate 231 near the tip of the rotary shaft 234, and is set to close when the mover 235 is located at the tip of the rotary shaft 234 and one terminal of the switch 237 is pushed. Has been done. The left and right virtual image projection devices 201L and 201R and the arms 212L and 212R are arranged symmetrically with respect to the rotation axis 234.

FIG. 12 is a perspective view showing the vicinity of the step motor 232. Rotation of the rotary shaft 2 by rotation of the step motor 232.
34 rotates, and the mover 235 moves in the left-right direction in the figure, that is, H.
Move in the front-back direction of MD2. The left and right arms 212L and 212R also move accordingly. In FIG. 12, the switch 237
Is open, but the mover 235 is near the left end of the rotation axis in the figure,
When the switch 237 is moved to the vicinity of the front end of the rotary shaft 234 of the HMD 2, the switch 237 is closed by the mover 235. That is, the switch 237 is provided at a position to be closed when the convergence is set to infinity, and the left and right virtual image projection device 2
It is used as a reference for rotation of 01L and 201R.

In such a structure, when the mover 235 moves back and forth due to the rotation of the step motor 232, the arms 212L and 212R move to the pin 21 in the middle of the arm.
5L and 215R move along the cam grooves 216L and 216R, and the left and right virtual image projection devices 201L and 201R also move to the cam grooves 20.
It moves according to the shapes of 9L, 211L, 209R, and 211R.
FIG. 9 shows a state in which an image of a subject at infinity is displayed and observed. The mover 235 exists near the tip of the rotary shaft 234, and the switch 237 is closed. At this time, the optical axes 206L of the left and right eyepieces 203L and 203R,
206R are parallel, and the left and right eyeballs EL,
Pass through the center of ER. In addition, the left and right image display devices 202L,
The display surface of 202R exists on the same plane.

FIG. 10 shows a state of the HMD 2 when the step motor 232 is rotated and the mover 235 is moved to the motor 232 side. This corresponds to a state of displaying and observing an image of a short-distance subject. At this time, the optical axes 206L and 206R of the eyepieces 203L and 203R are tilted inward from the parallel, and the vergence angle is set so that the observation object exists on the intersection of the extension lines of the optical axes 206L and 206R. Optical axes 206L and 206R are image display devices 202L and 2
Passing through the center of 02R is maintained as in FIG.

The left and right lens holding lens barrels 205L and 205R are arranged in the cam groove 209 together with the diopter correction lens barrels 207L and 207R.
It is moving inward along L and 209R. The left and right display device holding lens barrels 204L and 204R are also cam grooves 211L and 211R.
It is moving inward along, but is approaching the eyepieces 203L and 203R with it. Lens holding lens barrel 2
This movement of the 05L, 205R and the display device holding lens barrels 204L, 204R is caused by the cam grooves 209L, 211L, 209R, 21R.
This is due to the 1R shape. Cam grooves 209L, 209R
Is formed in an arc shape centering on the eyes EL and ER of the observer, and the left and right virtual image projectors 201L and 201R rotate symmetrically about the left and right eyes EL and ER.

In FIG. 9, the convergence is set for the image of the subject at infinity, and the distance between the eyepieces 203L and 203R and the image display devices 202L and 202R is S ₀ . On the other hand, when the convergence is set for the image of the short-distance subject as shown in FIG. 10, S ₁ is set, and the distance between the eyepieces 203L and 203R and the image display devices 202L and 202R is linked with the expansion of the convergence angle. It becomes smaller, and the diopter is set corresponding to the subject distance. As a result, the sense of distance due to the convergence and the sense of distance due to the diopter are interlocked, and a more realistic image can be obtained. The diopter setting linked with the convergence is performed independently of the correction of the diopter deviation peculiar to the observer, which has already been described. Note that the switch 237 is open when the short distance is supported.

A perspective view of the HMD 2 is shown in FIG. As described above, the driving force of the step motor 232 causes the arm 2 to move.
The left and right virtual image projection devices 201L,
It is transmitted to 201R, and congestion and diopter are set. The HMD 2 configured in this way is covered with a case (not shown), and the case is equipped with a mounting member for mounting on the head.

In this stereoscopic vision system, the orientation of the three-dimensional camera is changed in accordance with the movement of the head of the observer wearing the HMD, the observation target noticed by the observer is captured by the three-dimensional camera, and the image is displayed on the HMD. It is displayed and it is necessary to detect the movement of the observer's head. Therefore, as shown in FIG. 11, a vertical angular velocity sensor 238 for detecting a vertical rotational angular velocity and a horizontal angular velocity sensor 239 for detecting a horizontal rotational angular velocity are provided on the HMD base plate 231. is set up. In this embodiment, a vibration gyro using piezoelectric ceramics is used as the angular velocity sensors 238 and 239.

The piezoelectric vibrating gyro detects rotational movement,
A voltage linearly corresponding to the angular velocity is output. The direction of the rotational movement can be known from the sign of the difference between the output voltage when the piezoelectric vibrating gyroscope performs the rotational movement and the output voltage when the piezoelectric vibrating gyroscope is at rest, and the magnitude of the angular velocity can be known from the absolute value of the difference. Therefore, by continuing to detect the output voltage of the piezoelectric vibrating gyro, the azimuth of the HMD equipped with this can be known.

The method of detecting the direction of the HMD in this embodiment will be described with reference to the example shown in FIG. 32A is a graph showing the output of the piezoelectric vibration gyro with respect to time,
The DC reference output is an output voltage when the HMD wearer keeps his / her head stationary. The difference between this DC reference voltage and the detected output voltage is proportional to the magnitude of the angular velocity. Therefore, in the figure, the area of the portion Sa surrounded by the curve of the DC reference voltage and the output voltage located above it represents the rotation angle in one direction, and is surrounded by the curve of the DC reference voltage and the output voltage located below it. The area of the broken portion Sb represents the rotation angle in the opposite direction. If the rotation angle with respect to time is represented by a graph, it will be B.

The output voltage of the piezoelectric vibrating gyro drifts while maintaining a linear relationship with the angular velocity due to changes in environmental conditions such as temperature. That is, the graph of A in FIG. 32 is translated in the vertical axis direction. Therefore, in order to correctly obtain the angular velocity by the piezoelectric vibration gyro, it is necessary to set the DC reference output that matches the environmental conditions. The DC reference output also differs depending on the individual piezoelectric vibration gyro. Therefore, the DC reference output must be set for each HMD.

In this embodiment, at the time of starting the stereoscopic system or at any time after starting, the HMD wearer keeps his head stationary for a certain period of time, detects the output voltage of the piezoelectric vibrating gyro during that time, and calculates the average value. Take and use as DC reference output. As a result, the fluctuation of the output voltage due to the above-mentioned environmental conditions such as temperature and the individual difference of the piezoelectric vibration gyro is corrected,
The MD rotation speed can always be detected correctly.

The azimuth change amount of the HMD can be strictly obtained by integrating the detected angular velocity with respect to time. However, since the configuration of the integrating circuit becomes complicated, the angular velocity is detected in a minute time period in this embodiment, and the product of the angular velocity and the period time is calculated by assuming that the angular velocity is constant during one period, and this is approximated. Specifically, it is the amount of change in the HMD direction. The shorter the angular velocity detection cycle, the higher the accuracy of the calculated azimuth change amount. In this embodiment, the angular velocity detection cycle is set to 1 msec, and the azimuth of the HMD can be obtained with sufficient accuracy. Further, since the HMD is provided with the vertical angular velocity sensor 238 and the horizontal angular velocity sensor 239, it is possible to detect rotation of the head of the HMD wearer in any direction,
You can know the direction of the HMD correctly.

FIG. 13 is a block diagram showing the functions of the stereoscopic system of this embodiment. The three-dimensional camera 1 has a taking lens 10
The left and right cameras 101L and 101R having 2L and 102R and image pickup elements 103L and 103R are provided. The image of the object O is captured by the taking lenses 102L and 102R.
The image is formed on L and 103R. The outputs of the image pickup devices 103L and 103R are electric signals and are left and right image processing units 115L and 115R.
The image processing is performed. In addition, the image sensor 103
L and 103R are driven by driver circuits 113L and 113R included in the video processing units 115L and 115R. The signals from the image processing units 115L and 115R are transmitted to the HMD 2 and also to the controller 3 that controls the three-dimensional camera 1 and the HMD 2.

The HMD 2 is provided with left and right virtual image projection devices 201L and 201R having display elements 202L and 202R and eyepieces 203L and 203R, and a subject image photographed by the three-dimensional camera 1 is observed by the observer's left and right eyeballs EL and ER. To serve.
As described above, this embodiment is a lightweight LCD suitable for HMDs.
Is used as a display element. The video signal of the left camera 101L is input to the drive circuit 214L of the display element 202L for the left eye, and the video signal of the right camera 101R is input to the drive circuit 214R of the display element 202R for the right eye. Drive circuit 2
Display elements 202L, 2L by signals from 14L, 214R
A subject image is displayed on 02R. Left and right display elements 20
The images displayed on 2L and 202R are the eyepieces 203L,
203R generates virtual images on the left and right eyeballs EL and ER of the observer. In addition, the HMD 2 is provided with the azimuth detecting device 240 including the vertical and horizontal angular velocity sensors 238 and 239, and the azimuth of the head of the observer wearing the HMD 2 is detected. The output of the azimuth detecting device 240 is sent to the controller 3.

In the controller 3, the left and right video processing section 115
The subject distance is detected based on the signals from L and 115R. The controller 3 determines the pins 109L and 109R and the cam groove 110 for the three-dimensional camera 1 according to the detected distance.
Convergence and focus setting unit 154 including L, 110R, step motor 132, etc. is controlled to set congestion and focus, and HM
Pins 208L, 210L, 208R, 21 for D2
0R, the cam grooves 209L, 211L, 209R, 211R, and the convergence / diopter setting unit 241 including the step motor 232 are controlled to set the convergence and diopter of the left and right virtual image projectors 201L, 201R. The controller 3 also controls the step motor 14 according to the output of the azimuth detecting device 240 of the HMD 2.
The camera azimuth setting mechanism 155 of the three-dimensional camera 1 including 6, 148 and the like is controlled to set the azimuth of the three-dimensional camera 1 in association with the orientation of the observer's head.

In the present embodiment, the controller 3 performs distance measurement, camera azimuth setting, camera convergence, focus setting, or HMD convergence, diopter setting.
The control device for performing these controls is the three-dimensional camera 1 and H.
You may make it the structure each built into MD2. Also, 3
It is also possible to record the distance information together with the left and right images of the two-dimensional camera on a video tape or the like and observe the reproduced image on the HMD. At this time, since the HMD can set the convergence and diopter according to the recorded distance information, in addition to real-time observation, it is possible to observe an image having a natural stereoscopic effect at any time after photographing.

FIG. 14 is a block diagram showing an electric circuit of the stereoscopic system of this embodiment. The three-dimensional camera 1 and the HMD 2 are controlled by the controller 3. A step motor 132 for setting convergence and focus of the three-dimensional camera is connected to the controller 3 via a motor drive circuit 156. The switch 134 is the one shown in FIG. 3, and when the convergence is set for an infinitely distant subject, the signal is used as a reference for the convergence setting, and the optical axes 106L and 106R of the left and right cameras are outside the parallel. The drive of the step motor 132 is controlled so as not to face the. The step motor 232 for setting the convergence and diopter of the HMD is connected to the controller 3 via the motor drive circuit 242. The switch 237 shown in FIG. 9 is closed when the left and right optical axes 206L and 206R of the HMD are set in parallel and is used as a reference for the convergence setting.
The drive of the step motor 232 is limited so as to prevent the 206R from facing outward.

The step motor 148 and the switch 153 for rotating the three-dimensional camera in the horizontal direction are those shown in FIG. The step motor 148 is a motor drive circuit 1
It is connected to the controller 3 via 57, and the point where the switch 153 is closed is used as the reference position for rotation. A step motor 146 for rotating the three-dimensional camera in the vertical direction is connected to the controller 3 via a motor drive circuit 158. For the vertical rotation, the signal from the horizontal sensor 144 shown in FIG. 6 is used as a reference. The outputs of the horizontal angular velocity sensor 239 and the vertical angular velocity sensor 238 installed on the HMD are the A / D conversion circuit 24.
After that, it is sent to the controller 3 as a digital signal.

Reference numeral 301 denotes a switch for generating a signal for starting the operation of this system, which is operated by an observer wearing the HMD. A switch 302 generates a signal for requesting the initial setting of the orientation of the three-dimensional camera and the HMD, which is also operated by the HMD wearer. Further, a buzzer 303 is provided to send a signal to the HMD wearer at the time of initial setting. The system start operation and initial setting will be described later.

Left and right CCD sensors 10 of the three-dimensional camera
3L and 103R are driver circuits 113L and 11 respectively.
Driven by 3R. Driver circuits 113L and 113
R is connected to a CCD drive circuit 116 that generates the timing for driving the CCD. Therefore, the left and right CCD sensors 103L and 103R are driven at the same timing. The output of the left CCD sensor 103L is processed by the left image processing circuit 117L and sent to the left LCD drive circuit 214L of the HMD. The LCD drive circuit 214L receives the video signal from the video processing circuit 117L and displays the video on the LCD 202L which is the left display element.

The video processing circuit 117L is the A / D conversion circuit 1
It is connected to the field memory circuit 119L via 18L. The video signal is converted into a digital value for one field by the A / D conversion circuit 118L and stored in the field memory circuit 119L. The field memory circuit 119L is connected to the controller 3 and can transfer digital data to the controller 3.

The output of the right CCD sensor 103R is processed by exactly the same configuration. That is, the signal processed by the video processing circuit 117R is displayed on the LCD 202R as an image via the LCD driving circuit 214R on the one hand and stored in the field memory circuit 119R via the A / D conversion circuit 118R on the other hand. Video processing circuit 117L, 117
The R, A / D conversion circuits 118L and 118R and the field memory circuits 119L and 119R are connected to the CCD drive circuit 116 for timing.

Next, the control process of the stereoscopic system of this embodiment will be described with reference to the flow charts of FIGS. After the three-dimensional camera is installed with the fixed base plate horizontal, the power is turned on. The controller is configured to reset automatically when the power is turned on.
The process starts from step # 105 of FIG. First,
In step # 110, processing such as port initialization and internal RAM initialization is performed, and then in step # 115, the motor, CCD,
Turn on the power supply to drive the LCD.

Then, the convergence of the HMD and the three-dimensional camera is set to the reference position of infinity. HM in step # 120
The convergence and diopter setting motor 232 of D is driven until the switch 237 is closed, the convergence and diopter of the HMD are set to infinity, and the congestion and focus setting motor 132 of the three-dimensional camera is closed by the switch 134 at # 125. Drive to set the convergence and focus of the 3D camera to infinity.

Next, the azimuth of the three-dimensional camera is set to the reference position. In step # 130, the vertical drive step motor 146 is driven until the horizontal switch 144 is closed,
It is assumed that no rotation is made in the vertical direction. As a result, the three-dimensional camera is set horizontally. # 135
Then, the horizontal drive motor 148 is driven until the switch 153 is closed, so that the motor is not rotated in the horizontal direction. As a result, the three-dimensional camera faces the reference azimuth and is set at a position where it can rotate 180 ° in both left and right directions.

Furthermore, the orientation of the observer's head, that is, HM
Initialize the parameter indicating the direction of D. Step # 14
At 0, the parameter XH1 indicating the horizontal direction and the parameter YH1 indicating the vertical direction are both set to 0.

After the system initialization is completed by the above process, it is then determined in step # 145 whether or not a system activation request has been issued. When the switch 301 is closed and an activation request is made by the HMD wearer, the process proceeds to the activation routine # 205 and the activation process is started. If the switch 301 is open and the activation request is not issued, the power source is turned off in # 150, and the state is maintained until the activation request is made in the sleep mode of # 155 which is the low power consumption mode.

16 and 17 show the startup process. First, in step # 210, the power source is turned on, and in # 215 and # 220, the three-dimensional camera is set to the reference direction. Next, in # 225, the orientation parameters XH1 and YH1 of the HMD are set to 0 and initialized. Then, check the HMD congestion and diopter # 230.
In step # 235, the distance is once set to infinity and then set again in correspondence with the predetermined distance. The convergence and focus of the three-dimensional camera are once set to infinity at # 240, and then set to the same predetermined distance as the HMD at step # 245.

As the above-mentioned predetermined distance, the most general subject distance of the subject observed using this stereoscopic system is selected. As a result, 3D camera and HMD
Will be set near the subject distance to be actually observed, and the time required for setting the three-dimensional camera and HMD after activation will be shortened. Therefore, the image observation can be performed immediately after the start-up. In the above operation, the distance is set to infinity, which is the reference position, and then the predetermined distance is set, so that the three-dimensional camera and the HMD can accurately perform the convergence, focus, and diopter setting. Instead of the most general subject distance, for example, three representative values corresponding to a long distance, a medium distance, and a short distance are stored and held, and an arbitrary one is selected by a switch operation to select a three-dimensional camera. May be set, convergence of focus and HMD, and diopter may be set.

After the three-dimensional camera and the HMD are set, the CCD sensor is driven in step # 250 in FIG. 17 to start the photographing by the three-dimensional camera. After the start of photographing, whether or not there is a request for initialization of the orientation of the HMD and the three-dimensional camera is determined in # 255. The orientation initialization request is made by the HMD wearer operating the switch 302.

When the switch 302 is off, initialization is not requested, and the process proceeds to # 260 to permit a timer interrupt for performing the process of setting the orientation of the three-dimensional camera every 1 msec. Next, in step # 265, the image information of the three-dimensional camera is A / D converted, and it is determined whether or not it is stored in one field. If it is not stored in the memory, the determination is repeated and waits until the memory is completed.

After the memory is completed, the digital data of the right distance measuring area and the left distance measuring area are input to the controller in # 270 and # 275, and in step # 280, the distance measuring operation is performed by the above-described method and the distance P between the left and right images is calculated. Ask for. Next, in # 285, it is determined whether or not the distance measurement has been correctly performed. When the reliability of the distance measurement is low due to the low contrast of the subject or the like, it is determined that the distance measurement is impossible, and the process branches to step # 315. If the reliability of distance measurement is high, the image distance P is compared with the reference image distance P _{0 in} # 290 to determine whether or not the congestion of the three-dimensional camera is set appropriately. When the image interval P matches the reference image interval P ₀ within the predetermined allowable range ΔP, it is determined that the congestion of the three-dimensional camera is correctly set, and the process proceeds to step # 315. When the image distance P exceeds the allowable range ΔP and deviates from the reference image distance P ₀ , it is determined that the congestion of the three-dimensional camera is not set correctly, and the congestion setting of the three-dimensional camera and the HMD is performed as follows. I do.

First, in step # 295, the convergence angle to be set is calculated. From the absolute value of the difference between the image interval P and the reference image interval P ₀ and the sign, the magnitude and direction of the deviation of the image interval are obtained, and the convergence angle to be set is calculated based on the convergence angle set at that time. To be done. Next, at # 300, the number of drive pulses of the convergence and focus setting step motor of the three-dimensional camera is obtained. In the stereoscopic system of the present embodiment, the congestion of the HMD is set in accordance with the congestion of the three-dimensional camera, so # 305
Then, the number of driving pulses of the HMD convergence and diopter setting step motor is also obtained. Then, in step # 310, each step motor is driven in the appropriate direction by the calculated number of pulses, and 3
The congestion of the two-dimensional camera and the HMD is set at the same time.

If the permissible range ΔP of the image interval is too large, even if the congestion setting is inadequate, it is determined that the congestion setting is correct and the congestion is not reset. On the contrary, it is too small. And congestion will be set more frequently than necessary. Therefore, the permissible range ΔP is set to a large value within a range that does not cause any inconvenience in convergence and focus setting in actual shooting and image observation.

At step # 315, the activation request switch 3
The setting status of 01 is determined. If the switch 301 remains on, the process returns to # 255 and the control process is continued. If the switch 301 is turned off, it is required to stop the system operation. In # 320, the timer interrupt is prohibited, in # 325, the power supply is turned off, and then the process goes to sleep in # 330. To do.

If it is determined in step # 255 that the azimuth initialization request has been made, the process branches to # 405 to initialize the azimuth of the three-dimensional camera and the azimuth of the HMD. This is to match the orientation of the three-dimensional camera with the orientation of the head of the HMD wearer at the start of use of the stereoscopic system. This is for re-matching the two-dimensional camera and the HMD when they do not match each other.

First, in # 410, the interruption of the 1 msec timer for performing the orientation setting processing of the three-dimensional camera is prohibited. Then, in # 415, the three-dimensional camera is set horizontally, and # 420
Use to return the horizontal rotation to the reference position. After that, a buzzer sounds at # 425 to notify the HMD wearer that the three-dimensional camera has been set to the initial orientation. HMD wearers should look straight ahead and do not move their heads. In this state, # 430 detects the horizontal rotation speed of the HMD, and # 435 detects the vertical rotation speed. The rotation speed detection of the HMD in this stationary state continues for 10 seconds,
If it is determined that 10 seconds have passed in # 440, # 44
At 5, horizontal and vertical rotation speed averaging processing is performed. Also, the buzzer is turned off at # 450, and HM
The D wearer is released from the state in which the head is stationary.

The human head slightly oscillates even at rest, and the average of the angular velocities in the horizontal and vertical directions can be obtained by the above method. The average value thus obtained can be regarded as the output of the angular velocity sensor substantially when the head of the HMD wearer is stationary. Step # 455
Then, these values are used to set the DC reference output of the piezoelectric vibration gyro that is the angular velocity sensor. Next, in # 460, the direction parameters XH1 and YH1 of the HMD are initialized to 0,
The control proceeds to step # 260 described above.

FIG. 18 shows a 1 msec timer interrupt process for matching the orientation of the three-dimensional camera with the orientation of the HMD. In this stereoscopic system, the control processing shifts to step # 505 at a cycle of 1 msec and the following processing is performed.
First, in step # 510, the horizontal rotation speed of the HMD is detected by the horizontal angular velocity sensor, and in step # 515, the vertical rotation speed is detected by the vertical angular velocity sensor.
Next, in # 520, the horizontal direction XH1 and the vertical direction YH1 are calculated. Specifically, these calculations are performed according to the following two equations. XH1 = XH1 + (XAS-XAD), ka YH1 = YH1 + (YAS-YAD), ka

Here, XAS is the rotational speed in the horizontal direction detected this time, and XAD is the rotational speed in the horizontal direction at rest determined in # 445 of FIG. 17, that is, the horizontal DC reference output. Similarly, YAS is the vertical rotation speed detected this time, and YAD is the vertical rotation speed at rest, that is, the vertical DC reference output. Also, ka is a constant for converting a voltage representing an angular velocity into an angle. Right side first
The terms XH1 and YH1 are the horizontal and vertical azimuths set last time, respectively, and the second term (XAS- XA
D) ・ ka and (YAS-YAD) ・ ka are the amount of change in the horizontal and vertical angles from the previous time to the current time, respectively.

Next, the direction XH of the HMD thus obtained
1, YH1 and azimuth XH set by 3D camera
2, YH2 are compared, and the azimuth of the camera is set if necessary. In step # 525, it is determined whether or not the absolute value of the difference between the horizontal direction XH1 of the HMD and the horizontal direction XH2 of the three-dimensional camera is within a predetermined allowable range ΔX, and if it exceeds the allowable range ΔX # Branch to 535. If it is within the allowable range ΔX, the process proceeds to # 530 and the vertical direction is compared. When the absolute value of the difference between the HMD vertical direction YH1 and the three-dimensional camera vertical direction YH2 exceeds the predetermined allowable range ΔY, the process proceeds to # 535. If it is within the allowable range ΔY, it means that the azimuths of the HMD and the three-dimensional camera are the same in the horizontal direction and the vertical direction, and the interrupt processing ends in # 555.

If it is determined in steps # 525 and # 530 that the azimuth of the HMD and the three-dimensional camera has a deviation exceeding the allowable range in either the horizontal direction or the vertical direction,
The process proceeds to step # 535, and processing for matching the orientation of the three-dimensional camera with the orientation of the HMD is performed. In # 540, the driving pulse numbers of the step motor for setting the horizontal direction and the step motor for setting the vertical direction of the three-dimensional camera are calculated, and these step motors are driven in # 545 to set the orientation of the three-dimensional camera. In addition, the parameter XH2 and YH2 representing the orientation of the three-dimensional camera in # 550 is the parameter X representing the orientation of the HMD.
Replace with H1 and YH1. After that, the process proceeds to # 555 to end the interrupt process and restart the process performed before the interrupt.

The permissible ranges ΔX and ΔY of the displacement between the HMD azimuth and the three-dimensional camera azimuth are set to a level at which the observer does not feel uncomfortable even if the three-dimensional camera does not follow the movement of the head. do it. Also, the horizontal allowable range Δ
The allowable range ΔY in the vertical direction with respect to X may be set to the same value or different values.

By the interrupt processing described above, the head orientation of the HMD wearer is detected in a cycle of 1 msec, and when the deviation from the orientation of the three-dimensional camera becomes a predetermined value or more, the orientation of the three-dimensional camera becomes the head of the HMD wearer. Is set to match the direction of. If the deviation is within a predetermined value, the orientation of the 3D camera is not changed, so there is no unnecessary fine orientation setting of the 3D camera linked to an unintentional fine movement of the head, and a stable stereoscopic image with no discomfort can be obtained. It can be provided to an observer wearing the HMD.

In the stereoscopic system of the present embodiment configured and controlled as described above, the orientation of the three-dimensional camera for shooting is linked to the orientation of the HMD for display,
It is possible to automatically photograph and display the observation object desired by the observer. Moreover, the convergence of the three-dimensional camera, the focus setting, the convergence of the HMD, and the diopter setting are appropriately and automatically set according to the distance from the three-dimensional camera to the object.

[0132]

According to the stereoscopic system of the present invention,
Since the congestion of the three-dimensional camera is variably set, the stereoscopic viewable shooting distance is expanded. Further, the congestion of the three-dimensional camera is set according to the shooting distance so that the shooting target forms an image at substantially the center of each of the left and right image pickup devices. To be done. Therefore, the HMD wearer can observe the object of interest in front of him. Further, since the congestion setting of the HMD is made along with the congestion setting of the three-dimensional camera and the two congestions match, the left and right images that always give a natural sense of distance are presented to the observer.

Further, in the stereoscopic system of the present invention, the distance to the object to be photographed is detected and the congestion of the camera and the congestion of the HMD are set according to the detected distance, so that
It is not necessary to perform the congestion setting operation of the two-dimensional camera, and the observer need not perform the congestion setting operation of the HMD. Therefore, both photographing and observation are extremely easy.

According to the stereoscopic system of claim 2,
Since the three-dimensional camera is also set in focus in association with the congestion setting, it is possible to always take a sharp image of a target of interest. Further, since the foreground and the background other than the object to be photographed near the center of the image sensor are blurred, the HM
The image observer of D can obtain a feeling close to that of human vision.

In the stereoscopic system according to the third aspect, since the diopter of the HMD is set in association with the convergence setting, it is possible to observe a natural stereoscopic image that is closer to human vision. In addition, there is no fear that congestion and diopter will be set inconsistently, and the unfamiliar H
Even an MD user can always observe a natural stereoscopic image.

[Brief description of drawings]

FIG. 1 is a plan view showing a configuration of a three-dimensional camera when capturing an object at infinity.

FIG. 2 is a plan view showing a configuration of a three-dimensional camera when photographing a short-distance subject.

FIG. 3 is a plan view showing a convergence and focus setting mechanism of a three-dimensional camera when photographing an infinite object.

FIG. 4 is a plan view showing a convergence and focus setting mechanism of a three-dimensional camera when photographing a short-distance subject.

FIG. 5 is an exploded perspective view of a three-dimensional camera.

FIG. 6 is a perspective projection view of a three-dimensional camera.

FIG. 7 is a perspective side view showing a vertical direction setting mechanism of a three-dimensional camera.

FIG. 8 is a plan view showing a horizontal direction setting mechanism of the three-dimensional camera.

FIG. 9 is a plan view showing an infinitely distant subject showing the configuration of the HMD.

FIG. 10 is a plan view showing the configuration of an HMD when a short-distance subject is displayed.

FIG. 11 is a perspective projection view of a main part of the HMD.

FIG. 12 is a perspective view of an essential part showing a convergence and diopter setting mechanism of the HMD.

FIG. 13 is a block diagram showing functions of the stereoscopic system.

FIG. 14 is a block diagram showing an electric circuit of the stereoscopic system.

FIG. 15 is a flowchart showing a reset routine of the stereoscopic system.

FIG. 16 is a flowchart showing a startup routine of the stereoscopic system.

FIG. 17 is a flowchart showing a startup routine of the stereoscopic system.

FIG. 18 is a flowchart showing an interrupt routine of the stereoscopic system.

FIG. 19 is a diagram showing a relationship between a congestion setting of a three-dimensional camera and a left-right image interval.

FIG. 20 is a diagram showing a distance measuring area and left and right images of a three-dimensional camera.

FIG. 21 is a diagram showing diopter of HMD.

FIG. 22 is a diagram showing HMD congestion.

FIG. 23 is a diagram showing an outline of the configuration of a stereoscopic system.

FIG. 24 is a diagram showing a relationship between a convergence setting of a three-dimensional camera and fusion of left and right images.

FIG. 25 is a diagram showing a relationship between congestion settings of a 3D camera and an HMD.

FIG. 26 is a diagram showing diopter correction of an HMD.

FIG. 27 is a plan view showing the second configuration of the three-dimensional camera when photographing an infinite object.

FIG. 28 is a plan view showing a second configuration of the three-dimensional camera when photographing a short-distance subject.

FIG. 29 is a side view showing the rotation of the human head.

FIG. 30 is a front view showing the human eye width.

FIG. 31 is a side view showing the vertical rotation of the human head.

FIG. 32 is a diagram showing detection of HMD azimuth by an angular velocity sensor.

[Explanation of symbols]

1 3D camera 2 HMD 3 controller (distance detection means, control means) 101L left camera 101R right camera 102L left shooting lens 102R right shooting lens 103L left CCD sensor (left image sensor) 103R right CCD sensor (right image sensor) 107L, 107R pin (camera congestion setting means) 111L, 111R gear (camera congestion setting means) 131 base plate 132 step motor (camera congestion drive means) 138 fixed base plate 139 rotating base plate 144 horizontal sensor 146 step motor 148 step motor 171L left camera 171R Right camera 172L Left shooting lens 172R Right shooting lens 173L Left CCD sensor (left image sensor) 173R Right CCD sensor (right image sensor) 182L, 182R Arm (camera congestion setting means) 191 Base plate 192 Up motor (camera convergence driving means) 195 mover (camera convergence setting means) 196 arm (camera convergence setting means) 201L left virtual image projection device 201R right virtual image projection device 202L left image display device (left image display device) 202R right image display device (Right image display element) 203L Left eyepiece (Left eyepiece optical system) 203R Right eyepiece (Right eyepiece optical system) 212L, 212R Arm (HMD convergence setting means) 232 Step motor (HMD convergence driving means) 235 Mover (HMD) Convergence setting means) 238, 239 Angular velocity sensor

Claims (3)

[Claims] 1. A three-dimensional camera having a pair of left and right cameras equipped with a taking lens and an image pickup device for receiving light transmitted through the taking lens; an image display device; and eyepiece optics for guiding the image light of the image display device to an eye. It is composed of an HMD that has a pair of left and right virtual image projectors with a system, and displays the left and right images of the object photographed by the left and right cameras of the three-dimensional camera on the left and right image display elements of the HMD to display the left and right virtual images of the left and right display images. In the stereoscopic system given to the human eye, the camera convergence setting means for changing the direction of the left camera and the right camera to set the convergence of the three-dimensional camera, and changing the orientations of the left virtual image projection device and the right virtual image projection device to H
HMD congestion setting means for performing MD congestion setting, camera convergence driving means for giving driving force to the camera congestion setting means, HMD congestion driving means for giving driving force to the HMD congestion setting means, and light reception at the left and right image pickup devices Based on the distance to the object to be photographed detected from the distance between the left and right images of the object to be photographed, and the distance to the object to be photographed detected by the distance detector, the light from the object to be photographed is approximately at the center of the image sensor. And a control means for controlling the camera convergence drive means so as to form an image on the HMD and for controlling the HMD convergence drive means so that the HMD congestion matches the congestion of the three-dimensional camera. system. 2. The focus setting of the three-dimensional camera is performed by changing a distance between a photographing lens of each of the left and right cameras and an image sensor in association with a congestion setting by a camera congestion setting unit. Stereoscopic system. 3. The diopter setting of the HMD is performed by changing the distance between the image display element of the left and right virtual image projection device and the eyepiece optical system in conjunction with the convergence setting by the HMD convergence setting means. The stereoscopic system according to claim 1 or claim 2.