JP2949116B1 - Display device using reflection optical system - Google Patents

Abstract

Abstract: [PROBLEMS] Just by wearing a small and light eyeglass-like device,
Provided is a display device using a reflective optical system capable of presenting an omnidirectional image with high realism to a user. An image generation system and image reflection systems are provided.
Is fixed in the room, the user wears the glasses-type image reproducing device 60, the target image is emitted from the image generating system 30, and passes through an optical path composed of a plurality of reflecting optical systems 40 and 50.
An image is formed on the reproducing device 60 worn by the user. At this time, the position of the user's head in the room is measured by the wide area information detection camera 20, and in conjunction with this, the image reflection system control device 1
1, the reflection optical systems 40 and 50 are also controlled, and the optimum optical path is always maintained. The user's position and orientation information is also transmitted to the image generation device 13 at the same time, and by linking the projection image to the image, the space image in any arbitrary direction is presented.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a display device using a reflection optical system, and more particularly, to a display device using a reflection optical system which can be worn on a user's head to display a realistic image of the entire environment. About.

[0002]

2. Description of the Related Art Conventionally, a method using a head-mounted display (HMD) and a method using a plurality of large screens have been used to present an image of the entire surroundings to a user in order to realize a high sense of reality.

The HMD has a small liquid crystal display (LCD) placed in front of the user and presents image information to the user via an appropriate optical system. The HMD measures the direction of the line of sight by an appropriate means, and displays an image in conjunction with the direction of the line of sight, so that it is possible to present video information of a space substantially around the entire circumference of the user. In addition, there is an advantage that videos from independent viewpoints can be presented to a plurality of users. Such HMDs include, for example, Robert O. Fisher,
“Optics for Head Mounted Displays” Information Di
splay 7 & 8 described in 1994.

On the other hand, in the method using a large screen, one or a plurality of rear projection type projector systems are used.
There is a method of forming a screen wall around a user and presenting image information over a wide field of view around the user. CA
In a device called VE, a device has been proposed in which a large screen is arranged for a total of four surfaces: front, left, right, and bottom. For example, Masayuki Nakajima announced “CA
VE New Stereoscopic Image Space ”(Journal of the Institute of Electronics, Information and Communication Engineers, Vo
l. 80. No. 8 pp. 888-890 (199
7)).

[0005]

SUMMARY OF THE INVENTION However, HMD
Since all the components that make up the device are mounted on the HMD, the feeling of mounting is greatly impaired by the weight and the connected cables. Further, the viewing angle is determined by parameters of the LCD size and the lens diameter. However, widening the viewing angle involves a trade-off between weight and overall size.

In a method using a large screen, a user's viewpoint position is measured in the same manner as in the HMD, and an image corresponding to the viewpoint position is presented on each surface to generate a realistic surrounding image. Unlike the HMD, there is no need to mount any device other than a sensor for position measurement. However, even if a plurality of users try to use CAVE at the same time, they can only present a video from a single viewpoint, and other users who do not follow the viewpoint lack realism. Further, in order to use the rear projection type projector on each surface, a large space for installing the projector is required in addition to the space where the user observes the image.

Therefore, a main object of the present invention is to arrange a plurality of video generation systems and reflection systems in a room, and allow a user to experience a virtual space by wearing only a small and lightweight video reproduction system in the shape of glasses. It is an object of the present invention to provide a display device using such a reflection optical system.

[0008]

The invention according to claim 1 is
A display device using a reflection optical system for presenting a generated image to a user, the image generation unit for generating an image, and an image reflection unit for reflecting the generated image toward a user. A head position recognizing unit for recognizing a user's head position, a reflection control unit for controlling an image reflecting unit so that an image is reflected on the recognized user's head position, And a spectacle-type image reproducing means for reproducing the image reflected by the image reflecting means on the screen surface.

According to a second aspect of the present invention, the image reflecting means of the first aspect is provided in a plurality to reflect an image generated by the image generating means on a plurality of optical paths, and is provided at different positions, respectively. The means selects one of the plurality of optical paths based on the recognition result of the head position recognizing means so that the image is reflected in front of the user's image reproducing means.

In the invention according to claim 3, the image generating means according to claim 1 or 2 generates images from different viewpoints for presentation to a plurality of users, and the image reflecting means includes:
A plurality of paths are provided to reflect each image generated by the image generating means toward different users.

In the invention according to a fourth aspect, the image generating means according to any one of the first to third aspects generates a stereoscopic image using a binocular parallax image.

The invention according to claim 5 further includes a face image tracking means for tracking the face image of the user and measuring the facial expression.

[0013]

FIG. 1 is a block diagram showing the overall configuration of an embodiment of the present invention. In one embodiment of the present invention, an image generation system 30 and a plurality of image reflection systems 4 are installed in a room.
0 and 50 are arranged. An image of the virtual space to be displayed is generated by the image generation system 30, and a plurality of image reflection systems 40,
The light is guided to the glasses-type image reproducing device 60 worn by the user on the head via the optical path constituted by 50. The user can experience a virtual space synthesized by a computer from an arbitrary position in the installed room. A wide area information detection camera 20 is installed on the ceiling in the room, and a user detection camera 32 is provided on the image generation system 30, and the position and orientation of the user are constantly measured using an optical method.

More specifically, the system control system 1
Numeral 0 controls the entire system, and includes, for example, an image processing device and a computer. Here, the posture of the user's head in the room is measured by the head position / posture recognition device 12 based on the output from the wide area information detection camera 20, and the result is transmitted to other subsystems. To detect a user, for example, a state in which no user is present is captured and stored in advance, and when the user enters the apparatus, the difference between the video and the stored video is extracted to extract the position of the user. it can. The user's gaze direction is measured from the provisional optical path calculation result and the result of the user detection camera 32 described later.

The image reflection system controller 11 includes an image generation system 30
The optimum optical path between the user and the user is calculated in real time, and the image reflection systems 40 and 50 are controlled. The image generation device 13 generates an image of a space linked to the movement of the user based on information on the position of the head of the user and information on the distance to the user from the optical path calculation results of the image reflection systems 40 and 50.

The image generation system 30 projects an image generated by the image generation device 13 of the system control system 10 toward a user. Here, the projector includes a general projector 31 as a projection device, a focusing mechanism 33 for controlling an image forming position, and a user detection camera 32 for confirming the position and orientation of the user's face. The focusing mechanism 33 controls the composite lens mechanism by a motor, changes the projection image combining position according to the distance to the user, which changes from moment to moment, and controls the focal length so that all users can obtain an image without blur. I do. The user detection camera 32 is used to obtain feedback information for determining the line-of-sight direction and for fine adjustment. The user's face is always captured by the user detection camera 32, and is also used as a device for inputting a part of the user's facial expression to the system.

The image reflecting systems 40 and 50 are connected to the image generating system 30.
And an optical path for optically aligning the eyeglass-type image reproducing device 60 worn by the user. Image reflection system 40, 50
Is composed of a plurality of reflecting mirror units. Each unit has a structure in which a mirror is mounted on a head with a two-degree-of-freedom angle control device.
Is controlled in real time by the image reflection system control device 11. The optical path is not fixed, and the optical path is appropriately switched by a spare reflection system depending on the direction of the user's line of sight.

FIG. 2 is a view showing a glasses-type image reproducing apparatus, and FIG. 3 is a sectional view of an essential part along AA 'in FIG.

2 and 3, the spectacles type video reproducing apparatus 60 is constructed by attaching a diffusing screen surface 62 and a viewing lens 63 to a spectacles frame 61. For this reason, since there is no drive system or electric circuit as compared with the conventional HMD, there is no need for any cables that restrict the movement of the user, and it is possible to reduce the size and weight. For this reason, an image with a wider viewing angle than that of the conventional HMD can be presented by increasing the size of the diffusion surface.

FIG. 4 is a flowchart for explaining the operation of the embodiment of the present invention, and FIG. 5 is a diagram for explaining an optical path switching process for one user.

Next, a specific operation of the embodiment of the present invention will be described with reference to FIGS. In this embodiment, in order to select a temporary optimal optical path for the user,
The wide area information detection camera 20 measures the position of the user's head. When the position of the user's head is measured, the projector 33 is selected by the image generation system 30, and the system control system 10 calculates the temporary optimum optical path. Based on the calculation result, the image reflection system control device 11 controls the image reflection systems 40 and 50.

Thereafter, the user detection camera 32 focuses on the user's face, and the output of the user detection camera 32 confirms the direction of the user's head. If the user is facing the direction of the optical path formed by the image reflection systems 40 and 50, the selection of the projector 31 and the determination of the optimal optical path are determined. For example, if the user is facing as shown in FIG. 5A, the reflection systems A and B are selected in the optical path 1 to project an image in front of the user. But,
If the user is facing in the direction shown in FIG. 5B, the optical path 2 is switched between the reflection systems A and C. The same switching operation is performed when an obstacle enters the optical path. When the optimum optical path is selected in this way, the video signal generated by the video generation device 13 is input to the projector 31, and the video generated by the projector 31 is transmitted to the user via the video reflection systems 40 and 50. Glasses-type video playback device 6
It is projected on 0.

FIG. 6 is a diagram showing an example in which the same independent video is provided to three users. In the example shown in FIG. 6, images are projected individually by three projectors A, B, and C, and the image from projector A is reflected by reflection systems A and B, and is projected to user 1 through optical path 1. . The image from the projector B is guided to the optical path 3 by the reflection systems C and D, and is projected to the user 2. The image from the projector C is guided to the optical path 2 by the reflection systems E and F,
It is projected on.

FIG. 7 is a diagram showing an example of providing a stereoscopic viewing environment to one user. In the example shown in FIG. 7, an image having a binocular parallax image is projected by two projectors A and B. That is, the image of the projector A reaches the user via the optical path 2 via the reflection systems A and B, and the image from the projector B reaches the user via the optical path 1 via the reflection systems C and D. Thereby, the stereoscopic video is presented to the user's glasses-type video playback device 60. Further, if the image generation system and the reflection system are further provided, a plurality of users can simultaneously observe stereoscopic images from different viewpoints in the same room.

The following effects can be realized by extending the above-described embodiments. In other words, when a user uses the present invention, an ultra-light spectacle-type image reproducing device 60 that hardly burdens the user when using the present invention.
, And by simply entering the room where the device 60 is installed, the user can experience a virtual space developed all around. At this time,
The device on the user side does not require any power supply or signal line, is physically completely separated from the system, and can move freely in space.

In addition to being able to cope with a larger space than a CAVE system realizing the same function, the space required for installation is only a part like a room where a user is present, and almost no extra space is required.

Further, since a plurality of images can be independently presented, by adding an apparatus, a plurality of users can observe images in the same space corresponding to respective viewpoint positions accurately. Further, stereoscopic vision using binocular parallax can be realized.

Also, the user detection camera 32 used in the video generation system 30 can simultaneously measure the facial expression by tracking the face image of the user in the room. It can be expected that different images will be presented for each.

Further, the apparatus of the present invention is most effective for applications in which a plurality of users share a virtual experience in a relatively large space. For example, in an evacuation drill at the time of disaster, if this device is installed in a daily space and a flame can be synthesized by computer graphics to reproduce a realistic situation, the effect of the training can be enhanced. Further, if it is installed on the ground, it is also effective for a video presentation system of a ball game simulation system that requires virtual team play in a group.

[0030]

As described above, according to the present invention, the generated image is reflected toward the user, the position of the user's head is recognized, and the image is reflected at the recognized user's head position. By controlling the image reflection means in such a manner, the reflected image can be reproduced on the screen surface of the reproduction means worn by the user on the head. Therefore, the image reproducing means worn by the user is only the screen surface and the optical system for reproducing the reflected image, and it is possible to reduce the size and weight without requiring any cables that always restrict the movement of the user. Becomes For this reason, by increasing the size of the diffusion surface of the video reproduction system, it is possible to present a video with a wider viewing angle than the conventional HMD. Further, by providing a plurality of image reflection means, a plurality of optical paths can be selected, and CAs used for similar purposes are used.
It can accommodate a wider space than the VE system, and the space required for installation is only the room where the user is located, and does not require an extra space. Furthermore, if two video generation systems are used, it is possible to independently present a video considering binocular disparity to both eyes of the user, and to present a stereoscopic video. Furthermore, if an image generation system is added, a plurality of users can simultaneously observe images from different viewpoints in the same room.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an overall configuration of an embodiment of the present invention.

FIG. 2 is a diagram illustrating an example of a glasses-type video playback device.

FIG. 3 is a line AA ′ of the glasses-type image reproducing apparatus shown in FIG. 2;
FIG.

FIG. 4 is a flowchart for explaining the operation of the embodiment of the present invention;

FIG. 5 is a diagram for explaining an optical path switching process for one user.

FIG. 6 is a diagram illustrating an example in which the same independent video is provided to three users.

FIG. 7 is a diagram illustrating an example of providing a stereoscopic viewing environment to one user.

[Explanation of symbols]

 Reference Signs List 10 system control system 11 image reflection system control device 12 head position and orientation recognition device 13 image generation device 20 wide area information detection camera 30 image generation system 31 projector 32 user detection camera 33 focusing mechanism 40, 50 image reflection system 60 glasses-type image Reproduction device 61 Eyeglass frame 62 Diffusing screen surface 63 Peep lens

────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Tsutomu Miyazato 5 Sanraya, Daiya, Seika-cho, Soraku-cho, Kyoto Prefecture, Japan A / T Intelligence and Video Research Laboratories Co., Ltd. (58) Field surveyed (Int.Cl . ^6, DB name) G02B 27/02 H04N 5/64 511 H04N 5/74 H04N 13/04 G09G 5/36 510

Claims (5)

(57) [Claims] 1. A display device using a reflection optical system for presenting a generated video to a user, comprising: a video generation unit for generating the video; and a user generating a video generated by the video generation unit. Image reflecting means for reflecting toward the head, head position recognizing means for recognizing the user's head position, and the image being reflected on the user's head position recognized by the head position recognizing means. Reflection control means for controlling the image reflection means, and glasses-type image reproduction means for the user to wear on the head and reproduce the image reflected by the image reflection means on its screen surface. Also, a display device using a reflection optical system. 2. The image reflection means is provided in a plurality to reflect an image generated by the image generation means on a plurality of optical paths, and each of the image reflection means is provided at a different position. 2. The reflection optics according to claim 1, wherein one of the plurality of optical paths is selected based on a recognition result of the position recognition unit so that an image is reflected in front of the image reproduction unit of the user. Display device using the system. 3. The image generating means generates images from different viewpoints for presentation to a plurality of users, and the image reflecting means transmits each image generated by the image generating means to a different user. The display device using the reflection optical system according to claim 1, wherein the display device is provided on a plurality of paths for reflecting light toward the light source. 4. The display device using a reflection optical system according to claim 1, wherein the image generation unit generates a stereoscopic image using a binocular parallax image. 5. The reflection optical system according to claim 1, further comprising a face image tracking means for tracking a face image of the user and measuring a facial expression thereof. Display device used.