JP2003141573A - Immersive display device and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an immersive display device capable of maintaining immersive feeling, and reducing asthenopia and VR inebriety when working by using two-dimensional objects of a word processor document, a drawing and a photograph or the like in a virtual world. SOLUTION: This immersive display device which displays a three-dimensional virtual world image generated by a computer in binocular vision so as to surround an user is provided with a synthetic information display part displaying the image to which a two-dimensional image and a three-dimensional virtual world image are synthesized, a three-dimensional information display part displaying the three-dimensional virtual world image, a three-dimensional information generating part generating the three-dimensional virtual world image based on the view position of the user, and an information synthesis part synthesizing the two-dimensional image and the three-dimensional virtual world image based on the position of the synthetic information display part and generating the image displayed on the synthetic information display part.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention can stereoscopically view a virtual world generated by three-dimensional computer graphics or the like, and a virtual real world photographed by using a stereo camera or the like so as to surround a user. The present invention relates to an immersive display device and method that provide a high immersive feeling when presented in a form.

[0002]

2. Description of the Related Art A virtual world created by three-dimensional computer graphics or a virtual world taken by using a stereo camera is displayed in a stereoscopic view to provide a high immersive feeling. There is a technique called HMD or IPT as a device that does this. HMD (Head Mo
The "undisplayed display" is a general term for glasses-type displays that are worn on the user's head. In combination with a magnetic sensor, etc., the
By smoothly switching the displayed images, it is possible to give the user a high sense of immersion. However, it has problems such as low resolution of display and narrow viewing angle.

On the other hand, IPT is an Immersive
It is an abbreviation for Projection Technology and is a general term for devices that project an image on a screen arranged so as to surround the user.

The pioneer of IPT, and the most typical example is CAVE (CAV) developed at the University of Illinois.
E Automatic Virtual Envir
There is an on-ment).

The CAVE uses 2.5 to 3 m square screens on three to four sides and installs them so as to surround the user. If the screen has three screens, it is arranged on the front surface, one of the left and right side surfaces, and the floor surface. A three-dimensional projector projects a stereoscopic image on the floor screen from above and on the other screens from behind. A position sensor with 6 degrees of freedom is attached to the user's head, and a stereoscopic image viewed from the user's viewpoint position is displayed on the screen. By wearing glasses capable of stereoscopic viewing and standing in CAVE, the user is surrounded by a stereoscopic image with a wide field of view and can obtain a highly immersive sensation.

For details of CAVE, refer to Reference 1 (Caroli
na Cruz-Neira, Daniel J. Sandinand Thomas A. DeFan
ti, "Surround-Screen Projection-Based Virtual Reali
ty: The Design and Implementation of the CAVE ", In
Proceedings of the 20th annual conference on Compu
ter graphics, 1993, Pages 135-142).

Stereoscopic vision in HMD and IPT is realized by displaying different images with parallax for each of the right and left eyes based on the principle of binocular parallax.

The HMD is realized by disposing separate displays on the front surfaces of the right eye and the left eye and displaying images suitable for the respective displays.

In the case of IPT, there are a so-called liquid crystal shutter system and a system using the principle of polarization. The liquid crystal shutter system alternately switches the images for the right eye and the image for the left eye at high speed and displays them on the screen. The left and right images are separated by using liquid crystal shutter glasses that alternately open and close the left and right shutters in synchronization with the display in response to a synchronization signal from an infrared emitter. In the method using polarized light, the left and right images are separated from each other by applying different polarized light to the left and right images and wearing polarized glasses corresponding to the polarized light.

[0010]

The conventional immersive display device is mainly intended to experience a three-dimensional virtual world. So, in the virtual world, for example, word processing documents,
When displaying a two-dimensional object such as a drawing or a photograph, it is necessary to display the object using stereoscopic vision.

In stereoscopic vision, the image for the right eye and the image for the left eye are displayed with a parallax to give the user a feeling that the image is projected from the screen. At this time, it is natural that the vergence of the eyes is matched with the three-dimensional object projected from the screen. However, since the actual image is displayed on the screen, it is natural to focus the eye on the screen. This phenomenon is called "congestion / regulation inconsistency". Convergence and accommodation are not contradictory in nature, so the rotation and focusing of the human eye are usually linked. Therefore, when an image in which the vergence stimulus and the accommodation stimulus are inconsistent, such as a stereoscopic image presented on the immersive display device, a person feels uncomfortable even though he is unconscious. This is considered to be one of the causes of fatigue for stereoscopic vision.

As described above, the stereoscopic image due to the binocular parallax gives a user a feeling of fatigue, which makes it difficult to use for a long time. In particular, when a word processing document or the like is presented near the user, there is a problem that binocular parallax increases and eye strain further increases.

Further, in the immersive display device, the image is dynamically displayed so that the correct image can be seen from the viewpoint position of the user as the user moves, but the reaction delay of the sensor for detecting the viewpoint position and direction and the display image. Because of the time it takes to generate
Generally, the display of video is delayed with respect to the actual movement of the user. The difference between the user's movement and the display is "VR (Virtual Realit), which is similar to motion sickness.
y) It is one of the factors that cause a symptom called "sickness." Such a symptom becomes more problematic when attention is paid to detailed information such as a word processing document than when observing the surrounding environment image.

In order to solve these problems, it is conceivable to bring a display device for displaying a two-dimensional object into an immersive display device. In this case, depending on the brought-in two-dimensional display device, Since a part of the immersive display device is shielded from the user's viewpoint, there is a problem that a three-dimensional virtual object in the virtual world existing at a position closer to the user's viewpoint than the two-dimensional display device is not displayed.

A first object of the present invention is to maintain the immersive feeling while working with a two-dimensional object such as a word processing document, a drawing, and a photograph in a virtual world, while maintaining eyestrain and eye strain. An object is to provide an immersive display device that reduces VR sickness.

A second object of the present invention is to allow a user to use an application used in a daily two-dimensional display in a virtual world without impairing the immersive feeling in the virtual world.

[0017]

In order to achieve the above object, an immersive display device for displaying a computer-generated three-dimensional virtual world image of the present invention in a stereoscopically visible manner so as to surround the user is provided by a user. A viewpoint position detection unit that detects the viewpoint position of the, a combined information display unit that displays an image in which a two-dimensional image and a three-dimensional virtual world image are combined, and a three-dimensional information display unit that displays a three-dimensional virtual world image, A two-dimensional information generation unit that generates a two-dimensional image, a three-dimensional information generation unit that generates a three-dimensional virtual world image based on the viewpoint position of the user obtained by the viewpoint position detection unit, and the combined information display unit. Based on the position, the 2D image generated by the 2D information generation unit and the 3D virtual world image generated by the 3D information generation unit are combined and displayed on the combined information display unit. And having an information combining unit for generating an image, a.

Further, a composite information display position detection unit for detecting the position of the composite information display unit is provided, and the 3D virtual world image is generated in the 3D information generation unit, and the 2D in the information combination unit. When synthesizing the 3D virtual world image with the 2D image generated by the information generator,
The position obtained by the combined information display position detection unit is used as the position of the combined information display unit.

Further, the three-dimensional information generation unit is characterized by generating a three-dimensional virtual world image projected on the floor surface by making the portion shielded by the combined information display unit a black image.

[0020]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a schematic view of a three-dimensional information display section and a composite information display section showing an embodiment of the present invention.

Reference numeral 101 is a right side screen projector, 102 is a floor side screen projector, 103 is a left side screen projector, 104 is a right side screen mirror, 105 is a floor side screen mirror, 1
Reference numeral 06 is a left side screen mirror, 107 is a left side screen, 108 is a floor screen, 109 is a left side screen, 110 is a front screen, and 111 is a composite information display section.

At the back of the front screen 110, a mirror 112 for the front screen and a projector 113 are installed like the left and right floor surfaces. Mirrors 104, 105,
Reference numerals 106 and 112 are installed in a narrow space to secure the projection distances of the projectors 101, 102, 103 and 113, respectively. When a sufficiently wide space can be secured, the projector directly projects on the screen. It becomes unnecessary by doing. Projectors 101, 102,
103, 113, mirrors 104, 105, 106, 11
2 and the screens 107, 108, 109 and 110 form a three-dimensional information display unit.

FIG. 2 is a block diagram showing an embodiment of the present invention.

201 is a floor screen projector,
Reference numeral 202 is a front screen projector, 203 is a floor screen, 204 is a front screen, 205 is a composite information display unit, 206 is a composite information display position detection unit for detecting the position of the composite information display unit, 207 is a user, 208
Is a viewpoint position detection unit for detecting the viewpoint position of the user,
209 and 210 are computers, 211 is a three-dimensional information generation unit,
Reference numeral 212 is an information composition unit, 213 is a three-dimensional virtual world model stored in the computer 209, 214 is a two-dimensional information generation unit, and 215 is a two-dimensional document stored in the computer 210. Although the configuration related to the left and right side screens is omitted for simplification, the configuration and the operating principle are the same as those of the front screen. Further, in the above description, the position of the composite information display position detection unit 205 is detected by the composite information display position detection unit 206, but the composite information display position detection unit 206 is not provided and the composite information display unit 205 is installed at a predetermined position. Good as a matter.

The viewpoint information of the user 207 is acquired by the viewpoint position detection unit 208 and sent to the computer 209. Similarly, the position / orientation information of the composite information display unit 205 is acquired by the composite information display position detection unit 206, and the computer 209
Sent to. A magnetic sensor or the like can be used as the viewpoint position detection unit 208 and the combined information display position detection unit 206.

The computer 209 is a three-dimensional information generator 211.
In, a virtual world image viewed from the viewpoint position of the user 207 is generated from the three-dimensional virtual world model 213 based on the acquired viewpoint information of the user 207. At this time, since the three-dimensional virtual world image projected by the floor screen projector 201 is shielded by the combined information display unit 205, the shielded portion is generated as a blackout image. This is to prevent the image displayed on the combined image display unit 205 from being difficult to see by projecting the three-dimensional image from the floor screen projector 201 onto the combined image display unit 205.

The portion shielded by the composite information display unit 205 can be easily calculated from the position and orientation of the composite information display unit 205 acquired by the composite information display position detection unit 206. The three-dimensional information generation unit 211 is the computer 2
Program that runs on 09, 3D virtual world model 21
3 can be configured as three-dimensional computer graphics data stored in the computer 209.

The generated three-dimensional virtual world image is sent to the floor screen projector 201 and the front screen projector 202, and the floor screen 20 is respectively received.
3. Projected on the front screen 204. The same applies to both the left and right side faces.

On the other hand, the computer 210 causes the two-dimensional information generation unit 214 to generate a two-dimensional image from the two-dimensional document 215. As the two-dimensional information generation unit 214 and the two-dimensional document 215, a combination of word processing software and its document file, an image viewer and an image file, or the like can be used.

The generated two-dimensional image is sent to the computer 209. In the computer 209, the user 2 acquired by the viewpoint position detection unit 208 in the information synthesis unit 212
07 viewpoint information and the position / orientation information of the combined information display unit 205 obtained by the combined information display position detection unit 206, the three-dimensional virtual world viewed from the viewpoint position of the user 207 to be displayed on the combined information display unit 205. The image is generated from the three-dimensional virtual world model 213 and is combined with the two-dimensional image sent from the computer 210. At this time, the 2D image is combined as if it exists at the position of the combined information display unit 205 in the 3D virtual world.

The information synthesizing unit 212 can be configured as a program that operates on the computer 209.

The combined image is sent to the combined information display unit 205 and displayed. A liquid crystal display provided with a lenticular lens for stereoscopic viewing, or a combination of a projector and a screen for projection from the top or bottom can be used as the combined information display unit. With such a configuration, it becomes possible to display a three-dimensional virtual object existing at a position closer to the user's viewpoint than the combined information display unit 205 without being blocked by the combined information display unit 205. For example, when a three-dimensional character flying in the virtual world passes over the composite information display unit,
It is possible to minimize interruptions in character display and maintain the user's immersive feeling in the virtual world.

Here, the three-dimensional information generation unit 211 is used here.
And a computer on which the information synthesizing unit 212 operates is a computer 209,
Although the computer in which the two-dimensional information generation unit operates is configured as a computer separate from the computer 210 has been shown, it is possible to configure all of them by a single computer.

FIG. 3 is a three-view drawing for explaining calculation of a portion shielded by the composite information display portion when a three-dimensional image is generated in the three-dimensional information generation portion.

201 is a floor screen projector,
203 is a floor screen, 205 is a composite information display unit, 2
Reference numeral 06 is a composite information display position detection unit.

Here, the floor screen projector 2
01 is a point light source, and the position of the light source is the origin. If the shape of the combined information display unit is simplified to a rectangle, if the length of each side of the combined information display unit 205 and the attachment position of the combined information display position detecting unit 206 are known, the combined information display position detecting unit 206 is known. From the obtained position / orientation information, the coordinates of each vertex of the composite information display unit 205 can be obtained.
Once the coordinates of each vertex are obtained, the floor screen 2
By obtaining the distance to 03 in advance, the projection point of each vertex on the floor screen 203 can be obtained by the similarity calculation. By connecting the respective vertices on the floor screen 203 with straight lines, the portion shielded by the combined information display unit 205 can be calculated.

[0038]

According to the present invention, in the virtual world,
Even when working with two-dimensional objects such as word processing documents, drawings, and photographs, the two-dimensional information is displayed as a two-dimensional image that does not require stereoscopic vision in the composite information display section, so you can see the two-dimensional information. In that case, theoretically, eye strain due to convergence / adjustment contradiction does not occur. In addition, since the two-dimensional information can be independent of the three-dimensional virtual space, the display shift due to the movement of the user's viewpoint does not occur.
I don't get sick. In addition to the two-dimensional information, the composite information display unit also displays a three-dimensional virtual object that is located closer to the user's viewpoint than the composite information display unit. It is possible to reduce natural breaks and maintain the user's immersive feeling in the three-dimensional virtual world.

Therefore, even when working with a two-dimensional object such as a word processing document, drawing, or photograph in the virtual world, the eyestrain is maintained while maintaining the user's immersive feeling in the three-dimensional virtual world. And VR sickness can be reduced.
Further, it becomes possible for the user to use the application used in the daily two-dimensional display in the virtual world without impairing the immersive feeling in the virtual world.

[Brief description of drawings]

FIG. 1 is a schematic view of a three-dimensional information display section and a composite information display section showing an embodiment of the present invention.

FIG. 2 is a configuration diagram showing an embodiment of the present invention.

FIG. 3 is a trihedral view for explaining calculation of a portion shielded by a combined information display unit.

[Explanation of symbols]

101 ... Right side screen projector, 102 ... Floor screen projector, 103 ... Left side screen projector, 104 ... Right side screen mirror, 105 ... Floor screen mirror, 106 ... Left side screen mirror, 107 ... Left side screen, 10
8 ... Floor screen, 109 ... Left side screen, 11
0 ... Front screen, 111 ... Composite information display unit, 201
... Floor screen projector, 202 ... Front screen projector, 203 ... Floor screen, 204
... front screen, 205 ... composite information display section, 206 ...
Composite information display position detection unit, 207 ... User, 208 ... Viewpoint position detection unit, 209, 210 ... Computer, 211 ... Three-dimensional information generation unit, 212 ... Information synthesis unit, 213 ... Three-dimensional virtual world model, 214 ... Two-dimensional Information generator 215 ... Two-dimensional document.

Continued front page    (72) Inventor Satoshi Ishibashi             2-3-1, Otemachi, Chiyoda-ku, Tokyo             Inside Telegraph and Telephone Corporation F-term (reference) 2H059 AA35 AA38                 5B050 EA12 EA19 EA27 FA02 FA06                 5C061 AA02 AA20 AA23 AB01 AB08                       AB14 AB16 AB20 AB21                 5C082 AA01 BA02 BA12 BA26 BA46                       BB25 CA55 CA81 DA42 MM02                       MM09

Claims (6)

[Claims] 1. An immersive display device for displaying a computer-generated three-dimensional virtual world image in a stereoscopically visible manner so as to surround the user, and a viewpoint position detection unit for detecting the viewpoint position of the user. , A composite information display unit for displaying an image in which a 2D image and a 3D virtual world image are combined, a 3D information display unit for displaying a 3D virtual world image, and a 2D information generation unit for generating a 2D image A three-dimensional information generation unit that generates a three-dimensional virtual world image based on the user's viewpoint position obtained by the viewpoint position detection unit; and a two-dimensional information generation unit based on the position of the composite information display unit. Combining the generated two-dimensional image and the three-dimensional virtual world image generated by the three-dimensional information generation unit,
An immersive display device, comprising: an information synthesizing unit that generates an image displayed on the synthetic information display unit. 2. A composite information display position detection unit for detecting the position of the composite information display unit, wherein the three-dimensional virtual world image is generated by the three-dimensional information generation unit and the two-dimensional image is generated by the information combination unit. When the 3D virtual world image is combined with the 2D image generated by the information generation unit, the position obtained by the combined information display position detection unit is used as the position of the combined information display unit. The immersive display device according to claim 1. 3. The three-dimensional information generation unit generates a three-dimensional virtual world image projected on the floor surface by using a blackout image as a portion shielded by the combined information display unit. Alternatively, the immersive display device according to item 2. 4. An immersive display method for displaying a computer-generated three-dimensional virtual world image in a stereoscopically visible manner surrounding a user, the user's viewpoint position being detected, and the detected user being detected. The 3D virtual world image is generated based on the viewpoint position of the 3D virtual world image, the generated 3D virtual world image is displayed on the 3D information display unit, the 2D image is generated, and the 2D image and the 3D virtual world image are displayed. The 2D image and the 3D virtual world image are combined based on the position of the combined information display unit that displays the combined image, and the image displayed on the combined information display unit is generated. An immersive display method characterized by displaying on an information display section. 5. The position of the composite information display unit is detected by a composite information display position detection unit, the 3D virtual world image is generated in the 3D information generation unit, and the 3D virtual world image is added to the 2D image. The immersive display method according to claim 4, wherein the detected position of the combined information display unit is used as the position of the combined information display unit when combining. 6. The three-dimensional virtual world image projected on the floor surface is generated by using a blackout image as a portion shielded by the composite information display unit.
Immersive display method described.