JP4052357B2 - Virtual environment experience display device - Google Patents

Description

  The present invention relates to a virtual environment experience display device that allows a virtual environment to be experienced with a wide viewing angle image.

  As a 3D image creation method, a 3D image can be obtained by displaying the right-eye and left-eye images only for the corresponding eyes, so in the past, an anaglyph-based 3D image display example using red-blue cellophane glasses and so on. Stereoscopic video display can be realized by display on a television screen, but if the area where the video is projected is small, the stereoscopic effect is remarkably impaired, so that a larger video display screen is more effective. For this reason, at the Expo (Tsukuba Expo, Osaka Expo), stereoscopic images were displayed on a huge flat screen of several hundred inches. In addition, a hemispherical screen has also appeared (IMAX Co., Ltd.) due to the fact that the peripheral screen is more difficult with a flat screen. However, all of them display images that have been previously photographed on film, and are not interactive for experiencing a virtual environment.

As interactive 3D image display, there is a real-time computer generated image displayed on a screen, and examples of combinations of multiple screens include arch screens (3 screens) and CAVE (5 screens). and so on. The arch-type screen is a cylindrical screen and can achieve a wide viewing angle of 180 degrees or more in the horizontal direction, but the distance from the projection means increases in the vertical direction at the top of the screen. As a result, problems such as distortion and color unevenness occur, so about half of the viewing angle of the experience person is covered. In addition, since the image creation means has no distortion correction function and the image is projected by the image creation method to a flat shape, it is regarded as a pseudo-planar screen, and the distortion of the image in the peripheral part impairs the sense of immersion and stereoscopic effect. It was. CAVE is a display means that surrounds the user with five cube-shaped screens as a screen, and can provide a high level of three-dimensionality and immersiveness, but due to the geometry of the cube, it impairs continuity at the joints of the screen. There are drawbacks. CAVE has a display correction function to make the display between screens continuous, but it is difficult to smoothly connect the images at the boundary surfaces of the screen surfaces, and the straight line bends at the boundary surfaces just by slightly shifting the viewpoint position. A visible phenomenon appears.
Japanese Patent Laid-Open No. 08-334845

  By covering the viewer's field of view with video, displaying a high-resolution stereoscopic video and displaying its accurate scale, the user can feel a deep immersive feeling as if he had entered the displayed world. It has been reported that the human viewing angle is physiologically maximum 210 degrees in the horizontal direction and 110 degrees in the vertical direction, so it is ideal to cover this entire field of view with images, but this is not possible with a flat screen. Is possible. Therefore, it is necessary to bend the screen to surround the experience person. Such an ideal shape is a sphere.

  Real-time and interactive video is created by a computer, but the computer video is usually created in a planar shape. When this image is displayed on the spherical screen as it is, a distorted image is obtained as shown in FIG. Therefore, as shown in FIG. 2B, it is necessary to distort the image by the computer in advance so that the distortion is eliminated when displayed on the spherical screen. This is called a distortion correction function.

  Further, when the display is performed by one projection unit, the video is enlarged, so that both the resolution and the luminance of the video are lowered. By using multiple projection means, high resolution and brightness can be maintained, but in this case, an image divided by the number of projection means is created, and the boundary portion of each image at the time of composition is smoothly connected. Video creation is required.

  The present invention has been made in view of these points, and an object of the present invention is to realize a virtual environment experience display device that can greatly improve the immersive feeling.

According to the first aspect of the present invention, in order to solve the above-mentioned problem, as shown in FIG. 1, a spherical screen 1 having a concave surface facing the experience person and a wide viewing angle, and the image as a planar image. The image creating means 2 to be created, the mapping process for pasting the image on the plane created by the image creating means 2 to a spherical shape matching the screen 1, and the image pasted on the spherical shape are projected again on the plane By performing the processing, distortion correction means 3 for distorting the image in advance so that the distortion is eliminated when displayed on the screen 1, and projection means for projecting the distortion-corrected image onto the screen 1 4, a camera for detecting the position of the experience person is arranged above the experience apparatus, the position of the experience person is detected from an image photographed by the camera, and the position of the experience person is distorted. Supplement By changing the parameters dynamically, it is characterized in that always has the ability to perform optimal correction.

Here, it is preferable that a plurality of the projecting units 4 are provided, and the distortion correcting unit 3 performs the distortion correction so that the joints of the plurality of images divided and projected onto the screen 1 by each projecting unit 4 are smoothly connected. Item 2). In this case, it is preferable that the plurality of projection means 4 be arranged at substantially equal distances from the center of the spherical screen 1 (Claim 4). Viewing angle of the screen 1 is preferably the same extent as the viewing angle of a human (claim 3).

In addition, since projectors capable of realizing a wide angle of view are generally expensive, if a projector to be used cannot achieve the angle of view required for the display device of the present invention, a convex mirror is placed on the optical path. To widen the angle of view (claim 5 ). In this case, distortion correction is indispensable, and if there is no distortion correction, an extremely distorted image can be seen by the experience person. However, if there is a distortion correction function as described in claim 1 of the present invention, a natural image can be shown to the experience person.

In the case of performing distortion correction, strict correction cannot be performed unless information on where the viewpoint viewed by the experience person is used. Therefore, a means for measuring the position of the experience person is required. In general, electromagnetic induction type or ultrasonic sensors are used to detect the position of the experience person. However, these require the experience person to wear the sensor, so the operability is good. Absent. Therefore, as described in claim 1 , an image from above of the experience person is photographed using a CCD camera, and the position of the experience person is calculated and obtained by the difference between this and the image when there is no experience person. If the distortion of the image is corrected using the obtained location information of the experience person, the experience object can be achieved without having a special sensor.

  According to the first aspect of the present invention, an image with distortion correction corresponding to a spherical surface is created and projected onto a spherical screen with a wide viewing angle so that the user can view the image. Has the effect of providing a high level of immersion.

According to the invention of claim 2 , by providing a plurality of projection means, there is an effect that the resolution and brightness of the video can be increased. In addition, since distortion correction is performed so that the joints when a plurality of images are combined are smoothly connected, the sense of immersion is not impaired even if the viewpoint moves slightly.

According to the invention of claim 4 , since the projection means is arranged at substantially equal distance from the center of the spherical screen, the image display capability of the plurality of projection means can be made substantially equal, Variation in luminance can be suppressed.

According to the third aspect of the present invention, since the viewing angle of the screen is set to the same level as the human viewing angle, the feeling of immersion can be improved.

According to the fifth aspect of the present invention, since the image projected by the plurality of projection means is once reflected by the convex mirror so as to project the wide viewing angle image on the spherical screen, it is generally expensive. There is an effect that a virtual environment experience device with a wide viewing angle can be realized without using a projector with an angle of view.

According to the first aspect of the present invention, the camera for detecting the position of the experience person is arranged above the experience apparatus, the position of the experience person is detected from an image taken by the camera, and the detected position of the experience person is set. On the other hand, since the distortion correction parameters are dynamically changed, there is an effect that distortion correction in an optimal state can always be realized even for a moving experience person.

  The overall configuration of the virtual environment experience display device of the present invention is shown in FIG. In FIG. 1, a screen 1 is a spherical wide viewing angle screen having a curvature in both a horizontal direction and a vertical direction, as shown in FIGS. 3 and 4, and a projection means 4 composed of six projectors. Thus, an image with a wide viewing angle is projected and displayed. The video creation means 2 is made up of a graphics computer and creates an interactive stereoscopic video in real time. The distortion correction means 3 is software having a distortion correction function on the computer, and predistorts the video image created by the video image creation means 2 as shown in FIG. Moreover, it has not only a correction for the spherical surface but also a distortion correction function that smoothly connects the joints of the images so that one whole image is composed of six separate screens. The video signal from each screen is passed to each of the six projectors, and the video is displayed on the spherical wide viewing angle screen 1.

  In the video creation means 2, in order to display a stereoscopic video, left and right videos are created separately using a graphics computer. In order to see this stereoscopically, as described in the prior art, the left and right images may be displayed on the eyes. Therefore, the left and right images created by the computer are switched and displayed at such a high speed that humans cannot recognize them. Then, a stereoscopic image synchronization signal for switching is sent from the computer to the stereoscopic glasses 5 by infrared rays, and the stereoscopic glasses 5 switch the left and right liquid crystal shutters at high speed in synchronization with the images. , And when the image of the left eye is reflected, it is operated to hide the right eye. In order to send a liquid crystal shutter control signal from the computer to the stereoscopic glasses 5, a plurality of infrared emitter devices 6 are arranged in the vicinity of the experience person.

  Furthermore, since the video is created in real time, it can be moved interactively. This movement is controlled by an operation means 7 such as a mouse, a keyboard, or a joystick. As a result, the experience person can experience the virtual environment from various viewpoints while freely moving in the displayed virtual environment. Also, as shown in FIG. 4, the screen 1 is larger than the experience person M, and the projection means 4 is suspended from the ceiling by the support member 8 so as not to disturb the view of the experience person M. Multiple users can experience the virtual environment at the same time, and can be used for planning and design of urban development and disaster prevention plans, for example.

  Conventionally, there is a case where the distortion correction function is realized by optical processing using a fisheye lens or the like, but in this function, it is realized by software. The superiority will be described. When a distortion correction function is realized with a lens, a lens suitable for the display system must be created in advance, and an image must be created and displayed using the lens. For this reason, it cannot cope with a change in the size or shape of the display system. On the other hand, if a distortion correction function is realized by software, it can be handled only by setting parameters in accordance with the display system. In addition, when the image is large enough to cover the field of view of the user with one projection means, the resolution and brightness of the displayed image decrease in inverse proportion to its display area, and the original purpose of stereoscopic effect and immersion is reduced. There arises a problem that it becomes impossible to give a feeling to the experienced person. When the distortion correction function by software is used, correction for smoothly connecting displays by a plurality of projection means can be performed, and an image can be composed of six areas as in the embodiment. However, it is impossible to smoothly connect the images with the display using the optical system.

  The principle of realizing the distortion correction function by software is as shown in FIG. With current computers, all images are created on a flat surface. Therefore, an image is once created on a flat surface, and is pasted in a spherical shape that matches the screen without displaying it as it is. This is called mapping processing. Furthermore, the process which projects it on a plane is performed, and it displays as an image | video. That is, the distortion correction function is realized by performing the process of creating the video twice.

6 and 7 show an embodiment of the fifth aspect of the present invention. 6 is a view of the apparatus from above, and FIG. 7 is a view of the apparatus from the side. About the screen 1, the horizontal cross section and the vertical cross section are shown, respectively. By placing the convex mirror 9 on the optical path of the projector 4 as shown in the figure, an image with a wide viewing angle can be projected onto the spherical screen 1. In the example of FIGS. 3 and 4, a wide viewing angle image is realized using a total of six projectors 4, whereas in this example, the horizontal and vertical projection angles are widened. A wide viewing angle image is realized by a total of two projectors 4.

In this way, when the image is widened using the convex mirror 9, correction of the distortion accompanying it is indispensable, but if the distortion generated by the convex mirror 9 is corrected simultaneously with the distortion correction described in FIG. It is possible to show natural images to the experienced person. In the case of performing distortion correction, strict correction cannot be performed unless information on where the viewpoint viewed by the experience person is used. Therefore, a means for measuring the position of the experience person is required. In general, electromagnetic induction type or ultrasonic sensors are used to detect the position of the experience person. However, these require the experience person to wear the sensor, so the operability is good. Absent. Therefore, as described in claim 1 , an image from above of the experience person is photographed using a CCD camera, and the position of the experience person is calculated and obtained by the difference between this and the image when there is no experience person. If the distortion of the image is corrected by using the position information of the experienced user, it is not necessary for the experienced person to have a special sensor. As a result, the obtained position coordinates are input to a computer that performs distortion correction, and by dynamically correcting the distortion, it is possible to always perform distortion correction in an optimal state even for an experienced user. .

It is a block diagram which shows the whole structure of the apparatus of this invention. It is explanatory drawing which compares and shows the example of a display by the conventional apparatus and the apparatus of this invention. It is a cross-sectional view which shows arrangement | positioning of the screen and projection means which are used for the apparatus of this invention. It is a longitudinal cross-sectional view which shows arrangement | positioning of the screen used for the apparatus of this invention, and a projection means. It is explanatory drawing which shows the principle of the distortion correction means used for the apparatus of this invention. It is a top view of one embodiment of the invention of claim 5 . It is an elevational view of an embodiment of the invention of claim 5 .

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Screen 2 Image | video creation means 3 Distortion correction means 4 Projection means

Claims (5)

A spherical screen with a wide viewing angle, with a concave surface facing the experience,
A video creation means for creating a video on a plane;
A mapping process for pasting the image on the plane created by the image creation unit onto a spherical shape matching the screen and a process for projecting the image pasted on the spherical shape onto the plane again are performed on the screen. Distortion correction means for distorting the image in advance so that the distortion disappears when displayed,
Projection means for projecting the distortion-corrected image onto the screen;
The camera for detecting the position of the experience person is arranged above the experience apparatus, the position of the experience person is detected from an image photographed by the camera, and distortion correction is performed on the detected position of the experience person. A virtual environment experience display device characterized by having a function of always performing optimal correction by dynamically changing a parameter . The virtual environment according to claim 1, wherein the projection unit includes a plurality of projection units, and the distortion correction unit performs distortion correction so that joints of a plurality of images divided and projected onto the screen are smoothly connected by the projection units. Experience display device. 3. The virtual environment experience display device according to claim 1, wherein the viewing angle of the screen is set to be approximately the same as a human viewing angle. 3. The virtual environment experience display device according to claim 2, wherein the plurality of projection means are arranged at substantially equal distances from the center of the spherical screen . Virtual environments experience display device of a plurality of claims 1 or 2, wherein the projecting the image of a wide viewing angle in a spherical shape of the screen by causing once reflected by the convex mirror an image projected from the projection means .

Images