JP2011070432A - Stereo viewer for automatically converting three-dimensional content to stereo content - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a stereo viewer which processes 3D content data for 2D display into data for a stereo display and outputs it. <P>SOLUTION: The stereo viewer includes: an input unit to input 3D content data; an image generation unit which generates 2D display images for left eye and right eye from the data; a data retention unit to retain data of positions and gaze directions of the left and right eyes. A distance between the left eye and the right eye is d and the gaze directions intersect each other with an angle of convergence in proportion to the distance. The image generation unit horizontally divides a left eye and right eye image storage area into two areas and generates 2D display images for left eye and right eye. When changing the gaze direction by angle θ, assuming that a horizontal direction is x direction, a vertical direction is y direction, and this side which is perpendicular to both of x and y directions is z direction, a difference between the left eye position (x<SB>l</SB>, y<SB>l</SB>, z<SB>l</SB>) and the right eye position (x<SB>r</SB>, y<SB>r</SB>, z<SB>r</SB>) is set as (-d×cos(θ), y<SB>l</SB>-(1/χ)×y<SB>r</SB>, d×sin(θ)). By adjusting an aspect ratio using a compressibility ratio χ in a program including the expression, an eye-friendly image is obtained. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a stereo viewer that is mainly provided on an Internet website and automatically converts 3D content for a normal display device into 3D content for a 3D display device.

  In general, the amount of information that can be expressed by a 2D (two-dimensional) image is larger than that of a text-only screen, and the information amount of 3D (three-dimensional) content is larger than that of 2D content. In the case of three-dimensional content, the contents may be intuitively understood without reading the text. The merit of expressing the Internet web in three dimensions is that a larger amount of information can be expressed as compared with 2D content. For these reasons, the number of sites that employ 3D representation is gradually increasing.

  Examples of languages for describing 3D objects include VRML (Virtual Reality Modeling Language) and X3D (a file format for expressing XML-based three-dimensional computer graphics) as a successor standard. . In addition, WPF (Windows Presentation Foundation) is also known.

  3D content is expressed in a virtual three-dimensional space using the above language. The viewer of the 3D content normally browses the 2D display device by displaying the scenery viewed from the viewpoint provided in the virtual three-dimensional space, but the viewpoint can be freely moved or moved to a free position. By placing it, it can be confirmed that it is 3D content and can be felt.

  The present invention relates to a stereo viewer that processes the data of the three-dimensional content into data for a three-dimensional display device so that the three-dimensional content of the three-dimensional content can be realized even when the image is stationary. is there. Since this 3D display device can display more like the real thing, it has already been used for computer games, training simulators, remote control display devices, etc. in addition to display for display, and its application. The field is expanding.

  The stereoscopic display device assumed by the present invention performs three-dimensional display by providing different images to the left and right eyes. Such a system uses, for example, glasses (1 ) The left and right images are projected with red and blue light superimposed, and the left and right images are projected with linear polarization orthogonal to each other, separated by glasses with red and blue color filters, (2) Polarization filter system in which linearly polarized light orthogonal to the left and right images is projected and overlapped, and this is separated by a polarizing filter. (3) Liquid crystal so that the left and right images are alternately shielded according to the image. A liquid crystal shutter method using glasses driven by a shutter is known. In addition, as a system that does not use glasses, (4) a parallax barrier is provided so that separate light beams are incident on the left and right eyes, and (5) a light beam reproduction type, but a lenticular lens or fly An integral display that reproduces the wavefront of a light beam using an eye lens array is known.

T.Okino, H.Murata, K.Taima, T.Iinuma, K.Oketani: “New television with 2D / 3D image conversion technologies”, Proceedings of SPIE-The International Society for Optical Engineers. Vol.2653 pp.96- 103 (1996) Shinji Araya: “VRML / X3D Technology Course”, http://www.fit.ac.jp/~araya/jis/, VRML Automation, sample 8. (September 8, 2008)

  The present invention realizes a stereo viewer that processes data of 3D content prepared for a 2D display device into data for a stereoscopic display device and outputs the data.

The stereo viewer of the present invention is a device that outputs respective display signals for presenting a left-eye image and a right-eye image for displaying a three-dimensional image to an observer. This device includes an input unit that inputs 3D content data that can display images viewed from different viewpoints on a 2D display device, and a predetermined left eye position, line-of-sight direction, and right eye of the 3D content data. An image generation unit that generates a left-eye two-dimensional display image and a right-eye two-dimensional display image viewed from the position and the line-of-sight direction, and synchronization of changes between the left-eye two-dimensional display image and the right-eye two-dimensional display image And a data holding unit that holds data of the left eye position and the line-of-sight direction, and the right eye position and the line-of-sight direction. The left eye position and the right eye position are separated from each other by a predetermined distance (d), and the left visual line direction and the right visual line direction intersect with each other with a convergence angle corresponding to the distance to the object. This convergence angle is smaller for more distant objects. In addition, the image generation unit includes the left-eye two-dimensional display image and the right-eye 2 in each area obtained by horizontally dividing the display image storage area into a left-eye image storage area and a right-eye image storage area. A three-dimensional display image is generated. This relates to the storage area of the image display device. When changing the line-of-sight direction, the horizontal direction from the left eye to the right eye is the x direction, the upward vertical direction of the display image perpendicular to the x direction is the y direction, and the near direction perpendicular to the x direction and the y direction is the z direction. When the line-of-sight direction is changed by the angle θ, the left eye position (x _l , y _l , z _l ) and the right eye position (x _r , y _r , z _r ) are converted as follows.

  Here, an image having a narrow width can be displayed by setting the compression rate χ to be larger than 0.5, and an image having a width can be displayed by making the compression rate χ smaller than that. By adjusting the value of the compression ratio χ, 3D content can be displayed under a plurality of stereoscopic display devices having different screen sizes. The output of the stereo viewer is output as a left-eye image and a right-eye image of a side-by-side display device such as a head-mounted display.

  When the present invention is applied to an anaglyph three-dimensional image display, the left eye image and the right eye image are output through an image conversion unit that performs color conversion.

  Further, when applied to a liquid crystal shutter type three-dimensional image display, for example, a field sequential liquid crystal display device, the left eye image and the right eye image are output through an image conversion unit that alternately outputs the left eye image and the right eye image.

  Further, in order to present the left-eye image and the right-eye image of the polarization filter type three-dimensional image display device, the left-eye image and the right-eye image are arranged in accordance with a region where polarized light that is orthogonal to each other is transmitted. Output through the image converter.

  In the parallax barrier method or the lenticular lens method, it is not necessary to use dedicated glasses, but the image for the left eye and the image for the right eye of the three-dimensional image display device are aligned with the vertical stripe of the parallax barrier or the lenticular lens. Are output through an image conversion unit that alternately arranges a left-eye image and a right-eye image in the form of vertical stripes in one cycle.

  Further, the movement in the horizontal direction as viewed from the observer is performed by the movement of the left eye position and the right eye position, and the movement in the vertical direction as viewed from the observer is performed by rotation in the three-dimensional content data. As described above, by decomposing the moving component in the three-dimensional image display, it becomes possible to easily create a program for the observer to observe at a free observation position.

  According to the present invention, stereoscopic image display can be performed in real time using data of 3D content prepared for a 2D display device on an Internet website.

A conventional method (non-stereoscopic browsing method) for browsing VRML contents of a website is shown in route 1, and route 2 shows a method using the stereo viewer of the present invention. It is a figure which shows until it produces and browses a three-dimensional content for an example of a three-dimensional animation video. It is a figure explaining the synchronous mechanism added to a stereo viewer for enabling a viewer to walk through the virtual world under a stereoscopic view. It is a figure which shows the flow until the VRML content downloaded from the website is displayed on a stereoscopic display device as a stereoscopic image. It is a figure which shows the relationship between the left-eye position coordinate which considered binocular parallax, and the right-eye position coordinate. A stereo viewer uses a WEB browser (IE) provided on a personal computer (Windows (registered trademark) personal computer) and an installed VRML plug-in “Cortona VRML Client” (hereinafter abbreviated as Cortona) to stereoscopically view VRML content. The program for performing the conversion is <1> a two-image generation program written in the HTML language, <2> a stereoscopic setting program written in the VRML language, and <3> an image written in the JavaScript (registered trademark) language. Although it is comprised from three of the synchronous programs, it is a figure for demonstrating cooperation of these programs, limiting to the walkthrough which implement | achieves the stereoscopic vision. FIG. FIG. FIG. FIG. FIG. FIG. FIG. FIG. 8 is a diagram illustrating a code 8; FIG. FIG.

  The stereo viewer of the present invention will be described below. By using this, for example, VRML (Virtual Reality Modeling Language: Virtual Reality Modeling Language) content on a website is browsed in a stereoscopic view while performing a walk-through by dialogue with a viewer. It will be possible. This stereo viewer enables normal non-stereoscopic VRML content to be directly converted into stereoscopic content. By using this, the viewer can browse the VRML content uploaded to the website in a stereoscopic manner while walking through under a WEB browser.

  A conventional method (non-stereoscopic browsing method) for browsing VRML content of a website is shown in route 1 of FIG. Typically, VRML content is embedded in a web page as part of a home page and uploaded to the website. The viewer installs software for drawing VRML content (hereinafter referred to as a VRML plug-in) in advance on the client personal computer as a plug-in of the WEB browser. In route 1, when a viewer accesses a web page from a WEB browser, the VRML plug-in analyzes the source code of the VRML content embedded therein and draws the virtual world on the screen.

  On the other hand, route 2 in FIG. 1 shows a method using the stereo viewer of the present invention. In this method, a stereoscopic (3D) display device is used instead of a normal two-dimensional (2D) display device, and a stereo viewer that stereoscopically displays VRML content is added to the route 1. The stereo viewer can also be realized by software. For example, a program running on a website can be described in three languages, HTML (HyperText Markup Language), VRML, and JavaScript (registered trademark).

  Describing in more detail about Route 2, when a viewer accesses a program (web page) in HTML language, which is one of the components of a stereo viewer, using a WEB browser, VRML content linked from the stereo viewer is displayed on the website. Downloaded from. The stereo viewer prepares a stereoscopic environment, and the VRML plug-in draws an image that can be viewed stereoscopically on the stereoscopic display device.

  The following shows how this stereo viewer generates a stereoscopically visible virtual world from the VRML content of a website.

The process of creating and viewing 3D content can be divided into the following processes when taking a 3D animated movie as an example (FIG. 2). Hereinafter, processes <1>, <2>, and the like indicate the processes mentioned here.
Process <1>: A stereoscopic display device having an appropriate stereoscopic format based on the principle of stereoscopic vision is selected.
Process <2>: A pair of left-eye and right-eye animation images is prepared in a format suitable for the display device.
Process <3>: Two right and left animation images are generated with parallax.
Process <4>: A mechanism is provided for drawing a pair of left and right animated images that flow in time order in synchronization with each other.
Process <5>: A moving image composed of animation images is browsed on a stereoscopic display device.

  The principle process for stereoscopically viewing VRML content on a website is the same as the above-described process for stereoscopically viewing a normal television program.

In the field of television, a technique for stereoscopically viewing a normal television program (non-stereoscopic program) has been developed by a process similar to that of a stereoscopic animation video. The process is as follows.
In the process <1>, a side-bye format is adopted as a stereoscopic format of the stereoscopic television.
In process <2>, the TV screen is divided into two parts, the left screen for the left eye and the right screen for the right eye.
It is assumed that the two images generated in the process <3> are displayed. In process <3>, a pair of images with left and right parallax is created from the two images, but it is difficult to devise a parallax algorithm for this purpose. At present, an algorithm (MTD method) for estimating the depth from the horizontal movement of the video (Non-patent Document 1), an algorithm using the background or color as a clue to the depth, and the like have been developed. In any case, in the process <3>, the depth determination is performed at high speed by the semiconductor arithmetic element. Also,
The left-right synchronization and drawing of the process <4> are realized simultaneously with the process <4> in the course of electronic high-speed processing.
In process <5>, the viewer receives a normal television program on the stereoscopic television. As a result, the program can be viewed stereoscopically.

  First, the stereoscopic format of the stereoscopic display device will be determined. A “side-by-side format” is adopted as the format of the stereoscopic display device to be displayed by the stereo viewer (process <1>). This format has long been used as a format for stereoscopic photography, but in the 21st century it was applied to liquid crystal stereoscopic display devices and 3D television broadcast receivers. (There are other top-and-bottom formats, field sequential formats, frame sequential formats, etc.). If the stereoscopic format does not match between the stereoscopic display device and the VRML content, stereoscopic display is impossible.

The flow of stereoscopic viewing using a side-by-side stereoscopic display device will be described along the contents of FIG.
(1) First, one screen of the display device is divided into two left and right screens.
(2) Next, a left-eye image frame is prepared on the left screen, and a right-eye image frame is prepared on the right screen (process <2>).
(3) At this time, the two images are compressed in half in the horizontal direction, and binocular parallax and convergence angle are added between the left and right images.
(4) Every time a walk-through is performed, drawing is performed while synchronizing the left and right images (process <4>).
(5) When browsing, the hardware function (or software function) is used to enlarge both the left and right images twice in the horizontal direction and superimpose both images. The viewer stereoscopically views this display screen (process <5>).

  On the VRML plug-in side on which the stereo viewer is responsible for drawing, two drawing engines on the left and right operate. When the viewer walks through the virtual world, the walkthrough information is given by dragging the mouse from either the left or right screen, but in this viewer, the information is given from the right screen. Needs to be instantly transmitted from the drawing engine to the drawing engine on the left screen. If means for enabling such transmission is prepared on the VRML plug-in side, the left and right drawing engines can draw the synchronized left and right images. This is the synchronization in the stereo viewer (process <4>).

  In order to synchronize, not only walk-through by mouse operation, but also various other instruction information can be considered, but by adding such a synchronization mechanism to the stereo viewer, viewers can see in stereoscopic view It is possible to walk through the virtual world (Fig. 3). As a VRML plug-in that provides means for enabling synchronization as described above, there is Cortona of Parallel Graphics Co., Ltd., and an example using this is shown below.

  FIG. 4 shows a flow until the VRML content downloaded from the website is displayed as a stereoscopic image on the stereoscopic display device. The overall image of the stereo viewer will be described with reference to FIG. FIG. 4 is divided into the following three parts. The part [I] is a stereoscopic target (VRML content), the part [II] is a stereo viewer, and the part [III] is a stereoscopic display device and a WEB browser (for browsing) ( IE) and a VRML plug-in (Cortona) in charge of actual drawing. More specifically, the input unit 1 acquires VRML content from the website 5 via the Internet. By synchronizing the positions of the left eye and right eye held in the data holding units 2L and 2R and the line-of-sight direction data via the synchronization unit 3, there is no shift in the walk-through. Based on these contents and data, the image generation units 4L and 4R generate left-eye and right-eye images, respectively. The generated images are drawn on the left and right screens 7L and 7R of the stereoscopic display device 7 via the drawing engines 6L and 6R.

This stereo viewer is composed of the following three element programs.
(1) Two-image generation program (described in HTML language: responsible for process <2>)
(2) Stereoscopic setting program (described in VRML language: responsible for process <3>)
(3) Image synchronization program (written in JavaScript (registered trademark) language: responsible for process <4>)
The outline of this element program is as follows.
(1 ′) The two-image generation program realizes process <2>. This is a web page described in the HTML language, and serves to generate a left and right two-image framework in the stereoscopic display device (using the HTML table function).
(2 ′) A stereoscopic vision setting program that realizes the process <3> is embedded in this web page (using the Embed function of the HTML language), and parallax is given to the images for the left and right eyes. This program is described in the VRML language and linked to the VRML content uploaded to the website of [III] via a URL (Uniform Resource Locator) (using the VRML language Inline function) ).
(3 ′) An image synchronization program is interposed between the two stereoscopic vision setting programs. This program written in JavaScript (registered trademark) language realizes the process <4> and keeps the left and right images synchronized. This image synchronization program tells Cortona's VRML Automation function information such as walkthroughs about stereoscopic viewing.
In the final stage, Cortona's left and right drawing engines that have received the stereoscopic information draw left and right images with parallax. Since the stereo viewer is described in a language related to the web, it can be uploaded to the website of [I].

  Next, a method for preparing two images of VRML content will be described. The VRML plug-in (Cortona) has a specification in which one drawing engine corresponds to one VRML content. Therefore, when two VRML contents having the same content are embedded in one web page, two VRML plug-in drawing engines are activated simultaneously to generate two images. Using this characteristic, the display screen is divided into left and right, and two images are drawn side by side on the left and right. Thereby, the three-dimensional format becomes a side-by-side format.

  The actual embedding of the image is performed by using an HTML language Table tag as shown in Code 1 of FIG. 7 (9) (only the minimum necessary contents are described for reasons of space). Two screens are set using a Td tag (may be an Object tag) in the Table tag. A stereoscopic setting program (described in the VRML language) to be linked to the VRML content is embedded therein using an Embed tag. The location is set by the src attribute. The entire Table tag is described in the Body tag part of the web page.

  Two VRML programs (program names are stereoL.wrl and stereoR.wrl) set in the scr attribute of the Embed tag are drawn on the left and right split screens by the two drawing engines of the VRML plug-in. However, strictly speaking, the two VRML programs only link to the VRML content on the website, and the virtual world of the VRML content of the link destination is drawn. The two VRML programs are stereoscopic vision setting programs as described above, and their role is to provide stereoscopic information to the linked VRML content. As a matter of course, if there is no parallax between the left and right images, there is no stereoscopic effect between the images.

  The overall code of the two-image generation program (program name is stereo.html) is as shown by code 2 in FIG. 8 (only the minimum necessary contents are described). This program is a program that constitutes a web page and is activated first in the stereo viewer. The activation is performed by a WEB browser (IE).

  When this web page is accessed, the WEB browser divides the screen into two horizontally and creates two images that can be viewed stereoscopically. Note that a JavaScript (registered trademark) program (program name: StereoAPI.ja) indicated in the src attribute of the Script tag is a program that synchronizes the left and right images (the image synchronization program described in Chapter 5). This program is loaded when the init method in the image synchronization program is called by the onload attribute of the Body tag when the execution of the web page is started.

Next, parallax settings that need to be provided between the left and right images will be described.
In a virtual world controlled by the VRML plug-in, there is a transparent avatar (one person) that walks through, and through that avatar's eyes (which has only one eye), the viewer can see the virtual world. Look. Therefore, the eyes of the two avatars existing in the two virtual worlds on the left and right are made to correspond to the left and right eyes responsible for stereoscopic vision, and binocular parallax and convergence angle are given to them.

  Also for avatar images, binocular parallax is the distance between the left and right eyes (the distance between human eyes is about 6.5 cm), and the difference between the left and right images drawn by this difference. Is called binocular parallax. This parallax creates a stereoscopic effect. The convergence angle refers to the angle formed by the eyes of both eyes. The vergence angle changes from a positive value to zero as the line of sight of both eyes is moved from the near side to the far side. By adjusting this angle, the perspective of the entire stereoscopic image can be determined. If the convergence angle is zero (that is, the lines of sight are parallel and do not intersect, and the intersection is at infinity from the avatar), the entire stereoscopic image appears to pop out of the screen, but the angle of convergence increases (that is, the line of sight). As the viewer moves in the same way, the entire stereoscopic image feels to be pulled into the back of the screen.

  The convergence angle for avatars can be given a negative value, unlike the convergence angle for humans. In this case, the line-of-sight intersection exists behind the avatar, and for the viewer viewing stereoscopically, as the convergence angle decreases from zero to the negative direction, the entire stereoscopic image is Compared to zero convergence angle, it seems to jump out from the screen.

  The parallax between the two images is determined by the binocular parallax values of the left and right eyes of the avatar. Since the avatar's eyes correspond to the Viewpoint node in the VRML language, a stereoscopic eye is created using the Viewpoint node. The position field of the Viewpoint node is responsible for the eye position, and the orientation field is responsible for the viewing direction. In the VRML language, a coordinate transformation node called a Transform node is prepared. This node is used to translate, rotate, reduce and enlarge coordinates, but this node can also be used in conjunction with the Viewpoint node. A method for determining the viewpoint of the left eye of the avatar using the VRML language will be described below with reference to code 3 in FIG. 9 (only the necessary minimum contents are described for the sake of space).

In this example, the left eye of the avatar is at the coordinate point (x, y, z) in the world coordinate system, and the line-of-sight direction is rotated about the y axis on the horizontal plane (δ = ± 1, This is an example of 0 ≦ θ ≦ 2π). The x value and the z value are values calculated from the binocular parallax and the coordinate point of the right eye, α is the convergence angle, and the line of sight is α radians (−a ₁ ≦ α ≦ a ₂ , where a ₁ and a ₂ are both positive values, practically show that rotate about the y-axis by about 5 degrees or less value). When walking through the virtual space, the x, y, z values that make up the position field value, and the δ and θ values that make up the orientation field value are both synchronized with those of the right eye in real time. To change.

  In the side-by-side stereoscopic display device to be displayed by the stereo viewer, the left and right images are doubled horizontally by the hardware function (or software function) when viewing the stereoscopic view as described above. Spreads across the screen and overlaps twice. Therefore, at the stage of drawing and generating both images, it is necessary to set the vertical to horizontal ratio of the images to 2: 1, which means that the field of view of the stereoscopic glasses is 1 to 2 with respect to the vertical to horizontal ratio. Is the same as The vertical to horizontal ratio of the visual field is changed by performing the coordinate transformation shown in code 4 in FIG. 10 on the entire code 3 in FIG. For χ in the second line of code 4 in FIG. 10, χ = 0.5.

  There are two stereoscopic vision setting programs for the left eye and for the right eye (program names are stereoL.wrl and stereoR.wrl). The overall images of the codes are as shown in code 5 in FIG. 11 and code 6 in FIG. (Only the minimum necessary contents are described for reasons of space). For χ in code 5, χ = 0.5. These two programs cooperate with the image synchronization program of code 7 in FIG. 13 to give parallax to the left and right images. This will be described later.

  The stereoscopic vision setting program has three functions as follows. (1) Set the stereoscopic information and the avatar information necessary for the walk-through execution in the right-eye stereoscopic setting program. These pieces of information are read by a synchronization program between the left and right images and used in the program. (2) As above, the avatar is worn with stereoscopic glasses. (3) The URL of the VRML content to be drawn is set. The content actually rendered as a virtual world is VRML content having this URL.

  In the actual walkthrough, the data of the right eye viewpoint position is reflected as data in the ProximitySensor node of the right eye stereoscopic setting program by the VRML plug-in. Then, the image synchronization program reads the data in real time, and a value obtained by adding binocular parallax to the data is transmitted to the left-eye Viewpoint node as the left-eye viewpoint position.

Here, synchronization between the left and right images is performed as follows.
In order for the viewer to freely move through the virtual world by controlling the movement of the avatar, it is necessary to link the pair of avatars in real time. Therefore, the two drawing engines of the VRML plug-in that draws two images are synchronized as follows.

  The left and right dialog mechanisms of the VRML plug-in can be operated independently, but the left-eye dialog mechanism is stopped, the avatar is operated only by the right-eye dialog mechanism, and the right-eye avatar is operated. If the viewpoint information can be transmitted as the viewpoint information with the parallax of the left-eye avatar, the movements of the left and right avatars can be integrated. Under these circumstances, the left and right drawing engines draw the left and right images simultaneously.

  The VRML language does not provide a mechanism for exchanging information between the operating left and right drawing engines. However, Parallel Graphics, which provides the VRML plug-in Cortona, supports this mechanism, which the company calls "VRML Automation". Therefore, an image synchronization program was created using the following VRML Automation in accordance with Non-Patent Document 2.

  VRML Automation is designed with the same concept as Document Object Model (DOM) and is used under JavaScript (registered trademark) language. Its syntax and usage is similar to DOM.

  First, an overview of the code of the image synchronization program (program name is StereoAPI.js) written in JavaScript (registered trademark) language is shown in code 7 of FIG. 13 (using the framework of sample 8 described in Non-Patent Document 2). (However, only the minimum necessary information regarding the walk-through is shown). However, χ in the code 7 is χ = 0.5.

  When the WEB browser accesses the two-image generation program (stereo.html) of code 2, this program is loaded at that time, and the init method is executed first. This method performs various initial settings (initialization of solidity, convergence angle, headlight, walkthrough speed, etc., but omits the description of the code), and after that, the walkThrough method is called to Drive. This driving is performed by the setInterval method of JavaScript (registered trademark) code 8 in FIG.

  Here, the call of the walkthrough method is repeated once every 30 milliseconds until the viewer completes the walkthrough of the virtual world. Inside the walkthrough method, the position information of the viewpoint of the right eye (whose node ID is ViewR) is obtained from the ProximitySensor node (whose node ID is PS) by the VRML Automation instruction of code 9 in FIG. 15, and the variable (xR, yR, zR ) Is set. Then, the position information of the right eye viewpoint is corrected by binocular parallax by the VRML Automation command of code 10 in FIG. 16, and the position information of the left eye is detected as the position information of the left eye viewpoint (its node ID is ViewL). It is transmitted to the Viewpoint node (whose node ID is VP). θ is a conversion angle regarding the line-of-sight direction obtained from other information, and χ in the code 10 is χ = 0.5. This transmission is repeated once every 30 milliseconds, and Cortona's left and right drawing engines are driven accordingly. Thereby, the drawing of the left and right images can be synchronized in real time. The convergence angle and headlight direction can also be synchronized by a similar method.

In the above description, the binocular parallax setting has been described without any annotations, but this will be described below. When the viewpoint position of the right eye of the avatar in the world coordinates is the coordinates (x _r , _yr , z _r ), and the direction of the line of sight is along the horizontal plane, the binocular parallax (distance) is d and the line-of-sight angle in the horizontal plane Is θ around the origin of the world coordinate system. From FIG. 5, the position coordinates of the left eye (x _l , y _l , z _l ) taking binocular parallax into account are as follows because the z axis is positive downward: Can be represented.

  Here, the y coordinate is multiplied by χ = 0.5 in order to double the visual field of the stereoscopic glasses in the horizontal direction and to make the stereoscopic format of the VRML content a side-by-side format.

  Thus, the parallax setting in the stereo viewer is very simple. But be careful. That is, “the 3D glasses worn by the avatar are effective only when the avatar looks along the horizontal plane in the world coordinate system”.

  Most VRML content on the website simulates a landscape, and walk-through is mainly ground walking and horizontal flight, and even if the viewing direction is limited to a horizontal plane, there is no practical problem. Therefore, the 3D glasses worn by the avatar are limited to the direction of the line of sight only in the horizontal plane direction.

  However, it will be described below that stereoscopic viewing can be performed even when the avatar views the virtual world from a line-of-sight direction that is not parallel to the ground (when looking up or looking down).

  First, walk-through in a general VRML plug-in is realized by fixing objects constituting a virtual world other than an avatar to the world coordinate system, and only the avatar moves in the space of the world coordinate system. This is hereinafter referred to as “walk-through method by horizontal viewpoint movement”. Furthermore, the movement and rotation in space is relative, and even if the avatar is fixed to the world coordinate system and the objects that make up the other virtual world are moved or rotated in the world coordinate system space, When viewed from above, it seems to be doing a normal walkthrough.

  Therefore, in this embodiment, in order to enable browsing of the virtual world from any line-of-sight direction, the virtual world itself other than the avatar is directly rotated under the world coordinate system, and then the world coordinates are set by the avatar's stereoscopic glasses. The “walk-through method by rotating the virtual world” is used, which is viewed from the direction along the horizontal plane of the system. As a result, the viewer can view the virtual world from a viewing direction that is not horizontal with respect to the ground.

  The virtual world is rotated through buttons provided on the console of the stereo viewer. Inside the stereo viewer, rotation is performed by a CylinderSensor node and a SphereSensor node prepared in the VRML language, and the rotation information is transmitted to a drawing engine of Cortona which is a VRML plug-in for drawing. At the same time, correspondence to stereoscopic vision is performed using the stereoscopic glasses described above.

  The line-of-sight direction in the walk-through is separated into two directions, “horizontal direction” and “other directions”. The derivation and description of a large number of matrix arithmetic expressions necessary for the latter processing are avoided, and it is applied after Cortona. This is because it is left to the graphic driver. By this measure, the burden on the program creator can be reduced and the numerical calculation load can be suppressed.

  The stereo viewer stereoscopically displays VRML content using a WEB browser (for example, Internet Explorer: IE) provided in a personal computer (Windows (registered trademark) personal computer) and a VRML plug-in (Cortona) installed. The stereo viewer program includes: <1> a two-image generation program written in the HTML language, <2> a stereoscopic setting program written in the VRML language, and <3> an image synchronization written in the JavaScript (registered trademark) language. It consists of three programs. In FIG. 6, the cooperation of these programs will be described in association with the whole, limited to the walk-through for realizing stereoscopic viewing.

  When a viewer views VRML content in a stereoscopic view, a two-image generation program as a web page is accessed using a WEB browser (IE). In the two-image generation program, a stereoscopic setting program in which the URL of the VRML content to be browsed is registered is embedded. The IE prepares two images of the VRML content according to the information described on the web page, and activates the VRML plug-in (Cortona). Then, Cortona reads two stereoscopic image setting programs for left and right images, and displays the initial images of VRML content to be viewed on the two left and right screens at the link destination of the URL registered there (the thick broken lines in FIG. 6). Flow of). In accordance with this, the init method of the image synchronization program is activated and the stereoscopic vision preparation is set (the flow of the thick solid line in FIG. 6). Then, the initial screen becomes a stereoscopically viewable state by the action of the stereoscopic view setting program (the flow of the thick broken line in FIG. 6). Thereafter, the walkthrough method of the image synchronization program is repeatedly called every 30 milliseconds.

  When the viewer uses the mouse for a walk-through and drags the right screen (when the stereoscopic display device is stereoscopically visible, the left and right screens are overlapped, but at that time the right screen is composed) The viewpoint information of the avatar of the right image is transmitted to the stereoscopic image setting program for the right image (the flow of the thick solid line in FIG. 6). As a result, the Cortona drawing engine on the right screen updates and draws the right image, and the result of the walkthrough is immediately reflected on the right screen (the flow in the right direction of the thick broken line in FIG. 6).

  The walkthrough method of the image synchronization program monitors the movement of the viewpoint of the avatar in the right image in real time. When the walkthrough is performed, the viewpoint information of the right image avatar is obtained from the stereoscopic image setting program for the right image with the help of VRML Automation, and based on the information and binocular parallax, The viewpoint position is calculated. Then, with the help of VRML Automation, the viewpoint information is transmitted to the stereoscopic image setting program for the left image (the flow of the thick solid line in FIG. 6). Using the viewpoint as the viewpoint information of the avatar of the left image, the Cortona drawing engine on the left screen updates and draws the left image (the flow in the left direction of the thick broken line in FIG. 6). The walkthrough method repeats this cycle every 30 milliseconds, and the left and right images are synchronized in a stereoscopic state.

  Obviously, the same three-dimensional image can be obtained by the same scanning as described above even when the right image and the left image are interchanged.

  In order to use the above stereo viewer, it is necessary to solve operational problems. The first is to obtain the URL of the VRML content to be browsed. The second is to know the situation of the virtual world. Virtual worlds are generally not all created at the same geometric scale. One virtual world may be a landscape with a huge space that cannot be grasped without walking through, and another virtual world may be a simple virtual object that can be seen at a glance. It can be said that the former corresponds to a virtual world of landscape simulation, and the latter corresponds to a virtual world of an automobile model.

  A stereo viewer corresponding to various virtual worlds having different situations needs to be given geometric information of the virtual world in advance. For example, <1> the center position of the virtual world with respect to the world coordinate system, <2> the speed at which the avatar walks through, <3> the height of the step over which the avatar gets over, <4> the position of the avatar at the start of browsing, < Information such as 6> stericity, <7> angle of convergence, and <8> headlight intensity. The stereo viewer secures a table in which such information is registered in advance as parameters. When using a stereo viewer, it is necessary to prepare this table for each virtual world of VRML content.

  Also, for viewers who want to enjoy the virtual world stereoscopically using a stereo viewer, a collection of tables is built as a ledger (called a registered ledger), and it is convenient if it is prepared in advance. Easy to understand. This registration ledger is such that when a new virtual world is generated, its information (parameters) is added. This ledger has the same personality as the search ledger of a search engine site on the Internet. When a general viewer activates the stereo viewer under the WEB browser, the registration ledger allows the viewer to perform a smooth walk-through of the virtual world and sense an optimal stereoscopic effect.

  The present invention can be applied to an anaglyph method, a polarizing filter method, a liquid crystal shutter method, or a three-dimensional image display device such as a parallax barrier method or a lenticular lens method by including the following image conversion units. .

  (1) First, by providing an image conversion unit that performs color conversion between the left-eye image and the right-eye image generated as described above, the image can be used for anaglyph-type three-dimensional image display. (2) Further, in order to apply to a liquid crystal shutter type three-dimensional image display device, image conversion is performed by alternately outputting the left-eye image and the right-eye image generated as described above while changing the orthogonal polarization plane. May be provided. (3) Further, in the polarization filter type three-dimensional image display device, the regions through which polarized light orthogonal to each other are transmitted are arranged in horizontal stripes (or vertical stripes), and the left-eye image and the right-eye image are divided into the stripe-like regions. The image is output through an image conversion unit arranged in accordance with the left eye area and the right eye area. (4) In the case of the parallax barrier method or the lenticular lens method, the parallax barrier or the cylindrical lens is arranged in vertical stripes, and the vertical stripe-like left eye image and right eye image are alternately arranged in parallel during this one cycle. And output it through the image conversion unit.

  In the above description, the walk-through has been mainly described. However, the walk-through can be used for a stereo photo viewer and a stereo video viewer. In these viewers, it is often not possible to freely and smoothly zoom in and out on any part of a stereo image, but by applying this walkthrough, you can scale that part freely and smoothly. Is possible. At this time, when adjusting the vertical and horizontal sizes to χ in the second line of code 4 and the third line of code 5, the value slightly shifted from 0.5 as χ in the third line of code 10 It can be performed by using. It is apparent that the present invention can be easily applied not only to VRML contents but also to the production of stereoscopic image display software (stereo viewer) for photographs and video images.

DESCRIPTION OF SYMBOLS 1 Input part 2 Data holding part 3 Synchronizing part 4L, 4R Image generation part 5 Website 6 Browser 6L, 6R Drawing engine 7 Three-dimensional display apparatus 7L, 7R Screen

Claims (6)

An apparatus for outputting respective display signals for presenting a left-eye image and a right-eye image for displaying a three-dimensional image to an observer,
An input unit for inputting three-dimensional content data capable of displaying images viewed from different viewpoints on a two-dimensional display device;
An image generation unit for generating a left-eye two-dimensional display image and a right-eye two-dimensional display image as viewed from the predetermined left-eye position and line-of-sight direction and right-eye position and line-of-sight direction of the three-dimensional content data;
A synchronization unit that synchronizes changes between the left-eye two-dimensional display image and the right-eye two-dimensional display image;
A data holding unit for holding data of the left eye position and the line-of-sight direction and the right eye position and the line-of-sight direction,
The left eye position and the right eye position are separated by a predetermined distance (d),
The left visual line direction and the right visual line direction intersect with a convergence angle corresponding to the distance to the object,
The image generation unit includes the left-eye two-dimensional display image and the right-eye two-dimensional display image in respective regions obtained by horizontally dividing the display image storage region into a left-eye image storage region and a right-eye image storage region. And generate
When the horizontal direction from the left eye to the right eye is the x direction, the upward vertical direction of the display image perpendicular to the x direction is the y direction, and the near direction perpendicular to the x direction and the y direction is the z direction, the line-of-sight direction is the angle θ. Only for the left eye position (x _l , y _l , z _l ) and the right eye position (x _r , y _r , z _r ),

A stereo viewer that automatically converts 3D content into 3D content, displaying an image with an aspect ratio that is easy to see by changing the compression ratio χ.   The three-dimensional content according to claim 1, wherein the three-dimensional content is output to a stereoscopic content through an image conversion unit that performs color conversion in order to present an anaglyph-type left-eye image and right-eye image for displaying a three-dimensional image. Stereo viewer for automatic conversion.   2. The three-dimensional content according to claim 1, wherein the three-dimensional content is automatically converted into a three-dimensional content, which is output through an image conversion unit that alternately outputs a left-eye image and a right-eye image for liquid crystal shutter-type three-dimensional image display. Stereo viewer.   In order to present a left-eye image and a right-eye image of a polarization filter type three-dimensional image display device, an image conversion unit that arranges the left-eye image and the right-eye image according to a region where polarized light that is orthogonal to each other is transmitted The stereo viewer for automatically converting the three-dimensional content into a three-dimensional content according to claim 1, wherein the stereo viewer automatically outputs the three-dimensional content.   The left-eye image and the right-eye image of the parallax barrier type or lenticular lens type three-dimensional image display device are aligned with the vertical stripes of the parallax barrier or lenticular lens, and the vertical-striped left-eye image and right-eye during one period of the vertical stripes The stereo viewer for automatically converting 3D content to 3D content according to claim 1, wherein the image is output through an image conversion unit that alternately arranges images for use in parallel. In movement in 3D image display,
Movement in the horizontal direction when viewed from the observer is performed by moving the left eye position and right eye position,
Movement in the vertical direction when viewed from the observer is performed by rotation in the three-dimensional content data.
A stereo viewer for automatically converting three-dimensional content according to any one of claims 1 to 5 into stereoscopic content.

Images