JP7040080B2 - Virtual space image presentation system, virtual space image presentation method and program - Google Patents

Description

The present invention relates to a virtual space image presentation system, a virtual space image presentation method and a program.

Conventionally, as a method of allowing an observer to view a virtual object in a virtual space, an image of the virtual object to be viewed by the observer when the virtual space is viewed from a virtual camera corresponding to the viewpoint position of the observer is displayed. An image generation system for generating is known.
As a result, the observer can experience the virtual reality that he / she recognizes that he / she is located in the virtual space, which is utilized in fields such as games and digital archives.

Further, in recent years, due to advances in image processing algorithms that generate images and higher performance of hardware that runs applications for image processing algorithms, many contents that allow users to experience virtual reality in real time have become widespread.
Taking the walkthrough in the observer's virtual space as an example, it is possible to generate an image that allows you to experience virtual reality in real time. Can be viewed with a sense of reality.

In the contents of the virtual space described above, the image of the virtual object displayed to the observer includes a wide viewing angle image such as a panoramic image (see, for example, Patent Document 1). For this panoramic image, for example, a live-action image obtained by capturing a real space or an image generated from information of a three-dimensional shape model showing a virtual object is used.
At this time, when displaying an image to the observer using the information of the 3D shape model, the process of generating the image draws a large amount of the 3D shape model because the generated image has a wide viewing angle. There must be. For this reason, the drawing result of the panoramic image of the distant view is stored in advance, and the near view is drawn in real time and the distant view is drawn using the stored image, thereby improving the real-time property when generating the panoramic image. (See, for example, Patent Document 2).

In addition, as a method of separating a virtual object in the near view and a virtual object in the distant view and drawing each virtual object in the near view and the distant view, a distant view is once drawn on a plane in binocular disparity image generation for the purpose of stereoscopic viewing. A technique for reducing the deviation of the drawing position of a distant view object is disclosed by using the above-mentioned results (see, for example, Patent Document 3).

Japanese Unexamined Patent Publication No. 2016-62486 Japanese Patent No. 4806578 Japanese Unexamined Patent Publication No. 2012-59009

As in the walkthrough described above, the advantage of the observer viewing the virtual object of the 3D shape model in the virtual space in real time is that the virtual object or the viewpoint position in the virtual space depends on the viewpoint position of the observer. The point of view is that you can freely change the direction of your line of sight, and you can experience the realism of being in a virtual space.
However, as in each of Patent Document 1 and Patent Document 2, when the image of the drawing result of the panoramic image of the distant view object is prepared in advance, the virtual object to be changed in the virtual space and the viewpoint position of the observer are prepared. The movable range of the is limited, and the advantage of enjoying the above-mentioned sense of presence cannot be fully utilized.

On the other hand, for a virtual object in a distant view in a virtual space that is far away from the observer's viewpoint position, the appearance of the virtual object may hardly change even if the observer's viewpoint changes. Therefore, although the drawing result of the virtual object in the distant view is almost the same, the virtual object is drawn again in the next frame.
Therefore, as in Patent Document 3, if the virtual object of the near view and the distant view in the virtual space is drawn separately, the drawing load can be reduced by reusing the drawing result of the virtual object in the distant view.

However, Patent Document 3 does not aim to reduce the processing load in drawing a virtual object, and is premised on drawing on a plane.
Therefore, the advantage of drawing as a panoramic image as in each of Patent Document 1 and Patent Document 2, that is, processing when the line-of-sight direction is changed by preparing an image in the entire line-of-sight direction at a specific viewpoint position. The load cannot be reduced.
In addition, the advantage of separating the virtual object in the virtual space into the distant view and the near view, that is, when performing processing such as no need to redraw if the viewpoint movement is small, each virtual object of the distant view and the near view from the viewpoint position. It is necessary to manually set the distance to the boundary for separating.
Therefore, even if the distance to the boundary is adjusted in a predetermined scene, the distance to the boundary in another scene is not suitable when observed in another scene or a different device, and it is necessary to readjust the distance to this boundary. This may impose a burden on content creators and observers.

The present invention has been made in view of such a situation, and it is not necessary to draw a distant view object on a panoramic image in advance, the processing load due to drawing can be reduced, and when observing a virtual space. Virtual space image presentation system, virtual space image, which has no limit on the movable range with respect to the viewpoint position of the observer in, and does not need to readjust the distance to the boundary that separates the virtual object into the distant view and the near view depending on the scene. Provide presentation methods and programs.

In order to solve the above-mentioned problems, the virtual space image presentation system of the present invention generates a virtual space image showing a virtual object in the virtual space according to each of the viewpoint position and the line-of-sight direction of the observer, and observes the above. It is a virtual space image presentation system presented to a person, and is a distant view object extraction system that determines whether or not the virtual object is a distant view object arranged at a distance exceeding the distant view threshold from the viewpoint position of the observer in the virtual space. A distant view image generation unit that generates a distant view image that is an image obtained by drawing the virtual object that is the distant view object over a range wider than the viewing angle, and drawing of a distant view mapping object and a near view object to which the distant view image is mapped. Based on the virtual space image generation unit that generates the virtual space image by performing processing, the viewpoint position, and the distant view threshold, it is determined whether or not to redraw the distant view image in the distant view image generation unit. It is characterized in that the distant view image drawing determination unit is provided , and the distant view threshold is determined according to the sight of the observer .
The virtual space image presentation system of the present invention is a virtual space image presentation system that generates a virtual space image showing a virtual object in the virtual space according to each of the viewpoint position and the line-of-sight direction of the observer and presents the virtual space image to the observer. The distant view object extraction unit that determines whether or not the virtual object is a distant view object arranged at a distance exceeding the distant view threshold from the viewpoint position of the observer in the virtual space, and the distant view object. The virtual space image is created by performing drawing processing of a distant view image generation unit that generates a distant view image that is an image of a virtual object drawn over a wider range than the viewing angle, and a distant view mapping object and a near view object to which the distant view image is mapped. It is provided with a virtual space image generation unit for generating a distant view image, and a distant view image drawing determination unit for determining whether or not to redraw the distant view image in the distant view image generation unit based on the viewpoint position and the distant view threshold value. When the distant view image drawing determination unit determines that the viewpoint movement distance of the viewpoint position from the distant view drawing viewpoint position, which is the viewpoint position when the distant view image is drawn, exceeds a preset distant view drawing threshold. It is characterized in that it is determined to draw the distant view image, the viewpoint position at the time of drawing the distant view is changed to the viewpoint position, and the distant view drawing threshold is determined according to the sight of the observer.

In the virtual space image presentation system of the present invention, the distant view mapping object is composed of a plurality of primitive planes or celestial spheres, and is arranged at a position where the distance from the viewpoint position becomes the distant view threshold value. It is a feature.

In the virtual space image presentation system of the present invention, the distant view in which the viewpoint movement distance of the viewpoint position is preset from the distant view drawing viewpoint position which is the viewpoint position when the distant view image drawing determination unit draws the distant view image. When it is determined that the drawing threshold value has been exceeded, it is determined that the distant view image is drawn, the viewpoint position at the time of drawing the distant view is changed to the viewpoint position, and the distant view drawing threshold value is determined according to the visual acuity of the observer. It is characterized by that.

In the virtual space image presentation system of the present invention, when the distant view image drawing determination unit determines that the distant view image is drawn when the distant view object in the virtual space is changed, and draws the distant view image. It is characterized in that the viewpoint position at the time of drawing a distant view, which is the viewpoint position, is changed to the viewpoint position.

In the virtual space image presentation system of the present invention, when the distant view image generation unit determines that the distant view image drawing determination unit draws the distant view image, the distant view image presentation system is based on the viewpoint position after moving from the distant view drawing viewpoint position. It is characterized by drawing a distant view image of the distant view object.

In the virtual space image presentation system of the present invention, when the distant view image generation unit determines that the distant view image drawing determination unit does not draw a distant view, the distant view image is deformed according to the viewpoint movement, and the distant view mapping is performed. It is characterized by mapping to an object.

The virtual space image presentation system of the present invention is characterized in that the distant view threshold value is determined before the contents of the virtual space in which the virtual object is arranged are presented to the observer.
The virtual space image presentation system of the present invention is characterized in that the measurement content for measuring the visual acuity of the observer is displayed, and the distant view threshold is determined based on the visual acuity measured using the measurement content. And.

The virtual space image presentation system of the present invention is characterized in that the distant view drawing threshold value is determined before the contents of the virtual space in which the virtual object is arranged are presented to the observer.
The virtual space image presentation system of the present invention displays measurement content for measuring the visual acuity of the observer, and determines the distant view drawing threshold based on the visual acuity measured using the measurement content. It is a feature.

In the virtual space image presentation system of the present invention, when the virtual space image generation unit generates a binocular difference image for the observer to view a stereoscopic image, the viewpoint positions are the right eye and the left eye. It is set for each position of and, and is characterized in that the binocular disparity image is generated corresponding to each of the right eye and the left eye.

The virtual space image presentation method of the present invention is a virtual space image presentation method that generates a virtual space image showing a virtual object in the virtual space according to each of the viewpoint position and the line-of-sight direction of the observer and presents the virtual space image to the observer. A distant view object extraction process in which the distant view object extraction unit determines whether or not the virtual object is a distant view object arranged at a distance exceeding the distant view threshold from the viewpoint position of the observer in the virtual space. The distant view image is mapped to the distant view image generation process in which the distant view image generation unit generates a distant view image which is an image obtained by drawing the virtual object which is the distant view object over a range wider than the viewing angle, and the virtual space image generation unit maps the distant view image. The virtual space image generation process of generating the virtual space image by performing the drawing process of the distant view mapping object and the near view object, and the distant view image drawing determination unit determines the distant view based on the viewpoint position and the distant view threshold. It includes a distant view image drawing determination process for determining whether or not to redraw an image in the distant view image generation unit, and is characterized in that the distant view threshold is determined in correspondence with the observer's eyesight .

The program of the present invention generates a virtual space image showing a virtual object in the virtual space according to each of the viewpoint position and the line-of-sight direction of the observer, and computerizes the function of the virtual space image presentation system to be presented to the observer. A distant view object extraction means for determining whether or not the computer is a distant view object arranged in the virtual space at a distance exceeding the distant view threshold from the viewpoint position of the observer. A distant view image generation means for generating a distant view image, which is an image obtained by drawing the virtual object, which is the distant view object, over a range wider than the viewing angle, a distant view mapping object to which the distant view image is mapped, and a near view object drawing process. A distant view image drawing that determines whether or not to redraw the distant view image by the distant view image generation means based on the virtual space image generation means for generating the virtual space image, the viewpoint position, and the distant view threshold. It is a program that functions as a determination means and determines the distant view threshold value corresponding to the observer's eyesight .

As described above, according to the present invention, it is not necessary to draw a distant view object on a panoramic image in advance, the processing load due to drawing can be reduced, and the observer when observing a virtual space Virtual space image presentation system, virtual space image presentation method and program that have no limit on the movable range with respect to the viewpoint position and do not need to readjust the distance to the boundary that separates the virtual object into the distant view and the near view depending on the scene. Can be provided.

It is a block diagram which shows the structural example of the virtual space image presentation system by one Embodiment of this invention. It is a drawing explaining the drawing of the distant view image used for the virtual space image by one Embodiment of this invention. It is a conceptual diagram explaining the correspondence relation in each definition of a distant view threshold value TO and a distant view drawing threshold value TF. It is a figure which shows the distant view image CI generated by the drawing of the distant view object in FIG. It is a flowchart which shows the operation example of the virtual space image generation processing by the virtual space image presentation system of one Embodiment of this invention. It is a conceptual diagram explaining the drawing of the distant view image used for the virtual space image by one Embodiment of this invention. It is a figure which shows the distant view image CI generated by the drawing of the distant view object in FIG. It is a conceptual diagram explaining the drawing of the distant view image used for the virtual space image by one Embodiment of this invention. It is a figure which shows the distant view image CI generated by the drawing of the distant view object in FIG. It is a figure explaining the determination of presence / absence of update of a distant view image at the time of generating a binocular disparity image as a virtual space image.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a configuration example of a virtual space image presentation system according to an embodiment of the present invention. In FIG. 1, the virtual space image presentation system 1 includes an image generation server 10, an operation unit 11, a display unit 12, and a drawing information storage unit 13.

The operation unit 11 is an input means such as a keyboard or a mouse, and is a base of a virtual space image (or panoramic image) that the user (observer) causes the image generation server 10 to display on the display screen of the display unit 12 (described later). Enter the command etc. to specify the virtual space to be. Further, in addition to the keyboard or mouse described above, the operation unit 11 has a pointing device used for the user to operate the image displayed on the display screen of the display unit 12, such as a touch pad, a touch panel, and a stylus. Alternatively, it may be a trackball or the like.

Further, the operation unit 11 may include a device for acquiring the operating state of the user, for example, a motion sensor, a gyro sensor, or the like. Further, the operation unit 11 may be an HMD (Head Mounted Display) or the like equipped with a device such as a motion sensor. Further, the operation unit 11 outputs information (hereinafter, operation information) indicating an instruction or the like for designating the virtual space input by the user to the image generation server 10. This operation information includes information indicating a viewpoint position, a line-of-sight direction, and a viewing angle in a virtual space desired to be viewed by the user.

The image generation server 10 generates a virtual space image, which is an image of the virtual space to be displayed on the display unit 12, based on each of the user's viewpoint position and line-of-sight direction supplied from the operation unit 11. The image generation server 10 is a user in a virtual space based on virtual space information which is information about a virtual space (including a virtual object as a three-dimensional shape model in the virtual space) stored in the drawing information storage unit 22 described later. The virtual space image is generated as an image corresponding to the viewpoint position of.

Here, in the virtual space in the virtual space information, a plurality of virtual objects (sometimes referred to as objects) and a plurality of light sources for irradiating the virtual objects with light are arranged. The virtual space information includes information such as the three-dimensional shape and material (color, texture image) of the virtual object that can be arranged in the virtual space, the position and orientation of the virtual object arranged in the virtual space, and the light source. Information on each position, irradiation direction, color (wavelength) of the irradiated light, intensity of the irradiated light, and the like is included.

The operation unit 11 described above outputs information on the user's viewpoint position, line-of-sight direction, and viewing angle in the virtual space to the image generation server 10 based on the information operated and input by the user.
As a result, the image generation server 10 projects the virtual space onto a two-dimensional plane perpendicular to the line-of-sight direction in correspondence with the user's viewpoint position, line-of-sight direction, and viewing angle supplied from the operation unit 11, and the projected 2 Output a dimensional image as a virtual space image.
Then, the image generation server 10 outputs the generated virtual space image to the display unit 12. Further, the type of the virtual space image generated by the image generation server 10 from the virtual space information and the process of generating the virtual space image will be described later.

The display unit 12 is a display, and includes, for example, a liquid crystal display (LCD), an organic EL (Organic Electroluminescence) display, a CRT (Cathode Ray Tube), and the like. Further, as described above, the display unit 12 displays the virtual space image supplied from the image generation server 10 on its own display screen.
The image generation server 10, the operation unit 11, and the display unit 12 may all operate on the same computer, or each or a part of these components may operate on different computers and network. The virtual space image presentation system 1 may be configured by communicating with each other via the above.

The drawing information storage unit 13 stores data such as various computer programs used in the virtual space image presentation system 1 and virtual space information in advance. Further, the drawing information storage unit 13 also has a function as a temporary storage area used in various arithmetic processes and the like in the virtual space image presentation system 1. The drawing information storage unit 13 is a storage medium, for example, an HDD (Hard Disk Drive), a flash memory, an EEPROM (Electrically Erasable Programmable Read Only Memory), a RAM (Random Access Memory), a pseudo SRAM, and a ROM (Read Only Memory). , Or any combination thereof.

The virtual space information and the like stored in the drawing information storage unit 13 are referred to when the image generation server 10 generates a virtual space image. The drawing information storage unit 13 may be a database in the same computer as the image generation server 10 or a database in an external computer. When the drawing information storage unit 13 is a database in an external computer, the image generation server 10 refers to virtual space information and the like stored in the drawing information storage unit 13 via, for example, a network.

The image generation server 10 includes an operation information input unit 101, a distant view object extraction unit 102, a distant view image drawing determination unit 103, a distant view image generation unit 104, a virtual space image generation unit 105, and a virtual space image output unit 106. ..
The operation information input unit 101 outputs the operation information supplied from the operation unit 11 to each unit in the image generation server 10 as needed.
The distant view object extraction unit 102 selects each of the virtual objects in the virtual space as a distant view object and a near view object, respectively. At this time, the distant view object extraction unit 102 determines whether or not the distance between each of the virtual objects and the viewpoint position is equal to or greater than a preset distant view threshold value (distant view threshold value TO described later), and the distance is equal to or greater than the distant view threshold value. The virtual object having is extracted as a distant view object.

In the distant view image drawing determination unit 103, the moving distance of the viewpoint position from the distant view drawing viewpoint position (the distant view image drawing viewpoint position PP described later), which is the viewpoint position at the time when the current distant view image is drawn, is preset. When the distant view drawing threshold value or more is reached, it is determined that the current reference distant view image PI drawn at the viewpoint position at the time of distant view drawing is updated. Further, the distant view image drawing determination unit 103 determines that the distant view image is generated when the reference distant view image PI is not created.

FIG. 2 is a conceptual diagram illustrating drawing of a distant view image used for a virtual space image according to an embodiment of the present invention. FIG. 2 shows a plan view of the virtual space at the start of the content or at the time when the distant view image is drawn. That is, a diagram is shown in which a three-dimensional virtual space consisting of an X-axis, a Z-axis, and a Y-axis is projected onto a two-dimensional plane consisting of the X-axis and the Z-axis. The Y-axis is an axis in the height direction perpendicular to this two-dimensional plane. In FIG. 2, the distant view threshold TO is a threshold value for determining whether a three-dimensional virtual object in a three-dimensional space is a near view object or a distant view object, and is a threshold for determining whether a virtual object having a three-dimensional shape is a near view object or a distant view object. It is defined by the distance. As described above, the distant view object extraction unit 102 determines that the virtual object existing at the position where the distance from the viewpoint position CP is equal to or greater than the distant view threshold value TO is the distant view object FO (FO_1, FO_1), while the distant view object extraction unit 102 determines that the virtual object is less than the distant view threshold value TO. The virtual object existing at the position of is determined as the near view object NO (NO_1, NO_2).

The distant view mapping object FMO is an object installed in a virtual space for mapping a distant view image in which a distant view object FO is drawn. For example, the radius is a distant view threshold TO centered on the viewpoint position CP when drawing a distant view image. It is a celestial sphere whose distance is the distance from the viewpoint, or a three-dimensional screen body having a plurality of primitive planes whose distance from the viewpoint position CP is the distant view threshold TO.

The viewpoint position CP indicates the position of the viewpoint of the moving observer. Further, the viewpoint position PP at the time of drawing the distant view image indicates the position of the viewpoint position CP at the time when the reference distant view image PI is drawn. In FIG. 2, since it is the time when the distant view image is drawn, the viewpoint position PP at the time of drawing the distant view image exists at the same position as the viewpoint position CP.
The distant view drawing threshold TF is a threshold value for determining whether or not to update the reference distant view image PI, and is a viewpoint position after the observer moves from the distant view image drawing viewpoint position PP on which the reference distant view image PI is drawn. It is defined as the distance to the CP.

As described above, the distant view image drawing determination unit 103 updates the reference distant view image PI when the distance from the viewpoint position PP at the time of drawing the distant view image to the viewpoint position CP after the observer moves is equal to or larger than the distant view drawing threshold TF. .. On the other hand, when the distance from the viewpoint position PP at the time of drawing the distant view image to the viewpoint position CP after the observer moves is less than the distant view drawing threshold TF, the distant view image drawing determination unit 103 does not update the reference distant view image PI. Continue to use the reference distant view image PI.

FIG. 3 is a conceptual diagram illustrating the correspondence in the definitions of the distant view threshold value TO and the distant view drawing threshold value TF. FIG. 3 shows a plan view of the virtual space at the start of the content or at the time when the distant view image is drawn, as in FIG. That is, a diagram is shown in which a three-dimensional virtual space consisting of an X-axis, a Z-axis, and a Y-axis is projected onto a two-dimensional plane consisting of the X-axis and the Z-axis.
As described above, the distant view threshold value TO is a threshold value for the distance from the viewpoint position PP at the time of drawing the distant view image to the virtual object F. When the virtual object F is located at a distance equal to or greater than the distant view threshold TO from the viewpoint position PP at the time of drawing the distant view image, the virtual object F is determined as a distant view object and is extracted from the virtual space. On the other hand, when the virtual object F is located at a distance less than the distant view threshold TO from the viewpoint position PP at the time of drawing the distant view image, the virtual object F is determined as a near view object. Further, the distant view drawing threshold value TF is a threshold value for determining whether or not to update the reference distant view image PI. When the distance moved by the observer's viewpoint position CP from the distant view drawing threshold PP is equal to or greater than the distant view drawing threshold TF, the reference distant image PI is updated, while when the distance is less than the distant view drawing threshold TF, the reference distant image PI is updated. Do not update.

For each of the distant view threshold value TO and the distant view drawing threshold value TF, the creator of the content, the observer of the content, or the like may input arbitrary values.
However, in the present embodiment, a virtual object determined to be a distant view object is drawn, a distant view image is generated, this distant view image is mapped to the distant view mapping object FMO, and the viewpoint position CP of the observer is the distant view drawing threshold value. The distant view object is not updated unless it is moved from the viewpoint position PP when drawing the distant view image to a distance of TF or more. Hereinafter, the distance traveled by the observer's viewpoint position CP from the viewpoint position PP when drawing a distant view image may be referred to as a distance LP.

Therefore, when the distance moved from the viewpoint position PP at the time of drawing the distant view image by the observer's viewpoint position CP is less than the distant view drawing threshold value TF, it is more difficult to recognize the change in the distant view image corresponding to each visual sense of the observer. It is necessary to make it a state.
Therefore, in the present embodiment, each of the distant view threshold value TO and the distant view drawing threshold value TF is set corresponding to the visual acuity of the observer and corresponds to each visual sense of the observer.

That is, the distant view drawing threshold value TF may be obtained by inputting each of the visual acuity value E and the distant view threshold value TO [m] by the content creator, the content observer, or the like, and based on those values.
For example, the distant view drawing threshold TF [m] is expressed by the following equation (1) based on each of the visual acuity value E and the distant view threshold TO.
TF = TO * sin ((π) / (180 * 60 * E)) / 2 ... (1)
According to the above equation (1), for example, when the visual acuity value E = 1.0 and the distant view threshold value TO = 100 [m], the distant view drawing threshold value TF ≈ 0.0145 [m] is obtained. Then, the operating range having a radius of about 0.0145 [m] from the viewpoint position PP at the time of drawing the distant view image is set as a region where the change in the appearance of the virtual object separated by 100 [m] or more is hard to notice. As a result, in the virtual space, the distant view mapping object FMO is provided at a distance of 100 [m] from the viewpoint position PP at the time of drawing the distant view image.

Further, the distant view threshold value TO may be obtained by inputting the visual acuity value E and the distant view drawing threshold value TF [m] by the content creator, the content observer, or the like, and based on those values.
For example, the distant view threshold TO [m] is expressed by the following equation (2) based on each of the visual acuity value E and the distant view drawing threshold TF.
TO = 2 * TF / (sin ((π) / (180 * 60 * E))) ... (2)
According to the above equation (2), for example, when the visual acuity value E = 1.0 and the distant view drawing threshold TF = 0.1 [m], the distant view threshold value TO ≈688 [m] becomes 688 [m] or more. The distant virtual object becomes a distant view object in which the change in visual acuity is hard to notice in the operating range with a radius of 0.1 [m] from the viewpoint position PP at the time of drawing the distant view image, and the distant view threshold TO is set as 688 [m].

As described above, by setting the range to which the observer's visual acuity and the viewpoint position are to be moved, or the distance to the observer's visual acuity and the virtual object to be drawn as a distant view object, the virtual object in the virtual space is based on human vision. Since each of the distant view threshold value TO and the distant view drawing threshold value TF can be set as parameters that make it difficult to notice the change in the appearance of the object, it is possible to set the parameters regardless of the scene.

Returning to FIG. 1, when the distant view image drawing determination unit 103 determines that the distant view image is to be updated, the distant view image generation unit 104 draws the distant view object extracted by the distant view object extraction unit 102 and outputs the distant view image CI to the operation unit 11. It is generated based on the operation information including each of the user's viewpoint position and line-of-sight direction supplied from. Then, the distant view image generation unit 104 writes and stores the generated distant view image CI as a reference distant view image PI in the drawing information storage unit 13.

On the other hand, when the distant view image generation unit 104 does not determine that the distant view image drawing determination unit 103 updates the distant view image, the distant view image generation unit 104 deforms the reference distant view image PI according to the movement from the viewpoint position PP to the viewpoint position CP at the time of drawing the distant view image. I do. This transformation process is, for example, affine transformation, in which the image area in the moving direction (in the direction in which the distance becomes smaller) from the viewpoint position PP at the time of drawing a distant view image is enlarged and the image area in the opposite direction is reduced. It is a process to do. The distant view image generation unit 104 writes and stores the modified distant view image PI as the distant view image CI in the drawing information storage unit 13.

FIG. 4 is a diagram showing a distant view image CI generated by drawing a distant view object in FIG. 2. In the distant view image CI, each of the distant view objects FO_1 and FO_2 is drawn corresponding to the distance from the viewpoint position PP at the time of drawing the distant view image and the line-of-sight direction of the observer at the viewpoint position PP at the time of drawing the distant view image.

Returning to FIG. 1, the virtual space image generation unit 105 installs the distant view mapping object FMO as a celestial sphere having a radius of, for example, a distant view threshold TO centered on the viewpoint position CP. Then, the virtual space image generation unit 105 reads out the reference distant view image PI generated by the distant view image generation unit 23 from the drawing information storage unit 13, and reads the read reference distant view image PI from the inner surface (viewpoint position CP) of the distant view mapping object FMO. Map to the surface facing with).
Then, the virtual space image generation unit 105 simultaneously displays the distant view mapping object FMO to which the reference distant view image PI is mapped and the near view object NO in the virtual space on a plane perpendicular to the line-of-sight direction of the observer, that is, the display unit 12. A virtual space image I is generated by projecting and drawing on a two-dimensional plane corresponding to the display screen of.

The virtual space image output unit 106 outputs the virtual space image I generated by the virtual space image generation unit 105 to the display unit 12.

FIG. 5 is a flowchart showing an operation example of a virtual space image generation process by the virtual space image presentation system according to the embodiment of the present invention.
Step S101:
The operation information input unit 101 reads from the operation unit 11, for example, operation information including at least the viewpoint position CP of the observer, the line-of-sight direction, the distant view drawing threshold TF, and the distant view threshold TO.

Step S102:
The distant view image drawing determination unit 103 reads the data of the virtual space including the virtual object from the drawing information storage unit 13, and specifies the position (position in the three-dimensional coordinate system) where the observer's viewpoint position CP in the virtual space is arranged. do. At this time, the distant view image drawing determination unit 103 also reads the viewpoint position PP at the time of drawing the distant view image from the drawing information storage unit 13.

Step S103:
The distant view image drawing determination unit 103 determines whether or not the read virtual space data has been changed from the previously read virtual space data. The distant view image drawing determination unit 103 confirms whether or not there is a change, for example, when the three-dimensional shape of the virtual object changes, when the existing virtual object disappears, or when a non-existing virtual object appears. I do.
At this time, the distant view image drawing determination unit 103 does not need to generate the distant view image if the read virtual space data has not been changed from the previously read virtual space data, so the process proceeds to step S104. On the other hand, if the read virtual space data is changed from the immediately read virtual space data, the distant view image drawing determination unit 103 needs to generate a distant view image, so the process proceeds to step S107.

Step S104:
The distant view image drawing determination unit 103 refers to the drawing information storage unit 13 to determine whether or not the reference distant view image PI does not exist (whether or not it does not exist).
At this time, the distant view image drawing determination unit 103 does not need to draw the distant view image when the reference distant view image PI does not exist in the drawing information storage unit 13, so the process proceeds to step S105. On the other hand, when the reference distant view image PI exists in the drawing information storage unit 13, the distant view image drawing determination unit 103 needs to draw the distant view image, so the process proceeds to step S107.

Step S105:
The distant view image drawing determination unit 103 determines whether or not the distance LP between the viewpoint position PP at the time of drawing the distant view image and the viewpoint position CP of the observer is equal to or greater than the distant view drawing threshold value TF.
At this time, the distant view image drawing determination unit 103 processes the distant view image because it is not necessary to change the distant view image when the distance LP between the viewpoint position PP at the time of drawing the distant view image and the viewpoint position CP of the observer is less than the distant view drawing threshold value TF. To step S106. On the other hand, when the distance LP between the viewpoint position PP at the time of drawing the distant view image and the viewpoint position CP of the observer is equal to or larger than the distant view drawing threshold value TF, the distant view image drawing determination unit 103 needs to change the distant view image. Proceed to step S107.

Step S106:
Since the distant view image generation unit 104 does not determine that the distant view image drawing determination unit 103 updates the distant view image, the distant view image PI is read from the drawing information storage unit 13.
Then, the distant view image generation unit 104 performs deformation processing of the shape of the reference distant view image PI according to the movement from the viewpoint position PP at the time of drawing the distant view image to the viewpoint position CP, generates the distant view image CI, and generates the drawing information storage unit. Write to 13 and memorize it.

FIG. 6 is a conceptual diagram illustrating drawing of a distant view image used for a virtual space image according to an embodiment of the present invention. In FIG. 6, since the distance LP, which is the distance between the viewpoint position PP at the time of drawing the distant view image and the viewpoint position CP after the observer moves, is less than the distant view drawing threshold value TF, the distant view image drawing determination unit 103 updates the distant view image. Do not do.
Therefore, the distant view image generation unit 104 performs deformation processing of the reference distant view image PI of the distant view objects FO_1 and FO_2, and generates a distant view image CI as a result of the deformation.

FIG. 7 is a diagram showing a distant view image CI generated by drawing a distant view object in FIG. FIG. 7A is a reference distant view image PI generated according to the distance from each of the distant view objects FO_1 and FO_2 from the viewpoint position PP at the time of drawing the distant view image in FIG.
FIG. 7B is a distant view image CI generated by transforming the reference distant view image PI according to the distances from the viewpoint position CP in FIG. 6 to the distant view objects FO_1 and FO_2, respectively. Since the distance between the distant view object FO_1 and the distant view object FO_2 is reversed, the distant view object FO_1 is transformed into a shape that looks larger than the distant view object FO_2.

Step S107:
The distant view image drawing determination unit 103 writes and stores the viewpoint position CP of the observer as the viewpoint position PP at the time of drawing the distant view image in the drawing information storage unit 13. As a result, the distant view image drawing determination unit 103 updates the viewpoint position PP at the time of drawing the distant view image by the viewpoint position CP.

Step S108:
The distant view object extraction unit 102 reads the viewpoint position PP and the distant view threshold value TO at the time of drawing the distant view image from the drawing information storage unit 13. Then, the distant view object extraction unit 102 extracts a virtual object whose distance from the viewpoint position PP at the time of drawing the distant view image in the virtual space is equal to or greater than the distant view threshold value TO, and sets the distant view objects FO_1 and FO_2. Further, the distant view object extraction unit 102 sets a virtual object whose distance from the viewpoint position PP at the time of drawing the distant view image in the virtual space is less than the distant view threshold value TO as a near view object.

Further, the distant view image generation unit 104 reads the viewpoint position PP at the time of drawing the distant view image from the drawing information storage unit 13 and generates a distant view image CI to be mapped (pasted) to the distant view mapping object FMO.
Then, the distant view image generation unit 104 draws each of the distant view objects FO_1 and FO_2 corresponding to the viewpoint position PP at the time of drawing the distant view image, generates the distant view image CI, and uses the drawing information storage unit 13 as the reference distant view image PI. Write to and memorize.

FIG. 8 is a conceptual diagram illustrating drawing of a distant view image used for a virtual space image according to an embodiment of the present invention. In FIG. 8, since the distance LP, which is the distance between the viewpoint position PP at the time of drawing the distant view image and the viewpoint position CP after the observer moves, is equal to or higher than the distant view drawing threshold value TF, the distant view image drawing determination unit 103 updates the distant view image. I do. Here, the virtual object determined to be the distant view object FO_2 becomes the near view object NO_3 due to the movement of the viewpoint position CP. The distant view object extraction unit 102 extracts only the distant view object FO_1 as a distant view object.
Therefore, the distant view image generation unit 104 draws the distant view object FO_1 to generate the distant view image CI, and uses the distant view image CI as the reference distant view image PI to update the distant view image.

FIG. 9 is a diagram showing a distant view image CI generated by drawing a distant view object in FIG. FIG. 9 is a reference distant view image PI generated according to the distance from the distant view object FO_1 from the distant view drawing viewpoint position PP in FIG.
As described above, in the distant view objects FO_1 and FO_2 in FIG. 6, since the distant view object FO_2 becomes the near view object NO_3, only the distant view object FO_1 is drawn in the reference distant view image PI of FIG.

Step S109:
The distant view image generation unit 104 writes (overwrites) the generated distant view image CI in the drawing information storage unit 13 as a reference distant view image PI. As a result, the distant view image generation unit 104 updates the reference distant view image PI, which is a distant view image of the distant view object in the virtual space.

Step S110:
The virtual space image generation unit 105 installs the distant view mapping object FMO as a celestial sphere whose radius is the distant view threshold TO, for example, centered on the viewpoint position CP.
Then, the virtual space image generation unit 105 reads the reference distant view image PI from the drawing information storage unit 13 when the distant view image is updated, and reads the distant view image CI from the drawing information storage unit 13 when the distant view image is not updated. The virtual space image generation unit 105 maps the read reference distant view image PI or distant view image CI to the distant view mapping object FMO.

The virtual space image generation unit 105 simultaneously displays the distant view mapping object FMO to which the reference distant view image PI is mapped and the near view object NO in the virtual space on a plane perpendicular to the line-of-sight direction of the observer, that is, the display unit 12. A virtual space image I is generated by projecting and drawing on a two-dimensional plane corresponding to a screen.
The virtual space image output unit 106 outputs the virtual space image I generated by the virtual space image generation unit 105 to the display unit 12, and returns the process to step S101.

According to the present embodiment, when the distance LP from the viewpoint position PP at the time of drawing the distant view image to the viewpoint position CP after the observer moves is less than the preset distant view drawing threshold TF, the drawing is already performed. Only by transforming the referenced distant view image PI, it is not necessary to draw the distant view object on the panoramic image in advance each time, and the processing load due to drawing can be significantly reduced.

Further, according to the present embodiment, unlike the conventional case, the necessary distant view image is not prepared in advance, but the distant view image is generated and updated as necessary, so that the observation when observing the virtual space is performed. There is no limit to the range of movement with respect to the viewpoint position of the person, and it can be effectively used for applications such as walkthroughs.

Further, according to the present embodiment, the distant view threshold value as the distance to the boundary that separates the virtual object into the distant view and the near view in the virtual space is set in advance based on the distant view drawing threshold value corresponding to the observer's eyesight. It is not necessary to adjust the distance to the boundary that separates the virtual object into the distant view and the near view for each scene of the content, and it is possible to reduce the addition to the observer and the creator of the content.

Further, in the present embodiment, the configuration in which the visual acuity of the observer is input from the operation unit 11 has been described, but as shown below, the image generation server 10 has the visual characteristics of the observer (for example, the visual acuity of the observer). It may be configured to perform detection. That is, before presenting the content, the image generation server 10 presents the measurement content on the display screen of the display unit 12 so as to determine the distant view threshold value TO or the distant view drawing threshold value TF, which makes it difficult for the observer to notice the change in appearance. , Let the observer observe. The measurement content presents, for example, two images drawn by shifting the viewpoint position by the distant view drawing threshold TF × 2 with respect to the measurement object installed at the distance D to the observer while increasing the distance D. Then, the content is such that the distance D, which is determined for the first time by the observer that there is no change in the appearance of the measurement objects of the two images, is set as the distant view threshold TO.
Further, two images drawn by shifting the viewpoint position by the distance D × 2 with respect to the measurement object separated by the distant view threshold TO are presented to the observer while increasing the distance D, and the observer has two images. The content is such that the distance D, which is determined for the first time that the appearance of the image measurement object has changed, is set as the distant view drawing threshold TF.

Further, in the above-described embodiment, the configuration for generating the virtual space image to be displayed on the display screen of the display unit 12 has been described, but the display unit for the left eye and the right side of the HMD (Head Mounted Display) have been described. It can also be used to generate a binocular disparity image to be displayed on the ocular display unit.
In the process for generating the binocular parallax image, only the points different from the embodiments already described will be described below.

Here, as the viewpoint position CP, any of the intermediate positions between the right eye ER, the left eye EL, the right eye ER, and the left eye EL may be used. In the following description, the viewpoint position CP is, for example, an intermediate position between the right eye ER and the left eye EL.
FIG. 10 is a diagram illustrating determination of whether or not a distant view image is updated when a binocular parallax image as a virtual space image is generated.
In FIG. 10, the range in which the distant view image is not updated is the inside of the circle (excluding the arc portion of the circumference) centered on the viewpoint position PP at the time of drawing the distant view image and having the distant view drawing threshold value TF as the radius. When the distance between the viewpoint position PP at the time of drawing the distant view image and either the right eye ER or the left eye EL becomes equal to or greater than the distant view drawing threshold TF due to the movement of the observer, the distant view image drawing determination unit 103 determines the distant view image. Judge to update. Also in this case, the new viewpoint position PP at the time of drawing the distant view image is a numerical value updated by the viewpoint position CP at the position intermediate between the right eye ER and the left eye EL.

At this time, the distant view drawing threshold TF needs to be set larger than the distance between the right eye ER and the left eye EL.
Then, as described above, the distant view image generation unit 104 generates a distant view image CI of the distant view object extracted by the distant view object extraction unit 102, and uses it as a reference distant view image PI.
The virtual space image generation unit 105 makes each of the distant view mapping object FMO and the near view object N to which the distant view image is mapped correspond to the viewpoint position PP at the time of drawing the distant view image, respectively, for the right eye image and the left eye image. It is generated by drawing as a virtual space image of, and the generated virtual space image is output as a binocular disparity image.

A program for realizing the function of the virtual space image presentation system 1 of FIG. 1 in the present invention is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into the computer system and executed. By doing so, when the observer observes the virtual space, the virtual space image generation process at the viewpoint position in the virtual space presented to the observer may be performed. The term "computer system" as used herein includes hardware such as an OS and peripheral devices.
Further, the "computer system" shall also include a WWW system provided with a homepage providing environment (or display environment). Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, and a storage device such as a hard disk built in a computer system. Furthermore, a "computer-readable recording medium" is a volatile memory (RAM) inside a computer system that serves as a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, it shall include those that hold the program for a certain period of time.

Further, the program may be transmitted from a computer system in which this program is stored in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the "transmission medium" for transmitting a program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. Further, the above program may be for realizing a part of the above-mentioned functions. Further, a so-called difference file (difference program) may be used, which can realize the above-mentioned function in combination with a program already recorded in the computer system.

The virtual space image presentation system, which allows the observer to freely move the viewpoint position, can present high-definition and interactive video content by low-load drawing processing, and is suitable for image presentation in the entertainment industry and medical field. It can be applied.
That is, if the processing load of the virtual space image presentation system can be reduced according to the present invention, it is possible to contribute to the provision of higher-definition and interactive new contents by allocating the reduced load to other resources.

1 ... Virtual space image presentation system 10 ... Image generation server 11 ... Operation unit 12 ... Display unit 13 ... Drawing information storage unit 101 ... Operation information input unit 102 ... Distant view object extraction unit 103 ... Distant view image drawing determination unit 104 ... Distant view image generation Part 105 ... Virtual space image generation part 106 ... Virtual space image output part TF ... Far view drawing threshold TO ... Far view threshold

Claims (14)

It is a virtual space image presentation system that generates a virtual space image showing a virtual object in the virtual space and presents it to the observer according to each of the viewpoint position and the line-of-sight direction of the observer.
A distant view object extraction unit that determines whether or not the virtual object is a distant view object arranged at a distance exceeding the distant view threshold value from the viewpoint position of the observer in the virtual space.
A distant view image generation unit that generates a distant view image, which is an image obtained by drawing the virtual object, which is the distant view object, over a range wider than the viewing angle.
A virtual space image generation unit that generates the virtual space image by performing drawing processing of the distant view mapping object and the near view object to which the distant view image is mapped.
It is provided with a distant view image drawing determination unit that determines whether or not to redraw the distant view image in the distant view image generation unit based on the viewpoint position and the distant view threshold value .
The distant view threshold is determined according to the visual acuity of the observer.
A virtual space image presentation system characterized by this. It is a virtual space image presentation system that generates a virtual space image showing a virtual object in the virtual space and presents it to the observer according to each of the viewpoint position and the line-of-sight direction of the observer.
A distant view object extraction unit that determines whether or not the virtual object is a distant view object arranged at a distance exceeding the distant view threshold value from the viewpoint position of the observer in the virtual space.
A distant view image generation unit that generates a distant view image, which is an image obtained by drawing the virtual object, which is the distant view object, over a range wider than the viewing angle.
A virtual space image generation unit that generates the virtual space image by performing drawing processing of the distant view mapping object and the near view object to which the distant view image is mapped.
It is provided with a distant view image drawing determination unit that determines whether or not to redraw the distant view image in the distant view image generation unit based on the viewpoint position and the distant view threshold value .
When the distant view image drawing determination unit determines that the viewpoint movement distance of the viewpoint position exceeds a preset distant view drawing threshold value from the distant view drawing viewpoint position which is the viewpoint position when the distant view image is drawn. It is determined that the distant view image is drawn, and the viewpoint position at the time of drawing the distant view is changed to the viewpoint position.
The distant view drawing threshold is determined according to the visual acuity of the observer.
A virtual space image presentation system characterized by this. When the distant view image drawing determination unit determines that the viewpoint movement distance of the viewpoint position from the distant view drawing viewpoint position, which is the viewpoint position when the distant view image is drawn, exceeds a preset distant view drawing threshold value. It is determined that the distant view image is drawn, and the viewpoint position at the time of drawing the distant view is changed to the viewpoint position.
The distant view drawing threshold is determined according to the visual acuity of the observer.
The virtual space image presentation system according to claim 1 . Claims 1 to 3 are characterized in that the distant view mapping object is composed of a plurality of primitive planes or celestial spheres, and is arranged at a position where the distance from the viewpoint position becomes the distant view threshold value. The virtual space image presentation system according to any one of the above. The distant view image drawing determination unit
When the distant view object in the virtual space is changed, it is determined that the distant view image is drawn, and the viewpoint position at the time of drawing the distant view, which is the viewpoint position when the distant view image is drawn, is changed to the viewpoint position. The virtual space image presentation system according to any one of claims 1 to 4 , wherein the virtual space image presentation system is characterized. The distant view image generation unit
When the distant view image drawing determination unit determines to draw the distant view image, the distant view object is based on the viewpoint position after moving from the distant view drawing viewpoint position which is the viewpoint position when the distant view image is drawn. The virtual space image presentation system according to any one of claims 1 to 5 , wherein a distant view image is drawn. The distant view image generation unit
Claims 1 to 6 , wherein when the distant view image drawing determination unit determines that the distant view is not drawn, the distant view image is deformed according to the movement of the viewpoint and mapped to the distant view mapping object. The virtual space image presentation system according to any one of the above. The virtual according to any one of claims 1 to 7, wherein the distant view threshold value is determined before presenting the contents of the virtual space in which the virtual object is arranged to the observer. Spatial image presentation system. Claims 1 to 8 , wherein the measurement content for measuring the visual acuity of the observer is displayed, and the distant view threshold value is determined based on the visual acuity measured using the measurement content. The virtual space image presentation system according to any one of the items . The virtual space image presentation according to claim 2 or 3 , wherein the distant view drawing threshold value is determined before the content of the virtual space in which the virtual object is arranged is presented to the observer. system. 2 . Alternatively, the virtual space image presentation system according to claim 10 . The virtual space image generation unit
When generating a binocular disparity image for the observer to appreciate a stereoscopic image, the viewpoint position is set for each position of the right eye and the left eye, and the right eye and the left eye are set. The virtual space image presentation system according to any one of claims 1 to 11, wherein the binocular disparity image is generated corresponding to each of the above. It is a virtual space image presentation method that generates a virtual space image showing a virtual object in a virtual space and presents it to the observer according to each of the viewpoint position and the line-of-sight direction of the observer.
A distant view object extraction process in which the distant view object extraction unit determines whether or not the virtual object is a distant view object arranged at a distance exceeding the distant view threshold value from the viewpoint position of the observer in the virtual space.
A distant view image generation process in which the distant view image generation unit generates a distant view image, which is an image obtained by drawing the virtual object, which is the distant view object, over a range wider than the viewing angle.
A virtual space image generation process in which the virtual space image generation unit generates the virtual space image by drawing a distant view mapping object and a near view object to which the distant view image is mapped.
The distant view image drawing determination unit includes a distant view image drawing determination process of determining whether or not to redraw the distant view image in the distant view image generation unit based on the viewpoint position and the distant view threshold value.
The distant view threshold is determined according to the visual acuity of the observer.
A virtual space image presentation method characterized by this. It is a program that generates a virtual space image showing a virtual object in the virtual space according to each of the viewpoint position and the line-of-sight direction of the observer, and causes a computer to execute the function of the virtual space image presentation system presented to the observer. ,
The computer
A distant view object extraction means for determining whether or not the virtual object is a distant view object arranged at a distance exceeding the distant view threshold value from the viewpoint position of the observer in the virtual space.
A distant view image generation means for generating a distant view image, which is an image obtained by drawing the virtual object, which is the distant view object, over a range wider than the viewing angle.
A virtual space image generation means for generating the virtual space image by performing drawing processing of the distant view mapping object and the near view object to which the distant view image is mapped.
Based on the viewpoint position and the distant view threshold value, the distant view image is made to function as a distant view image drawing determination means for determining whether or not to redraw the distant view image by the distant view image generation means .
The distant view threshold is determined according to the visual acuity of the observer.
program.

Images