JP2017199237A - Virtual space display system, game system, virtual space display program and game program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a virtual space display system and the like that enhance a feeling (immersive feeling) as if actually entering the virtual space generated in the game of a virtual reality.SOLUTION: A virtual space display system includes: a head mount display; a first detection unit; an image generation unit; and a real object and a second detection unit. The head mount display is mounted by a user in a predetermined actual space. The first detection unit detects a position and a posture of the head mount display in a real space. The image generation unit generates a virtual space corresponding to the real space, and generates an image of the virtual space that must be viewable via the head mount display by a user on the basis of the detection result of the first detection unit. The real object is arranged in the real space. The second detection unit detects the position and the posture of the real body in the real space. The image generation unit generates the image of a virtual object corresponding to the real body in the virtual space by the position and the posture based on the detection result of the second detection unit.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a virtual space display system, a game system, a virtual space display program, and a game program for displaying an image of a virtual space on a head mounted display worn by a user in a predetermined real space.

  In recent years, games (game systems) related to virtual reality (virtual reality) using a head-mounted display have been realized (see Non-Patent Document 1, for example). In such a game, an image of the virtual space is displayed on a head mounted display worn by the user. And the image of the virtual space displayed on the same display changes according to the movement of the user moving. Thereby, the user can obtain a feeling (immersive feeling) as if the user actually entered the virtual space.

http://game.watch.impress.co.jp/docs/series/vrgaming/20150818_716564.html

  In the above-mentioned game related to virtual reality, a feeling (immersive feeling) as if it has actually entered the virtual space is obtained visually and auditorily, but in order to further improve the immersive feeling, it is related to other senses such as tactile sensation. Information needs to be given to the user.

  The present invention provides a virtual space display system, a game system, a virtual space display program, and a game program that improve a feeling (immersion feeling) as if actually entering a virtual space generated by a virtual reality game or the like. With the goal.

  The virtual space display system provided by the first aspect of the present invention includes a head mounted display, a first detection unit, an image generation unit, a real object, and a second detection unit. The user mounts the head mounted display in a predetermined real space. A 1st detection part detects the position and attitude | position in the real space of a head mounted display. An image generation part produces | generates the virtual space corresponding to real space, and produces | generates the image of the virtual space which a user should see via a head mounted display based on the detection result of a 1st detection part. Real objects are placed in real space. The second detection unit detects the position and orientation of the real object in real space. The image generation unit generates an image of the virtual object corresponding to the real object with a position and orientation based on the detection result of the second detection unit.

  A game system provided by the second aspect of the present invention includes the virtual space display system, and executes a game in a virtual space.

A virtual space display program provided by the third aspect of the present invention provides a computer for a virtual space display system that displays a virtual space image on a head mounted display worn by a user in a predetermined real space. First detection means for detecting the position and orientation in the space, a virtual space corresponding to the real space is generated, and based on the detection result of the first detection means, the virtual space that the user should see through the head mounted display It functions as an image generation means for generating an image and a second detection means for detecting the position and orientation of the real object arranged in the real space in the real space. The image generation means generates an image of the virtual object corresponding to the real object with a position and orientation based on the detection result of the second detection means.
A game program provided by the fourth aspect of the present invention includes the virtual space display program, and causes a computer to execute a game in a virtual space.

  According to this invention, the virtual object of the real object arranged in the same real space as the user wearing the head mounted display is arranged in the virtual space. Moreover, in the real space, it is arranged at a position in the virtual space corresponding to the positional relationship between the user and the real object. That is, the real object in the real space and the virtual object in the virtual space are linked. Therefore, when the user moves toward and touches the virtual object, the user touches the real object even in the real space. In addition to visual and auditory senses, a sense of touching the virtual object in the virtual space (tactile sense) Can be given to the user. Thereby, the feeling (immersion feeling) as if it actually entered the virtual space can be improved.

It is a figure showing composition of a game system of an embodiment to which this invention is applied. It is a block diagram of a game terminal device, a head mounted display, and a detection device of the game system. It is a figure which shows an example of the game image displayed on a head mounted display. It is a figure which shows an example of the game image displayed on a head mounted display. It is a figure which shows an example of the game image displayed on a head mounted display. It is a figure which shows the positional relationship in the real space of the user and object which mounted | wore the head mounted display. It is a figure which shows the positional relationship in the real space of the user and object which mounted | wore the head mounted display. It is a figure which shows the positional relationship in the real space of the user and object which mounted | wore the head mounted display. It is a functional block diagram of the game system. It is a flowchart which shows the space generation process and image generation process which the game system performs.

  A game system (virtual space display system) according to an embodiment of the present invention will be described with reference to the drawings. The configuration of the present invention is not limited to the embodiment. Further, the processing procedures of the respective flowcharts are not limited to the embodiment and are in no particular order as long as no contradiction occurs.

  FIG. 1 is a diagram showing the configuration of the game system 1 of this embodiment. FIG. 2 is a block diagram showing the configuration of the game terminal device 2, the head mounted display 3, and the detection device 5 of this embodiment.

  The game system 1 is installed in a room R having a floor surface F. The room R is surrounded by the wall surface W on all sides, but the wall surface W is not essential in the present invention. The game system 1 includes a game terminal device 2, a head mounted display (HMD) 3, a laser irradiation device 4 (4A, 4B), and an object (real object) 6. The head mounted display 3 has a function of sensing its position and posture. The object 6 is provided with a detection device 5 that detects the position and orientation of the object 6. The game system 1 executes a predetermined virtual reality (VR) game (hereinafter referred to as a VR game) using these devices.

  The game terminal device 2 reads and executes the game program 70 from the storage unit 27 (see FIG. 2) in which the game program is stored, and generates a three-dimensional virtual space corresponding to the real space including the room R. Then, the game image including the generated virtual space image is displayed on the head mounted display 3. As the game terminal device 2, for example, a dedicated game machine or a personal computer can be applied.

  In the VR game of this embodiment, the position and orientation of the head mounted display 3 worn by the user P is detected, so that the game image in the virtual space corresponding to the movement (movement, change in posture, etc.) of the user P is recorded on the head. It is displayed on the mount display 3 (see FIG. 3). That is, the game image in the virtual space corresponding to the line of sight of the user P is displayed, and the user P can feel as if he / she actually entered the virtual space.

  In the VR game of this embodiment, a virtual object A (see FIG. 3) corresponding to the object 6 arranged in the same real space as the user P is also arranged in the virtual space. Similarly to the head mounted display 3, the position and orientation of the object 6 are detected, and the virtual object A is arranged at a position in the virtual space corresponding to the object 6. That is, the virtual object A is linked to the object 6 with respect to the position and orientation. For example, when the object 6 is lifted in the real space, the virtual object A in the virtual space is also lifted in the same manner.

  The object 6 is not limited to a movable object that can be lifted or moved by the user P, and may be a fixed object. In that case, even if force is applied by the user P, the object 6 does not move, so the virtual object A also does not move.

  In FIG. 2, the head mounted display 3 has an eye mask 52 that covers both eyes of the user P, and a left-eye display 46 and a right-eye display 47 are arranged on the left and right in the eye mask. When the user P puts the eye mask 52 against the head (in front of both eyes) and the head mounted display 3 is mounted on each of the left eye display 46 and the right eye display 47, the eye mask 52 is attached to the real space of the user P. The field of view is blocked, and the left-eye display 46 and the right-eye display 47 are positioned in front of the left eye and right eye of the user P. As a result, the user P views only the virtual space displayed on the left-eye display 46 and the right-eye display 47.

  The laser irradiation devices (light emitting units) 4A and 4B scan the room R with an infrared laser alternately. The head cloak display 3 and the detection device 5 each include a plurality of light receiving sensors 50 and 65 for receiving the infrared laser, and the heads are based on which light receiving sensors 50 and 65 receive the laser at which scan timing. The positions and orientations of the mount display 3 and the object 6 are calculated (detected). In addition, the laser irradiation apparatus 4 is not limited to two, What is necessary is just one or more. In this embodiment, the object 6 is a box-shaped cube and has a weight that can be carried by the user P. The shape of the object 6 is not particularly limited to this. In FIG. 1, the detection device 5 is provided on the upper surface of the object 6, but the position where the detection device 5 is provided is not limited to the upper surface. Any position that can receive an infrared laser is acceptable. Further, the detection device 5 does not need to be provided on the entire object 6. The game program 70 should just memorize | store the positional relationship of the detection apparatus 5 and the virtual object A. FIG.

  The game machine 2 has a control unit 20, an operation unit 26, a storage unit 27, and a wireless communication circuit unit 28 on a bus 29. The control unit 20 includes a CPU 21, a ROM (flash memory) 22, a RAM 23, an image processor 24 and an audio processor 25. A video RAM (VRAM) 30 is connected to the image processor 24.

  The wireless communication circuit unit 28 communicates with the head mounted display 3 and the detection device 5 by Wi-Fi communication. The operation unit 26 detects an operation of the keyboard or the like by the user and inputs the operation signal to the CPU 21. The storage unit 27 is a hard disk, for example, and stores a game program 70. The game program 70 includes game data such as the shape and image of the virtual object A necessary for the progress of the VR game, in addition to the program main body.

  In the RAM 23, a work area used when the CPU 21 executes the game program 70 is set. The work area includes various parameters generated as the VR game progresses. The ROM 22 stores a basic program for executing the game program 70.

  The image processor 24 includes a GPU (Graphics Processing Unit) that can generate a game image. The image processor 24 generates a virtual space in accordance with an instruction from the CPU 21, and draws an image obtained by photographing the virtual space with a virtual camera on the VRAM 30 as a game image. In the present embodiment, a left-eye game image and a right-eye game image corresponding to the user's line of sight estimated based on the position and orientation information of the head mounted display 3 are generated. The generated game image information is transmitted to the head mounted display 3.

  The sound processor 25 has a DSP (Digital Signal Processor) that generates game sound. The sound processor 25 transmits the generated game sound to the head mounted display 3. The game sound is output from the earphone connected to the head mounted display 3.

  The head mounted display 3 includes a control unit 40, a wireless communication circuit unit 44, a VRAM 45, an amplifier 48, and (a plurality of) light receiving sensors 50 on a bus 51. The control unit 40 includes a CPU 41, a ROM (flash memory) 42, and a RAM 43. A left-eye display 46 and a right-eye display 47 are connected to the VRAM 45. An earphone terminal 49 is connected to the amplifier 48.

  The wireless communication circuit unit 44 communicates with the game terminal device 2 by Wi-Fi communication. A game image received from the game terminal device 2 is drawn on the VRAM 45 according to an instruction from the CPU 41. The drawn game image is output to the left-eye display 46 and the right-eye display 47. The game sound received from the game terminal device 2 is output to the amplifier 48. The amplifier 48 amplifies the game sound signal and outputs it to the earphone terminal 49. The earphone terminal 49 is connected to a stereo earphone worn by the user P on both ears, and game sound is output to the user P by the earphone.

  A plurality of light receiving sensors 50 are arranged on the peripheral surface of the head mounted display 3 (eye mask 52), detect each infrared laser irradiated from the laser irradiation device 4, and input a detection signal indicating the detection to the CPU 41. As described above, the CPU 41 detects the position and posture of the head mounted display 3 in the real space based on the timing when the infrared laser is scanned by the laser irradiation device 4 and which light receiving sensor 50 receives the laser. Then, information regarding the detected position and orientation is transmitted to the game terminal device 2.

  The detection device 5 includes a control unit 60, a wireless communication circuit unit 64, and a light receiving sensor 65 on a bus 66. The control unit 60 includes a CPU 61, a ROM (flash memory) 62 and a RAM 63. The wireless communication circuit unit 64 communicates with the game terminal device 2 by Wi-Fi communication.

  The light receiving sensor 65 is the same sensor as the light receiving sensor 50 of the head mounted display 3. For example, a plurality of light receiving sensors 65 are arranged on the upper surface of the object 6. Similarly to the light receiving sensor 50, the light receiving sensor 65 detects the infrared laser emitted from the laser irradiation device 4 and inputs a detection signal indicating the detection to the CPU 61. The CPU 61 detects the position and orientation of the object 6 in the real space based on the timing when the infrared laser is scanned by the laser irradiation device 4 and which light receiving sensor 60 receives the laser. Then, information regarding the detected position and orientation is transmitted to the game terminal device 2.

  Next, a scene of a game executed by the cooperation of the game system 1 and the game program 70 will be described. First, the user P wearing the head mounted display 3 and the object 6 are in a room R that is a real space, and each has the positional relationship shown in FIG. It is assumed that the user P is facing the direction of the object 6 (virtual object A in the virtual space). At this time, the game image shown in FIG. The game images shown in FIGS. 3A to 3C are three-dimensionally displayed to the user P by the left and right parallax on the head mounted display 3. Therefore, a left-eye image and a right-eye image having parallax are created as game images, but only one of them is displayed in this figure. In FIG. 3D, the object 6 is arranged on the floor surface F of the room R, but the arrangement location is not limited to the floor surface F. For example, it may be on a table installed on the floor F or in the air.

  From the state of FIG. 3D, when the user P moves in the direction approaching the object 6 and becomes the positional relationship of FIG. 3E, that is, when the user P moves in the direction approaching the virtual object A in the virtual space. The game image as shown in FIG. 3B is displayed on the head mounted display 3. At this distance, the user P can hold the object 6 with both hands. When the user P reaches out and touches the virtual object A that is visible in the head mounted display 3, the object 6 is present at the position in the real space corresponding to the virtual object A. The object 6 will be touched. Therefore, the user P can feel as if he / she actually touched the virtual object A.

  When the user P lifts the virtual object A visible in the head mounted display 3, the user P grasps and lifts the real space object 6 as shown in FIG. Therefore, since the user P feels the feel and weight of the object 6, the user P can feel as if the virtual object A is actually lifted and moved. As a result, the object 6 moves, so that the virtual object A also moves corresponding to the movement of the object 6 based on the position detection of the detection device 5.

  Further, when the virtual object A (object 6) is moved, a hint (message) HM may appear on the floor F of the virtual space in which the virtual object A is arranged as shown in FIG. Good. Alternatively, the hint HM may appear on the back surface of the virtual object A. Accordingly, the user P can advance the VR game with reference to the hint HM. That is, when the object 6 is rotated, the virtual object A is also rotated. As a result, the back surface of the virtual object A can be seen by the user P, and the hint HM described therein can be seen.

  FIG. 4 is a functional block diagram of the game system 1. In this figure, only the functions related to the present invention in the game system 1 are shown. The game system 1 is realized by the cooperation of the game terminal device 2, the head mounted display 3 and the detection device 5, and the game program 70 stored in the game terminal device 2.

  The game system 1 functions as a head mounted display detection unit (first detection unit) 110, an object detection unit (second detection unit) 111, an image generation unit 112, and the like. The head mounted display detection unit 110 includes the laser irradiation device 4, the head mounted display 3, and the like, and detects the position and orientation of the head mounted display 3. The object detection unit 111 includes a laser irradiation device 4, a detection device 5, and the like, and detects the position and orientation of the object 6. The image generation unit 112 includes a virtual space generation unit 113 and a virtual object generation unit 114, and controls generation of a game image as the VR game progresses.

  The virtual space generation unit 113 includes the control unit 20 of the game terminal device 2 and the like, and generates a virtual space where the virtual object A exists. The virtual object generation unit 114 generates the appearance (texture) of the virtual object A and arranges it at a position in the virtual space corresponding to the position of the object 6 detected by the object detection unit 111. The image generation unit 112 includes the control unit 20 of the game terminal device 2 and the like, is generated by the virtual space generation unit 113, and is a game image based on the line of sight of the user P in the virtual space (where the virtual object A is arranged) ( Left-eye game image, right-eye game image).

Next, the above-described space generation processing and image generation processing will be described with reference to flowcharts. FIG. 5 is a flowchart showing the space generation processing and image generation processing of this embodiment.
The space generation process and the image generation process are repeatedly executed by the game progress control unit in units of one frame (1/60 seconds), for example.

  The image generation unit 112 updates the position and orientation of the virtual object A based on information on the position and orientation of the object 6 detected by the object detection unit 111 (step S10). Next, the position and posture of the user P in the three-dimensional virtual space are updated based on the information on the position and posture of the head mounted display 3 detected by the head mounted display detection unit 110, and the updated user P in the virtual space is updated. A left-eye game image and a right-eye game image are generated based on the line of sight of the user P according to the position and orientation (step S20).

  As described above, the virtual object A of the object 6 arranged in the same real space as the user P wearing the head mounted display 3 is arranged at the position of the virtual space corresponding to the positional relationship between the user P and the object 6. The That is, the real space object 6 and the virtual space virtual object A are linked. Therefore, in addition to vision and hearing, the user P can be given a feeling (tactile sense) of touching the virtual object A in the virtual space. Thereby, the feeling (immersion feeling) as if the user actually entered the virtual space can be further improved.

  In the above-described embodiment, the position and orientation of the object 6 are detected, but at least the position of the object may be detected. In the above-described embodiment, one object 6 has been described. However, two or more objects may be arranged in the real space, and virtual objects that are linked to the respective objects may be arranged in the virtual space. Furthermore, the virtual objects to be linked may not be all objects in the virtual space, and some objects may be linked as virtual objects.

  In the above-described embodiment, the light receiving sensor is disposed on the head mounted display 3 and the object 6 to detect the position and the like. However, the position is not particularly limited as long as the position and the like can be detected. For example, the head mounted display 3 and the object 6 may be provided with light emitting elements, and the light receiving elements may be provided at predetermined positions in the room R.

<Other embodiments>
In the above-described embodiment, the object 6 and the virtual object A are always linked, but the present invention is not particularly limited to this. The interlock may be temporarily released based on a predetermined condition. The predetermined condition includes, for example, that the distance between the object 6 (virtual object A) and the user P is not less than a predetermined distance. In this case, when the distance is equal to or greater than the predetermined distance, for example, the interlock is released and the virtual object A is not placed in the virtual space (not displayed). Alternatively, the virtual object A may be controlled to float in the virtual space without being linked to the position of the object 6. When the distance between the user P and the object 6 is equal to or less than the predetermined distance, the floating virtual object A is lowered and moved to a position corresponding to the position of the object 6 to start the interlocking with the object 6. .

  In the above-described embodiment, the object 6 whose shape does not change has been described, but an object whose shape changes may be used, and the shape of the virtual object may be changed according to the shape change of the object. For example, there is a rod-like object having a telescopic mechanism.

  In the above-described embodiment, the object 6 has been described. However, the present invention can be applied to any object. For example, a push button device may be used. In this case, the push button device and the game terminal device 2 are arranged on the wall surface W (or floor surface F) of the room as shown in FIG. 1 and the operation signal is transmitted to the game terminal device 2. And are connected by wire. And the pseudo button of the push button device is also arranged on the wall surface W (or floor surface F) of the virtual space. For example, when the user P presses the pseudo push button, a boss monster appears in front of the user P in the virtual space. Also in this case, the user P can feel as if he / she actually pressed the pseudo push button.

  Furthermore, in the above-described embodiment, the appearance of the user P is not generated in the virtual space, but may appear in the virtual space. In this case, for example, a light receiving sensor that receives an infrared laser may be provided in each part of the user P's body to detect the position and posture of the user P.

  The object 6 may have a vibration function. For example, it may be used for an effect in accordance with the progress of the game by vibrating when an explosion or the like occurs in the virtual space. In addition, the object 6 may be provided with a drive unit so that it can run on its own. In this case, the position of the freely moving object 6 may be detected to cause the virtual object A to follow in the virtual space. Then, the object 6 may be moved. The object 6 may be a chair on which the user P sits.

  INDUSTRIAL APPLICABILITY The present invention is useful for improving a feeling (immersion feeling) as if the user actually entered the virtual space in a virtual space display system such as a virtual reality game.

DESCRIPTION OF SYMBOLS 1 Game system 2 Game terminal device 3 Head mounted display 4 (4A, 4B) Laser irradiation apparatus 5 Detection apparatus 43 Memory | storage part 70 Game program 110 Head mounted display detection part 111 Object detection part 112 Image generation part 113 Virtual space generation part 114 Virtual Object generator A Virtual object R Room F Floor W Wall

Claims (7)

A head mounted display worn by the user in a predetermined real space;
A first detector for detecting the position and orientation of the head-mounted display in the real space;
An image generation unit that generates a virtual space corresponding to the real space, and generates an image of the virtual space that the user sees through the head mounted display based on a detection result of the first detection unit;
A virtual space display system comprising:
A real object disposed in the real space;
A second detector for detecting the position and orientation of the real object in the real space;
Further comprising
The virtual space display system, wherein the image generation unit generates an image of a virtual object corresponding to the real object in the virtual space with a position and orientation based on a detection result of the second detection unit. The real object is arranged to be movable in the real space,
The virtual space display system according to claim 1, wherein the image generation unit changes the position and orientation of the virtual object in the virtual space according to a change in a detection result of the second detection unit due to the movement. The virtual space has a virtual floor corresponding to the floor of the real space,
The virtual space display system according to claim 1, wherein the real object is arranged on the floor surface or a stand provided on the floor surface.   The virtual space display system according to claim 1, wherein the image of the virtual object is generated in substantially the same shape as the real object.   A game system that includes the virtual space display system according to claim 1 and that executes a game in the virtual space. A virtual space display system computer that displays a virtual space image on a head mounted display worn by a user in a predetermined real space,
First detection means for detecting the position and orientation of the head mounted display in the real space;
Image generating means for generating a virtual space corresponding to the real space, and generating an image of the virtual space that the user should be able to see through the head mounted display based on a detection result of the first detecting means;
Second detection means for detecting the position and orientation of the real object arranged in the real space in the real space;
Function as
The image generation unit is a virtual space display program that generates an image of a virtual object corresponding to the real object with a position and orientation based on a detection result of the second detection unit.   A game program comprising the virtual space display program according to claim 6 and causing a computer to execute a game in the virtual space.

Images