WO2022254585A1

WO2022254585A1 - Virtual space experience system

Info

Publication number: WO2022254585A1
Application number: PCT/JP2021/020889
Authority: WO
Inventors: 良哉尾小山
Original assignee: 株式会社Abal
Priority date: 2021-06-01
Filing date: 2021-06-01
Publication date: 2022-12-08
Also published as: JP7055527B1; JPWO2022254585A1

Abstract

Provided is a virtual space experience system that enables a plurality of users who have experienced different environments in a virtual space to experience the same environment. When the occurrence of a trigger event is recognized, an avatar coordinate determination unit 34 of a VR system S moves the coordinates of one among a first avatar A1 and a second avatar A2 to coordinates that match or are adjacent to the coordinates of the other among the first avatar A1 and the second avatar A2.

Description

Virtual space sensory system

The present invention relates to a virtual space bodily sensation system that allows the user to recognize that he or she exists in a virtual space displayed as an image.

Conventionally, a virtual space is generated by a server or the like, and a user recognizes an image of the virtual space through a head-mounted display (hereinafter sometimes referred to as "HMD"), and the user himself/herself exists in the virtual space. There is a virtual space sensory system that makes you realize that you are there.

This type of virtual space experience system recognizes the user's motion in the real space (for example, movement of coordinates, change of posture including orientation, etc.) by a motion capture device or the like, and responds to the user according to the recognized motion. There is an avatar that operates in a virtual space (see Patent Document 1, for example).

In the virtual space experience system described in Patent Document 1, multiple avatars are generated in the same virtual space so as to correspond to each of multiple users. In this virtual space bodily sensation system, the image of the virtual space recognized by each user is determined based on the coordinates of the avatar corresponding to the user. That is, even when users are experiencing virtual spaces at the same time, the image of the virtual space being visually recognized (and the environment of the virtual space being experienced) is different for each user.

Japanese Unexamined Patent Application Publication No. 2010157461

By the way, when a plurality of users experience the virtual space at the same time, and the environment of the virtual space experienced by each user is different, one user may experience the environment experienced by the other user. There are times when we want the user to experience it as well. Conversely, one user may want to experience the environment experienced by the other user.

SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and provides a virtual space experience system that enables a plurality of users experiencing different environments in a virtual space to experience the same environment. With the goal.

The virtual space experience system of the present invention is
a virtual space generation unit that generates a virtual space corresponding to the real space in which the first user and the second user exist;
an avatar generation unit that generates a first avatar corresponding to the first user and a second avatar corresponding to the second user in the virtual space;
a user coordinate recognition unit that recognizes the coordinates of the first user and the coordinates of the second user in the physical space;
Determining the coordinates of the first avatar in the virtual space based on the coordinates of the first user, and determining the coordinates of the second avatar in the virtual space based on the coordinates of the second user. Department and
a virtual space image determination unit that determines an image of the virtual space to be recognized by the first user and the second user based on the coordinates of the first avatar and the coordinates of the second avatar;
a trigger event recognition unit that recognizes the occurrence of the first trigger event;
A virtual space experience system comprising a virtual space image display that allows the first user and the second user to recognize an image of the virtual space,
The avatar coordinate determination unit converts the coordinates of one of the first avatar and the second avatar to the coordinates of the other of the first avatar and the second avatar when occurrence of the first trigger event is recognized. It is characterized by moving to matching coordinates or adjacent coordinates.

Here, the "image of the virtual space" includes images of the background of the virtual space, images of other avatars, images of objects that exist only in the virtual space, and images of objects that exist in the virtual space corresponding to the real space. Images, etc. are included.

As described above, in the virtual space experience system of the present invention, when the occurrence of the first trigger event is recognized, the avatar coordinate determination unit sets the coordinates of one of the first avatar and the second avatar to It is moved to coordinates that match or are adjacent to the coordinates of the other of the second avatar.

Here, the images of the virtual space recognized by the first user and the second user are determined based on the coordinates of the first avatar and the second avatar. Therefore, when the coordinates of the avatars match or are adjacent to each other, the images of the virtual space recognized by each user corresponding to those avatars are also matching or adjacent images (that is, similar images).

Therefore, according to the virtual space experience system of the present invention, when the occurrence of the first trigger event is recognized, the images of the virtual space recognized by each user can also be matched or adjacent images. The first user and the second user, who have experienced different environments in the virtual space until then, can now experience substantially the same environment.

In addition, the virtual space experience system of the present invention is
a virtual space generation unit that generates a virtual space corresponding to the real space in which the first user and the second user exist;
an avatar generation unit that generates a first avatar corresponding to the first user and a second avatar corresponding to the second user in the virtual space;
a user coordinate recognition unit that recognizes the coordinates of the first user and the coordinates of the second user in the physical space;
Determining the coordinates of the first avatar in the virtual space based on the coordinates of the first user, and determining the coordinates of the second avatar in the virtual space based on the coordinates of the second user. Department and
a virtual space image determination unit that determines an image of the virtual space to be recognized by the first user and the second user based on the coordinates of the first avatar and the coordinates of the second avatar;
a trigger event recognition unit that recognizes the occurrence of the first trigger event;
A virtual space experience system comprising a virtual space image display that allows the first user and the second user to recognize an image of the virtual space,
The virtual space image determining unit determines the image of the virtual space to be recognized by one of the first user and the second user when occurrence of the first trigger event is recognized. The image is characterized in that the coordinates of one of the two avatars are moved to coordinates that match or are adjacent to the coordinates of the other of the first and second avatars.

As described above, in the virtual space sensation system of the present invention, when the occurrence of the first trigger event is recognized, the virtual space image determining unit selects the virtual space image to be recognized by one of the first user and the second user. is an image when the coordinates of one of the first and second avatars are moved to coordinates that match or are adjacent to the coordinates of the other of the first and second avatars.

That is, when the occurrence of the first trigger event is recognized, the coordinates serving as the reference for determining the image in the virtual space to be recognized by one user are set to the coordinates of the avatar corresponding to that user. , coordinates that match or are adjacent to the coordinates of the avatar corresponding to the other user.

Thereby, when the occurrence of the first trigger event is recognized, one user and the other user are determined based on the coordinates that match or are adjacent to the coordinates of the avatar corresponding to the other user. You will recognize the image in the virtual space.

Therefore, according to the virtual space experience system of the present invention, after the occurrence of the first trigger event is recognized, one user can create a Therefore, the first user and the second user, who have experienced different environments in the virtual space until then, can now experience substantially the same environment.

Further, in the virtual space experience system of the present invention,
The avatar generation unit generates a ghost, which corresponds to one of the first user and the second user and is an avatar independent of the first avatar and the second avatar, in the virtual space;
The avatar coordinate determining unit determines the coordinates of the ghost based on coordinates obtained by causing a predetermined deviation in the coordinates of the first avatar or the second avatar corresponding to one of the first user and the second user. decide and
The virtual space image determination unit adds information indicating that the image of the virtual space to be recognized by the other of the first user and the second user corresponds to one of the first user and the second user. including an image of the added ghost,
Preferably, the first trigger event is a predetermined action performed on the ghost by the first avatar or the second avatar corresponding to the other of the first user and the second user.

Here, the ``information indicating that it is compatible'' to the user means, for example, direct information such as a message that is displayed at all times or at the request of the user, as well as a translucent shape of the corresponding avatar. It also includes indirect information indicated by adopting things as ghost shapes.

When creating an avatar corresponding to a user in the virtual space, it is not necessarily the case that only one avatar is created for each user. Therefore, in addition to an avatar that corresponds to the user and serves as a reference for determining an image to be recognized by the user, an avatar that corresponds to the user but is independent of the reference avatar (that is, the reference You may generate a ghost that is an avatar that is not a character).

Here, in the virtual space bodily sensation system of the present invention, the ghost is added with information indicating that it corresponds to a predetermined user. Therefore, other users can easily grasp which user the ghost corresponds to.

Then, when a predetermined action to be performed by the avatar on the ghost is set as a first trigger event, the user corresponding to the avatar can experience the environment of the user corresponding to the ghost as a result of executing the action. You will be able to intuitively understand what you can do.

Here, the "predetermined action" may be any action performed by the avatar based on the ghost. For example, when an avatar touches a ghost, when an avatar moves within a predetermined range based on a ghost, when an avatar operates an object in the virtual space, the ghost is selected as the target. (for example, an action of shooting a ghost with a camera-type object, etc.).

Further, in the virtual space experience system of the present invention, in the case of a configuration that generates a ghost,
The trigger event recognition unit recognizes occurrence of a second trigger event,
The avatar generation unit generates a first ghost, which is the ghost corresponding to the first user, in the virtual space when occurrence of the second trigger event is recognized,
After the occurrence of the second trigger event is recognized, the avatar coordinate determining unit determines the coordinates of the first avatar based on the coordinates that cause the predetermined deviation from the coordinates of the first user. , determining the coordinates of the first ghost in the virtual space based on the coordinates of the first user;
After the occurrence of the second trigger event is recognized, the virtual space image determination unit adds information indicating that the virtual space image to be recognized by the second user corresponds to the first user. including the image of the added first ghost;
Preferably, the first trigger event is a predetermined action performed by the second avatar on the first ghost.

For example, when the first user and the second user exist in the same area (for example, the same room) in the physical space, the coordinates of the first avatar are shifted from the coordinates of the first user by a predetermined amount. Even if the second user moves the second avatar corresponding to himself/herself so as not to contact the first avatar, the second user can contact the first user corresponding to the first avatar. There is a risk that it will be lost.

Then, in order to avoid such contact, a method is adopted in which the coordinates of the first ghost corresponding to the first user are set to the coordinates based on the coordinates of the first user (that is, the coordinates before the deviation occurs). There is Thereby, the second avatar (that is, the second user) moves so as to avoid the first ghost, so it is possible to prevent the second user from contacting the first user.

Then, when such a method is adopted, it is preferable to adopt the action performed by the second avatar on the first ghost as the first trigger event.

With such a configuration, the second user can intuitively understand that the action causes the first trigger event that shares the environment with the first user. As a result, it is possible to prevent the second user's sense of immersion in the virtual space from being hindered due to the occurrence of the first trigger event.

Further, in the virtual space experience system of the present invention, in the case of a configuration that generates a ghost,
The trigger event recognition unit recognizes occurrence of a second trigger event,
The avatar generation unit generates a second ghost, which is the ghost corresponding to the second user, in the virtual space when occurrence of the second trigger event is recognized,
After the occurrence of the second trigger event is recognized, the avatar coordinate determining unit determines the coordinates of the first avatar based on the coordinates that cause the predetermined deviation from the coordinates of the first user. determining the coordinates of the second ghost in the virtual space based on the coordinates obtained by causing the predetermined deviation from the coordinates of the second user;
After the occurrence of the second trigger event is recognized, the virtual space image determination unit adds information indicating that the virtual space image to be recognized by the first user corresponds to the second user. including the image of the added second ghost,
Preferably, the first trigger event is a predetermined action performed by the first avatar on the second ghost.

For example, when the first user and the second user exist in the same area (for example, the same room) in the physical space, the coordinates of the first avatar are shifted from the coordinates of the first user by a predetermined amount. Even if the first user moves the first avatar corresponding to himself/herself so as not to contact the second avatar, the second user corresponding to the second avatar is contacted. There is a risk that it will be lost.

Then, in order to avoid such contact, a method is adopted in which the coordinates of the fourth ghost corresponding to the second user are set to the coordinates of the second avatar with a deviation similar to that of the first avatar. There is As a result, the first avatar (that is, the first user) moves so as to avoid the fourth ghost, so it is possible to prevent the first user from contacting the second user.

When such a method is adopted, it is preferable to adopt the action performed by the first avatar on the second ghost as the first trigger event.

With such a configuration, the first user can intuitively understand that the action causes the first trigger event that shares the environment with the second user. As a result, it is possible to prevent the first user's sense of immersion in the virtual space from being hindered due to the occurrence of the first trigger event.

1 is a schematic diagram showing a schematic configuration of a VR system according to a first embodiment; FIG. FIG. 2 is a block diagram showing the configuration of a processing unit of the VR system in FIG. 1; 2 is a flowchart showing processing executed by the VR system of FIG. 1 in normal use; FIG. 2 is a schematic diagram showing states of a real space and a virtual space in normal use of the VR system of FIG. 1; FIG. 2 is a flow chart showing processing executed when the VR system in FIG. 1 expands the virtual space and after that; FIG. FIG. 2 is a schematic diagram showing the states of the real space and the virtual space when expanding the virtual space in the VR system of FIG. 1; FIG. 2 is a schematic diagram showing the state of the real space and the virtual space after the virtual space is expanded in the VR system of FIG. 1; 4 is a flow chart showing processing in the first embodiment that is executed when and after the VR system of FIG. 1 synchronizes the environment; FIG. 2 is a schematic diagram showing the states of the real space and the virtual space when the VR system in FIG. 1 synchronizes the environments; FIG. 2 is a schematic diagram showing the state of the real space and the virtual space after the VR system of FIG. 1 has synchronized the environments; 10 is a flowchart showing processing executed when the VR system of the second embodiment synchronizes environments; FIG. 12 is a schematic diagram showing the states of the real space and the virtual space when the VR system in FIG. 11 synchronizes the environments; FIG. 12 is a schematic diagram showing the state of the real space and the virtual space after the VR system of FIG. 11 has synchronized the environments; 12 is a schematic diagram of an image of the virtual space as perceived by the second user before the VR system of FIG. 11 synchronizes the environment; FIG. FIG. 12 is a schematic diagram of an image of the virtual space as perceived by the second user after the VR system of FIG. 11 has synchronized the environment;

[First embodiment]
A VR system S, which is a virtual space sensation system according to the first embodiment, will be described below with reference to FIGS. 1 to 10. FIG.

For a first user U1 and a second user U2 (hereinafter collectively referred to as “users U”) existing together in a predetermined area (for example, one room) of the physical space RS, the VR system S One virtual space VS corresponding to the area is made to recognize that they exist together through a first avatar A1 corresponding to the first user U1 and a second avatar A2 corresponding to the second user U2 (Fig. 4, etc.).

In addition, in this embodiment and a second embodiment described later, the number of users is assumed to be two in order to facilitate understanding. However, the virtual space experience system of the present invention is not limited to such a configuration, and the number of users may be three or more.

[Schematic configuration of the system]
First, a schematic configuration of the VR system S will be described with reference to FIG.

As shown in FIG. 1, a VR system S includes a plurality of markers 1 attached to a user U existing in a physical space RS, and a camera 2 that captures the user U (strictly speaking, the marker 1 attached to the user U). , a server 3 that determines the image and sound of the virtual space VS (see FIG. 4, etc.), and a head-mounted display (hereinafter referred to as "HMD 4") that allows the user to recognize the determined image and sound. .

In the VR system S, the camera 2, server 3, and HMD 4 can mutually transmit and receive information wirelessly, such as through the Internet network, public lines, and short-range wireless communication. However, any one of them may be configured to transmit and receive information to and from each other by wire.

A plurality of markers 1 are attached to the user U's head, both hands, and both feet via the HMD 4, gloves, and shoes that the user U wears. As will be described later, the plurality of markers 1 are used to recognize the coordinates and posture of the user U in the physical space RS (furthermore, actions (eg, movement of coordinates, changes in posture including orientation, etc.)). is. Therefore, depending on other devices that constitute the VR system S, the mounting position of the marker 1 may be changed as appropriate.

The camera 2 is installed so that the user U's operable range (that is, the range in which the user U can move, change the posture, etc.) in the physical space RS where the user U exists can be photographed from multiple directions.

The server 3 recognizes the sign 1 from the image captured by the camera 2, and recognizes the coordinates and orientation of the user U based on the position of the recognized sign 1 in the physical space RS. Also, the server 3 determines images and sounds to be recognized by the user U based on the coordinates and orientation.

The HMD 4 is worn on the user U's head. The HMD 4 includes a monitor 41 (virtual space image display) for allowing the user U to recognize the image of the virtual space VS determined by the server 3, and the sound of the virtual space VS determined by the server 3. A speaker 42 (virtual space audio generator) is provided for recognition by the user U (see FIG. 2).

When playing a game or the like using the VR system S, the user U recognizes only the images and sounds of the virtual space VS, and is made to recognize that the user U himself exists in the virtual space. That is, the VR system S is configured as a so-called immersive system.

It should be noted that the virtual space experience system of the present invention is not limited to such an immersive system. Therefore, for example, the configuration of the virtual space experience system of the present invention is applied to a system (so-called AR system) that superimposes and displays an image of the real space and an image of the virtual space to allow the user to recognize the augmented real space. You may

The VR system S is equipped with a so-called motion capture device composed of a sign 1, a camera 2, and a server 3 as a system for recognizing the coordinates of the user U in the real space RS.

It should be noted that the virtual space experience system of the present invention is not limited to such a configuration. For example, when a motion capture device is used, in addition to the configuration described above, a configuration in which the numbers of signs and cameras are different from those described above (for example, one each is provided) may be used.

Also, for example, a device that recognizes only the user's coordinates may be used instead of the motion capture device. Specifically, for example, a sensor such as GPS may be installed in the HMD, and the user's coordinates, posture, etc. may be recognized based on the output from the sensor. Also, such a sensor may be used in combination with the motion capture device as described above.

[Configuration of processing unit]
Next, with reference to FIG. 2, the configuration of the processing units included in the server 3 will be described in detail.

The server 3 is composed of one or more electronic circuit units including CPU, RAM, ROM, interface circuits, and the like. As shown in FIG. 2, the server 3 includes a display image generation unit 31, a user information recognition unit 32, and a trigger event recognition unit 33 as functions (processing units) implemented by the installed hardware configuration or programs. , an avatar coordinate determination unit 34 , a virtual space image determination unit 35 , and a virtual space sound determination unit 36 .

The display image generation unit 31 generates an image to be recognized by the user U through the monitor 41 of the HMD 4 . The display image generator 31 has a virtual space generator 31a, an avatar generator 31b, and a moving object generator 31c.

The virtual space generation unit 31a generates an image that serves as the background of the virtual space VS and an image of an object that exists in the virtual space VS.

The avatar generation unit 31b generates, in the virtual space VS, a first avatar A1 corresponding to the first user U1, and a first ghost G1, which is an avatar independent of the first avatar A1 (FIG. 6, etc.). reference). In addition, the avatar generation unit 31b generates a second avatar A2 and a second ghost G2, which is an avatar independent of the second avatar A2, corresponding to the second user U2 (see FIG. 6, etc.). .

The first avatar A1 and second avatar A2 (hereinafter collectively referred to as "avatar A"), and the first ghost G1 and second ghost G2 (hereinafter collectively referred to as "ghost G") are , operate in the virtual space VS in response to the corresponding user U's actions in the physical space RS (that is, coordinate movement and posture change).

The moving body generation unit 31c generates a moving body in the virtual space VS that has no corresponding object in the real space RS and that can be connected to the avatar in the virtual space VS.

Here, the “moving object” is an object that allows the user U to predict (whether consciously or unconsciously) the movement of the avatar that differs from the actual movement of the user when the avatar connects. If it is

For example, moving objects include objects used for movement in real space such as elevators, logs flowing in a river that can be jumped on, floors that are likely to collapse when standing on them, jumping platforms, wings that assist jumping, etc. is applicable. In addition, characters, patterns, and the like drawn on the ground and wall surfaces of the virtual space also correspond to moving objects.

Here, the "connection" between the avatar and the moving object means that the user can predict that the movement of the moving object, the change in the shape of the moving object, etc. will affect the coordinates of the avatar. .

For example, an avatar getting into an elevator, an avatar riding on a log flowing in a river, an avatar standing on a crumbling floor, an avatar standing on a jumping platform, an avatar wearing wings that assist jumping, etc. Applicable. In addition, when an avatar touches or approaches characters, patterns, or the like drawn on the ground or wall surface of the virtual space, this also corresponds to connection.

Image data of the user U including the sign 1 captured by the camera 2 is input to the user information recognition unit 32 . The user information recognition section 32 has a user posture recognition section 32a and a user coordinate recognition section 32b.

The user posture recognition unit 32a extracts the marker 1 from the input image data of the user U, and recognizes the posture of the user U based on the extraction result.

The user coordinate recognition unit 32b extracts the marker 1 from the input image data of the user U, and recognizes the coordinates of the user U based on the extraction result.

The trigger event recognition unit 33 recognizes that a predetermined trigger event has occurred or that the trigger event has been canceled when a condition predetermined by the system designer is satisfied.

Here, the trigger event may be one that the user is unaware of. Therefore, as a trigger event, for example, events caused by user actions such as the user performing a predetermined action in the real space (that is, the avatar corresponding to the user performing a predetermined action in the virtual space) are also applicable. In addition, it also applies to events that are not caused by the user's actions, such as the passage of a predetermined period of time.

The avatar coordinate determination unit 34 determines the coordinates of the avatar corresponding to the user U in the virtual space VS based on the coordinates of the user U in the physical space RS recognized by the user coordinate recognition unit 32b.

In addition, when the trigger event recognition unit 33 recognizes the occurrence of a predetermined trigger event, the avatar coordinate determination unit 34, regardless of the coordinates of the user, during a predetermined period or within a predetermined range, or within a predetermined period and within a predetermined range, Move the coordinates of the avatar corresponding to the user.

Further, when the trigger event recognition unit 33 recognizes the occurrence of a predetermined trigger event, the avatar coordinate determination unit 34 sets the avatar A corresponding to the user U for a predetermined period, a predetermined range, or a predetermined period and a predetermined range. The coordinates of the ghost G are determined based on the coordinates obtained by shifting the coordinates of .

Based on the coordinates of the avatar, the virtual space image determination unit 35 determines the image of the virtual space to be recognized by the user U corresponding to the avatar via the monitor 41 of the HMD 4.

Here, in addition to the image of the background of the virtual space, the "image of the virtual space" includes images of other avatars, images of ghosts, images of objects that exist only in the virtual space, and images in the virtual space corresponding to the real space. Images of existing objects, etc. are included.

It should be noted that the image of the ghost G in this embodiment is added with information indicating that it corresponds to the user U corresponding to the ghost G.

Based on the coordinates of the avatar, the virtual space audio determining unit 36 determines the audio to be recognized by the user U corresponding to the avatar via the speaker 42 of the HMD 4.

It should be noted that each processing unit that constitutes the virtual space experience system of the present invention is not limited to the configuration as described above.

For example, part of the processing unit provided in the server 3 in the above embodiment may be provided in the HMD 4. Alternatively, a plurality of servers may be used, or the servers may be omitted and the CPUs mounted on the HMDs may cooperate with each other. Also, a speaker other than the speaker mounted on the HMD may be provided. In addition to devices that affect sight and hearing, devices that affect smell and touch such as generating smells, wind, etc. according to the virtual space may also be included.

[Process to be executed]
Next, with reference to FIGS. 2 to 10, processing executed by the VR system S when the user U is allowed to experience the virtual space VS using the VR system S will be described.

[Processing during normal use]
First, with reference to FIGS. 2, 3, and 4, the processing executed by the VR system S in normal use (that is, in a state in which the occurrence of a trigger event, which will be described later) is not recognized, will be described.

In this process, first, the display image generation unit 31 of the server 3 generates the virtual space VS, the first avatar A1 and the second avatar A2, and the moving object (FIG. 3/STEP 100).

Specifically, the virtual space generation unit 31a of the display image generation unit 31 generates the virtual space VS and various objects existing in the virtual space VS. Also, the avatar generation unit 31b of the display image generation unit 31 generates a first avatar A1 corresponding to the first user U1 and a second avatar A2 corresponding to the second user U2. Further, the moving object generation unit 31c of the display image generation unit 31 generates a moving object such as an elevator VS1, which will be described later.

In the virtual space VS generated by the processing of STEP 100, as shown in FIG. 4, there are a first avatar A1, a second avatar A2, an elevator VS1 which is a moving body, and a real space RS. An object related to a trigger event such as a switch VS2 generated at a position corresponding to the whiteboard RS1 is placed.

Next, the avatar coordinate determination unit 34 of the server 3 determines the coordinates and posture of the first avatar A1 in the virtual space VS based on the coordinates and posture of the first user U1 in the physical space RS, and The coordinates and orientation of the second avatar A2 in the virtual space VS are determined based on the coordinates and orientation in the physical space RS (FIG. 3/STEP 101).

The coordinates and orientation of the first user U1 and the second user U2 in the processing after STEP 101 are the coordinates and orientation recognized by the user information recognition unit 32 of the server 3 based on the image data captured by the camera 2. .

Next, the virtual space image determining unit 35 and the virtual space audio determining unit 36 of the server 3 determine the image and audio to be recognized by the first user U1 based on the coordinates and posture of the first avatar A1 in the virtual space VS. , the image and sound to be recognized by the second user U2 are determined based on the coordinates and posture of the second avatar A2 in the virtual space VS (FIG. 3/STEP 102).

Next, the HMD 4 worn by the user U causes the monitor 41 mounted on the HMD 4 to display the determined image, and the speaker 42 mounted on the HMD 4 to generate the determined sound (FIG. 3/ STEP 103).

Next, the user information recognition unit 32 of the server 3 determines whether or not the movement of the coordinates or the change of the posture in the physical space RS of the first user U1 or the second user U2 has been recognized (FIG. 3/STEP 104).

If the movement of the coordinates or the change in the posture of the first user U1 or the second user U2 in the physical space RS is recognized (YES in STEP104), the process returns to STEP101, and the processes after STEP101 are executed again.

On the other hand, if the movement of the coordinates or the change in the posture of the first user U1 or the second user U2 in the physical space RS is not recognized (NO in STEP 104), the server 3 recognizes the signal instructing the end of the process. (FIG. 3/STEP 105).

If the signal instructing the end could not be recognized (NO in STEP 105), the process returns to STEP 104, and the processes after STEP 104 are executed again.

On the other hand, if the signal instructing the end is recognized (YES in STEP 105), the VR system S ends this process.

By the above processing, in the virtual space VS, the first avatar A1 corresponding to the first user U1, the second avatar A2 corresponding to the second user U2, the elevator VS1 which is a moving body, and the A plurality of objects are placed including a switch VS2, etc., to be set.

Then, the first user U1 and the second user U2 can visualize themselves via the corresponding first avatar A1 and second avatar A2 by the image displayed and the sound generated by the HMD 4 they wear. It comes to recognize that it exists in space VS and can move freely.

[Processing when expanding the virtual space and after that]
Next, referring to FIGS. 2 and 4 to 7, the VR system S executes Processing will be explained.

Here, prior to the explanation of the specific contents of the processing, the processing for expanding the virtual space VS so as to be wider than the real space RS corresponding to the virtual space VS, and the accompanying processing for avoiding contact. Give an overview.

When a predetermined trigger event (second trigger event in the present invention) occurs, the VR system S sets the correspondence relationship between the coordinates of the first user U1 and the coordinates of the first avatar A1, or the relationship between the coordinates of the first user U2 and the first avatar A1. 2 avatar A2 is configured to perform a process of causing a deviation in the correspondence relationship.

In this embodiment, for example, as shown in FIG. 4, when the first avatar A1 is riding (connected to) the elevator VS1, which is a moving object, the second avatar A2 pushing the switch VS2 is set as the trigger event.

Then, when the occurrence of the trigger event is recognized, as shown in FIGS. 4 and 6, the coordinates of the first avatar A1 corresponding to the first user U1 are displayed regardless of the movement of the coordinates of the first user U1. moves upwards. Specifically, the first avatar A1 moves from the first floor F1 defined in the virtual space VS to the second floor F2 by the elevator VS1.

As a result, in the VR system S, the first user U1 to whom the corresponding first avatar A1 has been moved and the second user U2 who recognizes the first avatar A1 are extended in the vertical direction from the physical space RS. The virtual space VS created can be recognized.

Here, if such a deviation is caused, the first user U1 and the second user U2 may not be able to properly grasp their mutual positional relationship in the physical space RS. As a result, there is a possibility that the first user U1 and the second user U2 may come into contact with each other unintentionally.

For example, in the state shown in FIG. 6, the second user U2 may mistakenly think that the first user U1 is moving upward in the real space RS as well as the first avatar A1. Then, as shown in FIG. 7, the second user U2 misunderstands that the second avatar A2 corresponding to him/herself in the virtual space VS can move below the elevator VS1, and moves the second avatar A2 below the elevator VS1. In some cases, they try to move themselves, as if they were moving to another area.

However, in the physical space RS, the first user U1 corresponding to the first butter A1 actually exists at the same height as the second user U2. Therefore, in that case, the first user U1 and the second user U2 come into contact in the physical space RS. As a result, there is a possibility that the feeling of immersion in the virtual space VS of the first user U1 and the second user U2 may be hindered.

Therefore, in the VR system S, when the occurrence of a predetermined trigger event is recognized, the virtual space VS is expanded so as to be wider than the real space RS corresponding to the virtual space VS, as described below. Unintended contact between the first user U1 and the second user U2 is prevented by executing the process and the process for contact avoidance.

In this process, first, the trigger event recognition unit 33 of the server 3 determines whether or not the occurrence of the trigger event has been recognized (FIG. 5/STEP 200).

Specifically, first, as shown in FIG. 4, the trigger event recognizing unit 33 sets the coordinates of the first avatar A1 corresponding to the first user U1 in the virtual space VS so that the first user U1 gets on the elevator VS1, which is a moving body. , in the physical space RS, it is determined whether or not the coordinates of the first user U1 have moved.

After that, the trigger event recognition unit 33 further causes the second avatar A2 corresponding to the second user U2 to touch the switch VS2 at a position near the switch VS2 in the virtual space VS. It is determined whether or not the second user U2 has moved to a predetermined posture (see FIG. 6).

Then, when the first avatar A1 is on the elevator VS1 and the second avatar A2 is in a posture of touching the switch VS, the trigger event recognition unit 33 recognizes that a trigger event has occurred.

If the occurrence of the trigger event is not recognized (NO in STEP 200), the process returns to STEP 200 and the judgment is executed again. This process is repeated at a predetermined control cycle.

On the other hand, when the occurrence of the trigger event is recognized (YES in STEP200), the avatar coordinate determination unit 34 of the server 3 moves the coordinates of the first avatar A1 (FIG. 5/STEP201).

Specifically, the avatar coordinate determination unit 34 determines the coordinates of the first user U1 according to the content of the deviation (that is, correction direction and correction amount) predetermined according to the type of the recognized trigger event. Move the coordinates of the first avatar A1 based on. This movement is performed independently of the movement of the coordinates of the first avatar A1 based on the movement of the coordinates of the first user U1. As a result, the correspondence relationship between the coordinates of the first user U1 and the coordinates of the first avatar A1 is deviated.

In this embodiment, the first avatar A1 moves upward from the first floor F1 to the second floor F2 integrally with the elevator VS1 so that the state shown in FIG. 4 changes to the state shown in FIG. .

Next, the display image generator 31 of the server 3 generates the first ghost G1 and the second ghost G2 (FIG. 5/STEP 202).

Specifically, the avatar generation unit 31b of the display image generation unit 31 generates a first ghost G1 corresponding to the first user U1 in the virtual space VS using the occurrence of a trigger event as a trigger, and also generates a first ghost G1 corresponding to the first user U2. generates a second ghost G2 corresponding to .

In this embodiment, as shown in FIG. 6, the first ghost G1 is configured as a translucent avatar having the same shape as the first avatar A1 in order to indicate that it corresponds to the first user U1. . A first information board G1a, which will be described later, is added to the first ghost G1. Similarly, the second ghost G2 is configured as a translucent avatar having the same shape as the second avatar A2 in order to indicate that it corresponds to the second user U2. A second information board G2a is added.

Also, in this embodiment, the coordinates of the generated first ghost G1 are set to match the coordinates of the first avatar A1 immediately before the process of STEP201 is executed. Also, the generated coordinates of the second ghost G2 are set so as to be independent of the coordinates of the second user U2.

Next, the avatar coordinate determination unit 34 of the server 3 determines the virtual space VS of the first avatar A1 and the first ghost G1 based on the coordinates and posture of the first user U1 in the physical space RS recognized by the user information recognition unit 32. Determine the coordinates and attitude in (FIG. 5/STEP 203).

Specifically, the avatar coordinate determination unit 34 recognizes the occurrence of the trigger event according to the content of the deviation (that is, correction direction and correction amount) predetermined according to the type of the recognized trigger event. By correcting the coordinates and orientation of the first avatar A1 determined by the processing up to the time when the trigger event occurs (the processing before the processing of STEP 201 is executed), the first avatar A1 after the occurrence of the trigger event is recognized. Determine the coordinates and posture of avatar A1.

In this embodiment, the avatar coordinate determination unit 34 moves the coordinates of the first avatar A1 based on the coordinates of the first user U1 upward by the height of the second floor F2 relative to the first floor F1. The coordinates are determined as the coordinates of the first avatar A1 after the occurrence of the triggering event is recognized. Also, the avatar coordinate determining unit 34 determines the posture of the first avatar A1 based on the posture of the first user U1 as the posture of the first avatar A1 after the occurrence of the trigger event is recognized.

In addition, the avatar coordinate determining unit 34 uses the same process as the process for determining the coordinates and orientation of the first avatar A1 before the occurrence of the trigger event is recognized to determine the coordinates of the first user U1. and the orientation, determine the coordinates and orientation of the first ghost G1.

Next, the virtual space image determining unit 35 and the virtual space audio determining unit 36 of the server 3 determine the image and audio to be recognized by the first user U1 based on the coordinates and posture of the first avatar A1 in the virtual space VS. (FIG. 5/STEP 204).

Next, the avatar coordinate determination unit 34 of the server 3 determines the virtual space VS of the second avatar A2 and the second ghost G2 based on the coordinates and posture of the second user U2 in the physical space RS recognized by the user information recognition unit 32. Determine the coordinates and attitude in (FIG. 5/STEP 205).

Specifically, the avatar coordinate determination unit 34 uses the same processing as that for determining the coordinates and posture of the second avatar A2 so far, based on the coordinates and posture of the second user U2, to determine the 2 Determine the coordinates and posture of avatar A2.

In addition, the avatar coordinate determination unit 34 determines the timing at which the occurrence of the trigger event is recognized, according to the details of the deviation (that is, correction direction and correction amount) predetermined according to the type of the recognized trigger event. The coordinates and orientation of the second avatar A2 determined by the previous processing are corrected to determine the coordinates and orientation of the second ghost G2.

In this embodiment, the avatar coordinate determination unit 34 moves the coordinates of the second avatar A2 based on the coordinates of the second user U2 upward by the height of the second floor F2 relative to the first floor F1. is determined as the coordinates of the second ghost G2 after the time when the occurrence of the trigger event was recognized.

Also, the avatar coordinate determination unit 34 determines the posture of the second avatar A2 based on the posture of the second user U2 as it is as the posture of the second ghost G2 after the occurrence of the trigger event is recognized.

Next, the virtual space image determining unit 35 and the virtual space audio determining unit 36 of the server 3 determine the image and audio to be recognized by the second user U2 based on the coordinates and posture of the second avatar A2 in the virtual space VS. (FIG. 5/STEP 206).

Next, the HMD 4 worn by the user U displays the determined image on the monitor 41 and emits sound from the speaker 42 (FIG. 5/STEP 207).

Next, the user information recognition unit 32 of the server 3 determines whether or not the movement of the coordinates or the change of the posture in the physical space RS of the first user U1 or the second user U2 has been recognized (FIG. 5/STEP 208).

If the movement of the coordinates or the change in the posture of the first user U1 or the second user U2 in the physical space RS is recognized (YES in STEP208), the process returns to STEP203, and the processes after STEP203 are executed again.

On the other hand, if no coordinate movement or posture change in the physical space RS of the first user U1 or the second user U2 is recognized (NO in STEP 208), the server 3 recognizes a signal instructing the end of processing. It is determined whether or not it has been done (FIG. 5/STEP 209).

If the signal instructing the end could not be recognized (NO in STEP 209), the process returns to STEP 208, and the processes after STEP 208 are executed again.

On the other hand, when the signal instructing the end is recognized (YES in STEP209), the VR system S ends the current process.

As described above, in the VR system S, when a trigger event occurs and a deviation occurs in the correspondence between the coordinates of the first user U1 and the coordinates of the first avatar A1 corresponding to the first user U1, A first ghost G1 corresponding to the coordinates of the first user U1 is generated, and a second ghost G2 is generated at coordinates obtained by correcting the coordinates corresponding to the coordinates of the second user U2 so as to correspond to the deviation.

Then, the image and sound of the virtual space VS recognized by the first user U1 include the image and sound of the second ghost G2. Also, the image and sound of the virtual space VS recognized by the second user U2 include the image and sound of the first ghost G1.

Therefore, after the occurrence of the trigger event is recognized, the first user U1 naturally moves to the first avatar A1 corresponding to the corrected coordinates of himself and the corrected coordinates of the second player when performing some action. It operates while avoiding contact with the corresponding second ghost G2. In addition, when the second user U2 performs some action, the second user U2 naturally avoids contact between the second avatar A2 corresponding to his coordinates and the first ghost G1 corresponding to the coordinates of the first user U1. do.

Thereby, contact between the first user U1 and the second user U2 in the physical space RS can be prevented.

In this embodiment, the first ghost G1 is configured as a translucent avatar having the same shape as the first avatar A1, and the second ghost G2 is configured as a translucent avatar having the same shape as the second avatar A2. there is This is to indicate that the first ghost G1 corresponds to the first user U1 and that the second ghost G2 corresponds to the second user U2.

Also, in this embodiment, the first ghost G1 is added with the first information board G1a, and the second ghost G2 is added with the second information board G2a. When the user U generates a trigger event (first trigger event in the present invention) in the processing for synchronizing the environments described later, the processing executed when the occurrence of the trigger event is recognized is This is to allow the user U to intuitively understand.

However, the method of displaying information indicating which user the avatar corresponds to in the virtual space experience system of the present invention and the method of displaying information related to ghosts are not limited to such configurations. It may be set as appropriate.

Therefore, for example, in the present embodiment, the information board may be omitted by simply matching the shapes of the first ghost G1 and the second ghost G2 with the shapes of the first avatar A1 and the second avatar A2. Conversely, only the information board may be displayed without displaying shapes corresponding to the shapes of the first avatar A1 and the second avatar A2.

[Processing when synchronizing the environment and afterward]
Next, with reference to FIGS. 2 and 8 to 10, processing to be executed when synchronizing the environment and after that will be described.

Here, prior to describing the details of the specific processing, an outline of the processing for synchronizing the environments in this embodiment will be described.

When the VR system S recognizes the occurrence of a predetermined trigger event (the first trigger event in the present invention), the VR system S adds the first user U1 and the second user U2 to the environment of one of the first user U1 and the second user U2. It is configured to perform processing for synchronizing one environment of the

Here, the "environment" refers to at least one of images and sounds recognized by the user corresponding to the avatar located at the coordinates of the virtual space. Also, "environmental synchronization" means that a plurality of users corresponding to a plurality of avatars come to recognize similar environments.

In this embodiment, for example, as shown in FIG. 9, the second avatar A2 touching the first ghost G1 corresponding to the first user U1 is set as the trigger event.

Then, when the occurrence of the trigger event is recognized, as shown in FIG. 10, regardless of the movement of the coordinates of the second user U2, the coordinates of the second avatar A2 corresponding to the second user U2 are moved to the second 1 Move to the coordinates adjacent to the coordinates of avatar A1. Specifically, the second avatar A2 moves from the first floor F1 to coordinates adjacent to the first avatar A1 on the second floor F2.

Accordingly, in the VR system S, after the occurrence of the trigger event is recognized, the environment of the first user U1 and the environment of the second user U2 are synchronized, and the first user U1 and the second user U2 can experience the same environment.

In the following description, it is assumed that the first user U1 corresponding to the first avatar A1 wishes to view, together with the second user U2, the rainbow VS3 that exists above the second floor F2 of the virtual space VS. A description will be given of the processing when the second user U2 tries to synchronize the environment with the first user U1 (that is, he himself tries to look at the rainbow VS3) when the second user U2 is notified via a message function or the like. .

In this process, first, the trigger event recognition unit 33 of the server 3 determines whether or not the occurrence of the trigger event has been recognized (FIG. 8/STEP 300).

Specifically, first, as shown in FIG. 9, the trigger event recognition unit 33 recognizes that the second avatar A2 corresponding to the second user U2 is the first avatar A1 corresponding to the first user U1 in the virtual space VS. It is determined whether or not it is in a posture to touch the . Then, when the second avatar A2 assumes a posture of touching the first avatar A1, the trigger event recognition unit 33 recognizes that a trigger event has occurred.

If the occurrence of the trigger event is not recognized (NO in STEP 300), the process returns to STEP 300 and the judgment is executed again. This process is repeated at a predetermined control cycle.

On the other hand, when the occurrence of the trigger event is recognized (YES in STEP 300), the avatar coordinate determination unit 34 of the server 3 moves the coordinates of the second avatar A2 to the coordinates adjacent to the coordinates of the first avatar A1 ( FIG. 5/STEP 301).

Specifically, the avatar coordinate determination unit 34 refers not only to the coordinates of the first avatar A1 but also to the posture, and determines the coordinates located to the side, front, or rear of the first avatar A1 (that is, adjacent coordinates). Recognize and move the second avatar A2 to the coordinates.

This movement is performed independently of the movement of the coordinates of the second avatar A2 based on the movement of the coordinates of the second user U2. It should be noted that this causes a deviation in the corresponding relationship between the coordinates of the second user U2 and the coordinates of the second avatar A2.

In this embodiment, as shown in FIG. 9, the second avatar A2 located on the first floor F1 is located on the second floor F2 and is next to the first avatar A1 on the left, as shown in FIG. Momentarily move so as to be located in

It should be noted that the history of the movement (eg, the time it takes to move, the route during the movement, etc.) may be set as appropriate. For example, when there is a third user different from the first user U1 and the second user U2, the movement speed of the second avatar A2 is slowed to some extent so that the movement route of the second avatar A2 can be recognized. By setting the velocity, it becomes possible for the third user to recognize the movement of the coordinates when the occurrence of the trigger event is recognized.

Next, the display image generator 31 of the server 3 erases the first ghost G1 and the second ghost G2 (FIG. 8/STEP 302).

In this embodiment, the movement of the second avatar A2 in STEP 301 eliminates the deviation in the corresponding relationship between the coordinates of the first avatar A1 and the coordinates of the second avatar A2, which is caused by the process of expanding the virtual space. be. That is, after the movement, the correspondence relationship between the coordinates of the first avatar A1 and the coordinates of the second avatar A2 is changed from the correspondence relationship before the deviation occurred (that is, the coordinates of the first user U1 and the coordinates of the second user U2). ).

Therefore, after executing the process in STEP302, unintended contact is suppressed even if the first ghost G1 and the second ghost G2 do not exist. Rather, when the first ghost G1 and the second ghost G2 are present, there is a risk that unintended contact may occur. Therefore, in this embodiment, the process of erasing the unnecessary first ghost G1 and second ghost G2 is performed.

Note that even after execution of the process in STEP 302, the correspondence relationship between the coordinates of the first avatar A1 and the coordinates of the second avatar A2 and the relationship between the coordinates of the first user U1 and the corresponding coordinates of the second user U2 are If there is no match, etc., at least one of the first ghost G1 and the second ghost G2 should continue to exist as required.

Next, the avatar coordinate determination unit 34 of the server 3 determines the first avatar A1 and the first ghost based on the coordinates and postures in the physical space RS of the first user U1 and the second user U2 recognized by the user information recognition unit 32. The coordinates and orientations of G1, the second avatar A2, and the second ghost G2 in the virtual space VS are determined (FIG. 8/STEP 303).

Specifically, the avatar coordinate determination unit 34 continues to determine the coordinates and orientation determined based on the coordinates and orientation of the first user by determining the details of the deviation that occurred in the process of expanding the virtual space VS (i.e., correction direction and correction amount) to determine the coordinates and orientation of the first avatar A1.

In addition, the avatar coordinate determination unit 34 determines the coordinates and posture determined based on the coordinates and posture of the second user as the content of the deviation (that is, the correction direction) predetermined according to the type of the recognized trigger event. and correction amount) to determine the coordinates and posture of the second avatar A2.

In this embodiment, the avatar coordinate determination unit 34 moves the coordinates of the second avatar A2 based on the coordinates of the second user U2 upward by the height of the second floor F2 relative to the first floor F1, In addition, the horizontal movement from the coordinates of the second avatar A2 at the time when the occurrence of the trigger event is recognized to the coordinates to the left of the first avatar A1 is the coordinates after the time when the occurrence of the trigger event is recognized. 2 Determined as coordinates of avatar A2.

Also, the avatar coordinate determination unit 34 determines the posture of the second avatar A2 based on the posture of the second user U2 as the posture of the second avatar A2 after the occurrence of the trigger event is recognized.

Next, the virtual space image determining unit 35 and the virtual space audio determining unit 36 of the server 3 determine the image and audio to be recognized by the first user U1 based on the coordinates and posture of the first avatar A1 in the virtual space VS. , the image and sound to be recognized by the second user U2 are determined based on the coordinates and posture of the second avatar A2 in the virtual space VS (FIG. 8/STEP 304).

Next, the HMD 4 worn by the user U displays the determined image on the monitor 41 and emits sound from the speaker 42 (FIG. 8/STEP 305).

Next, the user information recognition unit 32 of the server 3 determines whether or not the movement of the coordinates or the change of the posture in the physical space RS of the first user U1 or the second user U2 has been recognized (FIG. 8/STEP 306).

If the movement of the coordinates or the change in the posture of the first user U1 or the second user U2 in the physical space RS is recognized (YES in STEP306), the process returns to STEP303, and the processes after STEP303 are executed again.

On the other hand, if the movement of the coordinates or the change in posture of the first user U1 or the second user U2 in the physical space RS is not recognized (NO in STEP 306), the server 3 recognizes the signal instructing the end of the process. It is determined whether or not it has been completed (FIG. 8/STEP 307).

If the signal instructing the end could not be recognized (NO in STEP 307), the process returns to STEP 306, and the processes after STEP 306 are executed again.

On the other hand, if the signal instructing the end is recognized (YES in STEP307), the VR system S ends the current process.

As described above, in the VR system S, when the occurrence of the trigger event is recognized, the avatar coordinate determination unit 34 moves the coordinates of the second avatar A2 to coordinates adjacent to the coordinates of the first avatar A1. I am letting

Here, the images and sounds of the virtual space VS recognized by the first user U1 and the second user U2 are determined based on the coordinates of the first avatar A1 and the second avatar A1. Therefore, when the coordinates of the avatars match or are adjacent to each other, the images and sounds in the virtual space VS recognized by each user corresponding to those avatars are also matched or similar images and sounds (that is, similar image and sound).

Therefore, according to the VR system S, when the occurrence of the trigger event is recognized, the images and sounds in the virtual space recognized by each user can be matched or adjacent to each other. The first user U1 and the second user U2 who experienced different environments in the virtual space can now experience substantially the same environment.

In addition, in the present embodiment, a case has been described in which the second user U2 synchronizes his/her own environment with the environment of the first user U1. However, the virtual space experience system of the present invention is not limited to such a configuration. Anything that synchronizes with the other environment of the second user may be used.

Therefore, for example, in the present embodiment, even when the first avatar A1 touches the second ghost G2 corresponding to the second user U2, the coordinates of the second avatar A2 are set to the coordinates adjacent to the coordinates of the first avatar A1. It may be configured to move. That is, instead of the second avatar A2 moving itself to coordinates matching or adjacent to the coordinates of the first avatar A1, the first avatar A1 moves the coordinates of the second avatar A2 to coordinates matching or adjacent to its own coordinates. It may be configured to call to the coordinates that

Further, for example, in the present embodiment, when the second avatar A2 touches the first ghost G1 corresponding to the first user U1, the first avatar A1 itself has coordinates matching or adjacent to the coordinates of the second avatar A2. Alternatively, the first avatar A1 may call the coordinates of the second avatar A2 to coordinates that match or are adjacent to the coordinates of the first avatar A1.

In addition, a configuration for performing a process of moving one's own coordinates to coordinates matching or adjacent to the coordinates of another user and a process of calling other users' coordinates to coordinates matching or adjacent to the coordinates of one's own are performed. When implementing configuration and configuration at the same time, the action that becomes the trigger event may be made different according to the processing to be performed.

Specifically, for example, when the avatar corresponding to another user is touched with the right hand of the avatar corresponding to the user, the process of moving the user's own coordinates to coordinates matching or adjacent to the coordinates of the other user. , and when the left hand of the avatar corresponding to itself touches the avatar corresponding to another user, the other user's coordinates are called to the coordinates that match or are adjacent to the own coordinates. It should be configured as follows.

Further, in the present embodiment, when the occurrence of the trigger event is recognized, processing is performed to move the coordinates of the second avatar A2 not to the coordinates that match the coordinates of the first avatar A1 but to the coordinates that are adjacent to them. . This makes it easy for one user to include an avatar corresponding to the other user in the image recognized by one user when the coordinates of the second avatar A2 move, and for one user to move the other user closer to himself/herself. This is to make it easier for you to recognize what you have done.

However, the present invention is not limited to such a configuration, and the coordinates for moving one avatar may be matched with the coordinates of the other avatar. With this configuration, the degree of environmental synchronization can be further enhanced compared to the case of moving to adjacent coordinates.

Also, in this embodiment, after the occurrence of the trigger event is recognized, the second avatar A2 continues to be located on the second floor F2. That is, the state in which the correspondence relationship between the coordinates of the second user U2 and the coordinates of the second avatar A2 continues to be deviated based on the deviation caused by the movement when the occurrence of the trigger event is recognized.

However, the present invention is not limited to such a configuration, and may be configured to eliminate the deviation caused by the movement when the occurrence of the trigger event is recognized after that.

Therefore, for example, only during the period during which the trigger event continues (for example, in the above embodiment, during the period during which the second avatar A2 is touching the first avatar A1), or after the occurrence of the trigger event is recognized, a predetermined may be configured to move the avatar's coordinates for a period of time, and after the period elapses, return the avatar's coordinates to the coordinates before the occurrence of the triggering event was recognized.

[Second embodiment]
A VR system S, which is a virtual space experience system according to the second embodiment, will be described below with reference to FIGS. 2 and 11 to 15. FIG.

Note that the VR system S of this embodiment has the same configuration as the VR system S of the first embodiment, except for the processing that is executed when synchronizing the environment and after that.

Therefore, in the following explanation, only the processing that is executed when synchronizing the environment and afterward will be explained. In addition, configurations that are the same as or corresponding to the configuration of the VR system S of the first embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted.

[Processing when synchronizing the environment and afterward]
Processing executed when synchronizing the environments and after that will be described below with reference to FIGS. 2 and 11 to 15. FIG.

In this embodiment, for example, as shown in FIG. 12, touching the first ghost G1 corresponding to the first user U1 by the second avatar A2 is set as the trigger event.

Then, when the occurrence of the trigger event is recognized, as shown in FIG. 13, the environment recognized by the second user U2 is recognized by the first user U1 regardless of the coordinates of the second user U2. be consistent with the environment in which they exist. Specifically, the environment recognized by the second user U2 changes from one based on the coordinates of the second avatar A2 (see FIG. 14) to one based on the coordinates of the first avatar A1 (see FIG. 15).

Accordingly, in the VR system S, the environment of the first user U1 and the environment of the second user U2 are synchronized so that the first user U1 and the second user U2 can experience the same environment. ing.

As shown in FIG. 14, at the start of this process, the second user U2 recognizes an image of the first avatar A1 on the second floor F2 of the virtual space VS looking up. It is assumed that there is

In this process, first, the trigger event recognition unit 33 of the server 3 determines whether or not the occurrence of the trigger event has been recognized (FIG. 11/STEP 400).

Specifically, first, as shown in FIG. 12, the trigger event recognizing unit 33 recognizes that the second avatar A2 corresponding to the second user U2 is the first avatar A1 corresponding to the first user U1 in the virtual space VS. It is determined whether or not it is in a posture to touch the . Then, when the second avatar A2 assumes a posture of touching the first avatar A1, the trigger event recognition unit 33 recognizes that a trigger event has occurred.

When the occurrence of the trigger event is recognized (YES in STEP400), the avatar coordinate determination unit 34 of the server 3 fixes the coordinates of the second avatar A2 and the second ghost G2 (FIG. 11/STEP401).

In the present embodiment, in the subsequent processing of STEP402 to STEP404 (that is, the processing for synchronizing the environment), the images and sounds recognized by the second user U2 corresponding to the second avatar A2 and the second ghost G2 is determined based on the coordinates of the first avatar A1.

Therefore, if the second avatar A2 is able to move while the process is being executed, the coordinates of the second avatar A2 will unintentionally move after the process for synchronizing the environments. Become. As a result, after the process is finished, there is a possibility that the sense of immersion of the second user U2, who recognizes the image and sound based on the coordinates of the second avatar A2, is disturbed.

Also, if the second ghost G2 is able to move while the process is being executed, the coordinates of the second ghost G2 may unintentionally move after the process for synchronizing the environments is completed. Become. As a result, after the process ends, the position of the second ghost G2 suddenly changes in the image recognized by the first user U1, which may hinder the first user U1's sense of immersion. .

Therefore, in the VR system S, when recognizing the occurrence of the trigger event (that is, when the execution of the processing for synchronizing the environment is started), the processing of STEP 401 is executed, and the second avatar A2 and the second avatar A2 By fixing the coordinates of the ghost G2, such impediment to the sense of immersion is suppressed.

However, as shown in FIGS. 12 and 13, even while the process is being executed, the postures of the second avatar A2 and the second ghost G2 continue to change according to the posture of the second avatar A2. do.

This is because if even the posture is fixed, the correspondence relationship between the posture of the second user U2, the posture of the second avatar A2, and the posture of the second ghost G2 will be deviated after the processing for synchronizing the environment is completed. On the other hand, even if the posture of the second ghost G2 changes during the processing for synchronizing the environment, the immersive feeling of the first user U1 is not hindered.

However, the posture of the second avatar A2 may also be configured to match the posture of the first avatar A1. With this configuration, the image and sound recognized by the second user U2 are more consistent with the image and sound recognized by the first user U1. You will be able to experience a more synchronized environment.

Note that if the second user U2 moves while the coordinates of the second avatar A2 and the second ghost G2 are thus fixed, the coordinates of the second user U2, the coordinates of the second avatar A2, and the second ghost Unintentional deviation occurs in the correspondence relationship with the coordinates of G2. As a result, a factor that induces unintended contact occurs.

Therefore, as shown in FIG. 15, in the VR system S, a message to the effect that environment synchronization processing is being executed and a message to the effect that cautions against movement are displayed on the image recognized by the second user U2. . Accordingly, the VR system S suppresses the occurrence of such deviations.

Returning to the description of the processing to be executed, after executing the processing of STEP 401, the avatar coordinate determination unit 34 of the server 3 determines the coordinates and posture of the first user U1 in the physical space RS recognized by the user information recognition unit 32. , coordinates and orientations of the first avatar A1 and the first ghost G1 in the virtual space VS (FIG. 11/STEP 402).

Next, the virtual space image determining unit 35 and the virtual space audio determining unit 36 of the server 3 determine the image and audio to be recognized by the first user U1 and the second user U2 as the coordinates and posture of the first avatar A1 in the virtual space VS. (FIG. 5/STEP 403).

Specifically, the virtual space image determining unit 35 and the virtual space audio determining unit 36 determine the image and audio to be recognized by the first user U1 based on the coordinates and posture of the first avatar A1. Also, the virtual space image determination unit 35 and the virtual space audio determination unit 36 determine the image and audio to be recognized by the second user U2 based on the coordinates of the first avatar A1 and the posture of the second avatar A2.

The image and voice determined in this manner and recognized by the first user U1 are based on the coordinates and posture of the first avatar A1, and are based on the second floor F2 of the virtual space VS, and are looking at the rainbow VS3 in the sky above. become a thing.

In addition, the image and voice determined in this manner to be recognized by the second user U2 are recognized when positioned at coordinates behind the coordinates of the first avatar A1 as viewed from the first avatar A1. image and sound. Specifically, as shown in FIG. 15, the rainbow VS3 in the sky is viewed from behind the first avatar A1 on the second floor F2 of the virtual space VS.

In this embodiment, since the second user U2 enjoys the environment together with the first user U1, the first avatar A1 (and thus the first user U1) can be recognized in this process. The coordinates that serve as references for the images and sounds that are recognized by the second user U2 are the coordinates behind the coordinates of the first avatar A1 (that is, the adjacent coordinates).

However, if the second user U2 wants to experience an environment that is closer to the environment experienced by the first user U1, it is preferable to match the reference coordinates with the coordinates of the first avatar A1. . Also, when the reference posture is also matched with the first avatar A1, the second user U2 can experience the same environment as the first user U1 is experiencing.

Next, the HMD 4 worn by the user U causes the monitor 41 mounted on the HMD 4 to display the determined image, and the speaker 42 mounted on the HMD 4 to generate the determined sound (FIG. 11/ STEP 404).

Next, the trigger event recognition unit 33 of the server 3 determines whether or not it has recognized the cancellation of the trigger event (FIG. 11/STEP 405).

Specifically, first, the trigger event recognition unit 33 determines that the second avatar A2 corresponding to the second user U2 touches the first avatar A1 corresponding to the first user U1 in the virtual space VS (see FIG. 12). ) is released. Then, when the second avatar A2 releases the posture of touching the first avatar A1, the trigger event recognition unit 33 recognizes that the trigger event has been released.

Note that the release of the trigger event may be configured to be automatically recognized, for example, when a predetermined period of time has passed since the occurrence of the trigger event, or when another trigger event occurs (for example, The second avatar A2 may be configured to be recognized when the second avatar A2 is not touching the first ghost G1.

If the cancellation of the trigger event is not recognized (NO in STEP 405), the process returns to STEP 402, and the processes after STEP 402 are executed again.

On the other hand, if the release of the trigger event is recognized (YES in STEP405), the avatar coordinate determination unit 34 of the server 3 releases the fixation of the coordinates of the second avatar A2 and the second ghost G2 (FIG. 11/STEP406). .

If the occurrence of the trigger event is not recognized (NO in STEP 400), or after releasing the fixation of the coordinates of the second avatar A2 and the second ghost G2, the avatar coordinate determination unit 34 of the server 3 determines that the user Based on the coordinates and orientation of the first user U1 in the physical space RS recognized by the information recognition unit 32, the coordinates and orientation of the first avatar A1 and the first ghost G1 in the virtual space VS are determined (FIG. 11/STEP 407).

Next, the virtual space image determining unit 35 and the virtual space audio determining unit 36 of the server 3 determine the image and audio to be recognized by the first user U1 based on the coordinates and posture of the first avatar A1 in the virtual space VS. (FIG. 11/STEP 408).

Next, the avatar coordinate determination unit 34 of the server 3 determines the virtual space VS of the second avatar A2 and the second ghost G2 based on the coordinates and posture of the second user U2 in the physical space RS recognized by the user information recognition unit 32. Determine the coordinates and attitude in (FIG. 11/STEP 409).

Next, the virtual space image determining unit 35 and the virtual space audio determining unit 36 of the server 3 determine the image and audio to be recognized by the second user U2 based on the coordinates and posture of the second avatar A2 in the virtual space VS. (FIG. 11/STEP 410).

Next, the HMD 4 worn by the user U displays the determined image on the monitor 41 and emits sound from the speaker 42 (FIG. 11/STEP 411).

Next, the server 3 determines whether or not it has recognized a signal instructing the end of processing (FIG. 11/STEP 412).

If the signal instructing the end could not be recognized (NO in STEP 412), the process returns to STEP 400, and the processes after STEP 400 are executed again.

On the other hand, if the signal instructing the end is recognized (YES in STEP412), the VR system S ends the current process.

As described above, in the VR system S, when the occurrence of the trigger event is recognized, the virtual space image determining unit 35 and the virtual space audio determining unit 36 generate images and sounds of the virtual space VS to be recognized by the second user U2. is the image and sound when the coordinates of the second avatar A2 move to the coordinates adjacent to the coordinates of the first avatar A1.

That is, when the occurrence of the trigger event is recognized, the coordinates serving as the reference for determining the image and sound of the virtual space VS to be recognized by the second user U2 are set to the second avatar A2 corresponding to the second user U2. The coordinates are adjacent to the coordinates of the first avatar A1 corresponding to the first user U1 regardless of the position of the coordinates of .

Thus, when the occurrence of the trigger event is recognized, the second user U2 is determined together with the first user U1 based on the coordinates adjacent to the coordinates of the first avatar A1 corresponding to the first user U1. Images and sounds in the virtual space VS are recognized.

Therefore, according to the VR system S, after the occurrence of the triggering event is recognized, the second user U2 appears in the virtual space determined based on coordinates adjacent to the coordinates of the first avatar A1 corresponding to the first user U1 Since the images and sounds of the space VS are recognized, the first user and the second user who have experienced different environments in the virtual space VS can now experience substantially the same environment.

In addition, in the present embodiment, a case has been described in which the second user U2 synchronizes his environment with the environment of the first user U1. However, the virtual space experience system of the present invention is not limited to such a configuration. Anything that synchronizes with the other environment of the second user may be used.

Therefore, for example, in the present embodiment, when the first avatar A1 touches the second ghost G2 corresponding to the second user U2, the image and voice recognized by the first user U1 are moved to the coordinates of the second avatar A2. The determination may be based on matching coordinates or adjacent coordinates.

Also, for example, in the present embodiment, the second user U2 synchronizes his environment with the environment of the first user U1 who is another user. may be configured.

Specifically, for example, in the present embodiment, when the first avatar A1 touches the second ghost G2 corresponding to the second user U2, the image and voice recognized by the second user U2 are transferred to the first avatar A1. may be determined based on coordinates that match or are adjacent to the coordinates of . Conversely, when the second avatar A2 touches the first ghost G1 corresponding to the first user U1, the image and sound recognized by the first user U1 are changed to coordinates matching the coordinates of the second avatar A2 or adjacent to the coordinates of the second avatar A2. It may be configured so as to be determined using the coordinates as a reference.

Note that when simultaneously implementing a configuration for synchronizing one's own environment with another user's environment and a configuration for performing processing for synchronizing another user's environment with one's own environment, an operation that becomes a trigger event should be different depending on the processing to be performed.

Specifically, for example, when the avatar corresponding to the user touches the avatar corresponding to the other user with the right hand of the avatar corresponding to the user, a process of synchronizing the user's environment with the environment of the other user is executed, and the user can respond to the user itself. It is preferable to configure such that when an avatar corresponding to another user is touched with the avatar's left hand, a process of synchronizing the other user's environment with the own environment is executed.

In addition, a configuration for performing processing for synchronizing the environments by moving the coordinates as in the first embodiment and a configuration for performing processing for synchronizing the environments by changing the reference coordinates as in the present embodiment may be combined. When implemented at the same time, the action that becomes the trigger event should be changed according to the process to be performed.

Specifically, for example, when the avatar corresponding to the self touches the hand of the avatar corresponding to another user, the coordinates are moved to synchronize the environment, and the avatar corresponding to the self touches the hand of the avatar corresponding to the other user. When the shoulder of the avatar corresponding to the user is touched, it may be configured such that processing for synchronizing the environment is performed by changing the reference coordinates.

[Other embodiments]
Although the illustrated embodiment has been described above, the present invention is not limited to such a form.

For example, in the above-described embodiment, as shown in FIG. 4 and the like, the VR system S, which is a virtual space experience system, provides a first user U1 and a second user U2 who are present together in one room in the real space RS. A single virtual space VS corresponding to a room is configured to recognize each other through a first avatar A1 corresponding to a first user U1 and a second avatar A2 corresponding to a second user U2.

However, the virtual space experience system of the present invention is not limited to such a configuration, as long as avatars corresponding to multiple users can exist in the virtual space at the same time.

Therefore, for example, the first user and the second user may exist in different areas of the physical space (eg, their own rooms).

Further, in the above embodiment, when the processing for expanding the virtual space VS so as to be wider than the real space RS corresponding to the virtual space VS (the processing described with reference to the flowchart of FIG. 5) is executed, the environment are synchronized (the process described with reference to the flowchart of FIG. 8 or the flowchart of FIG. 11).

This is because when the user is made to perceive a virtual space that is expanded from the real space in this way, each user has more opportunities to experience different environments (and thus more opportunities to feel a desire to share the environment). is.

However, the virtual space experience system of the present invention is not limited to the configuration in which the process of synchronizing the environment is executed only when the process of expanding the virtual space is executed. user can also recognize that they are experiencing some kind of virtual space. Therefore, for example, avatars corresponding to each user do not have to exist in the same virtual space.

Specifically, a plurality of first avatars corresponding to a plurality of first users who are shoppers freely exist in a store existing in the virtual space, and a second avatar corresponding to a second user who is a store clerk. , a concierge counter existing in the virtual space. At this time, the virtual space in which the concierge counter exists may be the virtual space in which the store exists, or may be a virtual space different from the virtual space in which the store exists.

In the case of such a configuration, when one avatar itself is configured to move to coordinates matching or adjacent to the coordinates of the other avatar as in the first embodiment, the first When the occurrence of one trigger event is recognized, the second avatar moves from the concierge counter to the coordinates of the first avatar corresponding to the first user who caused the first trigger event among the plurality of first users. and move.

Also, in the case of such a configuration, only the viewpoint of one avatar is configured to move to coordinates that match or are adjacent to the coordinates of the other avatar as in the second embodiment. When the occurrence of the first trigger event is recognized, the image displayed to the second user corresponding to the second avatar is selected from the image of the concierge counter to display the first trigger of the plurality of first users. The image changes to be similar to the image displayed to the first user who caused the event.

In addition, when configured as described above, the first trigger event is, for example, an action in which the first avatar presses a call button, which is an object generated in the virtual space, or an action in which the second avatar presses a move button. etc.

Further, in the above embodiment, the second avatar A2 corresponding to the second user U2 corresponds to the first user U1, and is the first ghost located at the coordinates where the correspondence relation with the coordinates of the first user U1 is deviated. The action of touching G1 is taken as the first trigger event. In addition, the first avatar A1 corresponding to the first user U1 contacts the second ghost G2 corresponding to the second user U2 and located at coordinates that are deviated from the coordinates of the second user U2. can also be adopted as the first trigger event.

This is so that one user can intuitively understand that the action will cause a first trigger event that shares the environment with the other user. Furthermore, in order to generate the first trigger event, it is possible to prevent the user who is trying to generate the first trigger event from having a sense of immersion in the virtual space being hindered.

However, the first trigger event in the present invention is not limited to such a configuration, and may be appropriately set by a system designer or the like.

Therefore, for example, the first trigger event may be generated by touching an object (eg, a movement switch, a call switch, etc.) generated in the virtual space. Alternatively, the first trigger event may be generated when the avatar performs a specific action (eg, beckoning action).

Also, in the above embodiment, the trigger event is the action of the avatar corresponding to one user touching the ghost corresponding to the other user. However, the first trigger event in the present invention is not limited to such a configuration, and may be a predetermined action performed by the avatar corresponding to one user on the ghost corresponding to the other user. .

Here, the "predetermined action" may be any action performed by the avatar based on the ghost. For example, in addition to the action in which the avatar contacts the ghost as in this embodiment, the action in which the avatar moves within a predetermined range based on the ghost, and when the avatar operates an object existing in the virtual space. Another example is an operation of selecting a ghost as a target (for example, an operation of photographing a ghost with a camera-type object, etc.).

Also, in this embodiment, the ghost generated during the process of expanding the virtual space is used as a key for generating the trigger event. However, the ghost in the present invention is not limited to such a configuration, and may be an avatar that corresponds to the user and does not serve as a reference for images and sounds recognized by the user.

Therefore, for example, regardless of the execution of processing such as expanding the virtual space, from the beginning, together with the avatar that corresponds to the user and is the reference of the image and sound recognized by the user, an avatar that is independent of the avatar , and its independent avatar (for example, generated at coordinates different from the coordinates of the reference avatar) may be adopted as a ghost.

Specifically, for example, in a virtual space, an avatar corresponding to a dance instructor, an avatar corresponding to a student, and a miniature avatar corresponding to a student, which are present at the hands of the avatar corresponding to the instructor, are generated. In such a case, the miniature avatar may be employed as a ghost when the lecturer confirms the student's condition using the miniature avatar.

With such a structure, the instructor will be able to observe the student's condition from his/her own objective point of view and, if necessary, from the student's subjective point of view.

1... Sign, 2... Camera, 3... Server, 4... HMD, 31... Display image generator, 31a... Virtual space generator, 31b... Avatar generator, 31c... Moving object generator, 32... User information recognizer, 32a User posture recognition unit 32b User coordinate recognition unit 33 Trigger event recognition unit 34 Avatar coordinate determination unit 35 Virtual space image determination unit 36 Virtual space audio determination unit 41 Monitor (virtual space image display), 42... speaker (virtual space audio generator), A... avatar, A1... first avatar, A2... second avatar, F1... first floor, F2... second floor, G1... first ghost, G1a...first information board, G2...second ghost, G2a...second information board, RS...real space, RS1...whiteboard, S...VR system, U...user, U1...first user, U2...second user , VS... virtual space, VS1... elevator (moving object), VS2... switch, VS3... rainbow.

Claims

a virtual space generation unit that generates a virtual space corresponding to the real space in which the first user and the second user exist;
an avatar generation unit that generates a first avatar corresponding to the first user and a second avatar corresponding to the second user in the virtual space;
a user coordinate recognition unit that recognizes the coordinates of the first user and the coordinates of the second user in the physical space;
Determining the coordinates of the first avatar in the virtual space based on the coordinates of the first user, and determining the coordinates of the second avatar in the virtual space based on the coordinates of the second user. Department and
a virtual space image determination unit that determines an image of the virtual space to be recognized by the first user and the second user based on the coordinates of the first avatar and the coordinates of the second avatar;
a trigger event recognition unit that recognizes the occurrence of the first trigger event;
A virtual space experience system comprising a virtual space image display that allows the first user and the second user to recognize an image of the virtual space,
The avatar coordinate determination unit converts the coordinates of one of the first avatar and the second avatar to the coordinates of the other of the first avatar and the second avatar when occurrence of the first trigger event is recognized. A virtual space experience system characterized by moving to matching coordinates or adjacent coordinates.
a virtual space generation unit that generates a virtual space corresponding to the real space in which the first user and the second user exist;
an avatar generation unit that generates a first avatar corresponding to the first user and a second avatar corresponding to the second user in the virtual space;
a user coordinate recognition unit that recognizes the coordinates of the first user and the coordinates of the second user in the physical space;
Determining the coordinates of the first avatar in the virtual space based on the coordinates of the first user, and determining the coordinates of the second avatar in the virtual space based on the coordinates of the second user. Department and
a virtual space image determination unit that determines an image of the virtual space to be recognized by the first user and the second user based on the coordinates of the first avatar and the coordinates of the second avatar;
a trigger event recognition unit that recognizes the occurrence of the first trigger event;
A virtual space experience system comprising a virtual space image display that allows the first user and the second user to recognize an image of the virtual space,
The virtual space image determining unit determines the image of the virtual space to be recognized by one of the first user and the second user when occurrence of the first trigger event is recognized. A virtual space bodily sensation system characterized in that an image is generated when the coordinates of one of two avatars are moved to coordinates that match or are adjacent to the coordinates of the other of the first and second avatars.
In the virtual space sensation system according to claim 1 or claim 2,
The avatar generation unit generates a ghost, which corresponds to one of the first user and the second user and is an avatar independent of the first avatar and the second avatar, in the virtual space;
The avatar coordinate determining unit determines the coordinates of the ghost based on coordinates obtained by causing a predetermined deviation in the coordinates of the first avatar or the second avatar corresponding to one of the first user and the second user. decide and
The virtual space image determination unit adds information indicating that the image of the virtual space to be recognized by the other of the first user and the second user corresponds to one of the first user and the second user. including an image of the added ghost,
The virtual space, wherein the first trigger event is a predetermined action performed on the ghost by the first avatar or the second avatar corresponding to the other of the first user and the second user. sensory system.
In the virtual space experience system according to claim 3,
The trigger event recognition unit recognizes occurrence of a second trigger event,
The avatar generation unit generates a first ghost, which is the ghost corresponding to the first user, in the virtual space when occurrence of the second trigger event is recognized,
After the occurrence of the second trigger event is recognized, the avatar coordinate determining unit determines the coordinates of the first avatar based on the coordinates that cause the predetermined deviation from the coordinates of the first user. , determining the coordinates of the first ghost in the virtual space based on the coordinates of the first user;
After the occurrence of the second trigger event is recognized, the virtual space image determination unit adds information indicating that the virtual space image to be recognized by the second user corresponds to the first user. including the image of the added first ghost;
The virtual space experience system, wherein the first trigger event is a predetermined action performed by the second avatar on the first ghost.
In the virtual space sensation system according to claim 3 or claim 4,
The trigger event recognition unit recognizes occurrence of a second trigger event,
The avatar generation unit generates a second ghost, which is the ghost corresponding to the second user, in the virtual space when occurrence of the second trigger event is recognized,
After the occurrence of the second trigger event is recognized, the avatar coordinate determining unit determines the coordinates of the first avatar based on the coordinates that cause the predetermined deviation from the coordinates of the first user. determining the coordinates of the second ghost in the virtual space based on the coordinates obtained by causing the predetermined deviation from the coordinates of the second user;
After the occurrence of the second trigger event is recognized, the virtual space image determination unit adds information indicating that the virtual space image to be recognized by the first user corresponds to the second user. including the image of the added second ghost,
The virtual space experience system, wherein the first trigger event is a predetermined action performed by the first avatar on the second ghost.