JP7329114B1 - Information processing device, information processing method and program - Google Patents

Abstract

An object of the present invention is to make it easier for users using VR and users using AR to act together. An information processing device (3) causes a first display device (a display device used by a first user) to display a structure object, which is a VR object representing a structure in a virtual space associated with a real space. The VR control unit 332 and the second display device, which is a display device used by a second user different from the first user, have a first user object representing the avatar of the first user positioned at the position where the avatar exists in the virtual space. an acquisition unit 331 that acquires a device position indicating the position of the second display device in the real space from the second display device displayed at the real avatar position that is the corresponding position in the real space, and the VR control unit 332 , causing the first display device to display information about the position of the second user's avatar in the virtual space corresponding to the device position as an information VR object. [Selection drawing] Fig. 3

Description

The present invention relates to an information processing device, an information processing method, and a program.

In recent years, services that provide content related to VR (Virtual Reality) and AR (Augmented Reality) are becoming popular. Patent Literature 1 discloses a technique of providing a first user with VR-related content at a predetermined location in a virtual world and providing a second user with AR-related content in which the first user's avatar is arranged. ing.

Japanese Patent Publication No. 2016-522463

By using the above technology, a user existing at a predetermined location in real space and a user not present at the predetermined location can act together. However, since these users cannot directly see each other's entities, it is sometimes difficult for them to act together.

Accordingly, the present invention has been made in view of these points, and an object of the present invention is to make it easier for users using VR and users using AR to act together.

An information processing apparatus according to a first aspect of the present invention provides a structure object that is a VR object representing a structure in a virtual space associated with a real space. and a second display device which is a display device used by a second user different from the first user, wherein the first user object representing the avatar of the first user is the virtual space. position information for acquiring a device position indicating the position of the second display device in the real space from the second display device displayed at the real avatar position, which is the position in the real space corresponding to the position of the avatar in the real space an acquisition unit, wherein the VR control unit causes the first display device to display information about the position of the second user's avatar in the virtual space corresponding to the device position as an information VR object.

The VR control unit may control an action of the avatar based on control information for controlling the avatar of the first user in the virtual space, and the information processing device may control the first user object. is presented on the second display device at the real avatar position corresponding to the existence position of the avatar whose motion in the virtual space is controlled by the VR control unit based on the control information. You may further provide a part.

The VR control unit places a second user object, which is a VR object representing an avatar of the second user and different from the information VR object, at a position in the virtual space corresponding to the device position. It may be displayed on the first display device.

The VR control unit may display the first user object on the first display device based on control information for controlling the avatar of the first user in the virtual space, or A distance from the position of the first user object to the position of the second user object, or a distance from the position of the first user object in the virtual space to a position in the virtual space corresponding to the device position is predetermined. The information VR object may be displayed on the first display device when the information display threshold distance is exceeded.

The VR control unit may display the first user object on the first display device based on control information for controlling the avatar of the first user in the virtual space, or between the position of the first user object and the position of the second user object, or between the position of the first user object in the virtual space and a position in the virtual space corresponding to the device position; Triggered by the presence of the object, the information VR object may be displayed on the first display device.

The VR control unit may cause the first display device to display map information indicating a position where the second user object exists as the information VR object.

The VR control unit controls the distance from the position of the first user object to the position of the second user object, or from the position of the first user object in the virtual space to the position in the virtual space corresponding to the device position. may be displayed on the first display device as the information VR object.

The VR control unit may cause the first display device to display, as the information VR object, information indicating the direction of the second user object when the first user object is the starting point.

The VR control unit may cause the first display device to display the information VR object corresponding to the second user's avatar pre-associated with the first user.

An information processing method according to a second aspect of the present invention is a computer-executed display device in which a first user uses a structure object, which is a VR object representing a structure in a virtual space associated with a real space. and a second display device which is a display device used by a second user different from the first user, wherein the first user object showing the avatar of the first user is displayed on the first display device A device position indicating the position of the second display device in the real space is obtained from the second display device displayed at the real avatar position, which is the position in the real space corresponding to the existing position of the avatar in the virtual space. and, in the step of displaying the structure object, displaying information on the position of the second user's avatar in the virtual space corresponding to the device position as an information VR object on the first display device. .

A program according to a third aspect of the present invention provides a computer with a structure object, which is a VR object representing a structure in a virtual space associated with a real space. and a second display device which is a display device used by a second user different from the first user, wherein a first user object representing an avatar of the first user is the virtual A position for acquiring a device position indicating the position of the second display device in the real space from the second display device displayed at the real avatar position, which is the position in the real space corresponding to the existing position of the avatar in the space. The VR control unit causes the first display device to display information about the position of the second user's avatar in the virtual space corresponding to the device position as an information VR object.

ADVANTAGE OF THE INVENTION According to this invention, it is effective in the user using VR and the user using AR being able to make it easy to act together.

It is a figure for demonstrating the outline|summary of an information processing system. FIG. 4 is a diagram schematically showing an example of correspondence between virtual space and real space; 1 is a block diagram of an information processing device; FIG. It is a figure which shows an example of a structure of an event management database. FIG. 4 is a diagram schematically showing a VR image displayed on the first display device; FIG. 4 is a diagram schematically showing a VR image displayed on the first display device; FIG. 4 is a diagram schematically showing a VR image displayed on the first display device; It is a figure which represented typically the superimposition image displayed on a 2nd display device. It is a figure which represented typically the superimposition image displayed on a 2nd display device. It is a figure which represented typically the superimposition image displayed on a 2nd display device. It is a flow chart which shows the flow of an information processor.

[Overview of information processing system S]
FIG. 1 is a diagram for explaining an outline of an information processing system S. As shown in FIG. The information processing system S is a system used to provide an event providing service. The event providing service provides an avatar of a first user who does not exist in the action place in the real space and a second user who exists in the action place in the real space at a predetermined place in the real space (hereinafter referred to as "action place"). It is a service that provides events that make people act together.

The first user is, for example, a guide who guides the place of action, or an entertainer who takes a walk with the second user through an avatar, etc., and participates in the event providing service in response to a request from the operator who manages the event providing service, for example. User. A second user is a user different from the first user, and is, for example, a user who wishes to act with the first user and participates in the event providing service. The information processing system S has a first display device 1 , a second display device 2 and an information processing device 3 . The information processing system S may have multiple second display devices 2 .

The first display device 1 is a display device used by the first user, and is, for example, a smart phone, tablet, personal computer, head-mounted display, or the like. The first display device 1 displays information about VR. The first display device 1 includes an operation unit such as a touch panel, a keyboard, or a controller for accepting operations, a sound collecting unit such as a microphone for acquiring voice, a speaker for outputting voice, and for displaying information. and an output unit such as a liquid crystal display. The first display device 1 can communicate with the information processing device 3 via a network.

The second display device 2 is a display device used by the second user, such as a smart phone or a tablet. The second display device 2 displays information about AR. The second display device 2 includes an operation unit such as a touch panel, a keyboard, or a controller for receiving operations, a sound collecting unit such as a microphone for acquiring voice, a speaker for outputting voice, and for displaying information. an output unit such as a liquid crystal display, an imaging unit such as a camera for reading information, and a GPS (Global Positioning System) receiving unit for measuring the position of the second display device 2 . The second display device 2 can communicate with the information processing device 3 via a network.

The information processing device 3 is a device used to manage the event providing service, and is, for example, a server. The information processing device 3 can communicate with the first display device 1 and the second display device 2 via a network. The information processing device 3 executes a VR control process for executing a process of presenting VR-related information to the first user and an AR control process for executing a process of presenting AR-related information to the second user. . In this specification, the information processing system includes one information processing device 3 that executes VR control processing and AR control processing. A processing device 3 and a second information processing device 3 that executes an AR control process may be included.

The virtual space presented in the VR control process is associated with the real space. A VR object that is an object that indicates the predetermined object at the position of and that is displayed in the virtual space is presented. Further, the information processing device 3 causes the second display device 2 to display an object corresponding to the VR object in the real space at a position in the real space corresponding to the position in the virtual space where the VR object exists. Present the AR object.

FIG. 2 is a diagram schematically showing an example of correspondence between virtual space and real space. In the example shown in FIG. 2, the space R is a real space, and there are objects B (eg, structures such as buildings and roads) and a person H who is a second user. Space V is a virtual space, and object A is a VR object representing the first user's avatar.

The information processing device 3 presents, to the first display device 1, a VR object representing the object B at a position P1 in the space V corresponding to the position of the object B in the space R, and corresponding to the position of the person H in the space R. A VR object representing an avatar of a person H is presented at a position P2 in a space V where the space V is located. The information processing device 3 also presents an AR object indicating the object A at a position P3 in the space R corresponding to the position of the object A in the space V on the second display device 2 . By doing so, the information processing device 3 can allow the second user to experience a virtual experience of acting together with the first user.

Here, since the first user and the second user cannot directly see each other's entities in the real space, it may be difficult for the first user and the second user to act together. For example, if the first user and the second user move at different paces, one of the first user and the second user may lose sight of the other. Therefore, the information processing device 3 further presents information regarding the positions of the first user and the second user.

Specifically, the information processing device 3 further presents the information VR object to the first display device 1 and further presents the information AR object to the second display device 2 . The information VR object is a VR object that indicates information about the position of the avatar of the second user (person H in the example shown in FIG. 2), and is different from the VR object that indicates the object B and the VR object that indicates the avatar of the person H. It is a VR object. The information AR object is an AR object indicating information about the position in the real space corresponding to the position of the avatar of the first user in the virtual space (position P3 in the example shown in FIG. 2), and the AR indicating the avatar of the first user. It is an AR object different from the object.

By doing so, the information processing system S can allow the first user and the second user to grasp where each other is. Thereby, the information processing system S can reduce the occurrence of situations in which the first user and the second user lose sight of each other. As a result, the information processing system S can make it easier for the first user and the second user to act together.
The configuration of the information processing device 3 will be described below.

[Configuration of information processing device 3]
FIG. 3 is a block diagram of the information processing device 3. As shown in FIG. In FIG. 3, arrows indicate main data flows, and there may be data flows other than those shown in FIG. In FIG. 3, each block does not show the configuration in units of hardware (apparatus), but the configuration in units of functions. As such, the blocks shown in FIG. 3 may be implemented within a single device, or may be implemented separately within multiple devices. Data exchange between blocks may be performed via any means such as a data bus, network, or portable storage medium.

The information processing device 3 has a communication section 31 , a storage section 32 and a control section 33 . The information processing device 3 may be configured by connecting two or more physically separated devices by wire or wirelessly. Also, the information processing device 3 may be configured by a cloud that is a collection of computer resources.

The communication unit 31 has a communication controller for transmitting and receiving data between the first display device 1 and the second display device 2 via the network. The communication unit 31 notifies the control unit 33 of data received from the first display device 1 or the second display device 2 via the network. The communication unit 31 also transmits data output from the control unit 33 to the first display device 1 or the second display device 2 via the network.

The storage unit 32 is a storage medium including a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk drive, and the like. The storage unit 32 stores programs executed by the control unit 33 in advance. The storage unit 32 may be provided outside the information processing device 3, in which case data may be exchanged with the control unit 33 via a network.

The storage unit 32 stores a three-dimensional map for generating a virtual space, VR object information, information about the first user, and information about the second user. For example, the three-dimensional map has position coordinates corresponding to position coordinates in real space. The VR object information is, for example, information about a VR object representing structures such as buildings and roads that exist in the real space, and includes at least the attributes of the VR object (eg, shape, color, etc.) and the VR object in the virtual space. position and. The information about the first user includes, for example, identification information (ID: identifier) for identifying the first user, the name of the first user, and the like. Information about the second user includes, for example, the ID of the second user, the name of the second user, and the like.

The storage unit 32 stores an event management database that manages events in which the first user and the second user act together. FIG. 4 is a diagram showing an example of the configuration of an event management database. In the example shown in FIG. 4, the event management database associates and stores an event ID, start date and time, start place, VR participants, and AR participants.

The start date and time is the date and time when the event starts. The starting location is where the event starts. The VR participant is information indicating the first user participating in the event in VR, and is indicated by the ID of the first user, for example. The AR participants are information indicating one or more second users participating in the event in AR, and are indicated by the IDs of the second users, for example.

Returning to FIG. 3 , the control unit 33 has an acquisition unit 331 , a VR control unit 332 , an AR control unit 333 and a transmission unit 334 . The control unit 33 is a processor such as a CPU (Central Processing Unit), for example, and by executing a program stored in the storage unit 32, an acquisition unit 331, a VR control unit 332, an AR control unit 333, and a transmission unit 334 Function. Processing performed by the acquisition unit 331, the VR control unit 332, the AR control unit 333, and the transmission unit 334 will be described later.

[VR control processing]
First, VR control processing executed by the information processing device 3 will be described. The VR control unit 332 controls the action of the first user's avatar based on the control information. The control information is information for controlling the first user's avatar in the virtual space.

For example, when the first user performs an operation for moving in the virtual space on the first display device, the first display device 1 transmits information indicating the first user's operation to the information processing device 3 as control information. Then, when the acquisition unit 331 acquires the control information from the first display device 1, the VR control unit 332 moves the first user's avatar in the direction in the virtual space indicated by the control information. controls the behavior of

The control information may be information for moving the first user's avatar in the virtual space along a predetermined route. For example, the event management database further stores control information associated with events. In this case, the VR control unit 332 is triggered by the elapse of the start date and time of a certain event, and based on the control information associated with the event ID corresponding to the event, the first user's avatar controls the behavior of

The VR control unit 332 executes processing for presenting the VR object on the first display device 1 . Specifically, the VR control unit 332 causes the first display device 1 to display the structure object as the VR object. A structure object is a VR object that represents a structure in virtual space (a structure that exists in real space). More specifically, the VR control unit 332 displays the structure object at a position in the virtual space corresponding to the position of the structure in the real space.

For example, first, based on the three-dimensional map and the VR object information stored in the storage unit 32, the VR control unit 332 is a virtual space within a predetermined range, a virtual space corresponding to the position of the structure in the real space. A VR image is generated in which a virtual space including structural objects arranged at spatial positions is displayed. The predetermined range is a range determined according to the position of the first user's avatar (for example, a range of up to 100 meters from the position of the first user's avatar), or the behavior of the first user and the second user in the event providing service. It is an action range or the like that is predetermined as a range to perform. The transmitter 334 transmits the VR image generated by the VR controller 332 to the first display device 1 . Then, the first display device 1 displays the acquired VR image.

The VR control section 332 may cause the first display device 1 to generate a VR image. For example, first, the VR control unit 332 identifies a 3D map portion and a VR object information portion included in a predetermined range from the 3D map and VR object information stored in the storage unit 32 . The transmission unit 334 transmits the 3D map and the VR object information specified by the VR control unit 332 to the first display device 1 . Then, the first display device 1 generates a VR image in which a virtual space including a structure object is displayed based on the acquired 3D map and VR object information, and displays the generated VR image.

The VR control unit 332 may cause the first display device 1 to display the first user VR object as the VR object. The first user VR object is a VR object representing the first user's avatar, and is a VR object different from the structure object.

Specifically, the VR control unit 332 causes the first display device 1 to display the first user VR object based on the control information. More specifically, the VR control unit 332 displays the first user VR object at a position in the virtual space after controlling the action of the first user's avatar based on the control information.

The VR control unit 332 may cause the first display device 1 to display the second user VR object as the VR object. The second user VR object is a VR object representing the second user's avatar, and is a VR object different from the structure object and the first user VR object.

Specifically, first, the acquisition unit 331 functions as a position information acquisition unit and acquires the device position of the second display device 2 from the second display device 2 . The device position is information indicating the position of the second display device 2 in real space, for example, information indicating the position measured based on the information received by the GPS receiver of the second display device 2 . The device position is not limited to this, and may be information indicating a position specified based on a captured image of the surroundings captured by the imaging unit of the second display device 2. In this case, the second display device 2 captures the The position may be specified based on the image, or the information processing device 3 may specify the position based on the captured image. Then, the VR control unit 332 causes the first display device 1 to display the second user VR object at the position in the virtual space corresponding to the device position acquired by the acquisition unit 331 .

The VR control unit 332 causes the first display device 1 to display the information VR object as the VR object. The information VR object is a VR object different from the structure object, the first user VR object and the second user VR object.

Specifically, the VR control unit 332 displays information about the position of the second user's avatar in the virtual space corresponding to the device position of the second display device 2 in the real space on the first display device 1 as an information VR object. Let Information about the position is, for example, information indicating the position of the second user's avatar. More specifically, the VR control unit 332 causes the first display device 1 to display map information indicating the position where the second user VR object exists as the information VR object.

FIG. 5 is a diagram schematically showing a VR image displayed on the first display device 1. As shown in FIG. In the example shown in FIG. 5, an image V1 that is a VR image displayed on the first display device 1 displays an object X1 that is map information. A mark L1 indicating the position of the first user's avatar and a mark L2 indicating the position of the second user's avatar are displayed on the object X1. The information processing device 3 causes the first display device 1 to display such an information VR object, so that the first user can grasp where the second user VR object is.

The information about the position may be information indicating the distance from the first user to the second user. Specifically, the VR control unit 332 controls the distance from the position of the first user VR object to the position of the second user VR object, or the distance from the position of the first user VR object in the virtual space to the virtual space corresponding to the device position. Information indicating the distance to the position is displayed on the first display device 1 as an information VR object.

For example, first, the VR control unit 332 identifies the distance from the position in the real space corresponding to the position of the first user's avatar in the virtual space to the device position of the second display device 2 . The VR control section 332 may specify the distance from the position of the first user VR object in the virtual space to the position of the second user VR object in the virtual space. In addition, the VR control unit 332 moves from the position of the first user in the first space (either the virtual space or the real space) to a second space (where the first space is a virtual space) different from the first space. If it is a space, the second space is a real space, and if the first space is a real space, the second space is a virtual section). good too. The transmission unit 334 transmits information indicating the distance specified by the VR control unit 332 to the first display device 1 as position information. Then, the first display device 1 displays information indicating the distance as an information VR object.

FIG. 6 is a diagram schematically showing a VR image displayed on the first display device 1. As shown in FIG. In the example shown in FIG. 6, an image V2, which is a VR image displayed on the first display device 1, is displayed on the second display device 2 from a position in the real space corresponding to the position of the first user's avatar in the virtual space. , the distance to the device location is displayed. Note that a message indicating the distance from the position of the first user VR object to the position of the second user VR object may be displayed as the object X2 in the image V2. When the VR control unit 332 specifies the distance from the position in the real space corresponding to the position of the first user's avatar in the virtual space to the device position of the second display device 2, the image V2 includes an object X2 of A message indicating that the distance from the second user is too far or a message indicating a warning may be displayed. By causing the information VR object to be displayed on the first display device 1 by the information processing device 3, the first user can grasp how far the second user VR object is.

The information about the position may be information indicating the direction of the second user VR object. Specifically, the VR control unit 332 causes the first display device 1 to display, as an information VR object, information indicating the direction of the second user VR object when the first user VR object is the starting point.

For example, first, the VR control unit 332 identifies the direction of the second user VR object when the first user VR object is the starting point. The transmission unit 334 transmits information indicating the direction specified by the VR control unit 332 to the first display device 1 as information regarding the position. The first display device 1 then displays the information indicating the direction as an information VR object.

FIG. 7 is a diagram schematically showing a VR image displayed on the first display device 1. As shown in FIG. In the example shown in FIG. 7, an image V3, which is a VR image displayed on the first display device 1, includes an arrow object X3 indicating the direction of the second user VR object when the first user VR object is the starting point. is displayed. Note that an arrow indicating the position of the second user VR object with respect to the direction in which the first user VR object is facing may be displayed as the object X3 in the image V3. The information processing device 3 causes the first display device 1 to display such an information VR object, so that the first user can grasp which direction the second user VR object is.

The VR control unit 332 provides, as position information, map information indicating the position where the second user VR object exists, information indicating the distance from the first user to the second user, and the direction of the second user VR object. map information indicating the position where the second user VR object exists, information indicating the distance from the first user to the second user, and information indicating the second user VR object. and information indicating the direction of .

For example, the VR control unit 332 causes the first display device 1 to display the information VR object each time the acquisition unit 331 acquires the device position of the second display device 2 at predetermined intervals. The VR control section 332 may display the information VR object on the first display device 1 when a predetermined condition is satisfied.

Specifically, the VR control unit 332 corresponds to the distance from the position of the first user VR object in the virtual space to the position of the second user VR object, or the position of the first user VR object in the virtual space to the device position. The information VR object is displayed on the first display device 1 when the distance to the position in the virtual space becomes larger than the information display threshold distance. The information display threshold distance is a predetermined numerical value. The information display threshold distance is determined according to the place of action, such as a short distance in urban areas and towns with many people and buildings, and a long distance in suburbs and riverbeds with few people and buildings. can be different.

For example, every time the acquisition unit 331 acquires the device position of the second display device 2 at a predetermined interval, the VR control unit 332 changes the position of the second user VR object from the position of the first user VR object in the virtual space. It is determined whether or not the distance to is greater than the information display threshold distance. When the VR control unit 332 determines that the distance from the position of the first user VR object to the position of the second user VR object in the virtual space is greater than the information display threshold distance, the VR control unit 332 displays the information VR object on the first display device. 1 is displayed. By having the information processing device 3 display the information VR object on the first display device 1 under such conditions, the first user can recognize that the second user VR object is separated.

The VR control section 332 may display the information VR object on the first display device 1 when the second user VR object is blocked by the structure object. Specifically, the VR control unit 332 corresponds to between the position of the first user VR object and the position of the second user VR object in the virtual space, or between the position of the first user VR object and the device position in the virtual space. The information VR object is displayed on the first display device with the presence of the structure object between it and the position in the virtual space as a trigger.

For example, every time the acquisition unit 331 acquires the device position of the second display device 2 at predetermined intervals, the VR control unit 332 first determines the position of the first user VR object and the position of the second user VR object in the virtual space. It is determined whether or not there is a structure object between When the VR control unit 332 determines that a structure object exists between the position of the first user VR object and the position of the second user VR object in the virtual space, the VR control unit 332 transfers the information VR object to the first display device 1 to display. By having the information processing device 3 display the information VR object on the first display device 1 under such conditions, the first user can recognize that the second user VR object has disappeared.

For example, in a certain event, the VR control unit 332 executes processing for presenting a VR object to the first display device 1 when the start date and time of the event has passed. The VR control section 332 may cause the first display device 1 to display an information VR object corresponding to the second user's avatar who participated in the event.

Specifically, the VR control unit 332 causes the first display device 1 to display an information VR object corresponding to the second user's avatar previously associated with the first user. The VR control unit 332 displays, on the first display device 1, an information VR object corresponding to the avatar of the AR participant (second user) associated with the VR participant (first user) in the event management database, for example. Let By doing so, the information processing device 3 can suppress the display of the information VR object corresponding to the avatar of the second user who has not participated in the event.

Note that the VR control unit 332 may display the avatar of the second user participating in the event and the avatar of the second user not participating in the event in different modes. The VR control unit 332, for example, marks or flags the second user VR object corresponding to the second user participating in the event, or assigns a conspicuous color (eg, red, blue, etc.) to the second user VR object. ) is displayed. On the other hand, the VR control unit 332 may, for example, not color the second user VR object corresponding to the second user who is not participating in the event (for example, translucent), or make the second user VR object inconspicuous. It is displayed with a color (for example, white).

[AR control processing]
Next, AR control processing executed by the information processing device 3 will be described. The AR control unit 333 executes processing for presenting the AR object on the second display device 2 . Specifically, the AR control unit 333 presents the first user AR object as the AR object to the second display device 2 at the position of the real avatar. The first user AR object is an AR object representing the avatar of the first user. The real avatar position is a position in real space corresponding to the existing position of the first user's avatar in virtual space.

For example, first, the AR control unit 333 generates an AR image in which the first user AR object placed at the position of the real avatar is displayed. The transmission unit 334 transmits the AR image generated by the AR control unit 333 to the second display device 2 . Then, the second display device 2 displays a superimposed image in which the acquired AR image is superimposed on the captured image captured by the imaging unit of the second display device 2 .

The AR control section 333 may cause the second display device 2 to generate an AR image. For example, first, the transmission unit 334 transmits the AR object information regarding the first user AR object and the real avatar position to the second display device 2 . The AR object information is, for example, information indicating the shape, color, etc. of the first user AR object. Then, the second display device 2 generates an AR image in which the first user AR object placed at the real avatar position is displayed based on the obtained AR object information and the real avatar position, and displays the generated AR image. A superimposed image superimposed on the captured image captured by the imaging unit of the second display device 2 is displayed.

The AR control unit 333 converts the first user AR object into a real object corresponding to the position where the first user's avatar exists after the VR control unit 332 controls the motion of the first user's avatar in the virtual space based on the control information. Presented to the avatar position. Specifically, the AR control unit 333 presents the first user AR object at the real avatar position corresponding to the second position of the avatar of the first user moved from the first position of the avatar of the first user. do. The first existing position is the existing position in the virtual space before the VR control unit 332 moves the first user's avatar based on the control information. The second existing position is the existing position in the virtual space after the VR control unit 332 moves the first user's avatar based on the control information.

The AR control unit 333 controls the operation of the first user AR object by moving the first user AR object in the direction in the real space indicated by the control information, and controls the first user AR object based on the control information. may be presented at the actual avatar position after Specifically, the AR control unit 333 presents the first user AR object to the second real avatar position moved from the first real avatar position. The first real avatar position is the position in real space before the first user AR object is moved based on the control information. The second real avatar position is the position in real space after the first user AR object is moved based on the control information.

When there are multiple second users, the AR control unit 333 may present the other second user AR object as an AR object to one of the multiple second users. Specifically, the AR control unit 333 causes the second display device 2 of one of the plurality of second users to display the other second user AR object at the device position of the other second display device 2 . to display.

The AR control unit 333 further presents the information AR object to the second display device 2 as an AR object. The information AR object is an AR object different from the first user AR object and the second user AR object.

Specifically, the AR control unit 333 causes the second display device 2 to further display information about the real avatar position of the first user's avatar as an information AR object. Information about the real avatar position is, for example, information indicating the position of the first user's avatar. More specifically, the AR control unit 333 causes the second display device 2 to display map information indicating the position where the first user's avatar exists as an information AR object.

FIG. 8 is a diagram schematically showing a superimposed image displayed on the second display device 2. As shown in FIG. An image O1 displayed on the second display device 2 in the example shown in FIG. 8 is a superimposed image obtained by superimposing a captured image captured by the second display device 2 on an AR image. An object Y1, which is map information, is displayed in the image O1. A mark L3 indicating the position of the second user's avatar and a mark L4 indicating the position of the first user's avatar are displayed on the object Y1. By causing the second display device 2 to display such an information AR object by the information processing device 3, the second user can grasp where the first user AR object is.

Information about the real avatar position may be information indicating the distance from the first user to the second user. Specifically, the AR control unit 333 causes the second display device 2 to display information indicating the distance between the real avatar position and the device position as an information AR object.

For example, first, the AR control unit 333 identifies the distance from the real avatar position to the device position. The AR control unit 333 may specify the distance from the position of the first user VR object in the virtual space to the position of the second user VR object in the virtual space. Also, the AR control unit 333 controls the position of the first user in the first space (either the virtual space or the real space) to move to a second space (where the first space is a virtual space) different from the first space. If it is a space, the second space is a real space, and if the first space is a real space, the second space is a virtual section). good too. The transmission unit 334 transmits information indicating the distance specified by the AR control unit 333 to the second display device 2 as information on the real avatar position. Then, the second display device 2 displays information indicating the distance as an information VR object.

FIG. 9 is a diagram schematically showing a superimposed image displayed on the second display device 2. As shown in FIG. The image O2 displayed on the second display device 2 in the example shown in FIG. 9 is a superimposed image obtained by superimposing the captured image captured by the second display device 2 on the AR image. An object Y2 of a message indicating the distance from the first user's real avatar position to the device position of the second display device 2 is displayed in the image O2. Note that a message indicating the distance from the position of the first user VR object to the position of the second user VR object may be displayed as the object Y2 in the image O2. When the AR control unit 333 specifies the distance from the position in the real space corresponding to the position of the first user's avatar in the virtual space to the device position of the second display device 2, the image O2 includes an object Y2 of A message indicating that the distance from the first user is too far or a warning message may be displayed. By causing the second display device 2 to display such an information AR object by the information processing device 3, the second user can grasp how far the second user AR object is from the first user AR object.

The information about the real avatar position may be information indicating the direction of the first user AR object. Specifically, the AR control unit 333 displays information indicating the direction of the real avatar position of the first user with the device position of the second display device 2 as the starting point on the second display device 2 as an information AR object. Let

For example, first, the AR control unit 333 identifies the direction of the real avatar position of the first user when the device position of the second display device 2 is used as the starting point. The transmission unit 334 transmits information indicating the direction specified by the AR control unit 333 to the second display device 2 as information regarding the real avatar position. Then, the second display device 2 displays the information indicating the direction as an information AR object.

FIG. 10 is a diagram schematically showing a superimposed image displayed on the second display device 2. As shown in FIG. An image O3 displayed on the second display device 2 in the example shown in FIG. 10 is a superimposed image obtained by superimposing a captured image captured by the second display device 2 on an AR image. The image O3 displays an arrow object Y3 that indicates the direction of the real avatar position of the first user when the device position of the second display device 2 is the starting point. In the image O3, an arrow indicating the real avatar position of the first user with respect to the direction in which the imaging unit of the second display device 2 faces may be displayed as the object Y3. By causing the second display device 2 to display such an information AR object by the information processing device 3, the second user can grasp the direction in which the first user AR object is.

The AR control unit 333 provides map information indicating the position where the first user's avatar exists, information indicating the distance from the first user to the second user, and information indicating the direction of the first user AR object as position information. map information indicating the position where the first user's avatar exists, information indicating the distance from the first user to the second user, and the first user AR object and information indicating the direction of .

The AR control unit 333 causes the second display device 2 to display the information AR object each time the acquisition unit 331 acquires the device position of the second display device 2 at predetermined intervals, for example. The AR control unit 333 may cause the AR control unit 333 to display the information AR object when a predetermined condition is satisfied.

Specifically, when the distance between the first user's actual avatar position and the device position of the second display device 2 becomes greater than the information display threshold distance, the AR control unit 333 displays the information AR object on the second display device. 2 is displayed.

For example, every time the acquisition unit 331 acquires the device position of the second display device 2 at a predetermined interval, the AR control unit 333 first detects the distance from the real avatar position of the first user to the device position of the second display device 2. Determine whether the distance is greater than the information display threshold distance. When the AR control unit 333 determines that the distance from the first user's actual avatar position to the device position of the second display device 2 is greater than the information display threshold distance, the AR control unit 333 displays the information AR object on the second display device 2. display. By causing the information AR object to be displayed on the second display device 2 by the information processing device 3 under such conditions, the second user can recognize that the first user AR object is separated.

The AR control unit 333 may display the information AR object on the second display device 2 when the first user AR object is blocked by a structure. Specifically, the AR control unit 333 detects that a structure object exists between the position of the first user's avatar in the virtual space and the position in the virtual space corresponding to the device position of the second display device 2 . Triggered by the determination by the control unit 332 , the information AR object is displayed on the second display device 2 .

For example, every time the acquisition unit 331 acquires the device position of the second display device 2 at predetermined intervals, the VR control unit 332 first determines the position of the first user VR object and the position of the second user VR object in the virtual space. It is determined whether or not there is a structure object between Then, when the VR control unit 332 determines that a structure object exists between the position of the first user VR object and the position of the second user VR object in the virtual space, the AR control unit 333 controls the information AR object is displayed on the second display device 2 .

The AR control unit 333 causes the second display device 2 to display the information AR object when a structure exists between the first user's real avatar position and the device position of the second display device 2 . may For example, the storage unit 32 stores information indicating a map in real space. In this case, first, the AR control unit 333 refers to the map in the real space stored in the storage unit 32 each time the acquisition unit 331 acquires the device position of the second display device 2 at predetermined intervals, It is determined whether a structure exists between the real avatar position of the first user and the device position of the second display device 2 . Then, when the AR control unit 333 determines that a structure exists between the first user's real avatar position and the device position of the second display device 2, the AR control unit 333 displays the information AR object on the second display device 2. display. By having the information processing device 3 display the information VR object on the second display device 2 under such conditions, the second user can recognize that the first user AR object has disappeared.

For example, in a certain event, the AR control unit 333 executes processing for presenting an AR object to the second display device 2 when the start date and time of the event has passed. The AR control unit 333 may cause the second display device 2 to display an information AR object corresponding to the avatar of the first user who participated in the event.

Specifically, the AR control unit 333 displays the information AR object on the second display device 2 used by the second user pre-associated with the first user. For example, in the event management database, the AR control unit 333 stores the information AR object corresponding to the VR participant (first user) on the second display device of the AR participant (second user) associated with the VR participant. 2 is displayed. By doing so, the information processing device 3 can suppress display of the information AR object corresponding to the avatar of the first user who has not participated in the event in which the second user participates.

Note that the AR control unit 333 may display the avatar of the first user participating in the event and the avatar of the first user not participating in the event in different modes. For example, the AR control unit 333 puts a mark or flag on the first user AR object corresponding to the first user participating in the event, or colors the first user AR object in a conspicuous color (for example, red, blue, etc.). ) is displayed. On the other hand, the AR control unit 333 may, for example, not color the first user AR object corresponding to the first user who is not participating in the event (for example, translucent), or make the first user AR object inconspicuous. It is displayed with a color (for example, white).

[Processing flow of information processing device 3]
Next, the processing flow of the information processing device 3 will be described. FIG. 11 is a flow chart showing the flow of the information processing device 3. As shown in FIG. This flowchart starts when the acquisition unit 331 acquires the device position from the second display device 2 (S1).

The VR control unit 332 identifies the distance from the position of the first user's avatar in the virtual space to the position of the second user's avatar corresponding to the device position of the second display device 2 (S2). The VR control unit 332 determines whether or not the identified distance is greater than the information display threshold distance (S3).

When the VR control unit 332 determines that the specified distance is not greater than the information display threshold distance (NO in S3), the VR image includes the structure object and does not include the information VR object. A certain first VR image is presented on the first display device 1 (S4). Further, when the VR control unit 332 determines that the specified distance is not greater than the information display threshold distance (NO in S3), the AR control unit 333 controls the AR image including the first user AR object, , the first AR image, which is an AR image containing no information AR object, is presented on the second display device 2 (S5).

On the other hand, when the VR control unit 332 determines that the specified distance is greater than the information display threshold distance (YES in S3), the VR control unit 332 displays the second VR image that is a VR image including the structure object and the information VR object. Presented on the first display device 1 (S6). Further, when the VR control unit 332 determines that the specified distance is greater than the information display threshold distance (YES in S3), the AR control unit 333 displays the AR image including the first user AR object and the information AR object. is presented on the second display device 2 (S7).

[Effects of this embodiment]
As described above, the information processing device 3 presents information regarding the position of the second user's avatar to the first display device 1 as an information VR object, and presents the real avatar of the first user to the second display device 2 . Information about the location is presented as an information AR object. By doing so, the information processing device 3 can allow the first user and the second user to grasp where each other is. Thereby, the information processing device 3 can reduce the occurrence of situations in which the first user and the second user lose sight of each other. As a result, the information processing device 3 can make it easier for the first user and the second user to act together.

The present invention makes it possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, "Industry and technological innovation."

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist thereof. be. For example, all or part of the device can be functionally or physically distributed and integrated in arbitrary units. In addition, new embodiments resulting from arbitrary combinations of multiple embodiments are also included in the embodiments of the present invention. The effect of the new embodiment caused by the combination has the effect of the original embodiment.

1 first display device 2 second display device 3 information processing device 31 communication unit 32 storage unit 33 control unit 331 acquisition unit 332 VR control unit 333 AR control unit 334 transmission unit S information processing system

Claims (11)

A VR control unit for displaying a structure object, which is a VR (Virtual Reality) object representing a structure in a virtual space associated with a real space, on a first display device, which is a display device used by a first user;
A second display device used by a second user different from the first user, wherein a first user object representing an avatar of the first user corresponds to a position of the avatar in the virtual space. a position information acquisition unit that acquires a device position indicating the position of the second display device in the real space from the second display device displayed at the position of the real avatar, which is the position in the real space;
with
The VR control unit is a VR object indicating information about the position of the second user's avatar in the virtual space corresponding to the device position, and is a VR object different from the second user object indicating the second user's avatar. causing an information VR object, which is an object , to be displayed on the first display device;
Information processing equipment. The VR control unit controls the action of the avatar based on control information for controlling the avatar of the first user in the virtual space,
The first user object is displayed on the second display device at the real avatar position corresponding to the existence position of the avatar whose motion in the virtual space is controlled by the VR control unit based on the control information. Further comprising an AR (Augmented Reality) control unit presented to
The information processing device according to claim 1 . The VR control unit causes the first display device to display the second user object at a position in the virtual space corresponding to the device position.
The information processing device according to claim 1 . The VR control unit
displaying the first user object on the first display device based on control information for controlling the first user's avatar in the virtual space;
The distance from the position of the first user object in the virtual space to the position of the second user object, or the distance from the position of the first user object in the virtual space to the position in the virtual space corresponding to the device position is greater than a predetermined information display threshold distance, causing the information VR object to be displayed on the first display device;
The information processing apparatus according to claim 3. The VR control unit
displaying the first user object on the first display device based on control information for controlling the first user's avatar in the virtual space;
Between the position of the first user object and the position of the second user object in the virtual space, or between the position of the first user object in the virtual space and a position in the virtual space corresponding to the device position display the information VR object on the first display device, triggered by the presence of the structure object in
The information processing apparatus according to claim 3. The VR control unit displays map information indicating a position where the second user object exists on the first display device as the information VR object;
The information processing apparatus according to claim 3. The VR control unit controls the distance from the position of the first user object to the position of the second user object, or from the position of the first user object in the virtual space to the position in the virtual space corresponding to the device position. display information indicating the distance of the first display device as the information VR object;
The information processing apparatus according to claim 3. The VR control unit causes the first display device to display, as the information VR object, information indicating the direction of the second user object when the first user object is the starting point;
The information processing apparatus according to claim 3. The VR control unit causes the first display device to display the information VR object corresponding to the second user's avatar pre-associated with the first user.
The information processing device according to claim 1 . the computer runs
a step of displaying a structure object, which is a VR object representing a structure in a virtual space associated with the real space, on a first display device, which is a display device used by a first user;
A second display device used by a second user different from the first user, wherein a first user object representing an avatar of the first user corresponds to a position of the avatar in the virtual space. obtaining a device position indicating the position of the second display device in the real space from the second display device displayed at the real avatar position, which is the position in the real space;
with
In the step of displaying the structure object, a VR object indicating information about the position of the second user's avatar in the virtual space corresponding to the device position, the second user object indicating the second user's avatar. causing the first display device to display an information VR object that is a VR object different from the
Information processing methods. the computer,
A VR control unit that causes a first display device, which is a display device used by a first user, to display a structure object, which is a VR object representing a structure in a virtual space associated with a real space, and the first user; is a second display device used by a different second user, wherein a first user object representing an avatar of the first user is located at a position in the real space corresponding to the existing position of the avatar in the virtual space a position information acquisition unit that acquires a device position indicating the position of the second display device in the real space from the second display device displayed at the position of the real avatar,
function as
The VR control unit is a VR object indicating information about the position of the second user's avatar in the virtual space corresponding to the device position, and is a VR object different from the second user object indicating the second user's avatar. causing an information VR object, which is an object , to be displayed on the first display device;
program.

Images