JP7381680B1 - Information processing device, information processing method and program - Google Patents

Abstract

An object of the present invention is to make it easier for a user using VR and a user using AR to act together.
[Solution] An information processing device 3 displays an AR object representing an avatar whose motion is controlled based on control information for controlling the avatar of the first user in a virtual space associated with a real space. The AR control unit 333 includes an AR control unit 333 that displays a display device used by a second user different from the user at a real avatar position that is a position in real space corresponding to the position of the avatar in the virtual space, and the AR control unit 333 includes: Information regarding the actual avatar position is further presented on the display device as an information AR object that is an AR object different from the AR object.
[Selection diagram] Figure 3

Description

The present invention relates to an information processing device, an information processing method, and a program.

In recent years, services that provide content related to VR (Virtual Reality) and AR (Augmented Reality) are becoming popular. Patent Document 1 discloses a technology for providing a first user with VR-related content at a predetermined location in a virtual world, and providing a second user with AR-related content in which an avatar of the first user is placed. ing.

Special Publication No. 2016-522463

By using the above technology, a user who is present at a predetermined location in real space and a user who is not present at the predetermined location can act together. However, since these users cannot directly visually recognize each other's entities, it is sometimes difficult for these users to act together.

Therefore, the present invention has been made in view of these points, and an object of the present invention is to make it easier for users who use VR and users who use AR to act together.

An information processing device according to a first aspect of the present invention includes an AR object showing an avatar whose motion is controlled based on control information for controlling the avatar of a first user in a virtual space associated with a real space. to a display device used by a second user different from the first user, at a real avatar position that is a position in the real space corresponding to the existing position of the avatar in the virtual space. The AR control unit further presents information regarding the real avatar position to the display device as an information AR object that is an AR object different from the AR object.

The information processing device may further include a VR control unit that controls the motion of the avatar by moving the avatar in a direction in the virtual space indicated by the control information, and the AR control unit may The AR object may be presented at the real avatar position corresponding to the avatar's existing position after the VR control unit controls the movement of the avatar in the virtual space based on the control information.

The AR control unit controls the operation of the AR object by moving the AR object in a direction in the real space indicated by the control information, and controls the AR object after controlling it based on the control information. It may be presented at the position of the real avatar.

The information processing device displays a structure existing in the real space on the first display device that is a first display device used by the first user and is different from a second display device that is the display device. The apparatus may further include a VR control unit that presents a structure object, which is the VR object shown, at a position in the virtual space corresponding to a position of the structure.

The information processing apparatus may further include a position information acquisition unit that acquires a device position indicating the position of the second display device in the real space, and the AR control unit acquires a position of the avatar in the virtual space. and a position in the virtual space corresponding to the device position, when the VR control unit determines that the structure object exists, the information AR object may be displayed on the display device. good.

The information processing apparatus may further include a position information acquisition unit that acquires a device position indicating the position of the second display device in the real space, and the AR control unit is configured to acquire the position of the second display device in the real space. The information AR object may be displayed on the display device in response to the presence of the structure between the information AR object and the location.

The information processing device may further include a position information acquisition unit that acquires a device position indicating the position of a display device used by the second user in the real space, and the AR control unit may The information AR object may be displayed on the display device when the distance between the information AR object and the device position becomes larger than a predetermined information display threshold distance.

The AR control unit may display map information indicating a position where the avatar exists on the display device as the information AR object.

The information processing device may further include a position information acquisition unit that acquires a device position indicating the position of a display device used by the second user in the real space, and the AR control unit may and the device position may be displayed on the display device as the information AR object.

The AR control unit may cause the display device to display information indicating the direction of the real avatar position when the position of the display device is a starting point, as the information AR object.
The AR control unit may display the information AR object on a display device used by a second user associated with the first user in advance.

In the information processing method according to the second aspect of the present invention, the operation is controlled based on control information for controlling the avatar of the first user in the virtual space associated with the real space, which is executed by the computer. Presenting an AR object representing the avatar to a display device used by a second user different from the first user at a real avatar position that is a position in the real space corresponding to the existing position of the avatar in the virtual space. and in the step of presenting the AR object, the display device further presents information regarding the real avatar position as an information AR object that is an AR object different from the AR object.

A program according to a third aspect of the present invention is configured to cause a computer to use an AR indicative of an avatar whose motion is controlled based on control information for controlling an avatar of a first user in a virtual space associated with a real space. an AR control unit that causes an object to be presented on a display device used by a second user different from the first user at a real avatar position that is a position in the real space that corresponds to the existing position of the avatar in the virtual space; The AR control unit further causes the display device to present information regarding the real avatar position as an information AR object that is an AR object different from the AR object.

According to the present invention, it is possible to make it easier for a user who uses VR and a user who uses AR to act together.

FIG. 1 is a diagram for explaining an overview of an information processing system. FIG. 2 is a diagram schematically showing an example of correspondence between virtual space and real space. FIG. 2 is a block diagram of an information processing device. FIG. 2 is a diagram illustrating an example of the configuration of an event management database. FIG. 3 is a diagram schematically representing a VR image displayed on a first display device. FIG. 3 is a diagram schematically representing a VR image displayed on a first display device. FIG. 3 is a diagram schematically representing a VR image displayed on a first display device. FIG. 3 is a diagram schematically showing a superimposed image displayed on a second display device. FIG. 3 is a diagram schematically showing a superimposed image displayed on a second display device. FIG. 3 is a diagram schematically showing a superimposed image displayed on a second display device. 3 is a flowchart showing the flow of the information processing device.

[Overview of information processing system S]
FIG. 1 is a diagram for explaining an overview of the information processing system S. As shown in FIG. The information processing system S is a system used to provide an event providing service. The event providing service provides, at a predetermined place in real space (hereinafter referred to as "behavior place"), an avatar of a first user who does not exist at the action place in real space and a second user who exists at the action place in real space. This is a service that provides events that encourage people to act together.

The first user is, for example, a guide who shows the place of action, or a celebrity who takes a walk with the second user through an avatar, and, for example, participates in the event providing service in response to a request from a business that manages the event providing service. I am a user. The second user is a user different from the first user, and is, for example, a user who participates in the event providing service hoping to interact with the first user. The information processing system S includes a first display device 1, a second display device 2, and an information processing device 3. The information processing system S may include a plurality of second display devices 2.

The first display device 1 is a display device used by a first user, and is, for example, a smartphone, a tablet, a personal computer, a head-mounted display, or the like. The first display device 1 displays information regarding VR. The first display device 1 includes an operation unit such as a touch panel, keyboard, or controller for receiving operations, a sound collection unit such as a microphone for acquiring audio, a speaker for outputting audio, and a display unit for displaying information. It has an output section such as a liquid crystal display. The first display device 1 can communicate with the information processing device 3 via a network.

The second display device 2 is a display device used by a second user, and is, for example, a smartphone or a tablet. The second display device 2 displays information regarding AR. The second display device 2 includes an operation unit such as a touch panel, keyboard, or controller for receiving operations, a sound collection unit such as a microphone for acquiring audio, a speaker for outputting audio, and a speaker for displaying information. It has an output section such as a liquid crystal display, an imaging section such as a camera for reading information, and a GPS (Global Positioning System) receiving section for measuring the position of the second display device 2. The second display device 2 can communicate with the information processing device 3 via the network.

The information processing device 3 is a device used to manage an event providing service, and is, for example, a server. The information processing device 3 can communicate with the first display device 1 and the second display device 2 via a network. The information processing device 3 executes a VR control process for executing a process for presenting information regarding VR to a first user, and an AR control process for executing a process for presenting information regarding AR to a second user. . In this specification, an example will be described in which the information processing system includes one information processing device 3 that executes VR control processing and AR control processing, but is not limited thereto. It may include a processing device 3 and a second information processing device 3 that executes AR control processing.

The virtual space presented in the VR control process is associated with the real space, and the information processing device 3 displays the virtual space corresponding to the position in the real space where a predetermined object exists with respect to the first display device 1. A VR object that represents the predetermined object and is displayed in the virtual space is presented at the position. The information processing device 3 also displays an object corresponding to the VR object in the real space at a position in the real space corresponding to a position in the virtual space where the VR object exists, to the second display device 2. Present an AR object.

FIG. 2 is a diagram schematically showing an example of the correspondence between virtual space and real space. In the example shown in FIG. 2, space R is a real space, and object B (for example, structures such as buildings and roads) and person H, who is the second user, exist. Space V is a virtual space, and object A is a VR object representing the first user's avatar.

The information processing device 3 presents the first display device 1 with a VR object indicating the object B at a position P1 in the space V corresponding to the position of the object B in the space R, and the VR object corresponds to the position of the person H in the space R. A VR object representing an avatar of person H is presented at position P2 in space V. Furthermore, the information processing device 3 presents the second display device 2 with an AR object that indicates the object A at a position P3 in the space R that corresponds to the position of the object A in the space V. By doing so, the information processing device 3 can allow the second user to experience a simulated experience of acting together with the first user.

Here, since the first user and the second user cannot directly visually recognize each other in real space, it may be difficult for the first user and the second user to act together. For example, if the first user and the second user move at different paces, a situation may arise where either the first user or the second user loses sight of the other. Therefore, the information processing device 3 further presents information regarding the respective positions of the first user and the second user.

Specifically, the information processing device 3 further presents an information VR object to the first display device 1 and further presents an information AR object to the second display device 2. The information VR object is a VR object indicating information regarding the position of the avatar of the second user (person H in the example shown in FIG. 2), and is different from the VR object indicating object B and the VR object indicating the avatar of person H. It is a VR object. The information AR object is an AR object indicating information regarding a position in real space (position P3 in the example shown in FIG. 2) corresponding to the position of the first user's avatar in the virtual space, and is an AR object indicating the first user's avatar. This is an AR object different from the object.

By doing so, the information processing system S can make the first user and the second user know where each other is. Thereby, the information processing system S can reduce the occurrence of a situation in which the first user and the second user lose sight of each other. As a result, the information processing system S can make it easier for the first user and the second user to act together.
The configuration of the information processing device 3 will be described below.

[Configuration of information processing device 3]
FIG. 3 is a block diagram of the information processing device 3. In FIG. 3, arrows indicate main data flows, and data flows other than those shown in FIG. 3 may exist. In FIG. 3, each block shows the configuration not in hardware (device) units but in functional units. Therefore, the blocks shown in FIG. 3 may be implemented within a single device, or may be implemented separately within multiple devices. Data may be exchanged between blocks via any means such as a data bus, network, or portable storage medium.

The information processing device 3 includes a communication section 31, a storage section 32, and a control section 33. The information processing device 3 may be configured by two or more physically separate devices connected by wire or wirelessly. Further, the information processing device 3 may be configured by a cloud that is a collection of computer resources.

The communication unit 31 includes a communication controller for transmitting and receiving data between the first display device 1 and the second display device 2 via the network. The communication unit 31 notifies the control unit 33 of data received from the first display device 1 or the second display device 2 via the network. Furthermore, the communication unit 31 transmits the data output from the control unit 33 to the first display device 1 or the second display device 2 via the network.

The storage unit 32 is a storage medium including a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk drive, and the like. The storage unit 32 stores in advance a program to be executed by the control unit 33. The storage unit 32 may be provided outside the information processing device 3, and in that case, data may be exchanged with the control unit 33 via a network.

The storage unit 32 stores a three-dimensional map for generating a virtual space, VR object information, information regarding the first user, and information regarding the second user. For example, in a three-dimensional map, position coordinates corresponding to position coordinates in real space are determined. VR object information is, for example, information about VR objects that indicate structures such as buildings and roads that exist in real space, and includes at least the attributes of VR objects (e.g., shape, color, etc.) and the attributes of VR objects in virtual space. including location. The information regarding the first user includes, for example, identification information (ID: identifier) for identifying the first user, the name of the first user, and the like. The information regarding the second user includes, for example, the second user's ID, the second user's name, and the like.

The storage unit 32 stores an event management database that manages events in which the first user and the second user act together. FIG. 4 is a diagram showing an example of the configuration of the event management database. In the example shown in FIG. 4, the event management database stores an event ID, a start date and time, a start location, a VR participant, and an AR participant in association with each other.

The start date and time is the date and time when the event starts. The start location is the location where the event is started. The VR participant is information indicating a first user who participates in an event using VR, and is indicated by, for example, the ID of the first user. The AR participant is information indicating one or more second users who participate in an event in AR, and is indicated by, for example, the ID of the second user.

Returning to FIG. 3, the control unit 33 includes an acquisition unit 331, a VR control unit 332, an AR control unit 333, and a transmission unit 334. The control unit 33 is, for example, a processor such as a CPU (Central Processing Unit), and by executing a program stored in the storage unit 32, functions as an acquisition unit 331, a VR control unit 332, an AR control unit 333, and a transmission unit 334. Function. The processing performed by each of the acquisition unit 331, VR control unit 332, AR control unit 333, and transmission unit 334 will be described later.

[VR control processing]
First, the VR control process executed by the information processing device 3 will be described. The VR control unit 332 controls the movement of the first user's avatar based on the control information. The control information is information for controlling the first user's avatar in the virtual space.

For example, when the first user performs an operation to move in the virtual space on the first display device, the first display device 1 transmits information indicating the first user's operation to the information processing device 3 as control information. Then, when the acquisition unit 331 acquires the control information from the first display device 1, the VR control unit 332 moves the first user's avatar in the direction in the virtual space indicated by the control information. control the behavior of

The control information may be information for the first user's avatar to move along a predetermined route in the virtual space. For example, the event management database further stores control information corresponding to the event in association with the event. In this case, in a certain event, when the start date and time of the event has passed, the VR control unit 332 uses the avatar of the first user based on the control information associated with the event ID corresponding to the event. control the behavior of

The VR control unit 332 executes a process of presenting a VR object on the first display device 1. Specifically, the VR control unit 332 causes the first display device 1 to display the structure object as a VR object. The structure object is a VR object that represents a structure (a structure existing in real space) in virtual space. More specifically, the VR control unit 332 displays the structure object at a position in the virtual space that corresponds to the position of the structure in the real space.

For example, first, based on the three-dimensional map and VR object information stored in the storage unit 32, the VR control unit 332 creates a virtual space in a predetermined range that corresponds to the position of the structure in the real space. A VR image is generated in which a virtual space including a structural object placed at a spatial position is displayed. The predetermined range is a range determined according to the position of the first user's avatar (for example, a range of up to 100 meters from the position of the first user's avatar), or a range determined according to the position of the first user's avatar, or a range where the first user and the second user act in the event providing service. This is a predetermined action range, etc. The transmitter 334 transmits the VR image generated by the VR controller 332 to the first display device 1. The first display device 1 then displays the acquired VR image.

The VR control unit 332 may cause the first display device 1 to generate a VR image. For example, first, the VR control unit 332 specifies, from among the three-dimensional map and VR object information stored in the storage unit 32, a portion of the three-dimensional map and a portion of the VR object information that are included in a predetermined range. The transmitter 334 transmits the three-dimensional map and VR object information specified by the VR controller 332 to the first display device 1. Then, the first display device 1 generates a VR image in which a virtual space including the structural object is displayed based on the acquired three-dimensional map and VR object information, and displays the generated VR image.

The VR control unit 332 may cause the first user VR object to be displayed on the first display device 1 as the VR object. The first user VR object is a VR object that represents the first user's avatar and is different from the structure object.

Specifically, the VR control unit 332 causes the first user VR object to be displayed on the first display device 1 based on the control information. More specifically, the VR control unit 332 displays the first user VR object at a position in the virtual space after controlling the motion of the first user's avatar based on the control information.

The VR control unit 332 may display the second user VR object on the first display device 1 as the VR object. The second user VR object is a VR object that represents the second user's avatar, and is different from the structure object and the first user VR object.

Specifically, first, the acquisition unit 331 functions as a position information acquisition unit and acquires the device position of the second display device 2 from the second display device 2 . The device position is information indicating the position of the second display device 2 in real space, and is information indicating the position measured based on information received by the GPS receiving section of the second display device 2, for example. The device position is not limited to this, and may be information indicating a position specified based on a captured image of the surroundings captured by the imaging unit of the second display device 2. In this case, the second display device 2 captures an image of the surrounding area. The position may be specified based on the image, or the information processing device 3 may specify the position based on the captured image. Then, the VR control unit 332 causes the first display device 1 to display the second user VR object at a position in the virtual space that corresponds to the device position acquired by the acquisition unit 331.

The VR control unit 332 causes the first display device 1 to display an information VR object as a VR object. The information VR object is a VR object different from the structure object, the first user VR object, and the second user VR object.

Specifically, the VR control unit 332 displays information regarding the position of the second user's avatar in the virtual space that corresponds to the device position of the second display device 2 in the real space on the first display device 1 as an information VR object. let The information regarding the position is, for example, information indicating the position of the second user's avatar. More specifically, the VR control unit 332 causes the first display device 1 to display map information indicating the position where the second user VR object exists as an information VR object.

FIG. 5 is a diagram schematically showing a VR image displayed on the first display device 1. In the example shown in FIG. 5, an image V1 that is a VR image displayed on the first display device 1 displays an object X1 that is map information. A mark L1 indicating the position of the first user's avatar and a mark L2 indicating the position of the second user's avatar are displayed on the object X1. By causing the information processing device 3 to display such an information VR object on the first display device 1, the first user can grasp where the second user VR object is located.

The information regarding the position may be information indicating the distance from the first user to the second user. Specifically, the VR control unit 332 determines the distance from the position of the first user VR object to the position of the second user VR object, or the distance from the position of the first user VR object in the virtual space to the device position. Information indicating the distance to the position is displayed on the first display device 1 as an information VR object.

For example, first, the VR control unit 332 identifies the distance from the position in the real space corresponding to the position of the first user's avatar in the virtual space to the device position of the second display device 2. The VR control unit 332 may specify the distance from the position of the first user VR object in the virtual space to the position of the second user VR object in the virtual space. Further, the VR control unit 332 moves from the first user's position in the first space (either virtual space or real space) to a second space different from the first space (the first space is virtual space). If the second space is a real space, the second space is a real space, and if the first space is a real space, the second space is a virtual interval. Good too. The transmitter 334 transmits information indicating the distance specified by the VR controller 332 to the first display device 1 as information regarding the position. The first display device 1 then displays the information indicating the distance as an information VR object.

FIG. 6 is a diagram schematically showing a VR image displayed on the first display device 1. In the example shown in FIG. 6, the image V2, which is a VR image displayed on the first display device 1, is displayed on the second display device 2 from a position in real space corresponding to the position of the first user's avatar in the virtual space. The distance to the device location is displayed. Note that a message indicating the distance from the position of the first user VR object to the position of the second user VR object may be displayed as the object X2 in the image V2. When the VR control unit 332 specifies the distance from the position in the real space corresponding to the position of the first user's avatar in the virtual space to the device position of the second display device 2, the image V2 includes an object X2 as the object X2. A message indicating that the distance from the second user is too far or a message indicating a warning may be displayed. By causing the information processing device 3 to display such an information VR object on the first display device 1, the first user can grasp how far away the first user is from the second user VR object.

The information regarding the position may be information indicating the direction of the second user VR object. Specifically, the VR control unit 332 causes the first display device 1 to display information indicating the direction of the second user VR object when the first user VR object is the starting point, as an information VR object.

For example, first, the VR control unit 332 identifies the direction of the second user VR object when the first user VR object is the starting point. The transmitter 334 transmits information indicating the direction specified by the VR controller 332 to the first display device 1 as information regarding the position. Then, the first display device 1 displays the information indicating the direction as an information VR object.

FIG. 7 is a diagram schematically showing a VR image displayed on the first display device 1. In the example shown in FIG. 7, the image V3 which is the VR image displayed on the first display device 1 includes an arrow object X3 indicating the direction of the second user VR object when the first user VR object is the starting point. is displayed. Note that an arrow indicating the position of the second user VR object with respect to the direction in which the first user VR object is facing may be displayed as the object X3 in the image V3. By causing the information processing device 3 to display such an information VR object on the first display device 1, the first user can grasp in which direction the second user VR object is located.

The VR control unit 332 includes, as information regarding the position, map information indicating the position where the second user VR object exists, information indicating the distance from the first user to the second user, and information indicating the direction of the second user VR object. map information indicating the position where the second user VR object exists, information indicating the distance from the first user to the second user, and the second user VR object. information indicating the direction may be displayed at the same time.

For example, the VR control unit 332 causes the information VR object to be displayed on the first display device 1 every time the acquisition unit 331 acquires the device position of the second display device 2 at a predetermined interval. The VR control unit 332 may display the information VR object on the first display device 1 when a predetermined condition is satisfied.

Specifically, the VR control unit 332 corresponds to the distance from the position of the first user VR object in the virtual space to the position of the second user VR object, or from the position of the first user VR object in the virtual space to the device position. The information VR object is displayed on the first display device 1 when the distance to the position in the virtual space becomes larger than the information display threshold distance. The information display threshold distance is a predetermined value. The information display threshold distance is determined depending on the location of the activity, such as a short distance is determined in urban areas and towns where there are many people and buildings, and a long distance is determined in suburban areas and riverbeds where there are few people and buildings. May be different.

For example, first, every time the acquisition unit 331 acquires the device position of the second display device 2 at a predetermined interval, the VR control unit 332 changes the position of the second user VR object from the position of the first user VR object in the virtual space. It is determined whether the distance to is greater than the information display threshold distance. Then, when the VR control unit 332 determines that the distance from the position of the first user VR object to the position of the second user VR object in the virtual space is greater than the information display threshold distance, the VR control unit 332 displays the information VR object on the first display device. Display it on 1. By causing the information processing device 3 to display the information VR object on the first display device 1 under such conditions, the first user can recognize that the first user is separated from the second user VR object.

The VR control unit 332 may display the information VR object on the first display device 1 when the second user VR object is blocked by a structure object. Specifically, the VR control unit 332 corresponds between the position of the first user VR object and the position of the second user VR object in the virtual space, or between the position of the first user VR object and the device position in the virtual space. The information VR object is displayed on the first display device in response to the presence of the structure object between the position in the virtual space and the position in the virtual space.

For example, first, each time the acquisition unit 331 acquires the device position of the second display device 2 at a predetermined interval, the VR control unit 332 controls the position of the first user VR object and the position of the second user VR object in the virtual space. Determine whether a structure object exists between the two. When the VR control unit 332 determines that a structure object exists between the position of the first user VR object and the position of the second user VR object in the virtual space, the VR control unit 332 transfers the information VR object to the first display device 1. to be displayed. By causing the information processing device 3 to display the information VR object on the first display device 1 under such conditions, the first user can recognize that the second user VR object is no longer visible.

For example, in a certain event, the VR control unit 332 executes a process of presenting a VR object on the first display device 1 when the start date and time of the event has passed. The VR control unit 332 may cause the first display device 1 to display an information VR object corresponding to the avatar of the second user who participated in the event.

Specifically, the VR control unit 332 causes the first display device 1 to display an information VR object corresponding to the avatar of the second user associated with the first user in advance. For example, the VR control unit 332 displays, on the first display device 1, an information VR object corresponding to the avatar of the AR participant (second user) associated with the VR participant (first user) in the event management database. let By doing so, the information processing device 3 can suppress the display of the information VR object corresponding to the avatar of the second user who is not participating in the event.

Note that the VR control unit 332 may display the avatar of the second user participating in the event and the avatar of the second user not participating in the event in different manners. The VR control unit 332 may, for example, attach a mark or flag to the second user VR object corresponding to the second user participating in the event, or add a conspicuous color to the second user VR object (for example, red, blue, etc.). ) and display it. On the other hand, the VR control unit 332 may, for example, not color the second user VR object corresponding to the second user who is not participating in the event (for example, make it semi-transparent), or make the second user VR object inconspicuous. Display with a color (for example, white).

[AR control processing]
Next, the AR control process executed by the information processing device 3 will be described. The AR control unit 333 executes a process of presenting the AR object on the second display device 2. Specifically, the AR control unit 333 presents the first user AR object as the AR object to the second display device 2 at the real avatar position. The first user AR object is an AR object that represents the first user's avatar. The real avatar position is a position in real space that corresponds to the position of the first user's avatar in virtual space.

For example, first, the AR control unit 333 generates an AR image in which the first user AR object placed at the real avatar position is displayed. The transmitter 334 transmits the AR image generated by the AR controller 333 to the second display device 2. Then, the second display device 2 displays a superimposed image in which the acquired AR image is superimposed on the captured image captured by the imaging unit of the second display device 2.

The AR control unit 333 may cause the second display device 2 to generate an AR image. For example, first, the transmitter 334 transmits AR object information regarding the first user AR object and the real avatar position to the second display device 2. The AR object information is, for example, information indicating the shape, color, etc. of the first user AR object. Then, the second display device 2 generates an AR image in which the first user AR object placed at the actual avatar position is displayed, based on the acquired AR object information and the actual avatar position, and displays the generated AR image. A superimposed image superimposed on the captured image captured by the imaging unit of the second display device 2 is displayed.

The AR control unit 333 converts the first user AR object into an actual object corresponding to the position of the first user's avatar after the VR control unit 332 controls the movement of the first user's avatar in the virtual space based on the control information. Present at the avatar position. Specifically, the AR control unit 333 presents the first user AR object at the real avatar position corresponding to the second existence position of the first user's avatar that has been moved from the first existence position of the first user's avatar. do. The first existing position is the existing position in the virtual space before the VR control unit 332 moves the first user's avatar based on the control information. The second existing position is the existing position in the virtual space after the VR control unit 332 moves the first user's avatar based on the control information.

The AR control unit 333 controls the operation of the first user AR object by moving the first user AR object in a direction in real space indicated by the control information, and controls the first user AR object based on the control information. It may be displayed at the actual avatar position after the actual avatar position. Specifically, the AR control unit 333 presents the first user AR object at a second real avatar position moved from the first real avatar position. The first real avatar position is the position in real space before the first user AR object is moved based on the control information. The second real avatar position is the position in real space after moving the first user AR object based on the control information.

When there are multiple second users, the AR control unit 333 may present the other second user AR object as the AR object to one of the multiple second users. Specifically, the AR control unit 333 transmits the AR object of the other second user to the second display device 2 of one of the plurality of second users at the device position of the second display device 2 of the other second user. to be displayed.

The AR control unit 333 further presents the information AR object to the second display device 2 as an AR object. The information AR object is an AR object different from the first user AR object and the second user AR object.

Specifically, the AR control unit 333 further displays information regarding the real avatar position of the first user's avatar on the second display device 2 as an information AR object. The information regarding the actual avatar position is, for example, information indicating the position of the first user's avatar. More specifically, the AR control unit 333 causes the second display device 2 to display map information indicating the location where the first user's avatar exists as an information AR object.

FIG. 8 is a diagram schematically showing a superimposed image displayed on the second display device 2. As shown in FIG. In the example shown in FIG. 8, the image O1 displayed on the second display device 2 is a superimposed image obtained by superimposing a captured image captured by the second display device 2 on an AR image. An object Y1, which is map information, is displayed in the image O1. A mark L3 indicating the position of the second user's avatar and a mark L4 indicating the position of the first user's avatar are displayed on the object Y1. By causing the information processing device 3 to display such an information AR object on the second display device 2, the second user can grasp where the first user AR object is located.

The information regarding the actual avatar position may be information indicating the distance from the first user to the second user. Specifically, the AR control unit 333 causes the second display device 2 to display information indicating the distance between the real avatar position and the device position as an information AR object.

For example, first, the AR control unit 333 identifies the distance from the real avatar position to the device position. The AR control unit 333 may specify the distance from the position of the first user VR object in the virtual space to the position of the second user VR object in the virtual space. Furthermore, the AR control unit 333 moves from the first user's position in the first space (either virtual space or real space) to a second space different from the first space (the first space is virtual space). If the second space is a real space, the second space is a real space, and if the first space is a real space, the second space is a virtual interval. Good too. The transmitter 334 transmits information indicating the distance specified by the AR controller 333 to the second display device 2 as information regarding the real avatar position. Then, the second display device 2 displays the information indicating the distance as an information VR object.

FIG. 9 is a diagram schematically showing a superimposed image displayed on the second display device 2. As shown in FIG. In the example shown in FIG. 9, the image O2 displayed on the second display device 2 is a superimposed image in which the captured image captured by the second display device 2 is superimposed on the AR image. In the image O2, an object Y2 of a message indicating the distance from the first user's real avatar position to the device position of the second display device 2 is displayed. Note that a message indicating the distance from the position of the first user VR object to the position of the second user VR object may be displayed as the object Y2 in the image O2. When the AR control unit 333 specifies the distance from the position in the real space corresponding to the position of the first user's avatar in the virtual space to the device position of the second display device 2, the image O2 includes an object Y2 as an object Y2. A message indicating that the distance from the first user is too far or a message indicating a warning may be displayed. By causing the information processing device 3 to display such an information AR object on the second display device 2, the second user can grasp how far away the second user is from the first user AR object.

The information regarding the real avatar position may be information indicating the direction of the first user AR object. Specifically, the AR control unit 333 displays information indicating the direction of the first user's real avatar position when the device position of the second display device 2 is the starting point, on the second display device 2 as an information AR object. let

For example, first, the AR control unit 333 identifies the direction of the first user's real avatar position with the device position of the second display device 2 as the starting point. The transmitter 334 transmits information indicating the direction specified by the AR controller 333 to the second display device 2 as information regarding the real avatar position. Then, the second display device 2 displays information indicating the direction as an information AR object.

FIG. 10 is a diagram schematically showing a superimposed image displayed on the second display device 2. As shown in FIG. In the example shown in FIG. 10, the image O3 displayed on the second display device 2 is a superimposed image obtained by superimposing the captured image captured by the second display device 2 on the AR image. In the image O3, an arrow object Y3 indicating the direction of the first user's real avatar position when the device position of the second display device 2 is the starting point is displayed. Note that an arrow indicating the position of the first user's real avatar with respect to the direction in which the imaging unit of the second display device 2 is facing may be displayed as the object Y3 in the image O3. By causing the information processing device 3 to display such an information AR object on the second display device 2, the second user can grasp in which direction the first user AR object is located.

The AR control unit 333 includes, as position-related information, map information indicating the position where the first user's avatar exists, information indicating the distance from the first user to the second user, and information indicating the direction of the first user AR object. map information indicating the location where the first user's avatar exists, information indicating the distance from the first user to the second user, and the first user AR object. information indicating the direction may be displayed at the same time.

The AR control unit 333 causes the information AR object to be displayed on the second display device 2, for example, every time the acquisition unit 331 acquires the device position of the second display device 2 at a predetermined interval. The AR control unit 333 may cause the AR control unit 333 to display the information AR object when a predetermined condition is satisfied.

Specifically, the AR control unit 333 displays the information AR object on the second display device when the distance between the first user's real avatar position and the device position of the second display device 2 becomes larger than the information display threshold distance. Display on 2.

For example, first, every time the acquisition unit 331 acquires the device position of the second display device 2 at a predetermined interval, the AR control unit 333 controls the distance between the real avatar position of the first user and the device position of the second display device 2. It is determined whether the distance is greater than the information display threshold distance. Then, when the AR control unit 333 determines that the distance from the first user's real avatar position to the device position of the second display device 2 is greater than the information display threshold distance, the AR control unit 333 displays the information AR object on the second display device 2. Display. By causing the information processing device 3 to display the information AR object on the second display device 2 under such conditions, the second user can recognize that the second user is separated from the first user AR object.

The AR control unit 333 may display the information AR object on the second display device 2 when the first user AR object is blocked by a structure. Specifically, the AR control unit 333 controls VR when a structure object exists between the position of the first user's avatar in the virtual space and the position in the virtual space corresponding to the device position of the second display device 2. In response to the determination by the control unit 332, the information AR object is displayed on the second display device 2.

For example, first, each time the acquisition unit 331 acquires the device position of the second display device 2 at a predetermined interval, the VR control unit 332 controls the position of the first user VR object and the position of the second user VR object in the virtual space. Determine whether a structure object exists between the two. Then, when the VR control unit 332 determines that a structure object exists between the position of the first user VR object and the position of the second user VR object in the virtual space, the AR control unit 333 controls the information AR object. is displayed on the second display device 2.

The AR control unit 333 causes the second display device 2 to display an information AR object, triggered by the presence of a structure between the first user's real avatar position and the device position of the second display device 2. It's okay. For example, the storage unit 32 stores information indicating a map in real space. In this case, first, the AR control unit 333 refers to the map in real space stored in the storage unit 32 every time the acquisition unit 331 acquires the device position of the second display device 2 at predetermined intervals, It is determined whether a structure exists between the first user's real avatar position and the device position of the second display device 2. Then, when it is determined that a structure exists between the first user's real avatar position and the device position of the second display device 2, the AR control unit 333 displays the information AR object on the second display device 2. Display. By causing the information processing device 3 to display the information VR object on the second display device 2 under such conditions, the second user can recognize that the first user AR object is no longer visible.

For example, in a certain event, the AR control unit 333 executes a process of presenting an AR object on the second display device 2 when the start date and time of the event has passed. The AR control unit 333 may cause the second display device 2 to display an information AR object corresponding to the avatar of the first user who participated in the event.

Specifically, the AR control unit 333 causes the information AR object to be displayed on the second display device 2 used by the second user who is associated with the first user in advance. For example, the AR control unit 333 stores the information AR object corresponding to the VR participant (first user) in the event management database on the second display device of the AR participant (second user) associated with the VR participant. Display on 2. By doing so, the information processing device 3 can suppress the display of the information AR object corresponding to the avatar of the first user who is not participating in the event in which the second user participates.

Note that the AR control unit 333 may display the avatar of the first user participating in the event and the avatar of the first user not participating in the event in different modes. The AR control unit 333 may, for example, mark or flag the first user AR object corresponding to the first user participating in the event, or add a color that stands out to the first user AR object (for example, red, blue, etc.). ) and display it. On the other hand, the AR control unit 333 may, for example, not color the first user AR object corresponding to the first user who is not participating in the event (for example, make it semi-transparent), or make the first user AR object inconspicuous. Display with a color (for example, white).

[Processing flow of information processing device 3]
Next, the flow of processing of the information processing device 3 will be explained. FIG. 11 is a flowchart showing the flow of the information processing device 3. This flowchart starts when the acquisition unit 331 acquires the device position from the second display device 2 (S1).

The VR control unit 332 specifies the distance from the position of the first user's avatar in the virtual space to the position of the second user's avatar corresponding to the device position of the second display device 2 (S2). The VR control unit 332 determines whether the specified distance is greater than the information display threshold distance (S3).

If the VR control unit 332 determines that the specified distance is not greater than the information display threshold distance (NO in S3), the VR control unit 332 displays a VR image that includes a structural object but does not include an information VR object. A certain first VR image is presented on the first display device 1 (S4). Further, if the VR control unit 332 determines that the specified distance is not greater than the information display threshold distance (NO in S3), the AR control unit 333 determines that the specified distance is not greater than the information display threshold distance (NO in S3), , the first AR image, which is an AR image that does not include the information AR object, is presented on the second display device 2 (S5).

On the other hand, if the VR control unit 332 determines that the specified distance is greater than the information display threshold distance (YES in S3), the VR control unit 332 displays a second VR image that is a VR image that includes a structure object and an information VR object. It is presented on the first display device 1 (S6). Further, when the VR control unit 332 determines that the specified distance is greater than the information display threshold distance (YES in S3), the AR control unit 333 controls the AR image including the first user AR object and the information AR object. A second AR image is presented on the second display device 2 (S7).

[Effects of this embodiment]
As explained above, the information processing device 3 presents information regarding the position of the second user's avatar to the first display device 1 as an information VR object, and presents the first user's real avatar to the second display device 2. Information regarding location is presented as an information AR object. By doing so, the information processing device 3 can make the first user and the second user know where each other is. Thereby, the information processing device 3 can reduce the occurrence of a situation in which the first user and the second user lose sight of each other. As a result, the information processing device 3 can make it easier for the first user and the second user to act together.

Furthermore, the present invention makes it possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, "Create a foundation for industry and technological innovation."

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes can be made within the scope of the gist. be. For example, all or part of the device can be functionally or physically distributed and integrated into arbitrary units. In addition, new embodiments created by arbitrary combinations of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiment resulting from the combination have the effects of the original embodiment.

1 First display device 2 Second display device 3 Information processing device 31 Communication section 32 Storage section 33 Control section 331 Acquisition section 332 VR control section 333 AR control section 334 Transmission section S Information processing system

Claims (13)

An AR (Augmented Reality) object representing the avatar whose motion is controlled based on control information for controlling the avatar of the first user in the virtual space that is associated with the real space is provided to the first user who is different from the first user. 2. An AR control unit that displays a real avatar position, which is a position in the real space corresponding to a position of the avatar in the virtual space, on a display device used by two users ;
a position information acquisition unit that acquires a device position indicating the position of the display device in the real space;
Equipped with
The AR control unit determines the real avatar position based on the relationship between the real avatar position and the device position, or the relationship between the avatar's existing position in the virtual space and a position in the virtual space corresponding to the device position. further presenting information to display on the display device as an information AR object that is a different object from the AR object;
Information processing device. further comprising a VR (Virtual Reality) control unit that controls the movement of the avatar by moving the avatar in a direction in the virtual space indicated by the control information,
The AR control unit presents the AR object at the real avatar position corresponding to the avatar's existing position after the VR control unit controls the movement of the avatar in the virtual space based on the control information. ,
The information processing device according to claim 1. The AR control unit controls the operation of the AR object by moving the AR object in a direction in the real space indicated by the control information, and controls the AR object after controlling it based on the control information. presenting the real avatar at the position;
The information processing device according to claim 1. A structure that is a VR object that indicates a structure existing in the real space for the first display device that is a first display device used by the first user and is different from a second display device that is the display device. further comprising a VR control unit that presents a physical object at a position in the virtual space corresponding to a position of the structure;
The information processing device according to claim 1. The AR control unit is configured to perform an operation when the VR control unit determines that the structure object exists between a position of the avatar in the virtual space and a position in the virtual space corresponding to the device position. , displaying the information AR object on the display device;
The information processing device according to claim 4. The AR control unit causes the information AR object to be displayed on the display device, triggered by the presence of the structure between the real avatar position and the device position.
The information processing device according to claim 5. The AR control unit causes the information AR object to be displayed on the display device when the distance between the real avatar position and the device position becomes larger than a predetermined information display threshold distance.
The information processing device according to claim 1. The AR control unit causes the display device to display map information indicating a position where the avatar exists as the information AR object.
The information processing device according to claim 1. The AR control unit causes the display device to display information indicating a distance between the real avatar position and the device position as the information AR object.
The information processing device according to claim 1. The AR control unit causes the display device to display information indicating the direction of the real avatar position when the position of the display device is a starting point, as the information AR object.
The information processing device according to claim 1. The AR control unit displays the information AR object on a display device used by a second user associated with the first user in advance.
The information processing device according to claim 1. computer executes
A second user different from the first user uses an AR object representing the avatar whose motion is controlled based on control information for controlling the avatar of the first user in a virtual space associated with the real space. causing a display device to present the real avatar at a position in the real space that corresponds to the position of the avatar in the virtual space ;
obtaining a device position indicating the position of the display device in the real space;
Equipped with
In the step of presenting the AR object, based on the relationship between the real avatar position and the device position, or the relationship between the avatar's existing position in the virtual space and a position in the virtual space corresponding to the device position, further causing the display device to present information for indicating the avatar position as an information AR object that is a different object from the AR object;
Information processing method. computer,
A second user different from the first user uses an AR object representing the avatar whose motion is controlled based on control information for controlling the avatar of the first user in a virtual space associated with the real space. an AR control unit that causes a display device to display a real avatar position that is a position in the real space corresponding to a position of the avatar in the virtual space ;
a position information acquisition unit that acquires a device position indicating the position of the display device in the real space;
function as
The AR control unit determines the real avatar position based on the relationship between the real avatar position and the device position, or the relationship between the avatar's existing position in the virtual space and a position in the virtual space corresponding to the device position. further causing the display device to present information to be displayed as an information AR object that is an object different from the AR object;
program.

Images