JP2024045446A - Program, information processing device, information processing system, and information processing method - Google Patents

Abstract

[Problem] To provide a program capable of improving a user's virtual experience. [Solution] A program executed by a computer to provide a virtual experience to user B includes the steps of: executing a first action in a virtual space 200B; executing a second action on another user A different from user B based on the execution of the first action; and executing a third action in the virtual space 200B based on the reaction to the second action. [Selected Figure] Figure 20

Description

The present disclosure relates to a program, an information processing device, an information processing system, and an information processing method.

As in Non-Patent Document 1, a service is known in which a plurality of users sharing a virtual space can enjoy communication through body language using virtual hands and the like.

“Social VR Demo - Selfie Stick and 360 Photo Spheres - Oculus”, [online], April 13, 2016, Oculus, [Retrieved August 10, 2017], Internet <https://youtu.be/-pumFtAjgLY?t=5>

However, in services such as the one described above, there is room for improvement in order to improve the virtual experience for users.

The present disclosure aims to improve the virtual experience of users.

According to one aspect of the program disclosed in the present disclosure,
A computer for providing virtual experiences to users,
a step of executing a first action in a virtual space;
executing a second action for another user different from the user based on the execution of the first action;
and executing a third action in the virtual space based on the reaction to the second action.

This disclosure can improve the user's virtual experience.

1 is a schematic diagram showing a virtual space distribution system according to an embodiment of the present invention (hereinafter simply referred to as this embodiment). FIG. 2 is a schematic diagram showing a user terminal; FIG. 2 is a diagram showing the head of a user wearing an HMD. FIG. 2 is a diagram illustrating a hardware configuration of a control device. 10 is a flowchart showing a process of displaying a visual field image on an HMD. FIG. 2 is an xyz space diagram showing an example of a virtual space. State (a) is a yx plan view of the virtual space shown in FIG. 6 . State (b) is a zx plan view of the virtual space shown in FIG. FIG. 2 is a diagram showing an example of a visual field image displayed on an HMD. FIG. 2 is a diagram illustrating a hardware configuration of the server illustrated in FIG. 1 . State (a) is a diagram showing a virtual space provided to user A. State (b) is a diagram showing a virtual space provided to user B. FIG. 3 is a sequence diagram for explaining an example of a process of synchronizing the movements of avatars between user terminals. 11 is an example of a flowchart for explaining the information processing method according to the present embodiment in a case where user A removes the HMD. State (a) is an example of a flowchart for explaining a process for determining whether or not user A has removed the HMD using an HMD sensor. State (b) is an example of a flowchart for explaining a process for determining whether or not user A has removed the HMD using a wearing sensor. A figure showing an example of a virtual space provided to user B to explain the information processing method according to this embodiment. A figure showing another example of a virtual space provided to user B to explain the information processing method related to this embodiment. It is an example of the flowchart for demonstrating the information processing method based on this embodiment when user A wears HMD again. A figure showing another example of a virtual space provided to user B to explain the information processing method related to this embodiment. 7 is an example of a flowchart for explaining an information processing method according to Modification 1. FIG. A figure showing an example of a virtual space 200B provided to user B to explain an information processing method related to variant example 1. A figure showing an example of a virtual space 200B provided to user B to explain an information processing method related to variant example 1. 7 is a diagram illustrating a display example of a user terminal 5 associated with user A for explaining an information processing method according to modification 1. FIG. A figure showing an example of a virtual space 200B provided to user B to explain an information processing method related to variant example 1. 7 is a diagram illustrating an example of a virtual space 200B provided to user B for explaining an information processing method according to modification 1. FIG. 11 is an example of a flowchart for explaining an information processing method according to Modification 1 in a case where a user A does not wear an HMD within a certain time period after receiving read information. 11 is an example of a flowchart for explaining an information processing method according to Modification 2. 12 is a diagram showing an example of a virtual space 200B provided to user B for explaining an information processing method according to modification 2. FIG. A figure showing an example of a virtual space 200B provided to user B to explain an information processing method related to variant example 2. 12 is a diagram showing an example of a virtual space 200B provided to user B for explaining an information processing method according to modification 2. FIG.

[Description of embodiments illustrated by the present disclosure]
An overview of embodiments disclosed in the present disclosure will be described.
(1) A first user terminal having a first head mount device mounted on the head of a first user, and a second user terminal having a second head mount device mounted on the head of a second user. An information processing method executed by a computer in a virtual space distribution system comprising:
The information processing method includes:
(a) generating virtual space data defining a virtual space including a first avatar associated with the first user and a second avatar associated with the second user;
(b) updating a visual field image displayed on the second head-mounted device based on the movement of the second head-mounted device and the virtual space data;
(d) If non-wearing information indicating that the first user is not wearing the first head mount device is received, the first user is not wearing the first head mount device. An information processing method comprising: presenting the information to the second user by a second user terminal.

According to the above method, when non-wearing information indicating that the first user is not wearing the first head mounted device (hereinafter referred to as the first HMD) is received, the second user terminal allows the first user to wear the first HMD. It is presented to the second user that the user is not wearing the second user. In this way, the second user can easily understand that the first user is not wearing the first HMD while communicating with the first user in the virtual space. Therefore, a rich virtual experience can be provided to the user.

In particular, if the second user is not aware that the first user is not wearing the first HMD, the second user may feel uncomfortable with the first user not reacting. In this way, by making the second user aware that the first user is not wearing the first HMD, it is possible to avoid a situation in which the second user feels uncomfortable with the first user not reacting.

(2) The step (d)
The information processing method according to item (1), further comprising a step of visualizing, when the non-wearing information is received, on a field of view image displayed on the second head-mounted device, that the first user is not wearing the first head-mounted device.

According to the above method, when non-wearing information indicating that the first user is not wearing the first head mounted device (first HMD) is received, it is possible to determine that the first user is not wearing the first HMD. The information shown is visualized on the visual field image displayed on the second head mounted device (second HMD). In this way, while communicating with the first user in the virtual space, the second user visually recognizes information indicating that the first user is not wearing the first HMD through the second HMD. It is possible to easily understand that one user is not wearing the first HMD.

(3) (e) further comprising the step of updating the facial expression of the first avatar based on the facial information when facial information indicating the facial expression of the first avatar is received;
The step (d) includes:
Item (1) or (2), including the step of visualizing a first avatar whose facial expression is set in the first mode on a visual field image displayed on the second head mounted device when the non-wearing information is received. The information processing method described in 2).

According to the above method, when non-wearing information indicating that the first user is not wearing the first head-mounted device (first HMD) is received, the first avatar, whose facial expression is set to a first state, is visualized on the field of view image displayed on the second head-mounted device (second HMD). In this way, while communicating with the first user in the virtual space, the second user can easily understand that the first user is not wearing the first HMD by visually confirming that the facial expression of the first avatar is a predetermined expression.

In particular, when the first user is not wearing the first HMD, the facial expression of the first avatar is not updated at all. Therefore, unless the second user knows that the first user is not wearing the first HMD, the second user may feel uncomfortable with the first user. In this way, by making the second user understand that the first user is not wearing the first HMD, it is possible to avoid a situation where the second user feels uncomfortable with the first user who does not show any reaction. It becomes possible.

(4) The information processing method described in item (3), in which the first aspect is a facial expression of the first avatar that is set as a default.

According to the above method, when the non-wearing information indicating that the first user is not wearing the first head mounted device (first HMD) is received, the facial expression of the first avatar is set as the default. 1 avatar's facial expression. In this way, when the second user is communicating with the first user in the virtual space, the second user can visually confirm that the first avatar's facial expression is the default facial expression. , it is possible to easily understand that the first user is not wearing the first HMD.

(5) The information processing method according to item (3), wherein the first aspect is a facial expression selected in advance as an aspect indicating that the first user is not wearing the first head mounted device.

According to the above method, when the non-wearing information indicating that the first user is not wearing the first head mounted device (first HMD) is received, the facial expression of the first avatar is A facial expression selected in advance by the first user (hereinafter referred to as a selected facial expression) is set as an aspect indicating that the first user is not wearing the first head-mounted device. In this way, when the second user is communicating with the first user in the virtual space, by visually recognizing that the facial expression of the first avatar is the selected expression, the second user can It is possible to easily recognize that the first HMD is not attached.

(6) The first user terminal further includes a wearing sensor configured to detect whether the first user is wearing the first head mounted device,
When the wearing sensor detects that the first user is not wearing the first head mounted device, the first user terminal outputs the non-wearing information, and the second user terminal outputs the non-wearing information. The information processing method according to any one of items (1) to (5), wherein the non-wearing information is received from the first user terminal.

According to the above method, it is determined that the first user is not wearing the first head-mounted device (first HMD) in accordance with information transmitted from the wearing sensor of the first user terminal. Then, non-wearing information indicating that the first user is not wearing the first HMD is transmitted. In this way, it is possible to automatically determine that the first user is not wearing the first HMD in accordance with information output from the wearing sensor of the first user terminal.

(7) (f) The information processing method according to any one of items (1) to (6), further comprising a step of, when receiving wearing information indicating that the first head mounted device has been removed from the first user and then worn again, presenting to the second user by the second user terminal that the first user is wearing the first head mounted device.

According to the above method, when attachment information indicating that the first head mounted device (first HMD) has been attached again after being removed from the first user is received, the second user terminal allows the first user to attach the first HMD to the first HMD. It is presented to the second user that the second user is wearing the. In this way, the second user can easily understand that the first user has worn the first HMD again. Therefore, a rich virtual experience can be provided to the user.

(8) A program for causing a computer to execute the information processing method described in any one of items (1) to (7).

According to the above program, a rich virtual experience can be provided to the user.

(9) A first user terminal having a first head mount device attached to the head of the first user, and a second user terminal having the second head mount device attached to the head of the second user. A virtual space distribution system configured to carry out the information processing method according to any one of items (1) to (7).

The above virtual space distribution system can provide users with a rich virtual experience.

(10) a processor;
and a memory storing computer readable instructions,
When the computer-readable instructions are executed by the processor, the device performs the information processing method described in any one of items (1) to (7).

According to the above device, a rich virtual experience can be provided to the user.

(11) A computer for providing a virtual experience to a user, comprising:
moving an operation object in a virtual space in response to a movement of a part of the user's body;
executing a first action by the operation object on a virtual object whose action in the virtual space is controllable based on a movement of another user different from the user;
and executing a second action on the other user based on the execution of the first action.

The above program can improve the virtual experience of the user. In particular, by performing actions between multiple users who share a virtual space through intuitive operations using control objects, communication in the virtual space can be carried out smoothly without compromising the sense of immersion.

(12) The program described in item (11), which executes the second action when the first action is executed in a state in which the virtual object is not being controlled by the other user.

According to the above program, by performing a second action on a virtual object that is not controlled, other users can be attracted into the virtual space.

(13) The program according to item (11) or (12), wherein the first action includes an action of contacting the virtual object during the repetitive motion of the control object.

(14) The program described in item (13), in which the first action includes a contact action with the virtual object during the repetitive motion with an acceleration equal to or greater than a certain value.

(15) The program according to item (13) or (14), in which the first action includes passing through relative coordinates associated with the virtual object a predetermined number of times or more within a certain period of time.

These programs can prevent erroneous detection of the motion of the control object by not taking into account the possibility that the control object may accidentally come into contact with a virtual object.

(16) The program according to any one of items (11) to (15), wherein the second action includes a notification urging the other user to participate in the virtual space.

According to the above program, it is possible to easily make other users who share the virtual space aware that, for example, multi-play has been approached, and it is possible to promote the use of multi-play.

(17) transitioning the virtual object from a first state to a second state when a reaction to the second action is performed;
Item (16) is a program for causing the computer to execute a step of transitioning the virtual object to a third state if the virtual object is controlled by the other user within a certain period of time after the transition to the second state.

The above program allows the user who performed the first action to easily understand that there has been a reaction from another user, or that the virtual object has been controlled by another user.

(18) If the virtual object is not controlled by the other user within a predetermined period of time after the transition to the second state, the step of returning the virtual object to the first state may further include the step of returning the virtual object to the first state. The program according to item (17), to be executed by a computer.

According to the above program, the user who executed the first action can easily understand that the virtual object is not controlled by another user despite executing the second action.

(19) Any one of items (16) to (18), wherein the content of the notification differs depending on at least one of the strength of the operation of the first action with respect to the virtual object and the type of the first action. The program described in.

According to the above method, other users can grasp the level of desire for multiplayer from the user who performed the first action, and can decide whether or not to participate in the virtual space depending on the level of desire.

(20) executing the first action with the operation object on another virtual object whose action in the virtual space is computer-controllable;
A program described in any one of items (11) to (19), for causing the computer to further execute the step of transitioning a game provided in the virtual space from a first scene to a second scene based on the execution of the first action.

According to the above program, the game can be played without impairing the user's sense of immersion in the virtual space.

(21) An information processing device for providing a user with a virtual experience, comprising:
Under the control of a processor included in the information processing device,
moving an operation object in a virtual space in response to a movement of a part of the user's body;
executing a first action by the operation object on a virtual object whose action in the virtual space is controllable based on a movement of another user different from the user;
and executing a second action on the other user based on the execution of the first action.

According to the above information processing device, it is possible to improve a user's virtual experience.

(22) An information processing system for providing a user with a virtual experience, comprising:
Under the control of a plurality of processors included in the information processing system,
moving an operation object in a virtual space in response to a movement of a part of the user's body;
executing a first action by the operation object on a virtual object whose action in the virtual space is controllable based on a movement of another user different from the user;
and executing a second action on the other user based on the execution of the first action.

The above information processing system can improve the user's virtual experience.

(23) An information processing method implemented by a computer for providing a virtual experience to a user, comprising:
moving an operation object in a virtual space in response to a movement of a part of the user's body;
executing a first action by the operation object on a virtual object whose action in the virtual space is controllable based on a movement of another user different from the user;
and executing a second action on the other user based on the execution of the first action.

The above information processing method can improve the user's virtual experience.

(24) A computer for providing a virtual experience to a user, comprising:
executing a first action in a virtual space;
executing a second action on a user different from the user based on the execution of the first action;
and executing a third action in the virtual space based on a reaction to the second action.

According to the above program, the user's virtual experience can be improved. In particular, by performing a third action in virtual space based on a reaction to a second action performed on another user, we provide a new virtual experience where the boundary between real space and virtual space is seamless. can do.

(25) The program described in item (24), in which the first action is an action for identifying the other user, and the third action is an action executed by a virtual object associated with the other user in the virtual space.

According to the above program, actions can be executed between a plurality of users sharing a virtual space, thereby allowing smooth communication in the virtual space.

(26) A step of moving an operation object in the virtual space in response to a movement of a part of the body of the user is further performed by the computer.
The program according to item (24) or (25), wherein the first action is executed using the operation object.

According to the above program, a plurality of users sharing a virtual space can perform actions through intuitive operations using operation objects, without impairing the user's sense of immersion.

(27) The program described in any one of items (24) to (26), wherein the second action includes a notification encouraging the other user to join the virtual space.

The above program makes it possible for other users sharing the virtual space to easily recognize that, for example, multiplayer play has been proposed, thereby promoting the use of multiplayer play.

(28) From item (24), the third action includes transitioning an action executed by a virtual object associated with the other user in the virtual space from a first state to a second state. The program according to any one of (27).

The above method allows the user who performed the first action to easily know whether or not there have been any reactions from other users.

(29) An information processing device for providing a user with a virtual experience, comprising:
Under the control of a processor included in the information processing device,
executing a first action in a virtual space;
executing a second action on a user different from the user based on the execution of the first action;
and executing a third action in the virtual space based on a reaction to the second action.

The information processing device described above can improve the user's virtual experience.

(30) An information processing system for providing a virtual experience to a user,
By controlling a plurality of processors included in the information processing system,
a step of executing a first action in a virtual space;
executing a second action for another user different from the user based on the execution of the first action;
An information processing system that executes a third action in the virtual space based on a reaction to the second action.

The above information processing system can improve the user's virtual experience.

(31) An information processing method implemented by a computer for providing a virtual experience to a user, comprising:
executing a first action in a virtual space;
executing a second action on a user different from the user based on the execution of the first action;
and executing a third action in the virtual space based on a reaction to the second action.

The above information processing method can improve the user's virtual experience.

[Details of the embodiment shown in this disclosure]
Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. For the sake of convenience, the description of elements having the same reference numbers as elements already described in the description of the present embodiment will not be repeated.

First, the configuration of the virtual space distribution system 100 will be outlined with reference to FIG. 1. FIG. 1 is a schematic diagram of a virtual space distribution system 100 (hereinafter simply referred to as distribution system 100). As shown in FIG. 1, the distribution system 100 includes a user terminal 1A (first user terminal) operated by a user A (first user) wearing a head mounted device (HMD) 110 (first HMD); It includes a user terminal 1B (second user terminal) operated by user B (second user) wearing a second HMD) and a server 2. The user terminals 1A and 1B are communicably connected to a server 2 via a communication network 3 such as the Internet. Note that in this embodiment, the virtual space includes a VR (Virtual Reality) space. Further, hereinafter, for convenience of explanation, each of the user terminals 1A and 1B may be simply referred to collectively as the user terminal 1. Furthermore, each of users A and B may be simply referred to collectively as user U. Further, in this embodiment, it is assumed that the user terminals 1A and 1B have the same configuration.

Next, the configuration of the user terminal 1 will be described with reference to FIG. 2. FIG. 2 is a schematic diagram showing the user terminal 1. As shown in FIG. As shown in FIG. 2, the user terminal 1 includes a head mounted device (HMD) 110 attached to the head of the user U, headphones 116, a face camera 117, a microphone 118, a position sensor 130, and an external controller. 320 and a control device 120.

The HMD 110 includes a display unit 112, an HMD sensor 114, a gaze sensor 140, a face camera 113, and a wearing sensor 115. The display unit 112 includes a non-transmissive display device configured to completely cover the field of view (visual field) of the user U wearing the HMD 110. This allows the user U to be immersed in the virtual space by only viewing the visual field image displayed on the display unit 112. The display unit 112 may be configured to include a left-eye display unit configured to provide an image to the left eye of the user U, and a right-eye display unit configured to provide an image to the right eye of the user U. The HMD 110 may also include a transmissive display device. In this case, the transmissive display device may be temporarily configured as a non-transmissive display device by adjusting its transmittance.

The HMD sensor 114 is mounted near the display section 112 of the HMD 110. The HMD sensor 114 includes at least one of a geomagnetic sensor, an acceleration sensor, and a tilt sensor (angular velocity sensor, gyro sensor, etc.), and detects various movements (tilt, etc.) of the HMD 110 attached to the head of the user U. be able to.

Gaze sensor 140 has an eye tracking function that detects user U's line of sight. The gaze sensor 140 may include, for example, a right eye gaze sensor and a left eye gaze sensor. The right eye gaze sensor obtains information regarding the rotation angle of the eyeball of the right eye by irradiating, for example, infrared light onto the right eye of the user U and detecting the reflected light reflected from the right eye (especially the cornea and iris). You may. On the other hand, the left eye gaze sensor irradiates the left eye of the user U with, for example, infrared light and detects the reflected light from the left eye (especially the cornea and iris), thereby obtaining information about the rotation angle of the eyeball of the left eye. may be obtained.

The face camera 113 is configured to acquire images (particularly, moving images) showing the eyes (left eye and right eye) and eyebrows (left eyebrow and right eyebrow) of the user U when the HMD 110 is worn by the user U. The face camera 113 is disposed at a predetermined location inside the HMD 110 so as to be able to capture an image of the eyes and eyebrows of the user U. The frame rate of the moving images acquired by the face camera 113 may be higher than the frame rate of a field of view image (described later) displayed on the HMD 110.

The wearing sensor 115 is configured to generate event information for indicating whether the user U is wearing the HMD 110. Specifically, when the user U removes the HMD 110 (when the state of the HMD 110 transitions from the wearing state to the non-wearing state), the wearing sensor 115 generates event information and then controls the event information. to the device 120. Furthermore, when the user U wears the HMD 110 (when the state of the HMD 110 transitions from the non-wearing state to the wearing state), the wearing sensor 115 generates event information and sends the event information to the control device 120. Send. Furthermore, the configuration of the mounting sensor 115 is not particularly limited. For example, assume that when user U wears HMD 110, the elastic force of a spring provided in HMD 110 changes. In this case, the mounting sensor 115 may generate event information based on the change in the elastic force of the spring. Furthermore, it is assumed that when user U wears HMD 110, the current flowing between two pads provided on HMD 110 that contact a part of the body (nose area) changes. In this case, the attachment sensor 115 may generate event information based on the change in the current flowing between the two pads.

Headphones 116 (audio output section) are worn in the left ear and right ear of user U, respectively. Headphones 116 are configured to receive audio data (electrical signals) from control device 120 and output audio based on the received audio data. The microphone 118 (voice input unit) is configured to collect the voice uttered by the user U and generate voice data (electrical signal) based on the collected voice. Additionally, microphone 118 is configured to transmit audio data to controller 120.

The face camera 117 is configured to acquire an image (particularly a moving image) in which the user U's mouth and its surroundings are displayed. The face camera 117 may be placed at a position facing the user U's mouth so that it can image the user's U mouth and its surroundings. Further, the face camera 117 may be connected to the HMD 110. The frame rate of the moving image acquired by the face camera 117 may be higher than the frame rate of the visual field image displayed on the HMD 110.

The position sensor 130 is, for example, a position tracking camera, and is configured to detect the positions of the HMD 110 and the external controller 320. The position sensor 130 is communicatively connected to the control device 120 wirelessly or by wire, and is configured to detect information about the positions, inclination, or emission intensity of a plurality of detection points (not shown) provided on the HMD 110. Furthermore, the position sensor 130 is configured to detect information about the positions, inclination, and/or emission intensity of a plurality of detection points (not shown) provided on the external controller 320. The detection points are, for example, light-emitting units that emit infrared rays or visible light. The position sensor 130 may also include an infrared sensor and a plurality of optical cameras.

The external controller 320 is used to control the movement of the hands of an avatar displayed in a virtual space by detecting the movement of a part of the user U's body (a part other than the head, which is the user U's hand in this embodiment). The external controller 320 has a right-hand external controller 320R (hereinafter simply referred to as controller 320R) operated by the user U's right hand, and a left-hand external controller 320L (hereinafter simply referred to as controller 320L) operated by the user U's left hand. The controller 320R is a device that indicates the position of the user U's right hand and the movement of the fingers of the right hand. The right hand of the avatar present in the virtual space moves in response to the movement of the controller 320R. The controller 320L is a device that indicates the position of the user U's left hand and the movement of the fingers of the left hand. The left hand of the avatar present in the virtual space moves in response to the movement of the controller 320L.

The control device 120 is a computer configured to control the HMD 110. The control device 120 identifies the position information of the HMD 110 based on the information acquired from the position sensor 130, and can accurately match the position of the virtual camera in the virtual space (the position of the avatar) with the position of the user U wearing the HMD 110 in the real space based on the identified position information. Furthermore, the control device 120 identifies the operation of the external controller 320 based on the information acquired from the position sensor 130 and/or the sensor built in the external controller 320, and can accurately match the hand operation of the avatar displayed in the virtual space with the operation of the external controller 320 in the real space based on the identified operation of the external controller 320. In particular, the control device 120 identifies the operation of the controller 320L based on the information acquired from the position sensor 130 and/or the sensor built in the controller 320L, and can accurately match the left hand operation of the avatar displayed in the virtual space with the operation of the controller 320L in the real space (the left hand operation of the user U) based on the identified operation of the controller 320L. Similarly, the control device 120 can identify the movement of the controller 320R based on information acquired from the position sensor and/or the sensor built into the controller 320R, and based on the identified movement of the controller 320R, accurately match the movement of the right hand of the avatar displayed in the virtual space with the movement of the controller 320R in the real space (the movement of the right hand of the user U).

The control unit 121 can also associate the facial expression (state) of the user U with the facial expression (state) of the user U's avatar based on the video images acquired by the face cameras 113 and 117. The process of associating the facial expression of the user U with the facial expression of the avatar will be described in detail later.

The control device 120 can also identify the gaze of the right eye and the left eye of the user U based on the information transmitted from the gaze sensors 140 (left eye gaze sensor and right eye gaze sensor), and identify the gaze point which is the intersection of the gaze of the right eye and the gaze of the left eye. Furthermore, the control device 120 can identify the gaze of both eyes of the user U (user U's gaze) based on the identified gaze point. Here, the gaze of user U is the gaze of both eyes of user U, and coincides with the direction of the straight line passing through the midpoint of the line segment connecting user U's right eye and left eye and the gaze point.

Next, with reference to FIG. 3, a method for acquiring information regarding the position and tilt of the HMD 110 will be described. FIG. 3 is a diagram showing the head of user U wearing HMD 110. Information regarding the position and inclination of the HMD 110 in conjunction with the movement of the head of the user U wearing the HMD 110 can be detected by the position sensor 130 and/or the HMD sensor 114 mounted on the HMD 110. As shown in FIG. 2, three-dimensional coordinates (uvw coordinates) are defined around the head of the user U wearing the HMD 110. The vertical direction in which the user U stands upright is defined as the v-axis, the direction perpendicular to the v-axis and passing through the center of the HMD 110 is defined as the w-axis, and the direction orthogonal to the v-axis and the w-axis is defined as the u-axis. The position sensor 130 and/or the HMD sensor 114 are configured to measure angles around each uvw axis (i.e., yaw angle indicating rotation around the v axis, pitch angle indicating rotation around the u axis, and angle around the w axis). The tilt determined by the roll angle indicating rotation) is detected. The control device 120 determines angle information for controlling the visual axis of the virtual camera based on the detected angle changes around each uvw axis.

Next, the hardware configuration of the control device 120 will be described with reference to FIG. 4. FIG. 4 is a diagram showing the hardware configuration of the control device 120. As shown in FIG. 4, the control device 120 includes a control section 121, a storage section 123, an I/O (input/output) interface 124, a communication interface 125, and a bus 126. The control unit 121, the storage unit 123, the I/O interface 124, and the communication interface 125 are communicably connected to each other via a bus 126.

The control device 120 may be configured as a personal computer, smartphone, phablet, tablet, or wearable device separate from the HMD 110, or may be built into the HMD 110. In addition, some of the functions of the control device 120 may be installed in the HMD 110, and the remaining functions of the control device 120 may be installed in another device separate from the HMD 110.

The control unit 121 includes a memory and a processor. The memory is, for example, a ROM (Read Only Memory) in which various programs are stored, and a RAM (Random Access Memory) having multiple work areas in which various programs executed by the processor are stored. The processor is, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and/or a GPU (Graphics Processing Unit), and is configured to load a specified program from the various programs built into the ROM onto the RAM and execute various processes in cooperation with the RAM.

In particular, the control unit 121 may control various operations of the control device 120 by having the processor develop a control program on the RAM and execute the control program in cooperation with the RAM. The control unit 121 displays the visual field image on the display unit 112 of the HMD 110 based on the visual field image data. This allows the user U to immerse himself in the virtual space.

The memory unit (storage) 123 is, for example, a storage device such as a hard disk drive (HDD), a solid state drive (SSD), or a USB flash memory, and is configured to store programs and various data. The memory unit 123 may store a control program for causing a computer to execute at least a part of the information processing method according to this embodiment, or a control program for realizing sharing of a virtual space by multiple users. The memory unit 123 may also store an authentication program for user U and data related to various images and objects (e.g., avatars, etc.). Furthermore, a database including tables for managing various data may be constructed in the memory unit 123.

The I/O interface 124 is configured to communicably connect the HMD 110, the position sensor 130, the external controller 320, the headphones 116, the face camera 117, and the microphone 118 to the control device 120, respectively. For example, it is configured with a USB (Universal Serial Bus) terminal, a DVI (Digital Visual Interface) terminal, an HDMI (registered trademark) (High-Definition Multimedia Interface) terminal, and the like. Note that the control device 120 may be wirelessly connected to each of the HMD 110, the position sensor 130, the external controller 320, the headphones 116, the face camera 117, and the microphone 118.

The communication interface 125 is configured to connect the control device 120 to a communication network 3 such as a LAN (Local Area Network), a WAN (Wide Area Network), or the Internet. The communication interface 125 includes various wired connection terminals for communicating with external devices such as a server 2 via the communication network 3, and various processing circuits for wireless connections, and is configured to conform to a communication standard for communicating via the communication network 3.

Next, a process for displaying a visual field image on the HMD 110 will be described with reference to FIGS. 5 to 8. FIG. 5 is a flowchart showing the process of displaying the visual field image on the HMD 110. FIG. 6 is an xyz space diagram showing an example of the virtual space 200. State (a) in FIG. 7 is a yx plan view of the virtual space 200 shown in FIG. 6. State (b) in FIG. 7 is a zx plan view of the virtual space 200 shown in FIG. FIG. 8 is a diagram showing an example of the visual field image V displayed on the HMD 110.

As shown in FIG. 5, in step S1, the control unit 121 (see FIG. 4) generates virtual space data indicating a virtual space 200 including a virtual camera 300 and various objects. As shown in FIG. 6, the virtual space 200 is defined as a complete celestial sphere centered on a center position 210 (in FIG. 6, only the upper half of the celestial sphere is illustrated). Furthermore, in the virtual space 200, an xyz coordinate system is set with the center position 210 as the origin. The virtual camera 300 defines a visual axis L for specifying a visual field image V (see FIG. 8) displayed on the HMD 110. The uvw coordinate system that defines the field of view of the virtual camera 300 is determined to be linked to the uvw coordinate system defined around the head of the user U in real space. Further, the control unit 121 may move the virtual camera 300 within the virtual space 200 in conjunction with the movement of the user U wearing the HMD 110 in the real space.

Next, in step S2, the control unit 121 determines the field of view CV (see FIG. 7) of the virtual camera 300. Specifically, the control unit 121 acquires information about the position and inclination of the HMD 110 based on data indicating the state of the HMD 110 transmitted from the position sensor 130 and/or the HMD sensor 114. Next, the control unit 121 determines the position and orientation of the virtual camera 300 in the virtual space 200 based on the information about the position and inclination of the HMD 110. Next, the control unit 121 determines the visual axis L of the virtual camera 300 from the position and orientation of the virtual camera 300, and determines the field of view CV of the virtual camera 300 from the determined visual axis L. Here, the field of view CV of the virtual camera 300 corresponds to a portion of the virtual space 200 visible to the user U wearing the HMD 110 (in other words, corresponds to a portion of the virtual space 200 displayed on the HMD 110). The visual field CV has a first region CVa set as an angular range of a polar angle α centered on the visual axis L in the xy plane shown in state (a) of FIG. 7, and a second region CVb set as an angular range of an azimuth angle β centered on the visual axis L in the xz plane shown in state (b) of FIG. 7. The control unit 121 may specify the line of sight of the user U based on data indicating the line of sight of the user U transmitted from the gaze sensor 140, and may determine the direction of the virtual camera 300 (the visual axis L of the virtual camera) based on the specified line of sight of the user U and information regarding the position and inclination of the HMD 110. As will be described later, the control unit 121 may also determine the facial direction of the user U's avatar based on information regarding the position and inclination of the HMD 110.

In this way, the control unit 121 can specify the field of view CV of the virtual camera 300 based on data from the position sensor 130 and/or the HMD sensor 114. Here, when the user U wearing the HMD 110 moves, the control unit 121 updates the visual field CV of the virtual camera 300 based on data indicating the movement of the HMD 110 transmitted from the position sensor 130 and/or the HMD sensor 114. be able to. That is, the control unit 121 can update the visual field CV according to the movement of the HMD 110. Similarly, when the line of sight of the user U changes, the control unit 121 may update the field of view CV of the virtual camera 300 based on data indicating the line of sight of the user U transmitted from the gaze sensor 140. That is, the control unit 121 may change the visual field CV according to a change in the user's U line of sight.

Next, in step S3, the control unit 121 generates visual field image data indicating the visual field image V displayed on the display unit 112 of the HMD 110. Specifically, the control unit 121 generates visual field image data based on virtual space data that defines the virtual space 200 and the visual field CV of the virtual camera 300.

Next, in step S4, the control unit 121 displays the visual field image V on the display unit 112 of the HMD 110 based on the visual field image data (see FIG. 7). In this way, the visual field CV of the virtual camera 300 changes according to the movement of the user U wearing the HMD 110, and the visual field image V displayed on the display unit 112 of the HMD 110 changes, so that the user U can see the virtual space. You can immerse yourself in 200.

The virtual camera 300 may include a left-eye virtual camera and a right-eye virtual camera. In this case, the control unit 121 generates left-eye field of view image data showing a field of view image for the left eye based on the virtual space data and the field of view of the left-eye virtual camera. Furthermore, the control unit 121 generates right-eye field of view image data showing a field of view image for the right eye based on the virtual space data and the field of view of the right-eye virtual camera. After that, the control unit 121 displays a left-eye field of view image on the left-eye display unit based on the left-eye field of view image data, and displays a right-eye field of view image on the right-eye display unit based on the right-eye field of view image data. In this way, the user U can visually recognize the field of view image three-dimensionally due to the parallax between the left-eye field of view image and the right-eye field of view image. The virtual cameras may be arranged at the position of the eyes of an avatar operated by the user, as described later. For example, the left-eye virtual camera may be arranged at the left eye of the avatar, while the right-eye virtual camera may be arranged at the right eye of the avatar.

Furthermore, the processing in steps S1 to S4 shown in FIG. 5 may be executed for each frame (still image forming a moving image). For example, if the frame rate of the moving image is 90 fps, the processes of steps S1 to S4 may be repeatedly executed at intervals of ΔT=1/90 (seconds). In this way, since the processes of steps S1 to S4 are repeatedly executed at predetermined intervals, the field of view of the virtual camera 300 is updated according to the operation of the HMD 110, and the field of view image V displayed on the display unit 112 of the HMD 110 is updated. is updated.

Next, the hardware configuration of the server 2 shown in FIG. 1 will be described with reference to FIG. 9. FIG. 9 is a diagram showing the hardware configuration of the server 2. As shown in FIG. As shown in FIG. 9, the server 2 includes a control section 23, a storage section 22, a communication interface 21, and a bus 24. The control unit 23, the storage unit 22, and the communication interface 21 are connected to each other via a bus 24 so as to be able to communicate with each other. The control unit 23 includes a memory and a processor, where the memory includes, for example, a ROM and a RAM, and the processor includes, for example, a CPU, an MPU, and/or a GPU.

The storage unit 22 is, for example, a large-capacity HDD. The storage unit 22 may store a control program for causing a computer to execute at least a part of the information processing method according to the present embodiment, and a control program for realizing sharing of a virtual space by a plurality of users. The storage unit 22 may also store user management information for managing each user and data regarding various images and objects (for example, avatars, etc.). Communication interface 21 is configured to connect server 2 to communication network 3 .

Next, an example of a process for synchronizing the movements of the avatars 4A and 4B between the user terminal 1A and the user terminal 1B will be described with reference to Figs. 1, 10, and 11. State (a) of Fig. 10 is a diagram showing a virtual space 200A provided to the user A. State (b) of Fig. 10 is a diagram showing a virtual space 200B provided to the user B. Fig. 11 is a sequence diagram for describing an example of a process for synchronizing the movements of the avatars 4A and 4B between the user terminal 1A and the user terminal 1B. In this description, as a prerequisite, as shown in Fig. 10, the avatar 4A (first avatar) associated with the user terminal 1A (user A) and the avatar 4B (second avatar) associated with the user terminal 1B (user B) share the same virtual space. In other words, it is assumed that the user A and the user B share one virtual space via the communication network 3.

As shown in state (a) of FIG. 10, user A's virtual space 200A includes an avatar 4A and an avatar 4B. The avatar 4A is operated by the user A and is linked to the user A's movements. The avatar 4A has a left hand (a part of the virtual body of the avatar 4A) that is linked to the action of the controller 320L of the user terminal 1A (the action of the user A's left hand), and a left hand that is linked to the action of the controller 320R of the user terminal 1A (the action of the user A's right hand). User A has a right hand whose expression is linked to user A's facial expressions. The face of the avatar 4A has multiple facial parts (eg, eyes, eyebrows, mouth, etc.). The avatar 4B is operated by the user B and is linked to the user B's movements. The avatar 4B has a left hand that is linked to the operation of the controller 320L of the user terminal 1B that shows the action of the left hand of the user B, a right hand that is linked to the action of the controller 320R of the user terminal 1B that shows the action of the right hand of the user B, and a facial expression. It has a face that is linked to user B's facial expression. The face of avatar 4B has a plurality of facial parts (eg, eyes, eyebrows, mouth, etc.).

It is also possible that the avatar 4A is not visualized in the virtual space 200A provided to the user A. In this case, the avatar 4A placed in the virtual space 200A includes at least a virtual camera 300 that is linked to the movement of the HMD 110 of the user terminal 1A.

Furthermore, the positions of the avatars 4A and 4B may be specified according to the positions of the HMDs 110 of the user terminals 1A and 1B. Similarly, the face orientations of the avatars 4A, 4B may be specified according to the inclinations of the HMDs 110 of the user terminals 1A, 1B. Furthermore, the hand motions of the avatars 4A, 4B may be specified according to the motions of the external controllers 320 of the user terminals 1A, 1B. In particular, the actions of the left hands of the avatars 4A, 4B are specified according to the actions of the controllers 320L of the user terminals 1A, 1B, and the actions of the right hands of the avatars 4A, 4B are determined according to the actions of the controllers 320R of the user terminals 1A, 1B. may be specified. Further, the facial expressions of the avatars 4A and 4B may be specified according to the facial expressions (states) of the users A and B.

Further, a virtual camera 300 (FIG. 6) may be placed in each eye of the avatars 4A and 4B. In particular, a left-eye virtual camera may be placed on the left eye of the avatars 4A, 4B, and a right-eye virtual camera may be placed on the right eye of the avatars 4A, 4B. In the following description, it is assumed that the virtual camera 300 is placed at the eyes of the avatars 4A and 4B. Therefore, the visual field CV of the avatar 4A is assumed to match the visual field CV of the virtual camera 300 placed on the avatar 4A (see FIG. 7). Similarly, it is assumed that the visual field CV of the avatar 4B matches the visual field CV of the virtual camera 300 placed on the avatar 4B (see FIG. 7).

As shown in state (b) of FIG. 10, user B's virtual space 200B includes an avatar 4A and an avatar 4B. The positions of avatars 4A and 4B in virtual space 200A may correspond to the positions of avatars 4A and 4B in virtual space 200B.

It is also possible that the avatar 4B is not visualized in the virtual space 200B provided to user B. In this case, the avatar 4B placed in the virtual space 200B includes at least a virtual camera 300 that is linked to the movement of the HMD 110 of the user terminal 1B.

Next, referring to FIG. 11, in step S10, the control unit 121 of the user terminal 1A generates user A's voice data. For example, when user A inputs voice into the microphone 118 (voice input unit) of the user terminal 1A, the microphone 118 generates voice data indicating the input voice. Thereafter, the microphone 118 transmits the generated audio data to the control unit 121 via the I/O interface 124.

Next, in step S11, the control unit 121 of the user terminal 1A generates control information for the avatar 4A, and then combines the generated control information for the avatar 4A with audio data (user A's audio data) indicating the user A's voice. ) is sent to server 2. After that, the control unit 23 of the server 2 receives the control information of the avatar 4A and the voice data of the user A from the user terminal 1A (step S12).

Here, the control information for the avatar 4A is information necessary for controlling the actions of the avatar 4A. The control information for the avatar 4A includes information regarding the position of the avatar 4A (position information), information regarding the direction of the face of the avatar 4A (face orientation information), and information regarding the state of the hands (left hand and right hand) of the avatar 4A (hand information). ) and information (face information) regarding the facial expression of the avatar 4A.

The face information of the avatar 4A includes information indicating the states of a plurality of facial parts. Information indicating the states of a plurality of facial parts includes information (eye information) indicating the state of the eyes of the avatar 4A (the shape of the whites of the eyes, the position of the iris relative to the whites, etc.), and information about the eyebrows of the avatar 4A (position and shape of the eyebrows). etc.) (eyebrow information), and information (mouth information) indicating the state of the mouth of the avatar 4A (mouth position, shape, etc.).

In detail, the eye information of avatar 4A includes information indicating the state of avatar 4A's left eye and information indicating the state of avatar 4A's right eye. The eyebrow information of avatar 4A includes information indicating the state of avatar 4A's left eyebrow and information indicating the state of avatar 4A's right eyebrow.

The control unit 121 of the user terminal 1A acquires an image showing the eyes (left eye and right eye) and eyebrows (left eyebrow and right eyebrow) of the user A from the face camera 113 installed in the HMD 110, and then displays the acquired image. The state of user A's eyes (left eye and right eye) and eyebrows (left eyebrow and right eyebrow) is identified based on a predetermined image processing algorithm. Next, the control unit 121 generates information indicating the state of the eyes (left eye and right eye) of the avatar 4A and information indicating the state of the eyes (left eyelash and right eyelash) of the avatar 4A, based on the state of the eyes and eyebrows of the specified user A. Generate information indicating each state.

Similarly, the control unit 121 of the user terminal 1A acquires an image showing the mouth and surrounding area of the user A from the face camera 117, and identifies the state of the user A's mouth based on the acquired image and a predetermined image processing algorithm. Next, the control unit 121 generates information indicating the state of the mouth of the avatar 4A based on the identified state of the user A's mouth.

In this way, the control unit 121 of the user terminal 1A receives the eye information of the avatar 4A corresponding to the eyes of the user A, the eyebrow information of the avatar 4A corresponding to the eyebrows of the user A, and the eye information of the avatar 4A corresponding to the mouth of the user A. It becomes possible to generate mouth information.

In addition, the control unit 121 selects a plurality of types of faces stored in the storage unit 123 or memory based on the image captured by the face camera 113, the image captured by the face camera 117, and a predetermined image processing algorithm. User A's facial expression (for example, a smile) may be identified from among the expressions (for example, a smile, a sad expression, a neutral expression, an angry expression, a surprised expression, a troubled expression, etc.). Thereafter, the control unit 121 can generate facial information indicating the facial expression (for example, a smile) of the avatar 4A based on the specified facial expression of the user A. In this case, facial expression data including a plurality of types of facial expressions of the avatar 4A and facial information of the plurality of avatars 4A, each of which is associated with one of the plurality of types of facial expressions, is stored in the storage unit 123. It's okay.

For example, when the control unit 121 identifies the facial expression of the avatar 4A as a smile, it acquires face information indicating the smile of the avatar 4A based on the facial expression (smile) and expression data of the avatar 4A. Here, the face information indicating the smile of the avatar 4A includes eye information, eyebrow information, and mouth information of the avatar 4A when the facial expression of the avatar 4A is a smile.

Next, in step S13, the control unit 121 of the user terminal 1B generates control information for the avatar 4B, and then transmits the generated control information for the avatar 4B to the server 2. After that, the control unit 23 of the server 2 receives control information for the avatar 4B from the user terminal 1B (step S14).

Here, the control information of avatar 4B is information necessary for controlling the movement of avatar 4B. The control information of avatar 4B may include information about the position of avatar 4B (position information), information about the direction of the face of avatar 4B (face direction information), information about the state of the hands (left hand and right hand) of avatar 4B (hand information), and information about the facial expression of avatar 4B (face information).

The face information of the avatar 4B includes information indicating the states of a plurality of facial parts. Information indicating the states of a plurality of facial parts includes information (eye information) indicating the state of the eyes of the avatar 4B (shape of the whites of the eyes, position of the iris relative to the whites, etc.), and information (eye information) of the eyebrows of the avatar 4B (position and shape of the eyebrows). etc.) (eyebrow information), and information (mouth information) indicating the state of the mouth of the avatar 4B (mouth position, shape, etc.). The method of acquiring face information of avatar 4B is the same as the method of acquiring face information of avatar 4A.

Next, the server 2 transmits control information for avatar 4B to user terminal 1A (step S15), while transmitting control information for avatar 4A and user A's voice data to user terminal 1B (step S19). After that, after receiving the control information for avatar 4B in step S16, the control unit 121 of user terminal 1A updates the states of avatars 4A and 4B based on the control information for avatars 4A and 4B, and then updates the virtual space data indicating virtual space 200A (see state (a) in FIG. 10) (step S17).

Specifically, the control unit 121 of the user terminal 1A updates the positions of the avatars 4A, 4B based on the position information of the avatars 4A, 4B. The control unit 121 updates the face orientations of the avatars 4A and 4B based on the face orientation information of the avatars 4A and 4B. The control unit 121 updates the hands of the avatars 4A and 4B based on the hand information of the avatars 4A and 4B. The control unit 121 updates the facial expressions of the avatars 4A and 4B based on the facial information of the avatars 4A and 4B. In this way, the virtual space data indicating the virtual space 200A including the updated avatars 4A and 4B is updated.

Thereafter, the control unit 121 of the user terminal 1A specifies the visual field CV of the avatar 4A (virtual camera 300) according to the position and inclination of the HMD 110, and then uses the updated virtual space data and the visual field CV of the avatar 4A. Based on this, the visual field image displayed on the HMD 110 is updated (step S18).

Meanwhile, after receiving the control information of avatar 4A and the voice data of user A in step S20, the control unit 121 of user terminal 1B updates the states of avatars 4A and 4B based on the control information of avatars 4A and 4B, and then updates the virtual space data showing virtual space 200B (see state (b) in Figure 10) (step S21).

In detail, the control unit 121 of the user terminal 1B updates the positions of the avatars 4A and 4B based on the position information of the avatars 4A and 4B. The control unit 121 updates the facial orientation of the avatars 4A and 4B based on the facial orientation information of the avatars 4A and 4B. The control unit 121 updates the hands of the avatars 4A and 4B based on the hand information of the avatars 4A and 4B. The control unit 121 updates the facial expressions of the avatars 4A and 4B based on the facial information of the avatars 4A and 4B. In this way, the virtual space data indicating the virtual space including the updated avatars 4A and 4B is updated.

Then, the control unit 121 of the user terminal 1B determines the field of view CV of the avatar 4B (virtual camera 300) according to the position and inclination of the HMD 110, and updates the field of view image displayed on the HMD 110 based on the updated virtual space data and the field of view CV of the avatar 4B (step S22).

Thereafter, the control unit 121 of the user terminal 1B uses the received voice data of the user A, the information regarding the position of the avatar 4A included in the control information of the avatar 4A, the information regarding the position of the avatar 4B, and a predetermined voice processing algorithm. User A's voice data is processed based on this. After that, the control unit 121 transmits the processed audio data to the headphones 116 (audio output unit), and the headphones 116 output the user A's audio based on the processed audio data (step S23). In this way, voice chat (VR chat) between users (between avatars) can be realized in the virtual space.

In this embodiment, after the user terminals 1A and 1B transmit the control information of the avatar 4A and the control information of the avatar 4B to the server 2, the server 2 transmits the control information of the avatar 4A to the user terminal 1B, while the avatar 4B control information is transmitted to the user terminal 1A. In this way, it becomes possible to synchronize the movements of each avatar 4A, 4B between the user terminal 1A and the user terminal 1B.

Next, the information processing method according to this embodiment when the HMD 110 of user A is removed will be described mainly with reference to Figs. 12 and 14. Fig. 12 is an example of a flowchart for explaining the information processing method according to this embodiment when the HMD 110 is removed by user A. Fig. 14 is a diagram showing an example of a virtual space 200B provided to user B for explaining the information processing method according to this embodiment. Note that, for convenience of explanation, the process in which the control unit 121 of user terminal 1A transmits control information of avatar 4A to the server 2 and the process in which the control unit 121 of user terminal 1B transmits control information of avatar 4B to the server 2 are omitted in the flowchart shown in Fig. 12.

In this embodiment, as a precondition, it is assumed that avatar 4A and avatar 4B share the same virtual space, as shown in FIG. In other words, it is assumed that users A and B share one virtual space via the communication network 3.

The virtual space 200B of user B includes avatar 4A and avatar 4B. The control unit 121 of the user terminal 1B updates the virtual space data indicating the virtual space 200B. For ease of explanation, in this description, the illustration of the virtual space 200A provided to avatar 4A is omitted.

Referring to FIG. 12, in step S30, the control unit 121 of the user terminal 1A determines whether or not the user A has removed the HMD 110. In this regard, the control unit 121 may determine whether or not the user A has removed the HMD 110 based on the tilt information of the HMD 110 transmitted from the HMD sensor 114 or the event information transmitted from the wearing sensor 115. The HMD 114 and the wearing sensor 115 function as sensors configured to detect whether or not the HMD 110 is being worn.

With reference to state (a) in FIG. 13, a process for determining whether user A has removed HMD 110 using HMD sensor 114 will be described. State (a) in FIG. 13 is an example of a flowchart for explaining a process for determining whether user A has removed HMD 110 using HMD sensor 114. As shown in state (a) of FIG. 13, the control unit 121 receives tilt information indicating the tilt (roll angle, yaw angle, pitch angle) of the HMD 110 from the HMD sensor 114 (step S301). Next, the control unit 121 determines whether the tilt of the HMD 110 is larger than a predetermined tilt based on the received tilt information (S302). Here, the predetermined inclination may be appropriately set by the user. It is assumed that the inclination of the HMD 110 does not become larger than a predetermined inclination while the user A is wearing the HMD 110. In other words, while the user A is wearing the HMD 110, the inclination of the HMD 110 is less than or equal to the predetermined inclination, but when the user A removes the HMD 110, the HMD 110 is tilted significantly, so that the inclination of the HMD 110 does not exceed the predetermined inclination. It may be exceeded. From this point of view, a predetermined slope is set.

For example, when the control unit 121 determines that the pitch angle θ of the HMD 110 is greater than a predetermined angle θth as an example of the tilt of the HMD 110 (YES in step S302), it determines that the user A has removed the HMD 110 (in other words, the user A is not wearing the HMD 110) and executes the processing of step S31. On the other hand, when the control unit 121 determines that the pitch angle θ of the HMD 110 is equal to or less than the predetermined angle θth (NO in step S302), it determines that the user A is wearing the HMD 110 and waits until it receives tilt information of the HMD 110 from the HMD sensor 114.

Next, referring to state (b) of FIG. 13, a process of determining whether user A has removed the HMD 110 using the wearing sensor 115 will be described. State (b) of FIG. 13 is an example of a flowchart for describing a process for determining whether user A has removed the HMD 110 using the wearing sensor 115. As shown in state (b) of FIG. 13, the control unit 121 determines whether event information has been received from the wearing sensor 115 (step S303). Here, the event information transmitted from the wearing sensor 115 is information indicating that the state of the HMD 110 of the user terminal 1A has changed between a wearing state and a non-wearing state. For example, when user A removes the HMD 110 (when the state of the HMD 110 transitions from the wearing state to the non-wearing state), the wearing sensor 115 outputs the event information. Also, when user A wears the HMD 110 (when the state of the HMD 110 transitions from the non-wearing state to the wearing state), the wearing sensor 115 outputs the event information.

When the control unit 121 determines that event information has been received from the wearing sensor 115 (YES in step S303), it determines that user A has removed the HMD 110 (in other words, user A is not wearing the HMD 110) and then executes the processing of step S31. On the other hand, when the control unit 121 determines that event information has not been received from the wearing sensor 115 (NO in step S303), it determines that user A is wearing the HMD 110 and waits until event information is received from the wearing sensor 115.

Returning to FIG. 12, if the control unit 121 determines that the user A has removed the HMD 110 (in other words, the user A is not wearing the HMD 110) (step S30 YES), the control unit 121 generates information indicating that the user A is not wearing the HMD 110 (non-wearing information) (step S31). The control unit 121 then transmits the non-wearing information to the server 2 via the communication network 3 (step S32).

Next, after receiving the non-wearing information from the user terminal 1A, the server 2 transmits the non-wearing information to the user terminal 1B via the communication network 3 (step S33). After that, the control unit 121 of the user terminal 1B sets the facial expression of the avatar 4A placed in the virtual space 200B to a default facial expression (in other words, a facial expression set as a default) after receiving the non-wearing information from the server 2 (step S34). Here, the default facial expression of the avatar 4A refers to the facial expression of the avatar 4A in the initial state before starting to update the facial expression of the avatar 4A. If the default facial expression is a smile, the facial expression of the avatar 4A is set to a smile, as shown in FIG. 14. In addition, the control unit 121 updates the movement of the avatar 4B (including the facial expression of the avatar 4B) based on the control information of the avatar 4B.

In addition, in step S34, the control unit 121 may set the facial expression of the avatar 4A to an expression selected in advance by the user A, as a mode indicating that the user A is not wearing the HMD 110. In this case, information regarding the facial expression of the avatar 4A selected by the user A may be transmitted from the user terminal 1A to the user terminal 1B, and then stored in advance in the memory unit 123 of the user terminal 1B.

Next, as shown in FIG. 14, the control unit 121 generates a speech bubble object 42A associated with the avatar 4A in the virtual space 200B. The balloon object 42A displays information indicating that the user A is not wearing the HMD 110 (for example, "User A is away.").

In addition, the control unit 121 may generate a caption object 43A in the virtual space 200B instead of the speech bubble object 42A (see FIG. 15). As shown in FIG. 15, the caption object 43A displays information indicating that the user A is not wearing the HMD 110 (for example, "User A is away"). The caption object 43A may be placed near the avatar 4A, or may be linked to the movement of the field of view CV of the avatar 4B so that the caption object 43A is always placed within the field of view CV of the avatar 4B. Furthermore, a caption indicating information indicating that the user A is not wearing the HMD 110 may be superimposed on the field of view image displayed on the HMD 110.

After that, the control unit 121 updates the virtual space data indicating the virtual space 200B, and also updates the visual field CV of the avatar 4B according to the movement of the HMD 110 of the user terminal 1B. Next, the control unit 121 updates the visual field image data based on the updated virtual space data and the updated visual field CV of the avatar 4B, and displays the visual field image on the HMD 110 based on the updated visual field image data. (Step S36).

On the other hand, after transmitting the non-wearing information to the server 2 via the communication network 3, the control unit 121 of the user terminal 1A shifts from the active mode to the sleep mode (step S37). In this case, it is preferable that the mode of the control unit 121 is the sleep mode, while the mode of the sensor (the mounted sensor 115 or the HMD sensor 114) is the active mode.

According to this embodiment, when user A is not wearing the HMD 110, the fact that user A is not wearing the HMD 110 is presented to user B by user terminal 1B. In particular, according to this embodiment, when user A is not wearing the HMD 110, information indicating that user A is not wearing the HMD 110 is visualized on the field of view image displayed on the HMD 110 of user terminal 1B. In this regard, avatar 4A, whose facial expression is set to a default facial expression, and speech bubble object 42A are visualized on the field of view image displayed on the HMD 110 of user terminal 1B.

In this way, when user B is communicating with user A on virtual space 200B, by visually recognizing the default facial expression of avatar 4A (for example, a smile) and the information displayed in balloon object 42A, It is possible to easily understand that user A is not wearing HMD 110. Therefore, a rich virtual experience can be provided to the user.

In particular, if user B is not aware that user A is not wearing the HMD 110, the user may feel uncomfortable with user A not showing any reaction. In this way, by making user B aware that user A is not wearing the HMD 100, it is possible to avoid a situation in which user B feels uncomfortable with user A not showing any reaction (e.g., the uncanny valley phenomenon).

Further, according to the present embodiment, it is determined that the user A is not wearing the HMD 110 according to the information (tilt information or event information) transmitted from the sensor (HMD sensor 114 or wearing sensor 115) of the user terminal 1A. be identified. Thereafter, non-wearing information indicating that user A is not wearing the HMD 110 is transmitted to the server 2. In this way, it becomes possible to automatically specify that user A is not wearing HMD 110, according to the information output from the sensor of user terminal 1A.

Note that in this embodiment, the user terminal 1A transmits non-wearing information to the user terminal 1B via the server 2, but instead of the non-wearing information, the user terminal 1A transmits tilt information transmitted from the HMD sensor 114 to the user terminal 1B. It may also be sent to 1B. In this case, the control unit 121 of the user terminal 1B specifies that the user A is not wearing the HMD 110 based on the received tilt information, and then executes the processes specified in steps S34 to S36. good.

Next, mainly with reference to Figs. 16 and 17, an information processing method according to this embodiment in a case where user A who has removed the HMD 110 puts the HMD 110 back on will be described. Fig. 16 is an example of a flowchart for explaining the information processing method according to this embodiment in a case where user A puts the HMD 110 back on. Fig. 17 is a diagram showing an example of a virtual space 200B provided to user B for explaining the information processing method according to this embodiment.

Referring to FIG. 16, in step S40, the control unit 121 of the user terminal 1A determines whether or not the user A has put on the HMD 110 again. In this regard, the control unit 121 may determine whether or not the user A has put on the HMD 110 again based on the tilt information of the HMD 110 transmitted from the HMD sensor 114 or the event information transmitted from the wearing sensor 115. For example, when the control unit 121 determines that the event information has been received from the wearing sensor 115, it determines that the user A has put on the HMD 110, and then executes the processing of step S41. On the other hand, when the control unit 121 determines that the event information has not been received from the wearing sensor 115, it determines that the user A has not put on the HMD 110, and then waits until the event information is received from the wearing sensor 115.

If the determination result in step S40 is YES, the control unit 121 transitions from sleep mode to active mode (step S41). Next, the control unit 121 that has transitioned to active mode generates control information for the avatar 4A and generates wearing information indicating that the user A has worn the HMD 110 (step S42). After that, the control unit 121 transmits the control information and wearing information for the avatar 4A to the server 2 via the communication network 3 (step S43).

Next, after receiving the control information and wearing information of avatar 4A from user terminal 1A, server 2 transmits the control information and wearing information of avatar 4A to user terminal 1B via communication network 3 (step S44). Thereafter, control unit 121 of user terminal 1B updates the movement of avatar 4A based on the control information of avatar 4A, and updates the movement of avatar 4B based on the control information of avatar 4B (step S45). In particular, control unit 121 updates the facial expression of avatar 4A based on the facial information of avatar 4A included in the control information of avatar 4A, and updates the facial expression of avatar 4B based on the facial information of avatar 4B included in the control information of avatar 4B.

Next, as shown in FIG. 17, the control unit 121 generates a speech balloon object 44A associated with the avatar 4A in the virtual space 200B (step S46). The balloon object 44A displays information indicating that the user A is wearing the HMD 110 (for example, "User A has returned to his seat.").

After that, the control unit 121 updates the virtual space data indicating the virtual space 200B, and updates the field of view CV of the avatar 4B in accordance with the movement of the HMD 110 of the user terminal 1B. Next, the control unit 121 updates the field of view image data based on the updated virtual space data and the updated field of view CV of the avatar 4B, and displays the field of view image on the HMD 110 based on the updated field of view image data (step S47).

According to this embodiment, when user A removes the HMD 110 and then puts it back on, the user terminal 1B presents to user B that user A is wearing the HMD 110. In this way, user B can easily understand that user A has put the HMD 110 back on by visually checking the updated facial expression of the avatar 4A and the speech bubble object 44A. Therefore, a rich virtual experience can be provided to the user.

In the description of this embodiment, information indicating whether user A is wearing or not wearing the HMD 110 is visualized on the field of view image, but this embodiment is not limited to this. For example, audio guidance indicating that user A is not wearing the HMD 110 (e.g., "User A has left the room") or audio guidance indicating that user A is wearing the HMD 110 (e.g., "User A has returned to his/her seat") may be output from the headphones 116 (audio output unit) of the user terminal 1B. In this case, the control unit 121 of the user terminal 1A transmits audio guidance data together with the non-wearing information (or wearing information) to the user terminal 1B via the server 2.

In addition, in the description of this embodiment, it is assumed that the virtual space data indicating the virtual space 200B is updated on the user terminal 1B side, but the virtual space data may be updated on the server 2 side. Furthermore, it is assumed that the visual field image data corresponding to the visual field image is updated on the user terminal 1B side, but the visual field image data may be updated on the server 2 side. In this case, the user terminal 1B displays the visual field image on the HMD 110 based on the visual field image data transmitted from the server 2.

Furthermore, the order of processing defined by each step shown in Figures 12 and 16 is merely an example, and the order of these steps can be changed as appropriate.

(Modification 1)
Next, with reference to FIGS. 18 to 25, a first modification of the information processing method according to the present embodiment in the case where user A's HMD 110 is removed will be described. FIG. 18 is an example of a flowchart for explaining the information processing method according to the first modification. 19, 20, 22, and 23 are diagrams illustrating an example of a virtual space 200B provided to user B for explaining the information processing method according to the first modification. FIG. 21 is a diagram showing a display example of the user terminal 5 associated with user A for explaining the information processing method according to the first modification. FIG. 24 is an example of a flowchart for explaining an information processing method according to modification example 1 when user A does not wear the HMD within a certain period of time after receiving read information.

Referring to FIG. 18, the control unit 121 of the user terminal 1A determines whether the user A has removed the HMD 110 (step S50). The process of determining whether the user A has removed the HMD 110 using the HMD sensor 114 is as described in the above embodiment, so a detailed description thereof will be omitted.

If it is determined in step S50 that user A has removed the HMD 110 (in other words, user A is not wearing the HMD 110) (YES in step S50), the control unit 121 generates information indicating that user A is not wearing the HMD 110 (non-wearing information) (step S51). After that, the control unit 121 transmits the non-wearing information to the server 2 via the communication network 3 (step S52). Note that in step S51, the control unit 121 may generate positional relationship information indicating the positional relationship between the HMD 110 and the external controller 320, and transmit the positional relationship information together with the non-wearing information to the server 2.

Next, after receiving the non-wearing information from the user terminal 1A, the server 2 transmits the non-wearing information to the user terminal 1B via the communication network 3 (step S53).

On the other hand, the control unit 121 of the user terminal 1A, after transmitting the non-wearing information to the server 2 via the communication network 3, transitions from the active mode to the sleep mode (step S54). In this case, it is preferable that the mode of the control unit 121 is the sleep mode, while the mode of the sensor (wearing sensor 115 or HMD sensor 114) is the active mode.

Next, the control unit 121 of the user terminal 1B sets the posture of the avatar 4A placed in the virtual space 200B to a default posture (in other words, a posture set as a default) based on the non-wearing information received from the server 2 (step S55). In the first modified example, the virtual space is assumed to be a room in which a player searches for a partner for multiplayer play, but the present invention is not limited to this. Here, the default posture of the avatar 4A refers to the posture of the avatar 4A in its initial state. If the default posture is lying down, the posture of the avatar 4A is set to lying down on the floor, as shown in FIG. 19.

In addition, in step S55, the control unit 121 may set the posture of the avatar 4A to a posture selected in advance by the user A, as a mode indicating that the user A is not wearing the HMD 110. In this case, information regarding the posture of the avatar 4A selected by the user A may be transmitted from the user terminal 1A to the user terminal 1B, and then stored in advance in the memory unit 123 of the user terminal 1B.

In addition, if user 1B has received positional relationship information indicating the positional relationship between HMD 110 and external controller 320, in step S55, control unit 121 may set the posture of avatar 4A based on the positional relationship information. That is, when user A is not wearing HMD 110, HMD 110 and external controller 320 are often placed on the floor or desk in the real space. In this case, avatar 4A in virtual space 200B may be set to a posture as if it is collapsed on the floor, based on the positional relationship information between HMD 110 and external controller 320.

Next, the control unit 121 of the user terminal 1B sets the facial expression of the avatar 4A placed in the virtual space 200B to the default facial expression (in other words, the default facial expression) (step S56 ). When the default facial expression is the closed eyes state, the facial expression of the avatar 4A is set to the closed eyes state, as shown in FIG. 19.

In addition, in step S56, the control unit 121 may set the facial expression of the avatar 4A to an expression selected in advance by the user A, as a mode indicating that the user A is not wearing the HMD 110. In this case, information regarding the facial expression of the avatar 4A selected by the user A may be transmitted from the user terminal 1A to the user terminal 1B, and then stored in advance in the memory unit 123 of the user terminal 1B.

Next, the control unit 121 updates the virtual space data indicating the virtual space 200B, and updates the field of view CV of the avatar 4B in accordance with the movement of the HMD 110 of the user terminal 1B. Next, the control unit 121 updates the field of view image data based on the updated virtual space data and the updated field of view CV of the avatar 4B, and displays the field of view image on the HMD 110 based on the updated field of view image data (step S57).

In this way, since the posture and face of the avatar 4A are displayed in the default setting state (lying down with eyes closed) in the virtual space 200B, the user B can understand that the user A is wearing the HMD 110. Therefore, it can be easily recognized that the user is not participating in the content provided to the virtual space 200B.

Next, after updating the visual field image in step S56, the control unit 121 of the user terminal 1B causes the avatar 4B associated with user B to take a specific action ( It is determined whether the first action (an example of the first action) has been executed (step S58).

The specific action that the avatar 4B performs on the avatar 4A includes, for example, an action that involves physical interference between the avatars 4A and 4B. Physical interference between the avatars 4A and 4B is detected using a collider (collision object) for detecting collisions with other objects. Although not shown, the collider is associated with the bodies of the avatars 4A and 4B, and is used for collision determination (hit determination) between the avatars 4A and 4B. For example, when the virtual hand 400 of the avatar 4B for which a collider has been set comes into contact with the avatar 4A for which a collider has been set, it is determined that the avatar 4B and the avatar 4A have come into contact. The collider may be set in the entire range of the bodies of the avatars 4A, 4B, or may be set in specific parts of the bodies of the avatars 4A, 4B (for example, hands, feet, heads, shoulders, hips, etc.). In particular, in the virtual hand 400, a collider is set at each fingertip, and it is preferable that collision determination is also independently detected for each finger. Thereby, it is possible to accurately determine the collision between another object and the finger. Note that the collider is configured as a transparent body and is not displayed within the visual field image V (FIG. 8).

In the above example, the control unit 121 specifies the collision determination of the avatar 4B with respect to the avatar 4A based on the movement of the avatar 4B (for example, the movement of the virtual hand 400), but the invention is not limited to this example. For example, the control unit 121 may specify the hit determination for the avatar 4A based on the movement of the user B (for example, the movement of the user B's hand).

Specific actions that involve physical interference include, for example, an action in which the avatar 4B touches the avatar 4A using the virtual hand 400, an action in which the avatar 4B grasps and shakes the body of the avatar 4A, and an action in which the avatar 4A is hit (for example, This may include actions such as slapping the face or shoulder of the avatar 4A. The specific action may be an action that includes a repetitive movement, that is, an action that contacts the avatar 4A during the repetitive movement of the virtual hand 400. In this way, even if the avatar 4B accidentally touches the avatar 4A due to an erroneous operation by the user B, it can be detected that the action is not a specific action, and erroneous detection of the motion of the virtual hand 400 can be prevented. Can be done.

For example, as shown in Fig. 20, when avatar 4B grasps the body (e.g., the torso) of avatar 4A with virtual hand 400 and the virtual hand 400 makes a repetitive movement, the control unit 121 determines that a shaking action has been performed as the specific action. The grasping (selection) of the body of avatar 4A by the virtual hand 400 can be realized, for example, by user B operating (pressing) a key (not shown) on the external controller 320 with the collider of the virtual hand 400 in contact with the collider of the body of avatar 4A. This shaking action may include a contact action with avatar 4A during the repetitive movement of the virtual hand 400 with an acceleration equal to or greater than a certain value. For example, in the case of the state shown in FIG. 20, when the virtual hand 400 is moved back and forth in the z-axis direction while grasping the body of the avatar 4A, if the acceleration in the +z-axis direction is equal to or greater than a certain value and the acceleration in the -z-axis direction is equal to or greater than a certain value for a predetermined number of times in succession, this may be regarded as a shaking motion. However, the direction in which the virtual hand 400 is moved back and forth is not limited to the z-axis direction, and may be any direction. In addition, the shaking motion may include a motion in which the virtual hand 400 moves back and forth through one or more relative coordinates associated with the avatar 4A a predetermined number of times or more within a certain period of time. For example, in the case of the state shown in FIG. 20, the one or more relative coordinates may include a first relative coordinate set at an arbitrary position in the +z-axis direction and a second relative coordinate set at an arbitrary position in the -z-axis direction based on the part of the body of the avatar 4A grasped by the virtual hand 400. In this example, when the virtual hand 400 grasps the body of the avatar 4A and moves back and forth in the z-axis direction, the first relative coordinate is moved back and forth a predetermined number of times within a certain time, and the second relative coordinate is moved back and forth a predetermined number of times within a certain time, which may be considered a shaking motion. However, the direction in which the virtual hand 400 is moved back and forth is not limited to the z-axis direction, and may be any direction. In this case, the relative coordinate for the virtual hand 400 may also be set in any direction.

Note that it is preferable to determine that a specific action has been performed for an action that user B himself recognizes as shaking. In other words, since the direction of shaking is not uniform and is assumed to vary depending on the user, the moving direction of virtual hand 400 may be any direction and does not need to be limited to a specific direction. However, if it is necessary to strictly determine the movement of a specific action or to give reality to the movement of the avatar 4B (for example, to prevent the virtual hand 400 from sinking into the avatar 4A), the control unit 121 may also take into account the moving direction of the virtual hand 400 and determine that a shaking motion has occurred if a repetitive motion is made in a specific direction.

On the other hand, if the virtual hand 400 makes contact with the avatar 4A at a constant acceleration or higher (collider collision) while the avatar 4B is not grasping the body of the avatar 4A with the virtual hand 400, the control unit 121 performs a hitting action as a specific action. Identify that an action was performed. In order to prevent erroneous detection of the motion of the virtual hand 400, the tapping motion is particularly performed by repeatedly moving the virtual hand 400 at a constant acceleration or higher, and also by touching the avatar 4A multiple times (collider collision) in the process of the repetitive motion. ) (specifically, a reciprocating slap operation). For example, in the case of the state shown in FIG. 19, the virtual hand 400 is reciprocated in the y-axis direction, and the acceleration in the +y-axis direction is a certain value or more, and the acceleration in the -y-axis direction is more than a certain value. , when the collider of the virtual hand 400 and the collider of the face of the avatar 4A come into contact a predetermined number of times or more, a tapping motion may be performed. However, the direction in which the virtual hand 400 is reciprocated is not limited to the y-axis direction, and may be any direction. This reciprocating slapping motion includes, for example, the virtual hand 400 of the avatar 4B reciprocating the relative coordinates associated with the avatar 4A a predetermined number of times or more within a predetermined time, and the action of touching the avatar 4A multiple times within a predetermined time ( The identification may be made based on the occurrence of a hit determination. For example, in the case of the state shown in FIG. 19, the one or more relative coordinates are a first relative coordinate provided at an arbitrary position in the +y-axis direction and an arbitrary position in the -y-axis direction with respect to the face of the avatar 4A. A second relative coordinate provided at the position is mentioned. In this example, the virtual hand 400 is reciprocated in the y-axis direction, reciprocating the first relative coordinate a predetermined number of times or more within a certain time, and reciprocating the second relative coordinate a predetermined number of times or more within a certain time. In this state, if the collider of the virtual hand 400 and the collider of the face of the avatar 4A contact each other a predetermined number of times or more, a hitting motion may be performed. However, the direction in which the virtual hand 400 is reciprocated is not limited to the y-axis direction, and may be any direction. In this case, relative coordinates with respect to the virtual hand 400 may also be provided in any direction. However, the control unit 121 may take into consideration the moving direction of the virtual hand 400 and determine that a hitting motion has been made when a repetitive motion is made in a specific direction.

Note that the avatar 4A may be set to follow the movement of the avatar 4B. For example, as shown in FIG. 20, when the avatar 4B performs a shaking motion on the avatar 4A, the control unit 121 controls the movement of the avatar 4A so that the avatar 4A follows the shaking motion. Can be set. However, when the avatar 4B performs a shaking motion or a hitting motion on the avatar 4A, the avatar 4A may be set not to follow the motion, that is, not to move.

Next, when the control unit 121 determines that the avatar 4B has performed a specific action on the avatar 4A (YES in step S58), the control unit 121 generates information indicating that the specific action has been performed by the user B (action execution information) (step S60). After that, the control unit 121 transmits the action execution information to the server 2 via the communication network 3 (step S61).

Next, based on the action execution information received from the user terminal 1B, the server 2 provides information regarding a participation consultation notification (participation consultation notification information) for inviting the user A to participate in the content provided in the virtual space 200. is generated (step S62). The participation consultation notification is a notification for asking user A, who has removed the HMD 110, to participate in the content provided in the virtual space 200. The participation consultation notification information includes, for example, the identification information of the avatar 4B that executed the specific action, the identification information of the avatar 4A that received the specific action, the type of the specific action, the strength of the movement of the specific action, etc. Good too.

Next, the server 2 identifies the personal information of the user A based on the generated participation inquiry notification information (step S63). For example, the personal information of the user A (e.g., an email address or an SNS (Social Networking Service) address, etc.) is preregistered in an information table stored in the storage unit 22 of the server 2 and managed in association with the identification information of the avatar 4A. In step S63, the server 2 refers to the information table stored in the storage unit 22 and acquires the personal information of the user A based on the identification information of the avatar 4A included in the participation inquiry notification information. However, the personal information of the user A may be managed by a server different from the server 2. Examples of the server different from the server 2 include, but are not limited to, a server capable of distributing a game program to a user whose personal information is registered, that is, a server that serves as a platform for game distribution.

Next, the server 2 transmits the participation inquiry notification information to the destination (e.g., the email address of user A) acquired in step S63 (step S64). The participation inquiry notification information of user A can be transmitted, for example, by communication tools such as email, chat, and SNS.

Next, the user terminal 5 receives the participation inquiry notification information through the communication tool, and displays a notification of the receipt of the participation inquiry notification information on the display unit 6 (step S65). The user terminal 5 is a terminal device owned by user A, and may be, for example, a portable terminal such as a smartphone, a PDA (Personal Digital Assistant), a tablet or phablet computer, or a wearable device, or may be a personal computer, a video game device, or the like.

Next, the user terminal 5 activates the communication tool based on the input operation by the user A who saw the reception notification, and displays the message M1 on the display unit 6 as shown in FIG. 21 (step S66). The message M1 includes information indicating that user B has asked user A to participate in the virtual space 200 (e.g., "I have just received [Content name (e.g., name of content provided in the virtual space 200)"). ] includes messages such as "You are receiving communication from [opposite player name (for example, user B)]. Let's start [content] and go see him!"). Note that if the user terminal 5 is a personal computer or a video game device, and the video of the content provided in the virtual space 200 is also displayed on the display of these devices, a message regarding the participation consultation notification will be displayed on the display. may be displayed. Moreover, the message M1 may be a message urging the user A to attach the HMD 110 of the user terminal 1A.

Next, the user terminal 5 determines that user A has read message M1 based on the display of message M1 on the display unit 6, and generates read information indicating that message M1 has been read (step S67). The read information is generated based on user A's reaction to message M1, and may be generated based on, for example, an email reply or the awarding of "Like!" points. The user terminal 5 then transmits the read information to the server 2 via the communication network (step S68).

Next, after receiving the read information from the user terminal 5, the server 2 transmits the read information to the user terminal 1B via the communication network 3 (step S69).

Next, the control unit 121 of the user terminal 1B sets the facial expression of the avatar 4A placed in the virtual space 200B to the updated facial expression based on the read information received from the server 2 (step S70). For example, as shown in FIG. 22, the control unit 121 updates the facial expression of the avatar 4A, which had eyes closed (an example of a first state), to eyes open (an example of a second state).

In addition, in step S70, the control unit 121 may update the posture of the avatar 4A based on the read information received from the server 2. For example, the posture of the avatar 4A that is lying down may be updated to a sitting position.

Next, the control unit 121 updates the virtual space data indicating the virtual space 200B, and also updates the visual field CV of the avatar 4B according to the movement of the HMD 110 of the user terminal 1B. Next, the control unit 121 updates the visual field image data based on the updated virtual space data and the updated visual field CV of the avatar 4B, and displays the visual field image on the HMD 110 based on the updated visual field image data. (Step S71). Thereby, the user B can easily understand that the user A has reacted to the participation consultation notification by visually recognizing that the avatar 4A has opened its eyes.

On the other hand, in step S72, the server 2 sends read information to the user terminal 1A in addition to the user terminal 1B.

Next, the control unit 121 of the user terminal 1A determines whether the user A has worn the HMD 110 within a certain period of time after receiving the read information (step S73). The control unit 121 may determine whether the user A has worn the HMD 110 based on the tilt information of the HMD 110 transmitted from the HMD sensor 114 or the event information transmitted from the wearing sensor 115. The determination as to whether the user A has worn the HMD 110 is based on the tilt information indicating the tilt (roll angle, yaw angle, pitch angle) of the HMD 110 acquired by the HMD sensor 114, and determines whether the tilt of the HMD 110 is greater than a predetermined tilt. It can be determined whether it is small or not. As mentioned above, while the user A is wearing the HMD 110, the tilt of the HMD 110 is less than or equal to the predetermined tilt. Therefore, if the tilt of the HMD 110 is smaller than the predetermined tilt, the user It is determined that

If it is determined in step S73 that user A has put on the HMD 110 (YES in step S73), the control unit 121 of the user terminal 1A transitions from sleep mode to active mode (step S74). Next, the control unit 121 that has transitioned to active mode generates control information for the avatar 4A and generates wearing information indicating that user A has put on the HMD 110 (step S75). Thereafter, the control unit 121 transmits the control information and wearing information for the avatar 4A to the server 2 via the communication network 3 (step S76).

Next, after receiving the control information and wearing information of the avatar 4A from the user terminal 1A, the server 2 transmits the control information and wearing information of the avatar 4A to the user terminal 1B via the communication network 3 (step S77).

Next, the control unit 121 of the user terminal 1B updates the behavior of the avatar 4A based on the control information of the avatar 4A, and also updates the behavior of the avatar 4B based on the control information of the avatar 4B (step S78). In particular, the control unit 121 updates the posture of the avatar 4A based on the wearing information of the avatar 4A. Further, the control unit 121 may update the facial expression of the avatar 4B based on the face information of the avatar 4B included in the control information of the avatar 4B. Specifically, the control unit 121 sets the posture of the avatar 4A to an upright state (an example of the third state), for example, as shown in FIG. 23.

Note that, similarly to the embodiment shown in FIG. 17, the control unit 121 may generate a speech bubble object associated with the avatar 4A in the virtual space 200B. Information indicating that user A is wearing the HMD 110 (for example, "User A has returned to his seat.") is displayed in the balloon object.

After that, the control unit 121 updates the virtual space data indicating the virtual space 200B, and also updates the visual field CV of the avatar 4B according to the movement of the HMD 110 of the user terminal 1B. Next, the control unit 121 updates the visual field image data based on the updated virtual space data and the updated visual field CV of the avatar 4B, and displays the visual field image on the HMD 110 based on the updated visual field image data. (Step S79).

On the other hand, if it is determined in step S73 that user A is not wearing the HMD 110 within a certain period of time after receiving the read information (NO in step S73), the process proceeds to FIG. 24. Next, as shown in FIG. 24, the control unit 121 generates non-wearing information indicating that user A is not wearing the HMD 110 (step S80). Thereafter, the control unit 121 transmits the non-wearing information to the server 2 via the communication network 3 (step S81).

Next, after receiving the non-wearing information from the user terminal 1A, the server 2 transmits the non-wearing information to the user terminal 1B via the communication network 3 (step S82).

Next, the control unit 121 of the user terminal 1B sets the facial expression of the avatar 4A placed in the virtual space 200B to a default expression based on the non-wearing information received from the server 2 (step S83). For example, if the default facial expression is a closed eyes state, the facial expression of the avatar 4A is updated to a closed eyes state, as shown in FIG. In addition, in the process of step S83, the control unit 121 of the user terminal 1B measures a certain period of time from the reception of the read information, and if the control information and wearing information of the avatar 4A are not received within the certain period of time, the control unit 121 of the user terminal 1B measures the control information and the wearing information of the avatar 4A. The facial expression may be set as a default expression. If the eyes of the avatar 4A are open based on the read information in step S70 of FIG. 18, the facial expression of the avatar 4A is returned to the closed eye state in step S83. On the other hand, if the participation consultation notice has not been read (NO in step S66), the avatar 4A's eyes are kept closed, so the facial expression of the avatar 4A is changed in step S83. Not changed.

After that, the control unit 121 updates the virtual space data indicating the virtual space 200B, and also updates the visual field CV of the avatar 4B according to the movement of the HMD 110 of the user terminal 1B. Next, the control unit 121 updates the visual field image data based on the updated virtual space data and the updated visual field CV of the avatar 4B, and displays the visual field image on the HMD 110 based on the updated visual field image data. (Step S84).

As described above, the information processing method according to the first modification includes the steps of moving the virtual hand 400 in the virtual space 200 in response to the movement of the user B, performing an action (first action) such as a shaking action by the virtual hand 400 on the avatar 4A whose movement in the virtual space 200 can be controlled based on the movement of the user A, and performing a participation inquiry notification (second action) to ask the user A (the user terminal 5 associated with the user A) to participate in the virtual space 200 based on the execution of the action. According to this method, by performing a predetermined action between a plurality of users A and B who share the virtual space 200 by intuitive operation using the virtual hand 400, communication in the virtual space 200 can be smoothly performed without impairing the user's immersive feeling. Thus, according to the first modification, the virtual experience of the user can be improved by providing a user interface (so-called diegetic UI) that does not require menu operation or key operation for communication between users.

In particular, by transmitting a participation consultation notification to the user terminal 5 associated with user A based on a specific action performed by user B on avatar 4A, user B can participate in the content provided to virtual space 200. User A can easily recognize that he has been approached to participate. Thereby, it is possible to promote the use of content (for example, a multiplayer game) that requires multiple users to play simultaneously in the virtual space 200.

Further, in the information processing method according to the first modification, the specific action on the avatar 4A by the user B is executed in a state where the avatar 4A is not controlled by the user A (that is, in a state where the user A is not wearing the HMD 110). In this case, a participation consultation notification is sent to user A. Therefore, user A who is not wearing HMD 110 can be attracted to the content provided in virtual space 200, and participation in the content can be further promoted. In particular, in VR games, when there are few users (avatars) waiting in a room, that is, when there are few candidates for multiplayer partners, the users (avatars) enter the room for the time being, but candidates for multiplayer appear. Since users often take off their HMDs and wait until the end of the game, such waiting users can be effectively invited to participate in multiplayer VR games.

Furthermore, in the information processing method according to the first modification, when the user A makes a reaction to the participation consultation notification (for example, when the message M1 is marked as read), the avatar 4A in the virtual space 200B is , a step of transitioning from a state with eyes closed (first state) to a state with eyes open (second state), and after the avatar 4A transitions to the state with eyes open, user A The process may further include a step of transitioning the avatar 4A in the virtual space 200B to an upright state (third state) when the avatar 4A is in a controlled state. Thereby, user B can easily understand that there has been a reaction from user A and that user A is wearing the HMD 110 and the avatar 4A is being controlled.

On the other hand, if avatar 4A does not become controlled by user A within a certain period of time after avatar 4A transitions to an open-eyed state, avatar 4A in virtual space 200B may be returned to a closed-eyed state. This allows user B to easily understand that avatar 4A is not controlled by user A despite the participation inquiry notification being sent (i.e., user A is not participating in the content). Therefore, user B can shake avatar 4A again to inquire about participation in the virtual space again, or can inquire about participation in the virtual space of avatars associated with other users other than user A who share virtual space 200.

Note that the state where the avatar 4A is not controlled by the user A includes not only the case where the user A has removed the HMD 110 but also the case where the user A is not logged in to the content provided to the virtual space 200 in the first place. obtain. Note that if user A is not logged in to the content, the avatar 4A does not normally exist in the virtual space 200B provided by user B's user terminal 1B, but if user B has registered user A as a friend, For example, user A's avatar 4A can exist in virtual space 200B even if user A is not logged into the content. The presence or absence of this friend registration may be registered, for example, in the server 2 together with each user's personal information. In this case, the control unit 121 of the user terminal 1B can default the posture and expression of the avatar 4A placed in the virtual space 200B based on the fact that the user A has not logged in to the content (Fig. 18 step S54). In this way, the avatar 4A associated with the user A who is not logged in to the content is made to exist in the virtual space 200B, and the user A is approached to participate based on the execution of a specific action from the avatar 4B to the avatar 4A. By sending the notification, it is possible to provide motivation for user A to log into the content.

The content of the participation inquiry notification (message M1) may differ depending on the strength of the specific action of avatar 4B against avatar 4A. For example, multiple thresholds may be set in stages for the acceleration of the action of virtual hand 400, and the content of message M1 may be changed depending on the acceleration threshold. For example, message M1 may include information on the degree of desire to participate, and the information on the degree of desire to participate may be changed depending on the strength of the shaking action. Specifically, when the shaking action is intense, the message M1 may display content indicating that the degree of desire to participate from user B is high (for example, "Degree of desire to participate: ☆☆☆ (three stars)"). A predetermined threshold may be set for the number of times that virtual hand 400 passes through the relative coordinates associated with avatar 4A, and the strength of the motion of the specific action may be specified depending on the threshold of the number of passes.

The content of message M1 may also be changed when multiple avatars simultaneously (or in cooperation) perform a specific action on avatar 4A. In this case, it is preferable to provide dedicated colliders for each specific part of avatar A's body so that avatar A can be grabbed by multiple avatars at the same time. This allows multiple avatars to shake avatar 4A. Note that in this case, it is preferable to increase the participation desire level the more users perform specific actions on avatar 4A.

The content of message M1 may also differ depending on the type of specific action performed on avatar 4A. For example, if avatar 4B slaps avatar 4A, the participation desire level displayed in message M1 may be higher than if avatar 4B shakes avatar 4A.

In this way, by varying the content of the participation consultation notification depending on the strength of the specific action performed by the avatar 4B toward the avatar 4A and the type of the specific action, user A can respond to user B's desire to participate in the content. be able to grasp the height and low of Therefore, user A can decide whether to participate in the content depending on the degree of desire.

In addition, in this embodiment, an action involving physical interference is exemplified as the specific action by avatar 4B against avatar 4A, but this example is not limited to this. The specific action may be any action for identifying user A, and for example, the specific action performed by avatar 4B against avatar 4A may be identified by gaze, voice, laser pointer, facial expression (smile, glare, etc.), clapping, etc. Also, user B may select an avatar (user) that he/she wishes to ask to participate in the content from a list of avatars (list of users) displayed in virtual space 200B. In other words, the specific action performed by avatar 4B may be a direct action against avatar 4A or an indirect action. Furthermore, a direct action and an indirect action may be combined.

Furthermore, the timing at which the participation consultation notification is sent may be varied depending on the contents of the content provided to the virtual space 200. For example, if the number of users required to start multiplayer is three or more, the control unit 121 of user terminal 1B controls whether user B is the last avatar among the plurality of avatars of the number required to start multiplayer. When it is determined that a specific action has been performed on a specific action, the control unit 121 of the user terminal 1B collectively sends a participation consultation notification to each user associated with the plurality of avatars that have performed the specific action. You can do it like this. Note that even if the number of users required to start multiplayer is three or more, the control unit 121 of the user terminal 1B performs a specific action on each avatar at the timing when the avatar 4B performs a specific action on each avatar. Participation consultation notifications may be sent individually.

Furthermore, in the above embodiment, the facial expression of the avatar 4A whose eyes are closed is changed based on the reaction of the user A to the participation consultation notification (for example, the message M1 regarding the participation consultation notification has been read). , the eyes are updated to an open state, but this is not limited to this example. For example, the avatar 4A may be updated to have its eyes open based on the fact that the avatar 4B has performed a specific action on the avatar 4A. Thereby, user B can easily understand that the specific action has been appropriately performed on avatar 4A. Note that the avatar 4A may be updated to have its eyes open based on the user A wearing the HMD 110.

The information processing method according to the first modification includes a step of executing a predetermined action (an example of a first action) involving physical interference from user B to user A in the virtual space 200, a step of executing a participation inquiry notice (an example of a second action) to user A based on the execution of the action, and a step of updating the posture and/or facial expression of the avatar 4A in the virtual space 200 (an example of a third action) based on user A's reaction to the participation inquiry notice (for example, the message M1 related to the participation inquiry notice being read). According to this method, the posture and facial expression of the avatar 4A associated with user A in the virtual space 200B are updated based on user A's reaction to the participation inquiry notice transmitted to user A, thereby enabling smooth communication between users A and B in the virtual space 200. In this way, a seamless virtual experience that blurs the boundary between the real space and the virtual space can be provided by reflecting user A's reaction to the participation inquiry notice in the real space in the virtual space 200.

(Variation 2)
Next, a second modified example of the information processing method according to the present embodiment will be described with reference to Figs. 25 to 28. In the second modified example, an information processing method at the start of a game in which only the user B plays a game (a so-called single-play game) in the virtual space 200 (200B) is provided will be illustrated. Fig. 25 is an example of a flowchart for explaining the information processing method according to the second modified example. Figs. 26 to 28 are diagrams showing an example of the virtual space 200B provided to the user B for explaining the information processing method according to the second modified example. As shown in Fig. 26, in the second modified example, the virtual space 200B includes an avatar 4B and an avatar 4C. The avatar 4C is an avatar 4C (a so-called Non Player Character (NPC)) that can be controlled by the server 2.

Referring to FIG. 25, first, the control unit 121 of the user terminal 1B accepts a game start input based on an operation by the user B (step S100). Next, the control unit 121 transmits game start input information to the server 2 via the communication network 3 (step S101).

Next, the server 2 reads opening movie data from the storage unit 22 based on the game start input information received from the control unit 121 (step S102). Note that the opening movie is a video that is played in the virtual space 200B as a scenario scene (an example of a first scene) introduced at the start of the game. Next, the server 2 transmits opening movie data to the control unit 121 of the user terminal 1B via the communication network 3 (step S103).

Next, the control unit 121 of the user terminal 1B plays an opening movie in the virtual space 200B based on the opening movie data received from the server 2 (step S104).

During or after playing the opening movie, the control unit 121 of the user terminal 1B sets the posture and facial expression of the avatar 4C to the default posture and facial expression (step S105). Assuming that the default posture is lying down and the default facial expression is eyes closed, the posture of avatar 4C is set to lying down, and the facial expression of avatar 4C is set to be lying down, as shown in FIG. The facial expression is set to a state where the eyes are closed.

Next, the control unit 121 updates the virtual space data indicating the virtual space 200B, and updates the field of view CV of the avatar 4B in accordance with the movement of the HMD 110 of the user terminal 1B. Next, the control unit 121 updates the field of view image data based on the updated virtual space data and the updated field of view CV of the avatar 4B, and displays the field of view image on the HMD 110 based on the updated field of view image data (step S106).

Next, after updating the field of view image in step S106, the control unit 121 determines whether or not the avatar 4B associated with user B has performed a specific action (an example of a first action) on the avatar 4C associated with the server 2 (step S107). The process of determining the specific action is the same as in variant example 1, so a detailed description thereof will be omitted. Note that, in order to encourage the avatar 4B to perform a specific action on the avatar 4C, the control unit 121 may display a message such as "Please wake up avatar 4C" in the virtual space 200B after playing the opening movie in step S104.

Next, when the control unit 121 determines that the avatar 4B has performed a specific action on the avatar 4C (YES in step S107), the control unit 121 generates action execution information indicating that the specific action has been performed by the user B (step S108). After that, the control unit 121 transmits the action execution information to the server 2 via the communication network 3 (step S109).

Next, the server 2 generates scene transition information based on the action execution information received from the user terminal 1B (step S110). The scene transition information is information for transitioning the game provided in the virtual space 200B from a scenario scene to a game scene (an example of a second scene). Next, the server 2 transmits the scene transition information to the control unit 121 of the user terminal 1B via the communication network 3 (step S111).

Next, the control unit 121 of the user terminal 1B updates the settings of the posture and facial expression of the avatar 4C based on the scene transition information received from the server 2 (step S112). Specifically, for example, as shown in FIG. 28, the posture of the avatar 4C is set to a sitting up position, and the facial expression of the avatar 4C is set to an open-eyed position.

Next, the control unit 121 generates a speech bubble object 45A associated with avatar 4C in the virtual space 200B (step S113). The speech bubble object 45A displays information indicating that avatar 4C has joined the group of avatar 4B (e.g., "Avatar 4C has joined the group.").

After that, the control unit 121 updates the virtual space data indicating the virtual space 200B, and also updates the visual field CV of the avatar 4B according to the movement of the HMD 110 of the user terminal 1B. Next, the control unit 121 updates the visual field image data based on the updated virtual space data and the updated visual field CV of the avatar 4B, and displays the visual field image on the HMD 110 based on the updated visual field image data. (Step S114).

Then, based on the scene transition information, the control unit 121 starts displaying the game scene (step S115).

As described above, the information processing method according to the second modification includes a step of executing a specific action, such as a shaking action, by the virtual hand 400 on the avatar 4C whose action in the virtual space 200B can be computer-controlled, and a step of transitioning the game provided in the virtual space 200B from a scenario scene (first scene) to a game scene (second scene) based on the execution of the specific action. Conventionally, in order to start a game scene, a user interface (e.g., "Press Any Button") for progressing the game is displayed after the scenario scene is played back, and the game scene is started by the user pressing a specific button on the external controller 320. In contrast, according to the method according to the second modification, a game scene is started based on the execution of a specific action by the user B on the avatar 4C, so that the game can be progressed without impairing the user B's immersion in the virtual space 200B. Note that, in the second modification, the processing of the second modification is executed in a system having the server 2 and the user terminal 1B, but is not limited to this, and may be executed by the user terminal 1B alone.

Further, the information processing method for progressing the game according to the second modification is not limited to the scene when starting the game, but can also be applied to the scene where the game is restarted in the middle of the game. For example, in a case where a game played by a party made up of multiple avatars (for example, avatar 4B and avatar 4C) is provided in virtual space 200B, a specific action is executed from avatar 4B to avatar 4C after the game progress is interrupted. The game may be restarted when it is determined that the game has been played. Specifically, assume that in a scene where the party stays at an inn, the avatar 4C is still sleeping the next morning. In this state, the avatar 4B may perform a specific action such as shaking or hitting the avatar 4C, thereby waking up the avatar 4C and restarting the game. According to such a method, it is possible to restart the progress of the game without impairing user B's sense of immersion in the virtual space 200B.

Furthermore, the order of processing defined by each step shown in Figures 18, 24, and 25 is merely an example, and the order of these steps can be changed as appropriate.

In addition, in the above description of the embodiment and the modified example, it is assumed that the wearing or non-wearing of the HMD 110 is detected by the tilt information of the HMD 110 transmitted from the HMD sensor 114 or the event information transmitted from the wearing sensor 115, but this is not limited to the example. For example, the control unit 121 of the user terminal may detect the movement of the HMD 110 by the position sensor 130 and/or the HMD sensor 114, and when the movement draws a predetermined trajectory, may determine that the user has worn the HMD 110 or removed the HMD 110. In addition, the control unit 121 may detect the movement of the HMD 110 by the position sensor 130 and/or the HMD sensor 114, and may determine that the user is not wearing the HMD 110, for example, when the position of the HMD 110 does not change at all for a certain period of time. Furthermore, the control unit 121 may detect the wearing or non-wearing of the HMD 110 by detecting the relative positional relationship between the HMD 110 and the external controller 320. For example, the control unit 121 may determine that the user is not wearing the HMD 110 if the HMD 110 and the external controller 320 are separated by a certain distance or more.

Furthermore, in the description of the above embodiments and modifications, it is assumed that the virtual space data indicating the virtual space 200B is updated on the user terminal 1B side, but the virtual space data may be updated on the server 2 side. good. Further, although it is assumed that the visual field image data corresponding to the visual field image has been updated on the user terminal 1B side, the visual field image data may be updated on the server 2 side. In this case, the user terminal 1B displays the visual field image on the HMD 110 based on the visual field image data transmitted from the server 2.

Furthermore, the order of processing defined by each step shown in Figures 12 and 16 is merely an example, and the order of these steps can be changed as appropriate.

In addition, in this embodiment, a virtual space (VR space) in which the user is immersed by the HMD device has been described as an example, but a see-through HMD device may be used as the HMD device. In this case, a field of view image in which a part of the image constituting the virtual space is synthesized in the real space visually recognized by the user through the see-through HMD device may be output to provide the user with a virtual experience in an augmented reality (AR) space or a mixed reality (MR) space. In this case, an action may be exerted on a target object in the virtual space based on the movement of the user's hand instead of the virtual hand. Specifically, the processor may specify coordinate information of the position of the user's hand in the real space and define the position of the target object in the virtual space in relation to the coordinate information in the real space. This allows the processor to grasp the positional relationship between the user's hand in the real space and the target object in the virtual space 2, and to execute processing corresponding to the above-mentioned collision control between the user's hand and the target object. As a result, it becomes possible to exert an action on the target object based on the movement of the user's hand.

Further, in order to realize various processes executed by the control unit 121 of the user terminal 1 by software, a control program for causing a computer (processor) to execute various processes may be pre-installed in the storage unit 123 or memory. good. Alternatively, the control program may include a magnetic disk (HDD, floppy disk), optical disk (CD-ROM, DVD-ROM, Blu-ray (registered trademark) disk, etc.), magneto-optical disk (MO, etc.), flash memory (SD card, It may be stored in a computer-readable storage medium such as a USB memory, SSD, etc.). In this case, by connecting the storage medium to the control device 120, the control program stored in the storage medium is incorporated into the storage unit 123. Then, the control program incorporated in the storage unit 123 is loaded onto the RAM, and the processor executes the loaded program, so that the control unit 121 executes various processes.

The control program may also be downloaded from a computer on the communication network 3 via the communication interface 125. In this case as well, the downloaded control program is incorporated into the storage unit 123.

Although the embodiments of the present disclosure have been described above, the technical scope of the present invention should not be interpreted to be limited by the description of the present embodiments. This embodiment is an example, and those skilled in the art will understand that various changes can be made within the scope of the invention as set forth in the claims. The technical scope of the present invention should be determined based on the scope of the invention described in the claims and the scope of equivalents thereof.

1, 1A, 1B: User terminal 2: Server 3: Communication network 4A, 4B: Avatar 21: Communication interface 22: Storage unit 23: Control unit 24: Bus 42A, 44A, 45A: Speech bubble object 43A: Caption object 100: Virtual space distribution system (distribution system)
110: HMD (head mounted device)
112: Display unit 113: Face camera 114: HMD sensor 115: Wearing sensor 116: Headphones 117: Face camera 118: Microphone 120: Control device 123: Memory unit 124: I/O interface 125: Communication interface 126: Bus 130: Position sensor 140: Gaze sensor 200, 200A, 200B: Virtual space 210: Center position 300: Virtual camera 320: External controller 320L: External controller for left hand (controller)
320R: Right-hand external controller (controller)
400: Virtual hand U, A, B: User M1: Message

Claims (8)

computer,
providing means for providing a virtual experience to the user via a device operated by the user;
first execution means for executing an action for identifying another user different from the user in a virtual space;
a second execution unit that executes a process of notifying the other user to participate in the virtual space based on the execution of the action;
A program that functions as
The action includes a specific action by a first virtual object associated with the user on a second virtual object associated with the other user,
The notification includes an indication of a degree of desire indicating the degree to which the user requests the other user to participate in the virtual space,
The second execution means is configured to display the degree of desire included in the notification differently depending on at least one of the strength of the action and the type of the action with respect to the second virtual object. The specific action may include physical interference between the first virtual object and the second virtual object, a line of sight from the first virtual object to the second virtual object, a voice, a laser pointer, a facial expression, or applause. The program according to claim 1, comprising at least one. computer,
providing means for providing a virtual experience to the user via a device operated by the user;
a first execution means for executing a first action that is an action for identifying another user different from the user in a virtual space;
a second execution unit that executes a process of notifying the other user to participate in the virtual space based on the execution of the action;
A program that functions as
The action includes a specific action by a first virtual object associated with the user on a second virtual object associated with the other user,
The notification includes an indication of a degree of desire indicating the degree to which the user requests the other user to participate in the virtual space,
The second execution means is configured to display the degree of desire included in the notification differently depending on the strength of the action for the second virtual object. causing the computer to further function as a moving means for moving an operation object within the virtual space in response to a movement of a part of the body of the user;
The program according to claim 1 , wherein the action is executed using the operation object. The program according to claim 4, wherein the specific action includes an action of the first virtual object shaking or hitting the second virtual object using the control object. The program according to any one of claims 1 to 5, wherein the notification is sent to a terminal other than the device and the computer, the terminal being associated with the other user. computer,
providing means for providing a virtual experience to the user via a device operated by the user;
first execution means for executing an action for identifying another user different from the user in a virtual space;
a second execution unit that executes a process of notifying the other user to participate in the virtual space based on the execution of the action;
An information processing system that functions as a
The action includes a specific action by a first virtual object associated with the user on a second virtual object associated with the other user,
The notification includes an indication of a degree of desire indicating the degree to which the user requests the other user to participate in the virtual space,
An information processing system, wherein the display of the degree of desire included in the notification is varied depending on at least one of the strength of the action and the type of the action with respect to the second virtual object. computer,
providing means for providing a virtual experience to the user via a device operated by the user;
first execution means for executing an action for identifying another user different from the user in a virtual space;
a second execution unit that executes a process of notifying the other user to participate in the virtual space based on the execution of the action;
An information processing system that functions as a
The action includes a specific action by a first virtual object associated with the user on a second virtual object associated with the other user,
The notification includes an indication of a degree of desire indicating the degree to which the user requests the other user to participate in the virtual space,
An information processing system, wherein the degree of desire included in the notification is displayed differently depending on the strength of the action for the second virtual object.

Images