JP2022121451A - Program, information processing apparatus, information processing system, and information processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a program for improving virtual experience of a user.

SOLUTION: A program to be executed by a computer for providing a user B with virtual experience includes: a step of executing first action in a virtual space 200B; a step of executing second action with respect to another user A different from the user B, on the basis of the execution of the first action; and a step of executing third action in the virtual space 200B on the basis of a reaction to the second action.

SELECTED DRAWING: Figure 20

COPYRIGHT: (C)2022,JPO&INPIT

Description

The present disclosure relates to a program, an information processing device, an information processing system, and an information processing method.

As in Non-Patent Literature 1, a service is known in which a plurality of users sharing a virtual space can enjoy communication with body language using virtual hands or the like.

“Social VR Demo - Selfie Stick and 360 Photo Spheres - Oculus”, [online], April 13, 2016, Oculus, [searched August 10, 2017], Internet <https://youtu.be/ -pumFtAjgLY?t=5>

However, services such as those described in Non-Patent Document 1 above have room for improvement in order to improve the user's virtual experience.

The present disclosure aims to enhance a user's virtual experience.

According to one aspect of the program presented by the present disclosure,
computer to provide a virtual experience to the user,
performing a first action within the virtual space;
performing a second action for another user different from the user based on performing the first action;
and executing a third action in the virtual space based on the reaction to the second action.

According to the present disclosure, a user's virtual experience can be enhanced.

1 is a schematic diagram showing a virtual space delivery system according to an embodiment of the present invention (hereinafter simply referred to as this embodiment); FIG. 1 is a schematic diagram showing a user terminal; FIG. It is a figure which shows the head of the user wearing HMD. It is a figure which shows the hardware constitutions of a control apparatus. 4 is a flowchart showing processing for displaying a field-of-view image on the HMD; It is an xyz space diagram which shows an example of virtual space. State (a) is a yx plan view of the virtual space shown in FIG. State (b) is a zx plan view of the virtual space shown in FIG. FIG. 4 is a diagram showing an example of a field-of-view image displayed on the HMD; 2 is a diagram showing a hardware configuration of a server shown in FIG. 1; FIG. State (a) is a diagram showing a virtual space provided to user A. FIG. State (b) is a diagram showing a virtual space provided to user B. FIG. FIG. 11 is a sequence diagram for explaining an example of processing for synchronizing movements of avatars between user terminals; 6 is an example of a flowchart for explaining the information processing method according to the embodiment when User A removes the HMD. State (a) is an example of a flowchart for explaining processing for determining whether user A has removed the HMD using the HMD sensor. State (b) is an example of a flowchart for explaining processing for determining whether user A has removed the HMD using the wearing sensor. It is a figure which shows an example of the virtual space provided to the user B for demonstrating the information processing method which concerns on this embodiment. FIG. 10 is a diagram showing another example of the virtual space provided to User B for explaining the information processing method according to the present embodiment; 8 is an example of a flowchart for explaining the information processing method according to the embodiment when user A wears the HMD again. FIG. 10 is a diagram showing another example of the virtual space provided to User B for explaining the information processing method according to the present embodiment; 9 is an example of a flowchart for explaining an information processing method according to modification 1; FIG. 10 is a diagram showing an example of a virtual space 200B provided to user B for explaining an information processing method according to modification 1; FIG. 10 is a diagram showing an example of a virtual space 200B provided to user B for explaining an information processing method according to modification 1; FIG. 10 is a diagram showing a display example of the user terminal 5 associated with the user A for explaining the information processing method according to Modification 1; FIG. 10 is a diagram showing an example of a virtual space 200B provided to user B for explaining an information processing method according to modification 1; FIG. 10 is a diagram showing an example of a virtual space 200B provided to user B for explaining an information processing method according to modification 1; FIG. 10 is an example of a flowchart for explaining an information processing method according to Modification 1 when User A has not worn the HMD within a certain period of time after receiving already-read information; FIG. 10 is an example of a flowchart for explaining an information processing method according to modification 2; FIG. 10 is a diagram showing an example of a virtual space 200B provided to user B for explaining an information processing method according to modification 2; FIG. 10 is a diagram showing an example of a virtual space 200B provided to user B for explaining an information processing method according to modification 2; FIG. 10 is a diagram showing an example of a virtual space 200B provided to user B for explaining an information processing method according to modification 2;

[Description of Embodiments Presented by the Present Disclosure]
An outline of embodiments indicated by the present disclosure will be described.
(1) A first user terminal having a first head-mounted device worn on the head of a first user and a second user terminal having a second head-mounted device worn on the head of a second user An information processing method executed by a computer in a virtual space distribution system comprising:
The information processing method includes:
(a) generating virtual space data defining a virtual space including a first avatar associated with said first user and a second avatar associated with said second user;
(b) updating a field-of-view image displayed on the second head-mounted device based on the movement of the second head-mounted device and the virtual space data;
(d) when non-wearing information indicating that the first user is not wearing the first head mounted device is received, the first user is not wearing the first head mounted device; and presenting to the second user by a second user terminal.

According to the above method, when non-wearing information indicating that the first user is not wearing the first head-mounted device (hereinafter referred to as the first HMD) is received, the second user terminal causes the first user to wear the first HMD. is presented to the second user. Thus, the second user can easily recognize that the first user is not wearing the first HMD while communicating with the first user in the virtual space. Therefore, it is possible to provide the user with a rich virtual experience.

In particular, unless the second user is aware that the first user is not wearing the first HMD, the second user may feel uncomfortable with the first user who does not show any reaction. In this way, by making the second user understand that the first user is not wearing the first HMD, it is possible to avoid a situation in which the second user feels uncomfortable with the first user who does not show any reaction. It becomes possible.

(2) The step (d) is
An item ( 1) The information processing method described in 1).

According to the above method, when non-wearing information indicating that the first user is not wearing the first head mounted device (first HMD) is received, it is determined that the first user is not wearing the first HMD. The indicated information is visualized on the visual field image displayed on the second head mounted device (second HMD). In this way, the second user visually recognizes through the second HMD the information indicating that the first user is not wearing the first HMD while communicating with the first user in the virtual space. It is possible to easily grasp that one user is not wearing the first HMD.

(3) (e) further comprising updating the facial expression of the first avatar based on the facial information when facial information indicating the facial expression of the first avatar is received;
The step (d) includes
Item (1) or ( 2) The information processing method described in 2).

According to the above method, when non-wearing information indicating that the first user is not wearing the first head mounted device (first HMD) is received, the first avatar whose facial expression is set to the first mode is received. is visualized on the field-of-view image displayed on the second head mounted device (second HMD). In this way, the second user visually recognizes that the facial expression of the first avatar is the predetermined expression while communicating with the first user in the virtual space, thereby allowing the first user to is not wearing the first HMD.

In particular, when the first user is not wearing the first HMD, the facial expression of the first avatar is not updated at all. Therefore, unless the second user is aware that the first user is not wearing the first HMD, the second user may feel uncomfortable with the first user. In this way, by making the second user understand that the first user is not wearing the first HMD, it is possible to avoid a situation in which the second user feels uncomfortable with the first user who does not show any reaction. It becomes possible.

(4) The information processing method according to item (3), wherein the first aspect is a default facial expression of the first avatar.

According to the above method, when non-wearing information indicating that the first user is not wearing the first head-mounted device (first HMD) is received, the facial expression of the first avatar is set as default. 1 It becomes the facial expression of the avatar. In this way, the second user can visually recognize that the facial expression of the first avatar is the default facial expression while communicating with the first user in the virtual space. , it is possible to easily grasp that the first user is not wearing the first HMD.

(5) The information processing method according to item (3), wherein the first aspect is an expression selected in advance as an aspect indicating that the first user is not wearing the first head mounted device.

According to the above method, when the non-wearing information indicating that the first user is not wearing the first head-mounted device (first HMD) is received, the facial expression of the first avatar is changed so that the first user is not wearing the first head-mounted device (first HMD). 1 is set to a facial expression (hereinafter referred to as a selected facial expression) selected in advance by the first user as a mode indicating that the head-mounted device is not worn. In this way, the second user visually recognizes that the facial expression of the first avatar is the selected expression while communicating with the first user in the virtual space. It is possible to easily grasp that the first HMD is not worn.

(6) the first user terminal further comprises a wearing sensor configured to detect whether the first user is wearing the first head-mounted device;
When the wearing sensor detects that the first user is not wearing the first head mounted device, the first user terminal outputs the non-wearing information, and the second user terminal outputs The information processing method according to any one of items (1) to (5), wherein the received non-wearing information is received from the first user terminal.

According to the above method, it is specified that the first user is not wearing the first head mounted device (first HMD) according to the information transmitted from the wearing sensor of the first user terminal. Thereafter, non-wearing information indicating that the first user is not wearing the first HMD is transmitted. In this way, it is possible to automatically identify that the first user is not wearing the first HMD according to the information output from the wearing sensor of the first user terminal.

(7) (f) when receiving wearing information indicating that the first head mounted device has been removed from the first user and then worn again, the first user wears the first head mounted device; The information processing method according to any one of items (1) to (6), further comprising the step of presenting to the second user by the second user terminal that the second user is

According to the above method, when the wearing information indicating that the first head-mounted device (first HMD) has been removed from the first user and has been worn again, the second user terminal allows the first user to attach the first HMD to the first head-mounted device. is presented to the second user. Thus, the second user can easily recognize that the first user has put on the first HMD again. Therefore, it is possible to provide the user with a rich virtual experience.

(8) A program for causing a computer to execute the information processing method according to any one of items (1) to (7).

According to the above program, it is possible to provide the user with a rich virtual experience.

(9) a first user terminal having a first head-mounted device worn on the head of a first user, and a second user terminal having a second head-mounted device worn on the head of a second user; A virtual space delivery system, comprising: a virtual space distribution system configured to execute the information processing method according to any one of items (1) to (7).

According to the above virtual space delivery system, it is possible to provide users with a rich virtual experience.

(10) a processor;
a memory storing computer readable instructions, comprising:
An apparatus, wherein the apparatus performs the information processing method of any one of items (1) to (7) when the computer readable instructions are executed by the processor.

According to the above device, it is possible to provide the user with a rich virtual experience.

(11) to a computer for providing a virtual experience to a user;
moving an operation object in a virtual space according to the movement of the user's body part;
executing a first action by the manipulation object on a virtual object whose movement in the virtual space is controllable based on the movement of another user different from the user;
and executing a second action for the other user based on execution of the first action.

According to the above program, the user's virtual experience can be improved. In particular, it is possible to smoothly communicate in the virtual space without impairing the sense of immersion by performing actions among a plurality of users who share the virtual space through intuitive operations using operation objects.

(12) The program according to item (11), which executes the second action when the first action is executed while the virtual object is not controlled by the other user.

According to the above program, by performing the second action on the uncontrolled virtual object, it is possible to attract other users to the virtual space.

(13) The program according to item (11) or (12), wherein the first action includes an action of contacting the virtual object during the repetitive motion of the manipulation object.

(14) The program according to item (13), wherein the first action includes a contact motion to the virtual object in the course of the repetitive motion with acceleration equal to or greater than a certain value.

(15) The program according to item (13) or (14), wherein the first action includes an action of passing relative coordinates associated with the virtual object a predetermined number of times or more within a predetermined period of time.

According to these programs, it is possible to prevent erroneous detection of the motion of the manipulation object by not considering the case where the manipulation object accidentally contacts the virtual object.

(16) The program according to any one of items (11) to (15), wherein the second action includes notification prompting the other user to participate in the virtual space.

According to the above program, other users sharing the virtual space can easily recognize that, for example, multiplay is being offered, and use of multiplay can be promoted.

(17) transitioning the virtual object from a first state to a second state when a reaction to the second action is performed;
a step of transitioning the virtual object to a third state if the virtual object is controlled by the other user within a certain period of time after the transition to the second state. The program according to item (16), which is executed by the computer.

According to the above program, the user who has executed the first action can easily recognize that there is a reaction from another user or that the virtual object is controlled by another user.

(18) If the state in which the virtual object is controlled by the other user does not occur within a certain period of time after the transition to the second state, the step of returning the virtual object to the first state is further performed. A program according to item (17) for being executed by a computer.

According to the above program, the user who has executed the first action can easily understand that the virtual object will not be controlled by another user despite the execution of the second action.

(19) Any one of items (16) to (18), wherein the content of the notification differs according to at least one of the intensity of the motion of the first action on the virtual object and the type of the first action. program described in .

According to the above method, other users can grasp the level of desire for multiplay from the user who executed the first action, and decide whether or not to participate in the virtual space according to the level of desire. can do.

(20) executing the first action by the manipulation object on another virtual object whose movement in the virtual space can be controlled by a computer;
items (11) to (19) for causing the computer to further execute a step of transitioning the game provided in the virtual space from a first scene to a second scene based on execution of the first action; ).

According to the above program, the game can be progressed without impairing the user's sense of immersion in the virtual space.

(21) An information processing device for providing a user with a virtual experience,
Under the control of the processor included in the information processing device,
moving an operation object in a virtual space according to the movement of the user's body part;
executing a first action by the manipulation object on a virtual object whose movement in the virtual space is controllable based on the movement of another user different from the user;
and executing a second action for the other user based on execution of the first action.

According to the information processing apparatus, it is possible to improve the user's virtual experience.

(22) An information processing system for providing a virtual experience to a user,
Under the control of a plurality of processors included in the information processing system,
moving an operation object in a virtual space according to the movement of the user's body part;
executing a first action by the manipulation object on a virtual object whose movement in the virtual space is controllable based on the movement of another user different from the user;
and executing a second action for the other user based on execution of the first action.

According to the above information processing system, it is possible to improve the user's virtual experience.

(23) A computer-implemented information processing method for providing a virtual experience to a user, comprising:
moving an operation object in a virtual space according to the movement of the user's body part;
executing a first action by the manipulation object on a virtual object whose movement in the virtual space is controllable based on the movement of another user different from the user;
and executing a second action for the other user based on execution of the first action.

According to the above information processing method, it is possible to improve the user's virtual experience.

(24) to a computer for providing a virtual experience to a user;
performing a first action within the virtual space;
performing a second action for another user different from the user based on performing the first action;
and executing a third action in the virtual space based on the reaction to the second action.

According to the above program, the user's virtual experience can be improved. In particular, by executing the third action in the virtual space based on the reaction to the second action executed by another user, a new virtual experience is provided in which the boundary between the real space and the virtual space becomes seamless. can do.

(25) The first action is an action for identifying the other user, and the third action is an action executed by a virtual object associated with the other user in the virtual space. A program according to item (24).

According to the above program, actions are executed among a plurality of users who share the virtual space, thereby enabling smooth communication in the virtual space.

(26) causing the computer to further execute a step of moving an operation object within the virtual space according to the movement of the body part of the user;
The program according to item (24) or (25), wherein the first action is executed using the operation object.

According to the above program, a plurality of users sharing a virtual space can perform an action by an intuitive operation using an operation object, and the user's sense of immersion is not impaired.

(27) The program according to any one of items (24) to (26), wherein the second action includes notification prompting the other user to participate in the virtual space.

According to the above program, other users sharing the virtual space can easily recognize that, for example, multiplay is being offered, and use of multiplay can be promoted.

(28) from item (24), wherein the third action includes transitioning an action performed by the virtual object associated with the other user in the virtual space from a first state to a second state; The program according to any one of (27).

According to the above method, the user who has performed the first action can easily grasp whether or not there is a reaction by another user.

(29) An information processing device for providing a user with a virtual experience,
Under the control of the processor included in the information processing device,
performing a first action within the virtual space;
performing a second action for another user different from the user based on performing the first action;
and executing a third action in the virtual space based on the reaction to the second action.

According to the information processing apparatus, it is possible to improve the user's virtual experience.

(30) An information processing system for providing a virtual experience to a user, comprising:
Under the control of a plurality of processors included in the information processing system,
performing a first action within the virtual space;
performing a second action for another user different from the user based on performing the first action;
and executing a third action in the virtual space based on the reaction to the second action.

According to the above information processing system, it is possible to improve the user's virtual experience.

(31) A computer-implemented information processing method for providing a virtual experience to a user, comprising:
performing a first action within the virtual space;
performing a second action for another user different from the user based on performing the first action;
and executing a third action in the virtual space based on a reaction to the second action.

According to the above information processing method, it is possible to improve the user's virtual experience.

[Details of Embodiments Presented by the Present Disclosure]
Hereinafter, embodiments indicated by the present disclosure will be described with reference to the drawings. For the sake of convenience, the description of elements having the same reference numbers as those already described in the description of the present embodiment will not be repeated.

First, an outline of the configuration of the virtual space distribution system 100 will be described with reference to FIG. FIG. 1 is a schematic diagram of a virtual space distribution system 100 (hereinafter simply referred to as distribution system 100). As shown in FIG. 1, the distribution system 100 includes a user terminal 1A (first user terminal) operated by a user A (first user) wearing a head mounted device (HMD) 110 (first HMD), an HMD 110 ( A user terminal 1B (second user terminal) operated by a user B (second user) wearing a second HMD) and a server 2 are provided. User terminals 1A and 1B are communicably connected to a server 2 via a communication network 3 such as the Internet. In this embodiment, the virtual space includes a VR (Virtual Reality) space. Further, hereinafter, the user terminals 1A and 1B may be collectively referred to simply as the user terminal 1 for convenience of explanation. Furthermore, users A and B may be collectively referred to as user U in some cases. Also, in this embodiment, the user terminals 1A and 1B are assumed to have the same configuration.

Next, the configuration of the user terminal 1 will be described with reference to FIG. FIG. 2 is a schematic diagram showing the user terminal 1. As shown in FIG. As shown in FIG. 2, the user terminal 1 includes a head mounted device (HMD) 110 worn on the head of the user U, headphones 116, a face camera 117, a microphone 118, a position sensor 130, and an external controller. 320 and a controller 120 .

HMD 110 includes display unit 112 , HMD sensor 114 , gaze sensor 140 , face camera 113 , and mounting sensor 115 . The display unit 112 includes a non-transmissive display device configured to completely cover the field of view of the user U wearing the HMD 110 . Thereby, the user U can be immersed in the virtual space by viewing only the field-of-view image displayed on the display unit 112 . The display unit 112 includes a left-eye display unit configured to provide an image to the left eye of the user U, and a right-eye display unit configured to provide an image to the right eye of the user U. good too. Also, the HMD 110 may include a transmissive display device. In this case, the transmissive display device may be temporarily configured as a non-transmissive display device by adjusting its transmittance.

The HMD sensor 114 is mounted near the display unit 112 of the HMD 110 . The HMD sensor 114 includes at least one of a geomagnetic sensor, an acceleration sensor, and an inclination sensor (an angular velocity sensor, a gyro sensor, etc.), and detects various movements (inclination, etc.) of the HMD 110 worn on the head of the user U. be able to.

The gaze sensor 140 has an eye tracking function of detecting the user U's line of sight. Gaze sensor 140 may comprise, for example, a right eye gaze sensor and a left eye gaze sensor. The right-eye gaze sensor irradiates the right eye of the user U with infrared light, for example, and detects the reflected light reflected from the right eye (particularly, the cornea and iris), thereby obtaining information about the rotation angle of the eyeball of the right eye. You may On the other hand, the left-eye gaze sensor irradiates the left eye of the user U with infrared light, for example, and detects the reflected light reflected from the left eye (particularly, the cornea and iris), thereby obtaining information on the rotation angle of the left eyeball. may be obtained.

The face camera 113 is configured to acquire an image (in particular, a moving image) in which the user U's eyes (left eye and right eye) and eyebrows (left eyebrow and right eyebrow) are displayed while the HMD 110 is worn by the user U. It is The face camera 113 is arranged at a predetermined location inside the HMD 110 so as to be able to image the user U's eyes and eyebrows. The frame rate of the moving image acquired by the face camera 113 may be higher than the frame rate of the field-of-view image (described later) displayed on the HMD 110 .

The wearing sensor 115 is configured to generate event information indicating whether the user U is wearing the HMD 110 . Specifically, when the user U removes the HMD 110 (when the state of the HMD 110 transitions from the wearing state to the non-wearing state), the wearing sensor 115 generates event information and controls the event information. Send to device 120 . Further, when the user U wears the HMD 110 (when the state of the HMD 110 transitions from the non-wearing state to the wearing state), the wearing sensor 115 generates event information and then transmits the event information to the control device 120 . Send. Also, the configuration of the mounting sensor 115 is not particularly limited. For example, assume that the elastic force of the springs provided in the HMD 110 changes when the user U wears the HMD 110 . In this case, the mounting sensor 115 may generate event information based on changes in the elastic force of the spring. Furthermore, it is assumed that when the user U wears the HMD 110, the current flowing between the two pads provided on the HMD 110 and contacting a part of the body (nose portion) changes. In this case, the wear sensor 115 may generate event information based on changes in the current flowing between the two pads.

Headphones 116 (audio output unit) are attached to the left ear and right ear of the user U, respectively. Headphone 116 is configured to receive audio data (electrical signal) from control device 120 and output audio based on the received audio data. The microphone 118 (speech input unit) is configured to collect speech uttered by the user U and generate speech data (electrical signal) based on the collected speech. Additionally, microphone 118 is configured to transmit audio data to controller 120 .

The face camera 117 is configured to acquire an image (in particular, a moving image) in which the mouth of the user U and its surroundings are displayed. The face camera 117 may be arranged at a position facing the mouth of the user U so as to be able to image the mouth of the user U and its surroundings. Also, the face camera 117 may be connected to the HMD 110 . The frame rate of moving images acquired by the face camera 117 may be higher than the frame rate of the field-of-view image displayed on the HMD 110 .

The position sensor 130 is composed of, for example, a position tracking camera, and is configured to detect the positions of the HMD 110 and the external controller 320 . The position sensor 130 is communicably connected to the control device 120 wirelessly or by wire, and is configured to detect information on the position, inclination, or light emission intensity of a plurality of detection points (not shown) provided on the HMD 110. . Furthermore, the position sensor 130 is configured to detect information on the position, tilt and/or light emission intensity of a plurality of detection points (not shown) provided on the external controller 320 . The detection point is, for example, a light emitting part that emits infrared rays or visible light. Position sensor 130 may also include an infrared sensor and multiple optical cameras.

The external controller 320 detects the movement of a part of the body of the user U (a part other than the head, which is the hand of the user U in the present embodiment), and controls the movement of the hand of the avatar displayed in the virtual space. Used to control movement. The external controllers 320 include a right-hand external controller 320R (hereinafter simply referred to as controller 320R) operated by the user U's right hand and a left-hand external controller 320L (hereinafter simply referred to as controller 320L) operated by the user U's left hand. ) and The controller 320R is a device that indicates the position of the right hand of the user U and the movement of the fingers of the right hand. Also, the right hand of the avatar existing in the virtual space moves according to the movement of the controller 320R. The controller 320L is a device that indicates the position of the left hand of the user U and the movement of the fingers of the left hand. Also, the left hand of the avatar existing in the virtual space moves according to the movement of the controller 320L.

Control device 120 is a computer configured to control HMD 110 . Control device 120 specifies the position information of HMD 110 based on the information acquired from position sensor 130, and based on the specified position information, the position of the virtual camera (the position of the avatar) in the virtual space and the position of the virtual camera in the real world. The position of the user U wearing the HMD 110 in space can be accurately associated. Furthermore, the control device 120 identifies the operation of the external controller 320 based on the information acquired from the position sensor 130 and/or the sensor incorporated in the external controller 320, and based on the identified operation of the external controller 320 Thus, it is possible to accurately associate the motion of the hand of the avatar displayed in the virtual space with the motion of the external controller 320 in the real space. In particular, the control device 120 identifies the operation of the controller 320L based on the information acquired from the position sensor 130 and/or the sensor built into the controller 320L, and based on the identified operation of the controller 320L, performs a virtual The motion of the left hand of the avatar displayed in the space and the motion of the controller 320L (the motion of the left hand of the user U) in the real space can be accurately associated. Similarly, the control device 120 identifies the operation of the controller 320R based on the information acquired from the position sensor and/or the sensor incorporated in the controller 320R, and based on the identified operation of the controller 320R, the virtual space The motion of the right hand of the avatar displayed inside can be accurately associated with the motion of the controller 320R (the motion of the right hand of the user U) in the real space.

Further, the control unit 121 can associate the facial expression (state) of the user U with the facial expression (state) of the user U's avatar based on the moving images acquired by the face cameras 113 and 117 . The details of the process of associating the user U's facial expression and the avatar's facial expression with each other will be described later.

Further, the control device 120 identifies the line of sight of the right eye and the line of sight of the left eye of the user U based on the information transmitted from the gaze sensor 140 (the gaze sensor for the left eye and the gaze sensor for the right eye), and A gaze point, which is the intersection of the line of sight of the left eye, can be specified. Furthermore, the control device 120 can specify the line of sight of both eyes of the user U (the line of sight of the user U) based on the identified point of gaze. Here, the line of sight of the user U is the line of sight of both eyes of the user U, and coincides with the direction of a straight line passing through the middle point of the line segment connecting the right eye and the left eye of the user U and the gaze point.

Next, with reference to FIG. 3, a method of acquiring information about the position and tilt of the HMD 110 will be described. FIG. 3 is a diagram showing the head of the user U wearing the HMD 110. As shown in FIG. Information about the position and tilt of the HMD 110 linked to the movement of the head of the user U wearing the HMD 110 can be detected by the position sensor 130 and/or the HMD sensor 114 mounted on the HMD 110 . As shown in FIG. 2 , three-dimensional coordinates (uvw coordinates) are defined around the head of the user U wearing the HMD 110 . The vertical direction in which the user U stands upright is defined as the v-axis, the direction orthogonal to the v-axis and passing through the center of the HMD 110 is defined as the w-axis, and the direction orthogonal to the v-axis and the w-axis is defined as the u-axis. Position sensor 130 and/or HMD sensor 114 can measure angles about each uvw axis (i.e., yaw angle indicating rotation about the v axis, pitch angle indicating rotation about the u axis, tilt determined by roll angle indicating rotation) is detected. The control device 120 determines angle information for controlling the visual axis of the virtual camera based on the detected angular change around each uvw axis.

Next, the hardware configuration of the control device 120 will be described with reference to FIG. FIG. 4 is a diagram showing the hardware configuration of the control device 120. As shown in FIG. As shown in FIG. 4 , the control device 120 includes a control section 121 , a storage section 123 , an I/O (input/output) interface 124 , a communication interface 125 and a bus 126 . The control unit 121 , the storage unit 123 , the I/O interface 124 and the communication interface 125 are communicably connected to each other via a bus 126 .

The control device 120 may be configured separately from the HMD 110 as a personal computer, smartphone, phablet, tablet, or wearable device, or may be built into the HMD 110 . Also, part of the functions of the control device 120 may be installed in the HMD 110 and the remaining functions of the control device 120 may be installed in another device separate from the HMD 110 .

The control unit 121 has a memory and a processor. The memory includes, for example, a ROM (Read Only Memory) storing various programs and the like, and a RAM (Random Access Memory) having a plurality of work areas storing various programs executed by the processor. The processor is, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit) and/or a GPU (Graphics Processing Unit). It is configured to execute various processes in cooperation with.

In particular, the control unit 121 may control various operations of the control device 120 by having the processor expand the control program on the RAM and execute the control program in cooperation with the RAM. The control unit 121 displays the visual field image on the display unit 112 of the HMD 110 based on the visual field image data. Thereby, the user U can be immersed in the virtual space.

The storage unit (storage) 123 is, for example, a storage device such as a HDD (Hard Disk Drive), an SSD (Solid State Drive), or a USB flash memory, and is configured to store programs and various data. The storage unit 123 may store a control program for causing a computer to execute at least part of the information processing method according to this embodiment, and a control program for realizing sharing of virtual space by a plurality of users. In addition, the storage unit 123 may store an authentication program for the user U and data related to various images and objects (for example, avatars and the like). Furthermore, a database including tables for managing various data may be constructed in the storage unit 123 .

The I/O interface 124 is configured to communicably connect the HMD 110, the position sensor 130, the external controller 320, the headphones 116, the face camera 117, and the microphone 118 to the control device 120, For example, it includes a USB (Universal Serial Bus) terminal, a DVI (Digital Visual Interface) terminal, an HDMI (registered trademark) (High-Definition Multimedia Interface) terminal, and the like. Note that the control device 120 may be wirelessly connected to each of the HMD 110 , the position sensor 130 , the external controller 320 , the headphones 116 , the face camera 117 and the microphone 118 .

The communication interface 125 is configured to connect the control device 120 to a communication network 3 such as a LAN (Local Area Network), a WAN (Wide Area Network), or the Internet. The communication interface 125 includes various wired connection terminals for communicating with external devices such as the server 2 via the communication network 3 and various processing circuits for wireless connection. It is configured to conform to the communication standards of

Next, processing for displaying a field-of-view image on the HMD 110 will be described with reference to FIGS. 5 to 8. FIG. FIG. 5 is a flowchart showing processing for displaying a field-of-view image on the HMD 110. As shown in FIG. FIG. 6 is an xyz space diagram showing an example of the virtual space 200. As shown in FIG. State (a) of FIG. 7 is a yx plan view of the virtual space 200 shown in FIG. State (b) of FIG. 7 is a zx plan view of the virtual space 200 shown in FIG. FIG. 8 is a diagram showing an example of the field-of-view image V displayed on the HMD 110. As shown in FIG.

As shown in FIG. 5, in step S1, the control unit 121 (see FIG. 4) generates virtual space data representing a virtual space 200 including a virtual camera 300 and various objects. As shown in FIG. 6, the virtual space 200 is defined as an omnisphere centered on a center position 210 (only the upper half of the sphere is shown in FIG. 6). In the virtual space 200, an xyz coordinate system is set with the center position 210 as the origin. Virtual camera 300 defines visual axis L for specifying field-of-view image V (see FIG. 8) displayed on HMD 110 . The uvw coordinate system that defines the field of view of the virtual camera 300 is determined so as to interlock with the uvw coordinate system defined around the head of the user U in the real space. Further, the control unit 121 may move the virtual camera 300 within the virtual space 200 in conjunction with movement of the user U wearing the HMD 110 in the physical space.

Next, in step S2, the control unit 121 specifies the field of view CV (see FIG. 7) of the virtual camera 300. FIG. Specifically, the control unit 121 acquires information about the position and tilt of the HMD 110 based on data indicating the state of the HMD 110 transmitted from the position sensor 130 and/or the HMD sensor 114 . Next, the control unit 121 identifies the position and orientation of the virtual camera 300 within the virtual space 200 based on the information regarding the position and tilt of the HMD 110 . Next, the control unit 121 determines the visual axis L of the virtual camera 300 from the position and orientation of the virtual camera 300, and specifies the visual field CV of the virtual camera 300 from the determined visual axis L. FIG. Here, the visual field CV of the virtual camera 300 corresponds to a partial area of the virtual space 200 visible to the user U wearing the HMD 110 (in other words, a partial area of the virtual space 200 displayed on the HMD 110). ). Also, the field of view CV is divided into a first region CVa set as an angular range of a polar angle α centered on the visual axis L on the xy plane shown in state (a) of FIG. and a second region CVb set as an angle range of an azimuth angle β centered on the visual axis L on the xz plane shown. Note that the control unit 121 identifies the line of sight of the user U based on the data indicating the line of sight of the user U transmitted from the gaze sensor 140, and based on information about the identified line of sight of the user U and the position and inclination of the HMD 110, , the orientation of the virtual camera 300 (the visual axis L of the virtual camera) may be determined. Also, as will be described later, the control unit 121 may determine the orientation of the face of the avatar of the user U based on information regarding the position and tilt of the HMD 110 .

Thus, the control unit 121 can specify the field of view CV of the virtual camera 300 based on the data from the position sensor 130 and/or the HMD sensor 114 . Here, when the user U wearing the HMD 110 moves, the control unit 121 updates the field of view CV of the virtual camera 300 based on the data indicating the movement of the HMD 110 transmitted from the position sensor 130 and/or the HMD sensor 114. be able to. That is, the control unit 121 can update the field of view CV according to the motion of the HMD 110 . Similarly, when the line of sight of the user U changes, the control unit 121 may update the field of view CV of the virtual camera 300 based on data indicating the line of sight of the user U transmitted from the gaze sensor 140 . In other words, the control unit 121 may change the field of view CV according to the change in the line of sight of the user U.

Next, in step S<b>3 , the control unit 121 generates visual field image data representing the visual field image V to be displayed on the display unit 112 of the HMD 110 . Specifically, the control unit 121 generates visual field image data based on the virtual space data defining the virtual space 200 and the visual field CV of the virtual camera 300 .

Next, in step S4, the control section 121 displays the visual field image V on the display section 112 of the HMD 110 based on the visual field image data (see FIG. 7). In this way, the visual field CV of the virtual camera 300 changes according to the movement of the user U wearing the HMD 110, and the visual field image V displayed on the display unit 112 of the HMD 110 changes. 200 can be immersed.

Note that the virtual camera 300 may include a left-eye virtual camera and a right-eye virtual camera. In this case, the control unit 121 generates left-eye view image data representing a left-eye view image based on the virtual space data and the view of the left-eye virtual camera. Furthermore, the control unit 121 generates right-eye visual field image data representing a right-eye visual field image based on the virtual space data and the visual field of the right-eye virtual camera. After that, the control unit 121 displays the left eye view image on the left eye display unit based on the left eye view image data, and displays the right eye view image on the right eye display unit based on the right eye view image data. display. In this manner, the user U can visually recognize the field-of-view image three-dimensionally due to the parallax between the left-eye field-of-view image and the right-eye field-of-view image. Note that the virtual camera may be placed at the eye position of the avatar operated by the user, as will be described later. For example, a left-eye virtual camera may be placed on the avatar's left eye, while a right-eye virtual camera may be placed on the avatar's right eye.

Further, the processing of steps S1 to S4 shown in FIG. 5 may be executed for each frame (still image forming a moving image). For example, when the moving image has a frame rate of 90 fps, the processes of steps S1 to S4 may be repeatedly performed at intervals of ΔT=1/90 (seconds). As described above, since the processing of steps S1 to S4 is repeatedly executed at predetermined intervals, the field of view of the virtual camera 300 is updated according to the operation of the HMD 110, and the field of view image V displayed on the display unit 112 of the HMD 110 is displayed. is updated.

Next, the hardware configuration of the server 2 shown in FIG. 1 will be described with reference to FIG. FIG. 9 is a diagram showing the hardware configuration of the server 2. As shown in FIG. As shown in FIG. 9 , the server 2 includes a control section 23 , a storage section 22 , a communication interface 21 and a bus 24 . The control unit 23, the storage unit 22, and the communication interface 21 are communicably connected to each other via the bus 24. FIG. The control unit 23 includes a memory and a processor. The memory is composed of, for example, ROM and RAM, and the processor is composed of, for example, CPU, MPU and/or GPU.

The storage unit (storage) 22 is, for example, a large-capacity HDD or the like. The storage unit 22 may store a control program for causing a computer to execute at least part of the information processing method according to the present embodiment, and a control program for realizing sharing of a virtual space by a plurality of users. In addition, the storage unit 22 may store user management information for managing each user and data related to various images and objects (for example, avatars and the like). A communication interface 21 is configured to connect the server 2 to the communication network 3 .

Next, an example of processing for synchronizing the movements of the avatars 4A and 4B between the user terminals 1A and 1B will be described with reference to FIGS. State (a) of FIG. 10 is a diagram showing a virtual space 200A provided to user A. FIG. State (b) of FIG. 10 is a diagram showing a virtual space 200B provided to user B. FIG. FIG. 11 is a sequence diagram for explaining an example of processing for synchronizing the movements of the avatars 4A and 4B between the user terminals 1A and 1B. In this description, as a prerequisite, as shown in FIG. 10, an avatar 4A (first avatar) associated with the user terminal 1A (user A) and an avatar 4B (first avatar) associated with the user terminal 1B (user B) 2 avatars) share the same virtual space. In other words, it is assumed that user A and user B share one virtual space via communication network 3 .

As shown in state (a) of FIG. 10, user A's virtual space 200A includes an avatar 4A and an avatar 4B. The avatar 4A is operated by the user A and interlocks with the user A's actions. The avatar 4A has a left hand (part of the virtual body of the avatar 4A) linked to the action of the controller 320L of the user terminal 1A (the action of the left hand of the user A), and the action of the controller 320R of the user terminal 1A (the action of the right hand of the user A). It has a right hand that is interlocked with movement) and a face whose expression is interlocked with the facial expression of user A. The face of the avatar 4A has multiple face parts (eg, eyes, eyebrows, mouth, etc.). The avatar 4B is operated by the user B and interlocks with the user B's actions. The avatar 4B has a left hand linked to the motion of the controller 320L of the user terminal 1B indicating the motion of the left hand of the user B, a right hand linked to the motion of the controller 320R of the user terminal 1B indicating the motion of the right hand of the user B, and facial expressions. It has a face that works with the facial expression of user B. The face of the avatar 4B has multiple face parts (eg, eyes, eyebrows, mouth, etc.).

In addition, in the virtual space 200A provided to the user A, it is assumed that the avatar 4A may not be visualized. In this case, the avatar 4A placed in the virtual space 200A includes at least the virtual camera 300 linked with the movement of the HMD 110 of the user terminal 1A.

Also, the positions of the avatars 4A and 4B may be identified according to the positions of the HMDs 110 of the user terminals 1A and 1B. Similarly, the orientation of the faces of the avatars 4A, 4B may be identified according to the tilt of the HMDs 110 of the user terminals 1A, 1B. Furthermore, hand motions of the avatars 4A and 4B may be identified according to motions of the external controllers 320 of the user terminals 1A and 1B. In particular, the actions of the left hands of the avatars 4A and 4B are specified according to the actions of the controllers 320L of the user terminals 1A and 1B, and the actions of the right hands of the avatars 4A and 4B are specified according to the actions of the controllers 320R of the user terminals 1A and 1B. may be specified. Further, the facial expressions of the avatars 4A and 4B may be specified according to the facial expressions (states) of the users A and B.

Also, a virtual camera 300 (FIG. 6) may be arranged in each eye of the avatars 4A and 4B. In particular, a left-eye virtual camera may be arranged for the left eyes of the avatars 4A and 4B, and a right-eye virtual camera may be arranged for the right eyes of the avatars 4A and 4B. In the following description, it is assumed that the virtual camera 300 is positioned over the eyes of the avatars 4A and 4B. Therefore, it is assumed that the field of view CV of the avatar 4A matches the field of view CV (see FIG. 7) of the virtual camera 300 placed on the avatar 4A. Similarly, it is assumed that the field of view CV of the avatar 4B matches the field of view CV (see FIG. 7) of the virtual camera 300 placed on the avatar 4B.

As shown in state (b) of FIG. 10, user B's virtual space 200B includes avatar 4A and avatar 4B. The positions of avatars 4A and 4B within virtual space 200A may correspond to the positions of avatars 4A and 4B within virtual space 200B.

In addition, in the virtual space 200B provided to the user B, it is assumed that the avatar 4B may not be visualized. In this case, the avatar 4B placed in the virtual space 200B includes at least the virtual camera 300 linked with the movement of the HMD 110 of the user terminal 1B.

Next, referring to FIG. 11, the control unit 121 of the user terminal 1A generates voice data of the user A in step S10. For example, when the user A inputs voice into the microphone 118 (voice input unit) of the user terminal 1A, the microphone 118 generates voice data representing the input voice. After that, the microphone 118 transmits the generated audio data to the control section 121 via the I/O interface 124 .

Next, in step S11, the control unit 121 of the user terminal 1A generates control information for the avatar 4A, and then controls the generated control information for the avatar 4A and voice data indicating the voice of the user A (voice data of the user A). ) to the server 2. After that, the control unit 23 of the server 2 receives the control information of the avatar 4A and the voice data of the user A from the user terminal 1A (step S12).

Here, the control information for the avatar 4A is information necessary for controlling the action of the avatar 4A. The control information of the avatar 4A includes information on the position of the avatar 4A (position information), information on the direction of the face of the avatar 4A (face orientation information), and information on the state of the hands (left hand and right hand) of the avatar 4A (hand information). ) and information (face information) on the facial expression of the avatar 4A.

The face information of the avatar 4A has information indicating the states of a plurality of face parts. The information indicating the state of a plurality of face parts includes information (eye information) indicating the state of the eyes of the avatar 4A (shape of the white of the eye and the position of the iris relative to the white of the eye, etc.) and information of the eyebrows of the avatar 4A (the position and shape of the eyebrow). etc.) and information (mouth information) indicating the state of the mouth of the avatar 4A (position and shape of the mouth, etc.).

Specifically, the eye information of the avatar 4A includes information indicating the state of the left eye of the avatar 4A and information indicating the state of the right eye of the avatar 4A. The eyebrow information of the avatar 4A has information indicating the state of the left eyebrow of the avatar 4A and information indicating the state of the right eyebrow of the avatar 4A.

The control unit 121 of the user terminal 1A acquires an image showing the eyes (left eye and right eye) and the eyebrows (left eyebrow and right eyebrow) of the user A from the face camera 113 mounted on the HMD 110, and then the acquired image and the state of user A's eyes (left eye and right eye) and eyebrows (left eyebrow and right eyebrow) based on a predetermined image processing algorithm. Next, the control unit 121 generates information indicating the state of the eyes (left eye and right eye) of the avatar 4A and the state of the eyebrows (left eyebrow and right eyebrow) of the avatar 4A based on the specified eye and eyebrow states of the user A. Each generates information indicating the state.

Similarly, the control unit 121 of the user terminal 1A acquires an image showing the mouth of the user A and its surroundings from the face camera 117, and then, based on the acquired image and a predetermined image processing algorithm, the mouth of the user A. identify the state of Next, the control unit 121 generates information indicating the state of the mouth of the avatar 4A based on the identified state of the mouth of the user A. FIG.

In this way, the control unit 121 of the user terminal 1A controls the eye information of the avatar 4A corresponding to the eyes of the user A, the eyebrow information of the avatar 4A corresponding to the eyebrows of the user A, and the avatar 4A corresponding to the mouth of the user A. It is possible to generate mouth information of

In addition, the control unit 121 uses the image captured by the face camera 113, the image captured by the face camera 117, and a plurality of types of faces stored in the storage unit 123 or memory based on a predetermined image processing algorithm. The user A's facial expression (eg, smiling face) may be specified from among the facial expressions (eg, smiling face, sad face, expressionless face, angry face, surprised face, embarrassed face, etc.). After that, the control unit 121 can generate facial information indicating the facial expression (for example, smile) of the avatar 4A based on the specified facial expression of the user A. In this case, a plurality of types of facial expressions of the avatar 4A and facial expression data including facial information of a plurality of avatars 4A each associated with one of the plurality of types of facial expressions are stored in the storage unit 123. may

For example, when the facial expression of the avatar 4A is identified as a smiling face, the control unit 121 acquires facial information indicating the smiling face of the avatar 4A based on the facial expression (smiling face) of the avatar 4A and facial expression data. Here, the face information indicating the smiling face of the avatar 4A includes eye information, eyebrow information and mouth information of the avatar 4A when the facial expression of the avatar 4A is smiling.

Next, in step S<b>13 , the control unit 121 of the user terminal 1</b>B generates control information for the avatar 4</b>B and transmits the generated control information for the avatar 4</b>B to the server 2 . After that, the control unit 23 of the server 2 receives the control information of the avatar 4B from the user terminal 1B (step S14).

Here, the control information of the avatar 4B is information necessary for controlling the action of the avatar 4B. The control information of the avatar 4B includes information on the position of the avatar 4B (position information), information on the direction of the face of the avatar 4B (face orientation information), and information on the state of the hands (left hand and right hand) of the avatar 4B (hand information). ) and information (face information) on the facial expression of the avatar 4B.

The face information of the avatar 4B has information indicating the state of a plurality of face parts. The information indicating the state of the plurality of face parts includes information (eye information) indicating the state of the eyes of the avatar 4B (shape of the white of the eye and the position of the iris relative to the white of the eye, etc.) and information of the eyebrows of the avatar 4B (the position and shape of the eyebrow). etc.) and information (mouth information) indicating the state of the mouth of the avatar 4B (position and shape of the mouth, etc.). The method of obtaining the face information of the avatar 4B is the same as the method of obtaining the face information of the avatar 4A.

Next, the server 2 transmits the control information of the avatar 4B to the user terminal 1A (step S15), and transmits the control information of the avatar 4A and the voice data of the user A to the user terminal 1B (step S19). After that, after receiving the control information for the avatar 4B in step S16, the control unit 121 of the user terminal 1A updates the states of the avatars 4A and 4B based on the control information for the avatars 4A and 4B, and then controls the virtual space 200A. (Refer to state (a) in FIG. 10) is updated (step S17).

Specifically, the control unit 121 of the user terminal 1A updates the positions of the avatars 4A and 4B based on the position information of the avatars 4A and 4B. The control unit 121 updates the face directions of the avatars 4A and 4B based on the face direction information of the avatars 4A and 4B. The control unit 121 updates the hands of the avatars 4A, 4B based on the hand information of the avatars 4A, 4B. The control unit 121 updates the facial expressions of the avatars 4A and 4B based on the facial information of the avatars 4A and 4B. Thus, the virtual space data representing the virtual space 200A including the updated avatars 4A and 4B are updated.

After that, the control unit 121 of the user terminal 1A identifies the field of view CV of the avatar 4A (virtual camera 300) according to the position and tilt of the HMD 110, and then sets the updated virtual space data to the field of view CV of the avatar 4A. Based on this, the visual field image displayed on the HMD 110 is updated (step S18).

On the other hand, after receiving the control information of the avatar 4A and the voice data of the user A in step S20, the control unit 121 of the user terminal 1B updates the states of the avatars 4A and 4B based on the control information of the avatars 4A and 4B. Above, the virtual space data representing the virtual space 200B (see state (b) in FIG. 10) is updated (step S21).

Specifically, the control unit 121 of the user terminal 1B updates the positions of the avatars 4A and 4B based on the position information of the avatars 4A and 4B. The control unit 121 updates the face directions of the avatars 4A and 4B based on the face direction information of the avatars 4A and 4B. The control unit 121 updates the hands of the avatars 4A, 4B based on the hand information of the avatars 4A, 4B. The control unit 121 updates the facial expressions of the avatars 4A and 4B based on the facial information of the avatars 4A and 4B. Thus, the virtual space data representing the virtual space containing the updated avatars 4A and 4B are updated.

After that, the control unit 121 of the user terminal 1B identifies the visual field CV of the avatar 4B (virtual camera 300) according to the position and tilt of the HMD 110, and then sets the updated virtual space data to the visual field CV of the avatar 4B. Based on this, the visual field image displayed on the HMD 110 is updated (step S22).

After that, the control unit 121 of the user terminal 1B receives the voice data of the user A, the information about the position of the avatar 4A included in the control information of the avatar 4A, the information about the position of the avatar 4B, and a predetermined voice processing algorithm. Based on this, the voice data of user A is processed. After that, the control unit 121 transmits the processed audio data to the headphone 116 (audio output unit), and the headphone 116 outputs the voice of the user A based on the processed audio data (step S23). In this way, voice chat (VR chat) between users (between avatars) can be realized in the virtual space.

In this embodiment, after the user terminals 1A and 1B transmit the control information of the avatar 4A and the control information of the avatar 4B to the server 2 respectively, the server 2 transmits the control information of the avatar 4A to the user terminal 1B, while the avatar 4B to the user terminal 1A. Thus, it is possible to synchronize the movements of the avatars 4A and 4B between the user terminal 1A and the user terminal 1B.

Next, mainly referring to FIGS. 12 and 14, an information processing method according to the present embodiment when user A's HMD 110 is removed will be described. FIG. 12 is an example of a flowchart for explaining the information processing method according to this embodiment when the user A removes the HMD 110 . FIG. 14 is a diagram showing an example of a virtual space 200B provided to user B for explaining the information processing method according to this embodiment. In the flowchart shown in FIG. 12, the control unit 121 of the user terminal 1A transmits the control information of the avatar 4A to the server 2, and the control unit 121 of the user terminal 1B transmits the control information of the avatar 4B to the server 2. are omitted for convenience of explanation.

In this embodiment, as a precondition, as shown in FIG. 14, the avatar 4A and the avatar 4B shall share the same virtual space. In other words, it is assumed that users A and B share one virtual space via the communication network 3 .

User B's virtual space 200B includes an avatar 4A and an avatar 4B. The control unit 121 of the user terminal 1B updates virtual space data indicating the virtual space 200B. In this description, illustration of the virtual space 200A provided to the avatar 4A is omitted for convenience of description.

Referring to FIG. 12, in step S30, the control unit 121 of the user terminal 1A determines whether or not the user A has removed the HMD 110. FIG. In this respect, the control unit 121 may determine whether the user A has removed the HMD 110 based on the tilt information of the HMD 110 transmitted from the HMD sensor 114 or the event information transmitted from the wearing sensor 115 . HMD 114 and wearing sensor 115 function as sensors configured to detect whether HMD 110 is worn.

A process of determining whether or not user A has removed the HMD 110 using the HMD sensor 114 will be described with reference to state (a) of FIG. 13 . State (a) of FIG. 13 is an example of a flowchart for explaining processing for determining whether user A has removed the HMD 110 using the HMD sensor 114 . As shown in state (a) of FIG. 13, the control unit 121 receives tilt information indicating the tilt (roll angle, yaw angle, pitch angle) of the HMD 110 from the HMD sensor 114 (step S301). Next, the control unit 121 determines whether the tilt of the HMD 110 is greater than a predetermined tilt based on the received tilt information (S302). Here, the predetermined inclination may be appropriately set by the user. Assume that the tilt of HMD 110 does not exceed a predetermined tilt while user A is wearing HMD 110 . That is, while the user A is wearing the HMD 110, the inclination of the HMD 110 is less than or equal to the predetermined inclination. may exceed. A predetermined inclination is set from such a point of view.

For example, when the control unit 121 determines that the pitch angle θ of the HMD 110 is larger than the predetermined angle θth as an example of the inclination of the HMD 110 (YES in step S302), the user A removed the HMD 110 (in other words, , the user A is not wearing the HMD 110), and then the process of step S31 is executed. On the other hand, when the control unit 121 determines that the pitch angle θ of the HMD 110 is equal to or less than the predetermined angle θth (NO in step S302), the control unit 121 identifies that the user A is wearing the HMD 110, and then controls the HMD sensor. It waits until it receives the tilt information of the HMD 110 from 114 .

Next, the process of determining whether user A has removed the HMD 110 using the wearing sensor 115 will be described with reference to state (b) of FIG. 13 . State (b) of FIG. 13 is an example of a flowchart for explaining processing for determining whether user A has removed the HMD 110 using the wearing sensor 115 . As shown in state (b) of FIG. 13, the control unit 121 determines whether event information has been received from the wearing sensor 115 (step S303). Here, the event information transmitted from the wearing sensor 115 is information indicating that the state of the HMD 110 of the user terminal 1A has changed between the wearing state and the non-wearing state. For example, when the user A removes the HMD 110 (when the state of the HMD 110 transitions from the wearing state to the non-wearing state), the wearing sensor 115 outputs event information. Also, when the user A wears the HMD 110 (when the state of the HMD 110 transitions from the non-wearing state to the wearing state), the wearing sensor 115 outputs event information.

When the control unit 121 determines that the event information has been received from the wearing sensor 115 (YES in step S303), the control unit 121 indicates that the user A has removed the HMD 110 (in other words, the user A has not worn the HMD 110). After specifying, the process of step S31 is executed. On the other hand, if the control unit 121 determines that the event information has not been received from the wearing sensor 115 (NO in step S303), it specifies that the user A is wearing the HMD 110, and then receives the event information. Wait until received from sensor 115 .

Next, returning to FIG. 12, when the control unit 121 determines that the user A has removed the HMD 110 (in other words, the user A does not wear the HMD 110) (step S30 YES), the user A removes the HMD 110 is not worn (non-wearing information) (step S31). After that, the control unit 121 transmits non-wearing information to the server 2 via the communication network 3 (step S32).

Next, after receiving the non-wearing information from the user terminal 1A, the server 2 transmits the non-wearing information to the user terminal 1B via the communication network 3 (step S33). Thereafter, after receiving the non-wearing information from the server 2, the control unit 121 of the user terminal 1B changes the facial expression of the avatar 4A placed in the virtual space 200B to the default facial expression (in other words, the default facial expression). facial expression) (step S34). Here, the default facial expression of the avatar 4A refers to the facial expression of the avatar 4A in the initial state before the update of the facial expression of the avatar 4A is started. When the default facial expression is a smiling face, the facial expression of the avatar 4A is set to a smiling face, as shown in FIG. Also, the control unit 121 updates the action of the avatar 4B (including the facial expression of the avatar 4B) based on the control information of the avatar 4B.

In step S<b>34 , the control unit 121 may set the facial expression of the avatar 4</b>A to an expression preselected by the user A as a mode indicating that the user A is not wearing the HMD 110 . In this case, the information about the facial expression of the avatar 4A selected by the user A may be transmitted from the user terminal 1A to the user terminal 1B and then stored in advance in the storage unit 123 of the user terminal 1B.

Next, as shown in FIG. 14, the control unit 121 generates a balloon object 42A associated with the avatar 4A in the virtual space 200B. The speech balloon object 42A displays information indicating that the user A is not wearing the HMD 110 (for example, "User A is away.").

Note that the control unit 121 may generate a telop object 43A in the virtual space 200B instead of the balloon object 42A (see FIG. 15). As shown in FIG. 15, the telop object 43A displays information indicating that the user A is not wearing the HMD 110 (for example, "User A is away."). The telop object 43A may be arranged near the avatar 4A, or may be linked with the movement of the avatar 4B's visual field CV so that it is always arranged within the avatar 4B's visual field CV. Furthermore, a telop indicating information indicating that user A is not wearing HMD 110 may be superimposed on the field-of-view image displayed on HMD 110 .

After that, the control unit 121 updates the virtual space data indicating the virtual space 200B, and updates the field of view CV of the avatar 4B according to the movement of the HMD 110 of the user terminal 1B. Next, the control unit 121 updates the visual field image data based on the updated virtual space data and the updated visual field CV of the avatar 4B, and then displays the visual field image on the HMD 110 based on the updated visual field image data. (step S36).

On the other hand, after transmitting the non-wearing information to the server 2 via the communication network 3, the control unit 121 of the user terminal 1A shifts from the active mode to the sleep mode (step S37). In this case, it is preferable that the mode of the control unit 121 is the sleep mode, and the mode of the sensor (the wearing sensor 115 or the HMD sensor 114) is the active mode.

According to this embodiment, when the user A is not wearing the HMD 110, the user terminal 1B presents to the user B that the user A is not wearing the HMD 110. FIG. In particular, according to this embodiment, when the user A is not wearing the HMD 110, the information indicating that the user A is not wearing the HMD 110 is visualized on the field-of-view image displayed on the HMD 110 of the user terminal 1B. be. In this regard, the avatar 4A whose facial expression is set to the default facial expression and the balloon object 42A are visualized on the field-of-view image displayed on the HMD 110 of the user terminal 1B.

In this way, while user B is communicating with user A in virtual space 200B, by visually recognizing the default facial expression (for example, a smile) of avatar 4A and the information displayed in balloon object 42A, It can be easily recognized that the user A is not wearing the HMD 110 . Therefore, it is possible to provide the user with a rich virtual experience.

In particular, if user B does not understand that user A is not wearing the HMD 110, user B may feel uncomfortable with user A who does not show any reaction. In this way, by making user B understand that user A is not wearing the HMD 100, it is possible to avoid a situation in which user B feels uncomfortable with user A who does not show any reaction (for example, the uncanny valley effect). It becomes possible to

Further, according to the present embodiment, it is determined that the user A is not wearing the HMD 110 according to information (tilt information or event information) transmitted from the sensor (the HMD sensor 114 or the wearing sensor 115) of the user terminal 1A. identified. Thereafter, non-wearing information indicating that the user A is not wearing the HMD 110 is transmitted to the server 2 . In this manner, it is possible to automatically identify that the user A is not wearing the HMD 110 according to the information output from the sensor of the user terminal 1A.

In this embodiment, the user terminal 1A transmits the non-wearing information to the user terminal 1B via the server 2, but instead of the non-wearing information, the tilt information transmitted from the HMD sensor 114 is sent to the user terminal. It may be sent to 1B. In this case, the control unit 121 of the user terminal 1B may specify that the user A is not wearing the HMD 110 based on the received tilt information, and then execute the processing defined in steps S34 to S36. good.

Next, mainly referring to FIGS. 16 and 17, an information processing method according to the present embodiment when user A who has removed the HMD 110 puts on the HMD 110 again will be described. FIG. 16 is an example of a flowchart for explaining the information processing method according to this embodiment when the user A wears the HMD 110 again. FIG. 17 is a diagram showing an example of a virtual space 200B provided to user B for explaining the information processing method according to this embodiment.

Referring to FIG. 16, in step S40, the control unit 121 of the user terminal 1A determines whether the user A has put on the HMD 110 again. In this regard, the control unit 121 may determine whether the user A has worn the HMD 110 again based on the tilt information of the HMD 110 transmitted from the HMD sensor 114 or the event information transmitted from the wearing sensor 115. . For example, when determining that the event information has been received from the wearing sensor 115, the control unit 121 specifies that the user A has worn the HMD 110, and then executes the process of step S41. On the other hand, when the control unit 121 determines that the event information has not been received from the wearing sensor 115 , the control unit 121 identifies that the user A is not wearing the HMD 110 and then waits until the event information is received from the wearing sensor 115 . stand by.

When the determination result of step S40 is YES, the control unit 121 transitions from the sleep mode to the active mode (step S41). Next, the control unit 121, which has shifted to the active mode, generates control information for the avatar 4A and also generates wearing information indicating that the user A has worn the HMD 110 (step S42). After that, the control unit 121 transmits the control information and the wearing information of the avatar 4A to the server 2 via the communication network 3 (step S43).

Next, after receiving the control information and the mounting information of the avatar 4A from the user terminal 1A, the server 2 transmits the control information and the mounting information of the avatar 4A to the user terminal 1B via the communication network 3 (step S44). After that, the control unit 121 of the user terminal 1B updates the action of the avatar 4A based on the control information of the avatar 4A, and updates the action of the avatar 4B based on the control information of the avatar 4B (step S45). In particular, the control unit 121 updates the facial expression of the avatar 4A based on the facial information of the avatar 4A included in the control information of the avatar 4A, and updates the facial expression of the avatar 4B based on the facial information of the avatar 4B included in the control information of the avatar 4B. to update the facial expression of the avatar 4B.

Next, as shown in FIG. 17, the control unit 121 generates a balloon object 44A associated with the avatar 4A in the virtual space 200B (step S46). The balloon object 44A displays information indicating that the user A has put on the HMD 110 (for example, "User A has returned to his seat.").

After that, the control unit 121 updates the virtual space data indicating the virtual space 200B, and updates the field of view CV of the avatar 4B according to the movement of the HMD 110 of the user terminal 1B. Next, the control unit 121 updates the visual field image data based on the updated virtual space data and the updated visual field CV of the avatar 4B, and then displays the visual field image on the HMD 110 based on the updated visual field image data. (step S47).

According to this embodiment, when the user A removes the HMD 110 and then puts it on again, the user terminal 1B presents to the user B that the user A is wearing the HMD 110 . In this way, user B can easily recognize that user A has put on the HMD 110 again by visually recognizing the updated facial expression of avatar 4A and balloon object 44A. Therefore, it is possible to provide the user with a rich virtual experience.

In addition, in the description of the present embodiment, information indicating that the user A is wearing or not wearing the HMD 110 is visualized on the field-of-view image, but the present embodiment is not limited to this. For example, voice guidance indicating that user A is not wearing the HMD 110 (for example, "user A has left the seat") or voice guidance indicating that user A is wearing the HMD 110 (for example, "user A is I have returned to my seat.") may be output from the headphone 116 (audio output unit) of the user terminal 1B. In this case, the control unit 121 of the user terminal 1A transmits the non-wearing information (or wearing information) and the voice guidance data to the user terminal 1B via the server 2 .

Also, in the description of the present embodiment, it is assumed that the virtual space data representing the virtual space 200B is updated on the user terminal 1B side, but the virtual space data may be updated on the server 2 side. Further, it is assumed that the view image data corresponding to the view image is updated on the user terminal 1B side, but the view image data may be updated on the server 2 side. In this case, the user terminal 1</b>B displays the visual field image on the HMD 110 based on the visual field image data transmitted from the server 2 .

Also, the order of processing defined in each step shown in FIGS. 12 and 16 is merely an example, and the order of these steps can be changed as appropriate.

(Modification 1)
Next, Modified Example 1 of the information processing method according to the present embodiment when user A's HMD 110 is removed will be described with reference to FIGS. 18 to 25 . FIG. 18 is an example of a flowchart for explaining an information processing method according to Modification 1. As shown in FIG. 19, 20, 22 and 23 are diagrams showing an example of a virtual space 200B provided to user B for explaining the information processing method according to Modification 1. FIG. FIG. 21 is a diagram showing a display example of the user terminal 5 associated with the user A for explaining the information processing method according to Modification 1. As shown in FIG. FIG. 24 is an example of a flowchart for explaining the information processing method according to Modification 1 when User A has not worn the HMD within a certain period of time after receiving the already-read information.

Referring to FIG. 18, the control unit 121 of the user terminal 1A determines whether or not the user A has removed the HMD 110 (step S50). The process of determining whether or not the user A has removed the HMD 110 using the HMD sensor 114 is as described in the above embodiment, so detailed description thereof will be omitted.

In step S50, if it is determined that user A has removed the HMD 110 (in other words, user A is not wearing the HMD 110) (YES in step S50), the control unit 121 controls whether the user A has worn the HMD 110. Information (non-wearing information) indicating that the device is not mounted is generated (step S51). After that, the control unit 121 transmits non-wearing information to the server 2 via the communication network 3 (step S52). In step S<b>51 , the control unit 121 may generate positional relationship information indicating the positional relationship between the HMD 110 and the external controller 320 and transmit the positional relationship information to the server 2 together with the non-wearing information.

Next, after receiving the non-wearing information from the user terminal 1A, the server 2 transmits the non-wearing information to the user terminal 1B via the communication network 3 (step S53).

On the other hand, after transmitting the non-wearing information to the server 2 via the communication network 3, the control unit 121 of the user terminal 1A shifts from the active mode to the sleep mode (step S54). In this case, it is preferable that the mode of the control unit 121 is the sleep mode, and the mode of the sensor (the wearing sensor 115 or the HMD sensor 114) is the active mode.

Next, based on the non-wearing information received from the server 2, the control unit 121 of the user terminal 1B changes the posture of the avatar 4A placed in the virtual space 200B to the default posture (in other words, default posture). (step S55). Modification 1 will be described on the assumption that the virtual space is a room in which to search for a multiplayer partner, but the present invention is not limited to this. Here, the default posture of the avatar 4A refers to the posture of the avatar 4A in its initial state. If the default posture is lying down, the posture of the avatar 4A is set to lying on the floor as shown in FIG.

In step S55, the control unit 121 may set the posture of the avatar 4A to a posture selected in advance by the user A as a mode indicating that the user A is not wearing the HMD 110. FIG. In this case, the information about the posture of the avatar 4A selected by the user A may be transmitted from the user terminal 1A to the user terminal 1B and then stored in advance in the storage unit 123 of the user terminal 1B.

Further, when the user 1B has received the positional relationship information indicating the positional relationship between the HMD 110 and the external controller 320, in step S55, the control unit 121 adjusts the posture of the avatar 4A based on the positional relationship information. May be set. That is, when the user A is not wearing the HMD 110, the HMD 110 and the external controller 320 are often placed on the floor or desk in the physical space. In this case, avatar 4</b>A in virtual space 200</b>B can be set to take a posture as if it were squashed on the floor, based on the positional relationship information between HMD 110 and external controller 320 .

Next, the control unit 121 of the user terminal 1B sets the facial expression of the avatar 4A placed in the virtual space 200B to the default facial expression (in other words, the default facial expression) (step S56). ). If the default facial expression is eyes closed, the facial expression of avatar 4A is set to eyes closed, as shown in FIG.

In step S<b>56 , the control unit 121 may set the facial expression of the avatar 4</b>A to an expression preselected by the user A as a mode indicating that the user A is not wearing the HMD 110 . In this case, the information about the facial expression of the avatar 4A selected by the user A may be transmitted from the user terminal 1A to the user terminal 1B and then stored in advance in the storage unit 123 of the user terminal 1B.

Next, the control unit 121 updates the virtual space data indicating the virtual space 200B, and updates the field of view CV of the avatar 4B according to the movement of the HMD 110 of the user terminal 1B. Next, the control unit 121 updates the visual field image data based on the updated virtual space data and the updated visual field CV of the avatar 4B, and then displays the visual field image on the HMD 110 based on the updated visual field image data. (step S57).

In this way, since the posture and face of avatar 4A are displayed in the default setting state (the state of lying down with eyes closed) in virtual space 200B, user B can see that user A is wearing HMD 110. Therefore, it can be easily recognized that the user is not participating in the content provided to the virtual space 200B.

Next, after updating the field-of-view image in step S56, the control unit 121 of the user terminal 1B causes the avatar 4B associated with the user B to perform a specific action ( An example of the first action) is executed (step S58).

The specific action performed by the avatar 4B on the avatar 4A includes, for example, an action involving physical interference between the avatar 4A and the avatar 4B. Physical interference between avatars 4A and 4B is detected using colliders (collision objects) to detect collisions with other objects. Although illustration is omitted, the colliders are associated with the bodies of the avatars 4A and 4B, and are used for collision determination (hit determination) between the avatars 4A and 4B. For example, when the virtual hand 400 of the avatar 4B with a collider touches the avatar 4A with a collider, it is determined that the avatars 4B and 4A have come into contact. A collider may be set to the entire body area of the avatars 4A and 4B, or may be set to specific parts of the body of the avatars 4A and 4B (for example, hands, feet, head, shoulders, waist, etc.). In particular, it is preferable that a collider is set for each fingertip in the virtual hand 400 and the hit determination is independently detected for each finger. This makes it possible to accurately determine the collision between another object and the finger. Note that the collider is configured as a transparent body and is not displayed in the field-of-view image V (FIG. 8).

In the above example, the control unit 121 specifies the hit determination of the avatar 4B with respect to the avatar 4A based on the motion of the avatar 4B (for example, the motion of the virtual hand 400), but the present invention is not limited to this example. For example, the control unit 121 may specify hit determination for the avatar 4A based on the movement of the user B (for example, movement of the hand of the user B).

Specific actions including actions involving physical interference include, for example, the action of the avatar 4B touching the avatar 4A using the virtual hand 400, the action of grabbing and shaking the body of the avatar 4A, and the action of hitting the avatar 4A (for example, slaps on the face or shoulders of the avatar 4A). The specific action may be a motion including repetitive motion, that is, a motion of contacting the avatar 4A during the repetitive motion of the virtual hand 400 . In this way, even if the avatar 4B accidentally touches the avatar 4A due to an erroneous operation by the user B, it is possible to detect that the action is not a specific action, thereby preventing erroneous detection of the motion of the virtual hand 400. can be done.

For example, as shown in FIG. 20 , when the avatar 4B grips the body (for example, the trunk) of the avatar 4A with the virtual hand 400 and the repetitive motion of the virtual hand 400 is performed, the control unit 121 Identify that a shaking motion has been performed as the action. Grasping (selection) of the body of avatar 4A by virtual hand 400 is performed by user B operating a key (not shown) of external controller 320 while the collider of virtual hand 400 and the collider of the body of avatar 4A are in contact, for example. It can be realized by pressing (pressing). This shaking motion may include a touching motion to the avatar 4A in the course of the repetitive motion of the virtual hand 400 with acceleration equal to or greater than a certain value. For example, in the state shown in FIG. 20, the virtual hand 400 is reciprocatingly moved in the z-axis direction with the virtual hand 400 gripping the body of the avatar 4A, and the acceleration in the +z-axis direction is equal to or greater than a certain value, and If the state in which the acceleration in the −z-axis direction is equal to or greater than a certain value continues for a predetermined number of times or more, it may be a shaking motion. However, the direction in which the virtual hand 400 is reciprocated is not limited to the z-axis direction, and may be any direction. Further, the shaking motion may include a motion in which the virtual hand 400 reciprocates one or more relative coordinates associated with the avatar 4A within a predetermined time more than a predetermined number of times. For example, in the case of the state shown in FIG. 20, the one or more relative coordinates are the first relative coordinates provided at any position in the +z-axis direction with reference to the part of the body of the avatar 4A grasped by the virtual hand 400. , and a second relative coordinate provided at an arbitrary position in the −z-axis direction. In this example, the virtual hand 400 is reciprocated in the z-axis direction while grasping the body of the avatar 4A with the virtual hand 400, reciprocating the first relative coordinates more than a predetermined number of times within a certain period of time, and moving the second relative coordinate back and forth. When the relative coordinates of are reciprocated more than a predetermined number of times within a certain period of time, the shaking motion may be performed. However, the direction in which the virtual hand 400 is reciprocated is not limited to the z-axis direction, and may be any direction. In this case, relative coordinates with respect to the virtual hand 400 may also be provided in any direction.

It should be noted that it is preferable to determine that a specific action has been performed with respect to an action that the user B himself recognizes as shaking. In other words, the shaking direction is not uniform and is assumed to vary from user to user. Therefore, the moving direction of the virtual hand 400 may be any direction, and does not need to be limited to a specific direction. However, if it is necessary to strictly determine the motion of a specific action or to add reality to the motion of the avatar 4B (for example, to prevent the virtual hand 400 from squeezing into the avatar 4A), the control unit 121 , taking into account the moving direction of the virtual hand 400, it may be determined that there is a shaking motion when a repetitive motion is performed in a specific direction.

On the other hand, when the avatar 4B does not grasp the body of the avatar 4A with the virtual hand 400 and the virtual hand 400 contacts the avatar 4A with a certain acceleration or more (collider collision), the control unit 121 selects the hit as the specific action. Identify that an action was performed. In order to prevent erroneous detection of the motion of the virtual hand 400, the tapping motion is performed by performing a repetitive motion of the virtual hand 400 with a constant acceleration or more, and in the process of the repetitive motion, the avatar 4A is touched multiple times (collider collision). ) (specifically, back-and-forth slapping). For example, in the state shown in FIG. 19, the virtual hand 400 is reciprocated in the y-axis direction, and the acceleration in the +y-axis direction is a certain value or more and the acceleration in the −y-axis direction is a certain value or more. , when the collider of the virtual hand 400 and the collider of the face of the avatar 4A make contact more than a predetermined number of times, a tapping motion may be performed. However, the direction in which the virtual hand 400 is reciprocated is not limited to the y-axis direction, and may be any direction. This back-and-forth slap motion is, for example, the virtual hand 400 of the avatar 4B reciprocating the relative coordinates associated with the avatar 4A more than a predetermined number of times within a given period of time, and touching the avatar 4A multiple times within a given period of time ( It may be specified by the fact that there was a hit judgment). For example, in the state shown in FIG. 19, the one or more relative coordinates are a first relative coordinate provided at an arbitrary position in the +y-axis direction and an arbitrary position in the -y-axis direction with respect to the face of the avatar 4A. A second relative coordinate provided at the position is included. In this example, the virtual hand 400 is reciprocated in the y-axis direction to reciprocate the first relative coordinate more than a predetermined number of times within a certain period of time, and reciprocate the second relative coordinate more than a predetermined number of times within a certain period of time. In this state, when the collider of the virtual hand 400 and the collider of the face of the avatar 4A come into contact a predetermined number of times or more, a tapping motion may be performed. However, the direction in which the virtual hand 400 is reciprocated is not limited to the y-axis direction, and may be any direction. In this case, relative coordinates with respect to the virtual hand 400 may also be provided in any direction. However, the control unit 121 may determine that there is a hitting motion when the repetitive motion is performed in a specific direction, taking into consideration the moving direction of the virtual hand 400 .

Note that the avatar 4A may be set to move following the movement of the avatar 4B. For example, as shown in FIG. 20, when the avatar 4B performs a shaking motion with respect to the avatar 4A, the control unit 121 controls the movement of the avatar 4A so that the avatar 4A is shaken following the shaking motion. can be set. However, when the avatar 4B shakes or hits the avatar 4A, the avatar 4A may be set not to follow the movement, that is, not to move.

Next, when the control unit 121 determines that the avatar 4B has performed a specific action on the avatar 4A (YES in step S58), information indicating that the specific action has been performed by the user B (action execution information) is generated (step S60). After that, the control unit 121 transmits action execution information to the server 2 via the communication network 3 (step S61).

Next, based on the action execution information received from the user terminal 1B, the server 2 provides information (participation inquiry notification information) for asking the user A to participate in the content provided in the virtual space 200. is generated (step S62). The participation inquiry notification is a notification for asking user A, who is removing the HMD 110, to participate in the content provided in the virtual space 200. FIG. The participation inquiry notification information includes, for example, the identification information of the avatar 4B that performed the specific action, the identification information of the avatar 4A that received the specific action, the type of the specific action, the strength of the action of the specific action, and the like. good too.

Next, the server 2 identifies the personal information of the user A based on the generated participation inquiry notification information (step S63). For example, user A's personal information (e.g., e-mail address, SNS (Social Networking Service) address, etc.) is registered in advance in an information table stored in the storage unit 22 of the server 2, and managed in association with the identification information of the avatar 4A. It is In step S63, the server 2 refers to the information table stored in the storage unit 22, and acquires the personal information of the user A based on the identification information of the avatar 4A included in the participation inquiry notification information. However, user A's personal information may be managed by a server different from server 2 . Examples of the server different from the server 2 include, but are not limited to, a server capable of distributing game programs to users whose personal information is registered, that is, a server serving as a game distribution platform. not something.

Next, the server 2 transmits participation inquiry notification information to the destination (for example, user A's email address) acquired in step S63 (step S64). As a method of transmitting the participation inquiry notification information to User A, for example, communication tools such as e-mail, chat, and SNS can be used.

Next, the user terminal 5 receives the participation inquiry notification information by the communication tool, and displays the reception notification of the participation inquiry notification information on the display unit 6 (step S65). The user terminal 5 is a terminal device owned by the user A. For example, it may be a portable terminal such as a smart phone, a PDA (Personal Digital Assistant), a tablet or phablet computer, a wearable device, or a personal computer. , a video game device, or the like.

Next, the user terminal 5 activates the communication tool based on the input operation by the user A who saw the reception notification, and displays the message M1 on the display unit 6 as shown in FIG. 21 (step S66). The message M1 is information indicating that the user B is asking the user A to participate in the virtual space 200 (for example, "Right now, [content name (for example, the name of the content provided to the virtual space 200) ] contains [Communication is being received from the other player name (for example, user B)]. Note that if the user terminal 5 is a personal computer or a video game device, and if the display of these devices also displays an image of the content provided in the virtual space 200, a message regarding the notification of participation request will be displayed on the display. may be displayed. Also, the message M1 may be a message prompting the user A to mount the HMD 110 of the user terminal 1A.

Next, based on the fact that the message M1 is displayed on the display unit 6, the user terminal 5 determines that the user A has finished reading the message M1, and generates read information indicating that the message M1 has been read. (step S67). The already-read information is generated based on the user A's reaction to the message M1, and may be generated based on, for example, replying to an e-mail or giving "like" points. After that, the user terminal 5 transmits the read information to the server 2 via the communication network (step S68).

Next, after receiving the read information from the user terminal 5, the server 2 transmits the read information to the user terminal 1B via the communication network 3 (step S69).

Next, the control unit 121 of the user terminal 1B sets the facial expression of the avatar 4A placed in the virtual space 200B to the updated facial expression based on the read information received from the server 2 (step S70). ). For example, as shown in FIG. 22, the control unit 121 updates the facial expression of the avatar 4A that was in a state with closed eyes (an example of a first state) to a state with open eyes (an example of a second state). .

In step S70, the control unit 121 may update the posture of the avatar 4A based on the read information received from the server 2. FIG. For example, the posture of the avatar 4A in a lying state may be updated to a sitting state.

Next, the control unit 121 updates the virtual space data indicating the virtual space 200B, and updates the field of view CV of the avatar 4B according to the movement of the HMD 110 of the user terminal 1B. Next, the control unit 121 updates the visual field image data based on the updated virtual space data and the updated visual field CV of the avatar 4B, and then displays the visual field image on the HMD 110 based on the updated visual field image data. (step S71). As a result, the user B can easily recognize that the user A has reacted to the invitation to participate by visually confirming that the avatar 4A has opened its eyes.

On the other hand, in step S72, the server 2 transmits read information to the user terminal 1A in addition to the user terminal 1B.

Next, the control unit 121 of the user terminal 1A determines whether or not the user A has worn the HMD 110 within a certain period of time after receiving the read information (step S73). The control unit 121 may determine whether the user A has worn the HMD 110 based on the tilt information of the HMD 110 transmitted from the HMD sensor 114 or the event information transmitted from the wearing sensor 115 . Whether or not user A wears HMD 110 is determined based on tilt information indicating the tilt (roll angle, yaw angle, pitch angle) of HMD 110 acquired by HMD sensor 114. It can be determined by whether it is small or not. As described above, while the user A is wearing the HMD 110, the inclination of the HMD 110 is equal to or less than the predetermined inclination. is identified.

When it is determined in step S73 that the user A has worn the HMD 110 (YES in step S73), the control unit 121 of the user terminal 1A transitions from sleep mode to active mode (step S74). Next, the control unit 121, which has shifted to the active mode, generates control information for the avatar 4A and also generates wearing information indicating that the user A has worn the HMD 110 (step S75). After that, the control unit 121 transmits the control information and the wearing information of the avatar 4A to the server 2 via the communication network 3 (step S76).

Next, after receiving the control information and the mounting information of the avatar 4A from the user terminal 1A, the server 2 transmits the control information and the mounting information of the avatar 4A to the user terminal 1B via the communication network 3 (step S77).

Next, the control unit 121 of the user terminal 1B updates the action of the avatar 4A based on the control information of the avatar 4A, and updates the action of the avatar 4B based on the control information of the avatar 4B (step S78). In particular, the control unit 121 updates the posture of the avatar 4A based on the wearing information of the avatar 4A. Also, the control unit 121 may update the facial expression of the avatar 4B based on the face information of the avatar 4B included in the control information of the avatar 4B. Specifically, for example, as shown in FIG. 23, the control unit 121 sets the posture of the avatar 4A to the standing up state (an example of the third state).

Note that, similarly to the embodiment shown in FIG. 17, the control unit 121 may generate a balloon object associated with the avatar 4A in the virtual space 200B. The balloon object displays information indicating that the user A has put on the HMD 110 (for example, "User A has returned to his seat.").

After that, the control unit 121 updates the virtual space data indicating the virtual space 200B, and updates the field of view CV of the avatar 4B according to the movement of the HMD 110 of the user terminal 1B. Next, the control unit 121 updates the visual field image data based on the updated virtual space data and the updated visual field CV of the avatar 4B, and then displays the visual field image on the HMD 110 based on the updated visual field image data. (step S79).

On the other hand, if it is determined in step S73 that the user A has not worn the HMD 110 within a predetermined period of time after receiving the read information (NO in step S73), the process proceeds to FIG. Next, as shown in FIG. 24, the control unit 121 generates non-wearing information indicating that the user A is not wearing the HMD 110 (step S80). After that, the control unit 121 transmits non-wearing information to the server 2 via the communication network 3 (step S81).

Next, after receiving the non-wearing information from the user terminal 1A, the server 2 transmits the non-wearing information to the user terminal 1B via the communication network 3 (step S82).

Next, the control unit 121 of the user terminal 1B sets the facial expression of the avatar 4A placed in the virtual space 200B to the default facial expression based on the non-wearing information received from the server 2 (step S83). For example, if the default facial expression is closed eyes, the facial expression of avatar 4A is updated to closed eyes as shown in FIG. It should be noted that the processing of step S83 is performed by the control unit 121 of the user terminal 1B measuring a certain period of time from the reception of the read information, and if the control information and the wearing information of the avatar 4A are not received within the certain period of time, the avatar 4A The facial expression may be set to a default expression. If the eyes of the avatar 4A are open based on the read information in step S70 of FIG. 18, the expression of the face of the avatar 4A is returned to the closed eyes in step S83. On the other hand, if the participation inquiry notification has not been read (NO in step S66), the avatar 4A keeps its eyes closed. Not changed.

After that, the control unit 121 updates the virtual space data indicating the virtual space 200B, and updates the field of view CV of the avatar 4B according to the movement of the HMD 110 of the user terminal 1B. Next, the control unit 121 updates the visual field image data based on the updated virtual space data and the updated visual field CV of the avatar 4B, and then displays the visual field image on the HMD 110 based on the updated visual field image data. (step S84).

As described above, according to the information processing method according to Modification 1, the step of moving the virtual hand 400 within the virtual space 200 according to the movement of the user B, and the step of moving the virtual hand 400 within the virtual space 200 based on the movement of the user A A step of executing an action (first action) such as a shaking motion with the virtual hand 400 for the avatar 4A whose motion can be controlled, and based on the execution of the action, the user A (the user terminal 5 associated with ) sending a participation invitation notification (second action) for asking participation in the virtual space 200 to the virtual space 200. According to this method, a plurality of users A and B who share the virtual space 200 execute a predetermined action through an intuitive operation using the virtual hand 400, thereby enabling communication within the virtual space 200 to This can be done smoothly without impairing the user's sense of immersion. Thus, according to Modification 1, by providing a user interface (so-called diegetic UI) that does not require menu operations or key operations for communication between users, users can experience virtual experiences. can be improved.

In particular, by transmitting a participation request notification to the user terminal 5 associated with the user A based on a specific action performed by the user B on the avatar 4A, the user B can request content provided to the virtual space 200. User A can easily recognize that participation has been requested. As a result, it is possible to promote the use of content (for example, a multiplayer game) that requires multiple users to play simultaneously in the virtual space 200 .

Further, in the information processing method according to Modification 1, in a state in which user A does not control avatar 4A (that is, user A does not wear HMD 110), user B performs a specific action on avatar 4A. In this case, a notice of invitation to participate is sent to User A. Therefore, user A, who is not wearing the HMD 110, can be attracted to the content provided in the virtual space 200, thereby further promoting participation in the content. In particular, in a VR game, when there are few users (avatars) waiting in the room, that is, when there are few multiplayer opponent candidates, the users (avatars) enter the room for the time being, but multiplayer candidates appear. In many cases, the users wait with their HMDs removed until the end of the game, so it is possible to effectively lure such waiting users to the multi-play of the VR game.

Further, in the information processing method according to Modification 1, when the user A reacts to the participation inquiry notification (for example, when the message M1 is already read), the avatar 4A in the virtual space 200B is , a step of transitioning from a state with eyes closed (first state) to a state with eyes open (second state); and a step of transitioning the avatar 4A in the virtual space 200B to a standing state (third state) when the avatar 4A is in a controlled state. Thereby, the user B can easily grasp that there is a reaction from the user A, or that the user A is wearing the HMD 110 and the avatar 4A is being controlled.

On the other hand, if the avatar 4A is not controlled by the user A within a certain period of time after the avatar 4A transitions to the state with the eyes open, the avatar 4A in the virtual space 200B is changed to the state with the eyes closed. You can return to As a result, the user B can easily understand that the avatar 4A is not controlled by the user A (that is, the user A does not participate in the content) even though the participation request notification has been sent. can be done. Therefore, the user B shakes the avatar 4A again to ask for participation in the virtual space again, or asks an avatar associated with a user other than the user A who shares the virtual space 200 to participate in the virtual space. You can make inquiries.

The state in which the avatar 4A is not controlled by the user A includes not only the case where the user A has removed the HMD 110 but also the case where the user A has not logged in to the content provided in the virtual space 200 in the first place. obtain. When User A has not logged in to the content, the avatar 4A does not normally exist in the virtual space 200B provided by User B's user terminal 1B. For example, the avatar 4A of the user A can exist in the virtual space 200B even if the user A has not logged in to the content. The presence or absence of this friend registration may be registered in the server 2 or the like together with the personal information of each user, for example. In this case, the control unit 121 of the user terminal 1B can default the posture and facial expression of the avatar 4A placed in the virtual space 200B based on the fact that the user A has not logged in to the content (see FIG. 18). step S54). In this way, the avatar 4A associated with the user A who is not logged in to the content exists in the virtual space 200B, and the user A is asked to participate based on the execution of the specific action from the avatar 4B to the avatar 4A. Sending a notification can provide motivation for user A to log into the content.

Further, the content of the participation request notification (message M1) may differ according to the intensity of the specific action performed by the avatar 4B on the avatar 4A. For example, a plurality of thresholds may be set in stages for the acceleration of the motion of the virtual hand 400, and the contents of the message M1 may be changed according to the thresholds of the acceleration. For example, the message M1 may include information about the degree of desire to participate, and the information about the degree of desire to participate may be changed according to the strength of the shaking motion. Specifically, when the shaking motion is intense, a message M1 indicating that the user B has a high degree of desire to participate (for example, "degree of desire to participate: ☆☆☆ (three stars)") is displayed. You may do so. A predetermined threshold may be set for the number of times the virtual hand 400 passes through the relative coordinates associated with the avatar 4A, and the strength of the motion of the specific action may be specified according to the threshold for the number of times the virtual hand 400 passes. .

Also, when a plurality of avatars simultaneously (or in cooperation) perform a specific action on the avatar 4A, the content of the message M1 may be changed. In this case, it is preferable to provide a dedicated collider for each specific part of the body of Avatar A so that Avatar A can be grasped by a plurality of avatars at the same time. This enables the avatar 4A to be shaken by a plurality of avatars. Note that, in this case, it is preferable to increase the degree of desire to participate as the number of users performing the specific action on the avatar 4A increases.

Also, the content of the message M1 may differ according to the type of specific action performed on the avatar 4A. For example, when the avatar 4B slaps the avatar 4A, the participation desire degree displayed in the message M1 may be higher than when the avatar 4B shakes the avatar 4A.

In this way, by varying the content of the participation request notification according to the intensity of the movement of the specific action on the avatar 4A by the avatar 4B and the type of the specific action, the user A can determine the degree of desire to participate in the content from the user B. It is possible to grasp the high and low of Therefore, user A can decide whether or not to participate in the content according to the degree of desire.

Also, in the present embodiment, an action involving physical interference is exemplified as the specific action by the avatar 4B on the avatar 4A, but the specific action is not limited to this example. The specific action may be an action for specifying the user A. For example, the specific action executed from the avatar 4B to the avatar 4A may be line of sight, voice, laser pointer, facial expression (smile, glaring, etc.). , applause, etc. Alternatively, the user B may select an avatar (user) whom the user B wishes to ask about participation in the content from an avatar list (user list) displayed in the virtual space 200B. That is, the specific action performed by the avatar 4B may be a direct action or an indirect action on the avatar 4A. Also, direct and indirect actions may be combined.

Further, the transmission timing of the notification of invitation to participate may be changed according to the contents of the content provided to the virtual space 200 . For example, when the number of users required to start multiplay is three or more, the control unit 121 of the user terminal 1B determines that user B is the last avatar among the plurality of avatars required to start multiplay. , the control unit 121 of the user terminal 1B collectively transmits a participation request notification to each user associated with each of the plurality of avatars that performed the specific action. You may do so. Even if the number of users required to start multiplay is three or more, the control unit 121 of the user terminal 1B controls each avatar at the timing when the avatar 4B performs a specific action on each avatar. You may make it transmit a notice of participation inquiry individually.

In the above embodiment, the facial expression of the avatar 4A with its eyes closed is changed based on User A's reaction to the participation inquiry notification (for example, the fact that the message M1 regarding the participation inquiry notification has been read). , is updated to open eyes, but is not limited to this example. For example, the avatar 4A may be updated to open its eyes based on the execution of a specific action on the avatar 4A by the avatar 4B. Thereby, the user B can easily grasp that the specific action has been appropriately performed on the avatar 4A. Note that the avatar 4A may be updated to open the eyes based on the fact that the user A wears the HMD 110 .

Further, the information processing method according to Modification 1 includes a step of executing a predetermined action (an example of a first action) involving physical interference from user B to user A in virtual space 200; Based on the step of executing a participation inquiry notification (an example of a second action) to user A, and user A's reaction to the participation inquiry notification (for example, message M1 regarding the participation inquiry notification has been read). and updating the posture and/or facial expression of the avatar 4A in the virtual space 200 (an example of the third action). According to this method, based on the user A's reaction to the participation inquiry notification sent to the user A, by updating the posture and facial expression of the avatar 4A associated with the user A in the virtual space 200B, Communication between users A and B within 200 can be performed smoothly. In this way, by reflecting the user A's reaction to the participation inquiry notification in the real space in the virtual space 200, it is possible to provide a seamless virtual experience that blurs the boundary between the real space and the virtual space. .

(Modification 2)
Next, Modified Example 2 of the information processing method according to the present embodiment will be described with reference to FIGS. 25 to 28. FIG. Modification 2 will exemplify an information processing method at the start of a game when a game (so-called single-play game) played only by user B is provided in the virtual space 200 (200B). FIG. 25 is an example of a flowchart for explaining an information processing method according to Modification 2. As shown in FIG. 26 to 28 are diagrams showing an example of a virtual space 200B provided to user B for explaining the information processing method according to Modification 2. FIG. Note that, as shown in FIG. 26, in Modification 2, the virtual space 200B includes an avatar 4B and an avatar 4C. The avatar 4C is an avatar 4C (so-called Non Player Character (NPC)) that can be controlled by the server 2 .

Referring to FIG. 25, first, control unit 121 of user terminal 1B receives a game start input based on an operation by user B (step S100). Next, the control unit 121 transmits game start input information to the server 2 via the communication network 3 (step S101).

Next, the server 2 reads the opening movie data from the storage unit 22 based on the game start input information received from the control unit 121 (step S102). The opening movie is a video played in the virtual space 200B as a scenario scene (an example of the first scene) introduced at the start of the game. Next, the server 2 transmits the opening movie data to the control unit 121 of the user terminal 1B via the communication network 3 (step S103).

Next, the control unit 121 of the user terminal 1B reproduces the opening movie in the virtual space 200B based on the opening movie data received from the server 2 (step S104).

During or after the opening movie is reproduced, the control unit 121 of the user terminal 1B sets the posture and facial expression of the avatar 4C to the default posture and facial expression (step S105). Assuming that the default posture is the lying state and the default facial expression is the state where the eyes are closed, the posture of the avatar 4C is set to the lying state as shown in FIG. The facial expression is set to a state in which the eyes are closed.

Next, the control unit 121 updates the virtual space data indicating the virtual space 200B, and updates the field of view CV of the avatar 4B according to the movement of the HMD 110 of the user terminal 1B. Next, the control unit 121 updates the visual field image data based on the updated virtual space data and the updated visual field CV of the avatar 4B, and then displays the visual field image on the HMD 110 based on the updated visual field image data. (step S106).

Next, after updating the field-of-view image in step S106, the control unit 121 causes the avatar 4B associated with the user B to perform a specific action (an example of a first action) on the avatar 4C associated with the server 2. is executed (step S107). Since the specific action determination process is the same as that of Modification 1, detailed description thereof will be omitted. In order to prompt the avatar 4B to perform a specific action on the avatar 4C, the control unit 121 displays a message such as "Please wake up the avatar 4C" in the virtual space 200B after the opening movie is reproduced in step S104. You may

Next, when the control unit 121 determines that the avatar 4B has performed the specific action on the avatar 4C (YES in step S107), the control unit 121 generates action execution information indicating that the user B has performed the specific action. (step S108). Thereafter, the control unit 121 transmits action execution information to the server 2 via the communication network 3 (step S109).

Next, the server 2 generates scene transition information based on the action execution information received from the user terminal 1B (step S110). The scene transition information is information for transitioning the game provided in the virtual space 200B from a scenario scene to a game scene (an example of a second scene). Next, the server 2 transmits scene transition information to the control unit 121 of the user terminal 1B via the communication network 3 (step S111).

Next, the control unit 121 of the user terminal 1B updates the setting of the posture and facial expression of the avatar 4C based on the scene transition information received from the server 2 (step S112). Specifically, for example, as shown in FIG. 28, the posture of the avatar 4C is set to a standing state, and the facial expression of the avatar 4C is set to open the eyes.

Next, the control unit 121 generates a balloon object 45A associated with the avatar 4C in the virtual space 200B (step S113). The balloon object 45A displays information indicating that the avatar 4C has joined the group of the avatar 4B (for example, "Avatar 4C has joined the group.").

After that, the control unit 121 updates the virtual space data indicating the virtual space 200B, and updates the field of view CV of the avatar 4B according to the movement of the HMD 110 of the user terminal 1B. Next, the control unit 121 updates the visual field image data based on the updated virtual space data and the updated visual field CV of the avatar 4B, and then displays the visual field image on the HMD 110 based on the updated visual field image data. (step S114).

After that, based on the scene transition information, the control unit 121 starts displaying the game scene (step S115).

As described above, the information processing method according to Modification 2 includes the step of performing a specific action such as a shaking motion with the virtual hand 400 on the avatar 4C whose motion in the virtual space 200B is computer-controllable, and and a step of transitioning the game provided in the virtual space 200B from the scenario scene (first scene) to the game scene (second scene) based on the execution of the action. Conventionally, in order to start a game scene, a user interface (for example, “Press Any Button”) for progressing the game is displayed after the scenario scene is reproduced, and the user presses a predetermined button on the external controller 320. The game scene had started. On the other hand, according to the method according to Modification 2, the game scene is started based on the execution of the specific action by User B with respect to avatar 4C. You can progress the game. In Modification 2, a case where the processing of Modification 2 is executed by a system having server 2 and user terminal 1B has been described as an example, but the present invention is not limited to this, and may be executed by user terminal 1B alone. good.

Further, the information processing method for advancing the game according to Modification 2 is not limited to the scene when the game is started, but can be applied to the scene where the game is restarted in the middle of the game. For example, when a game played by a party organized by a plurality of avatars (for example, avatars 4B and 4C) is provided in the virtual space 200B, after the game progress is interrupted, the avatar 4B executes a specific action to the avatar 4C. The game may be restarted when it is determined that the Specifically, in the scene where the party stays at the inn, it is assumed that the avatar 4C is still sleeping the next morning. In this state, the avatar 4B may perform a specific action, such as a shaking motion or a hitting motion, on the avatar 4C to wake up the avatar 4C and restart the progress of the game. According to such a method, the progress of the game can be resumed without impairing the user B's sense of immersion in the virtual space 200B.

Also, the order of processing defined in each step shown in FIGS. 18, 24, and 25 is merely an example, and the order of these steps can be changed as appropriate.

Further, in the description of the above embodiments and modifications, it is assumed that whether or not the HMD 110 is worn is detected based on the tilt information of the HMD 110 transmitted from the HMD sensor 114 or the event information transmitted from the wearing sensor 115. , but not limited to this example. For example, the control unit 121 of the user terminal detects the movement of the HMD 110 using the position sensor 130 and/or the HMD sensor 114, and when the movement draws a predetermined trajectory, the user wears the HMD 110, or the HMD 110 may be specified to have been removed. Further, the control unit 121 detects whether or not the HMD 110 is moving by the position sensor 130 and/or the HMD sensor 114, and, for example, when the position of the HMD 110 does not change for a certain period of time, the user wears the HMD 110. You may specify that you have not. Furthermore, the control unit 121 may detect whether or not the HMD 110 is attached by detecting the relative positional relationship between the HMD 110 and the external controller 320 . For example, the control unit 121 may identify that the user is not wearing the HMD 110 when the HMD 110 and the external controller 320 are separated by a certain distance or more.

Further, in the description of the above embodiments and modifications, it is assumed that the virtual space data representing the virtual space 200B has been updated on the user terminal 1B side. good. Further, it is assumed that the view image data corresponding to the view image is updated on the user terminal 1B side, but the view image data may be updated on the server 2 side. In this case, the user terminal 1</b>B displays the visual field image on the HMD 110 based on the visual field image data transmitted from the server 2 .

Also, the order of processing defined in each step shown in FIGS. 12 and 16 is merely an example, and the order of these steps can be changed as appropriate.

Further, in the present embodiment, the virtual space (VR space) in which the user is immersed by the HMD device has been described as an example, but a transmissive HMD device may be employed as the HMD device. In this case, an augmented reality (AR) space or a mixed reality space is generated by outputting a visual field image obtained by synthesizing a part of an image constituting a virtual space with a real space visually recognized by a user through a transmissive HMD device. A user may be provided with a virtual experience in (MR: Mixed Reality) space. In this case, instead of the virtual hand, the action on the target object in the virtual space may be generated based on the movement of the user's hand. Specifically, the processor may identify the coordinate information of the position of the user's hand in the real space and define the position of the target object in the virtual space in relation to the coordinate information in the real space. As a result, the processor can grasp the positional relationship between the user's hand in the real space and the target object in the virtual space 2, and execute processing corresponding to the above-described collision control or the like between the user's hand and the target object. Become. As a result, it is possible to give an effect to the target object based on the movement of the user's hand.

Further, in order to realize various processes executed by the control unit 121 of the user terminal 1 by software, a control program for causing a computer (processor) to execute various processes may be incorporated in advance in the storage unit 123 or memory. good. Alternatively, the control program may be a magnetic disk (HDD, floppy disk), optical disk (CD-ROM, DVD-ROM, Blu-ray (registered trademark) disk, etc.), magneto-optical disk (MO, etc.), flash memory (SD card, It may be stored in a computer-readable storage medium such as a USB memory, SSD, etc.). In this case, the control program stored in the storage medium is incorporated into the storage unit 123 by connecting the storage medium to the control device 120 . Then, the control program incorporated in the storage unit 123 is loaded onto the RAM, and the processor executes the loaded program, whereby the control unit 121 executes various processes.

Also, the control program may be downloaded from a computer on the communication network 3 via the communication interface 125 . In this case as well, the downloaded control program is incorporated into the storage unit 123 .

Although the embodiments of the present disclosure have been described above, the technical scope of the present invention should not be construed to be limited by the description of the embodiments. It should be understood by those skilled in the art that this embodiment is merely an example, and that various modifications of the embodiment are possible within the scope of the invention described in the claims. The technical scope of the present invention should be determined based on the scope of the invention described in the claims and their equivalents.

1, 1A, 1B: user terminal 2: server 3: communication network 4A, 4B: avatar 21: communication interface 22: storage unit 23: control unit 24: bus 42A, 44A, 45A: balloon object 43A: telop object 100: virtual Spatial distribution system (distribution system)
110: HMD (head mounted device)
112: Display unit 113: Face camera 114: HMD sensor 115: Mounting sensor 116: Headphones 117: Face camera 118: Microphone 120: Control device 123: Storage unit 124: I/O interface 125: Communication interface 126: Bus 130: Position Sensor 140: gaze sensors 200, 200A, 200B: virtual space 210: center position 300: virtual camera 320: external controller 320L: left hand external controller (controller)
320R: External controller for right hand (controller)
400: virtual hands U, A, B: user M1: message

Claims (8)

computer to provide a virtual experience to the user,
performing a first action within the virtual space;
performing a second action for another user different from the user based on performing the first action;
and executing a third action in the virtual space based on the reaction to the second action. 2. The first action is an action for specifying the other user, and the third action is an action executed by a virtual object associated with the other user in the virtual space. program described in . causing the computer to further execute a step of moving an operation object within the virtual space according to the movement of the body part of the user;
3. The program according to claim 1, wherein said first action is executed using said manipulation object. 4. The program according to any one of claims 1 to 3, wherein said second action includes a notification prompting said other user to participate in said virtual space. 5. Any one of claims 1 to 4, wherein the third action includes transitioning an action performed by the virtual object associated with the other user in the virtual space from a first state to a second state. 1. The program according to item 1. An information processing device for providing a virtual experience to a user,
Under the control of the processor included in the information processing device,
performing a first action within the virtual space;
performing a second action for another user different from the user based on performing the first action;
and executing a third action in the virtual space based on the reaction to the second action. An information processing system for providing a virtual experience to a user, comprising:
Under the control of a plurality of processors included in the information processing system,
performing a first action within the virtual space;
performing a second action for another user different from the user based on performing the first action;
and executing a third action in the virtual space based on the reaction to the second action. A computer-implemented information processing method for providing a virtual experience to a user, comprising:
performing a first action within the virtual space;
performing a second action for another user different from the user based on performing the first action;
and executing a third action in the virtual space based on a reaction to the second action.

Images