JP7379627B1 - Programs and information processing systems - Google Patents

Abstract

An object of the present invention is to provide a virtual space to a user in which the possibility of the user feeling uncomfortable is reduced. The program causes a computer to perform predetermined settings for a second character by a user operating a first character, with respect to a virtual space in which a character corresponding to the user can appear. At least one dummy character serving as a dummy of the first character is made to function as a setting reflecting means for appearing in the virtual space. Thereby, it is possible to provide the user with a virtual space in which the possibility that the user feels uncomfortable is reduced. [Selection diagram] Figure 19

Description

The present invention relates to a program and an information processing system.

BACKGROUND ART Services (for example, games, etc.) that allow interaction with other users in a virtual space have been known for some time (see, for example, Patent Document 1).

JP 2021-114036 Publication

Incidentally, in such services, it is required to provide the user with a virtual space in which the possibility of the user feeling uncomfortable is reduced.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a user with a virtual space in which the possibility of the user feeling uncomfortable is reduced.

According to one embodiment presented in the present disclosure,
computer,
Regarding a virtual space in which a character corresponding to the user can appear, based on a predetermined setting made for a second character by the user operating the first character, at least one dummy of the first character is set. A program is provided that functions as a setting reflection means for causing two dummy characters to appear in the virtual space.

According to the present invention, it is possible to provide a user with a virtual space in which the possibility that the user feels uncomfortable is reduced.

1 is a diagram schematically showing the configuration of an HMD system according to an embodiment. 1 is a block diagram illustrating an example of a hardware configuration of a computer according to an embodiment. FIG. FIG. 2 is a diagram conceptually representing a UVW visual field coordinate system set in an HMD according to an embodiment. FIG. 2 is a diagram conceptually representing one aspect of representing a virtual space according to an embodiment. FIG. 2 is a top view of a user's head wearing an HMD according to an embodiment. FIG. 3 is a diagram showing a YZ cross section of a viewing area viewed from the X direction in virtual space. FIG. 3 is a diagram showing an XZ cross section of a viewing area viewed from the Y direction in virtual space. FIG. 1 is a diagram showing a schematic configuration of a controller according to an embodiment. FIG. 3 is a diagram illustrating an example of yaw, roll, and pitch directions defined for a user's right hand according to an embodiment. FIG. 2 is a block diagram illustrating an example of the hardware configuration of a server according to an embodiment. 1 is a block diagram representing a computer as a modular configuration according to an embodiment; FIG. It is a sequence chart which shows the process performed in the HMD set according to a certain embodiment. FIG. 2 is a schematic diagram showing a situation in which each HMD provides a virtual space to a user in a network. It is a figure which shows the visual field image of user 5A in FIG. 12A. It is a sequence chart which shows the process performed in the HMD system according to a certain embodiment. 1 is a block diagram showing a detailed configuration of a module of a computer according to an embodiment. FIG. 1 is a block diagram representing a server according to an embodiment as a modular configuration. FIG. It is a flowchart which shows the display change process performed in the HMD system according to a certain embodiment. FIG. 4 is a diagram showing an example of the appearance and handle name of an avatar object according to an embodiment, in which (a) shows the avatar object before the predetermined settings are made, and (b) shows the avatar object after the predetermined settings are made. . It is a flowchart which shows the dummy placement process performed in the HMD system according to a certain embodiment. FIG. 2 is a schematic diagram showing a virtual space when dummy placement processing is executed in an HMD system according to an embodiment. It is a flowchart which shows the nuisance determination process (part 1) performed in the HMD system according to a certain embodiment. It is a flowchart which shows the nuisance determination process (part 2) performed in the HMD system according to a certain embodiment. It is a flowchart which shows the nuisance determination process (part 3) performed in the HMD system according to a certain embodiment. It is a flowchart which shows the nuisance determination process (part 4) performed in the HMD system according to a certain embodiment. It is a flowchart which shows the nuisance determination process (part 5) performed in the HMD system according to a certain embodiment.

Hereinafter, embodiments of this technical idea will be described in detail with reference to the drawings. In the following description, the same parts are given the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions thereof will not be repeated. In one or more embodiments shown in this disclosure, elements included in each embodiment can be combined with each other, and the resultant product of the combination also forms part of the embodiments shown in this disclosure.

[HMD system configuration]
The configuration of an HMD (Head-Mounted Device) system 100 will be described with reference to FIG. 1. FIG. 1 is a diagram schematically showing the configuration of an HMD system 100 according to this embodiment. The HMD system 100 is provided as a home-use system or a business-use system. HMD system 100 provides predetermined services to users.

The HMD system 100 includes a server 600, HMD sets 110A, 110B, 110C, and 110D, an external device 700, and a network 2. Each of HMD sets 110A, 110B, 110C, and 110D is configured to be able to communicate with server 600 and external device 700 via network 2. Hereinafter, the HMD sets 110A, 110B, 110C, and 110D are also collectively referred to as the HMD set 110. The number of HMD sets 110 that constitute the HMD system 100 is not limited to four, and may be three or less or five or more. The HMD set 110 includes an HMD 120, a computer 200, an HMD sensor 410, a display 430, and a controller 300. HMD 120 includes a monitor 130, a gaze sensor 140, a first camera 150, a second camera 160, a microphone 170, and a speaker 180. Controller 300 may include a motion sensor 420.

In one aspect, the computer 200 can be connected to the Internet or other network 2 and can communicate with the server 600 or other computers connected to the network 2. Examples of other computers include computers of other HMD sets 110 and external equipment 700. In another aspect, HMD 120 may include sensor 190 instead of HMD sensor 410.

HMD 120 is attached to the head of user 5 and can provide virtual space to user 5 during operation. More specifically, HMD 120 displays a right-eye image and a left-eye image on monitor 130, respectively. When each eye of the user 5 visually recognizes each image, the user 5 can recognize the image as a three-dimensional image based on the parallax between the eyes. HMD 120 may include either a so-called head-mounted display including a monitor, or a head-mounted device to which a smartphone or other terminal having a monitor can be attached.

The monitor 130 is realized, for example, as a non-transmissive display device. In one aspect, the monitor 130 is arranged on the main body of the HMD 120 so as to be located in front of both eyes of the user 5. Therefore, when the user 5 views the three-dimensional image displayed on the monitor 130, he or she can immerse himself in the virtual space. In one aspect, the virtual space includes, for example, a background, objects that can be operated by the user 5, and images of menus that can be selected by the user 5. In one aspect, the monitor 130 can be implemented as a liquid crystal monitor or an organic EL (Electro Luminescence) monitor included in a so-called smartphone or other information display terminal.

In another aspect, monitor 130 may be implemented as a transmissive display device. Examples of the transmissive monitor 130 include glasses-type monitors and contact lens-type monitors. The HMD 120 is not a closed type that covers the eyes of the user 5 as shown in FIG. 1, but may be an open type that does not cover the eyes of the user 5. The transmissive monitor 130 may be temporarily configured as a non-transmissive display device by adjusting its transmittance. The monitor 130 may include a configuration that simultaneously displays a portion of the image forming the virtual space and the real space. For example, the monitor 130 may display an image of the real space taken with a camera mounted on the HMD 120, or may make the real space visible by setting the transmittance of a part to be high.

In one aspect, monitor 130 may include a sub-monitor for displaying an image for the right eye and a sub-monitor for displaying an image for the left eye. In another aspect, the monitor 130 may be configured to integrally display the right eye image and the left eye image. In this case, monitor 130 includes a high speed shutter. The high-speed shutter operates so that images for the right eye and images for the left eye can be alternately displayed so that the images are only perceived by one eye.

In one aspect, HMD 120 includes a plurality of light sources (not shown). Each light source is realized by, for example, an LED (Light Emitting Diode) that emits infrared rays. HMD sensor 410 has a position tracking function for detecting movement of HMD 120. More specifically, HMD sensor 410 reads a plurality of infrared rays emitted by HMD 120 and detects the position and tilt of HMD 120 in real space.

In another aspect, HMD sensor 410 may be realized by a camera. In this case, the HMD sensor 410 can detect the position and tilt of the HMD 120 by performing image analysis processing using the image information of the HMD 120 output from the camera.

In another aspect, HMD 120 may include sensor 190 as a position detector instead of or in addition to HMD sensor 410. HMD 120 can use sensor 190 to detect the position and tilt of HMD 120 itself. For example, when the sensor 190 is an angular velocity sensor, a geomagnetic sensor, or an acceleration sensor, the HMD 120 can detect its own position and inclination using any of these sensors instead of the HMD sensor 410. As an example, when the sensor 190 is an angular velocity sensor, the angular velocity sensor detects the angular velocity around three axes of the HMD 120 in real space over time. The HMD 120 calculates temporal changes in the angle of the HMD 120 around the three axes based on each angular velocity, and further calculates the inclination of the HMD 120 based on the temporal changes in the angle.

Gaze sensor 140 detects the direction in which the right and left eyes of user 5 are directed. That is, the gaze sensor 140 detects the user's 5 line of sight. Detection of the direction of the line of sight is realized, for example, by a known eye tracking function. The gaze sensor 140 is realized by a sensor having the eye tracking function. In one aspect, gaze sensor 140 preferably includes a right eye sensor and a left eye sensor. The gaze sensor 140 may be, for example, a sensor that irradiates the right and left eyes of the user 5 with infrared light and detects the rotation angle of each eyeball by receiving reflected light from the cornea and iris of the irradiated light. . The gaze sensor 140 can detect the user's 5 line of sight based on each detected rotation angle.

The first camera 150 photographs the lower part of the user's 5 face. More specifically, the first camera 150 photographs the nose, mouth, etc. of the user 5. The second camera 160 photographs the eyes, eyebrows, etc. of the user 5. The housing on the user 5 side of the HMD 120 is defined as the inside of the HMD 120, and the housing on the opposite side of the HMD 120 from the user 5 is defined as the outside of the HMD 120. In one aspect, first camera 150 may be placed outside HMD 120, and second camera 160 may be placed inside HMD 120. Images generated by the first camera 150 and the second camera 160 are input to the computer 200. In another aspect, the first camera 150 and the second camera 160 may be implemented as one camera, and the face of the user 5 may be photographed with this one camera.

The microphone 170 converts the speech of the user 5 into an audio signal (electrical signal) and outputs it to the computer 200. The speaker 180 converts the audio signal into audio and outputs it to the user 5. In another aspect, HMD 120 may include earphones instead of speaker 180.

Controller 300 is connected to computer 200 by wire or wirelessly. Controller 300 accepts command input from user 5 to computer 200 . In one aspect, the controller 300 is configured to be graspable by the user 5. In another aspect, the controller 300 is configured to be attachable to the body or part of clothing of the user 5. In yet another aspect, the controller 300 may be configured to output at least one of vibration, sound, and light based on a signal transmitted from the computer 200. In yet another aspect, the controller 300 receives an operation from the user 5 for controlling the position and movement of an object placed in the virtual space.

In some aspects, controller 300 includes multiple light sources. Each light source is realized by, for example, an LED that emits infrared rays. HMD sensor 410 has a position tracking function. In this case, the HMD sensor 410 reads a plurality of infrared rays emitted by the controller 300 and detects the position and tilt of the controller 300 in real space. In another aspect, HMD sensor 410 may be realized by a camera. In this case, the HMD sensor 410 can detect the position and tilt of the controller 300 by executing image analysis processing using image information of the controller 300 output from the camera.

In a certain aspect, motion sensor 420 is attached to the user's 5 hand and detects the movement of the user's 5 hand. Attached to the user's 5 hand means, for example, that the motion sensor 420 is provided on the controller 300 that is configured to be graspable by the user 5. An example of the grip type controller 300 will be described later using FIG. 8. Further, the motion sensor 420 may be provided in a glove-shaped device (in other words, the glove-shaped controller 300) configured to be wearable by the user 5. In the case of the glove-shaped controller 300, the controller 300 does not fly away easily compared to the grip-type controller 300, so safety in real space can be ensured. The motion sensor 420 detects, for example, the rotational speed, number of rotations, and finger movements of the hand. The detected signal is sent to computer 200. Note that the glove-shaped controller 300 may be configured to be able to detect movements of a larger number of fingers (for example, all fingers) than the grip-type controller 300.
In another aspect, a sensor that is not attached to the user 5 may detect the movement of the user's 5 hand. For example, a signal from a camera photographing user 5 may be input to computer 200 as a signal representing user 5's action. The motion sensor 420 and the computer 200 are, for example, connected to each other wirelessly. In the case of wireless communication, the communication form is not particularly limited, and for example, Bluetooth (registered trademark) or other known communication methods may be used.

Display 430 displays an image similar to the image displayed on monitor 130. This allows users other than the user 5 wearing the HMD 120 to view the same images as the user 5. The image displayed on the display 430 does not need to be a three-dimensional image, and may be an image for the right eye or an image for the left eye. Examples of the display 430 include a liquid crystal display and an organic EL monitor.

Server 600 may send programs to computer 200. In another aspect, server 600 may communicate with other computers 200 to provide virtual reality to HMDs 120 used by other users. For example, when a plurality of users play a participatory game at an amusement facility, each computer 200 communicates signals based on the actions of each user with the other computers 200 via the server 600, so that multiple users can play a participatory game in the same virtual space. allows users to enjoy common games. Each computer 200 may communicate signals based on the actions of each user with other computers 200 without going through the server 600.

The external device 700 may be any device that can communicate with the computer 200. The external device 700 may be, for example, a device that can communicate with the computer 200 via the network 2, or a device that can directly communicate with the computer 200 through short-range wireless communication or a wired connection. Further, the external device 700 may be a device that can communicate with the server 600 via the network 2, for example. Examples of the external device 700 include, but are not limited to, a smart device, a PC (Personal Computer), and peripheral devices of the computer 200.

[Computer hardware configuration]
Referring to FIG. 2, computer 200 according to this embodiment will be described. FIG. 2 is a block diagram showing an example of the hardware configuration of computer 200 according to this embodiment. Computer 200 includes a processor 210, memory 220, storage 230, input/output interface 240, and communication interface 250 as main components. Each component is connected to a bus 260, respectively.

Processor 210 executes a series of instructions included in a program stored in memory 220 or storage 230 based on a signal given to computer 200 or based on the satisfaction of a predetermined condition. In one aspect, the processor 210 includes a CPU (Central Processing Unit), a GPU (Graphics
Processing Unit), MPU (Micro Processor Unit), FPGA (Field-Programmable Gate Array), and other devices.

Memory 220 temporarily stores programs and data. The program is loaded from storage 230, for example. The data includes data input to computer 200 and data generated by processor 210. In one aspect, memory 220 is implemented as RAM (Random Access Memory) or other volatile memory.

Storage 230 permanently holds programs and data. The storage 230 is implemented as, for example, a ROM (Read-Only Memory), a hard disk device, a flash memory, or other nonvolatile storage device. The programs stored in the storage 230 include a program for providing a virtual space in the HMD system 100, a simulation program, a game program, a user authentication program, and a program for realizing communication with other computers 200. The data stored in the storage 230 includes data and objects for defining a virtual space.

In another aspect, storage 230 may be realized as a removable storage device such as a memory card. In yet another aspect, a configuration may be used in which programs and data stored in an external storage device are used instead of the storage 230 built into the computer 200. According to such a configuration, for example, in a scene where a plurality of HMD systems 100 are used, such as in an amusement facility, it becomes possible to update programs and data all at once.

Input/output interface 240 communicates signals between HMD 120, HMD sensor 410, motion sensor 420, and display 430. The monitor 130, gaze sensor 140, first camera 150, second camera 160, microphone 170, and speaker 180 included in the HMD 120 can communicate with the computer 200 via the input/output interface 240 of the HMD 120. In one aspect, the input/output interface 240 is a USB (Universal Serial Bus), a DVI (Digital Visual
Interface), HDMI (registered trademark) (High-Definition Multimedia
Interface) is realized using other terminals. The input/output interface 240 is not limited to those described above.

In some aspects, input/output interface 240 may further communicate with controller 300. For example, input/output interface 240 receives input signals output from controller 300 and motion sensor 420. In another aspect, input/output interface 240 sends instructions output from processor 210 to controller 300. The command instructs the controller 300 to perform vibration, audio output, light emission, etc. Upon receiving the command, the controller 300 performs vibration, audio output, or light emission in accordance with the command.

Communication interface 250 is connected to network 2 and communicates with other computers (eg, server 600) connected to network 2. In one aspect, the communication interface 250 is, for example, a LAN (Local Area Network) or other wired communication interface, or a Wi-Fi (registered trademark), Bluetooth (registered trademark), NFC (Near Field Communication) or other wireless communication interface. It is realized as. Communication interface 250 is not limited to those described above.

In some aspects, processor 210 accesses storage 230, loads one or more programs stored in storage 230 into memory 220, and executes a series of instructions contained in the programs. The one or more programs may include the operating system of the computer 200, an application program for providing a virtual space, game software executable in the virtual space, and the like. Processor 210 sends a signal for providing a virtual space to HMD 120 via input/output interface 240. HMD 120 displays video on monitor 130 based on the signal.

In the example shown in FIG. 2, a configuration is shown in which the computer 200 is provided outside the HMD 120, but in another aspect, the computer 200 may be built into the HMD 120. As an example, a portable information communication terminal (for example, a smartphone) including the monitor 130 may function as the computer 200.

The computer 200 may have a configuration that is commonly used by a plurality of HMDs 120. According to such a configuration, for example, the same virtual space can be provided to a plurality of users, so each user can enjoy the same application as other users in the same virtual space.

In one embodiment, in the HMD system 100, a real coordinate system that is a coordinate system in real space is set in advance. The real coordinate system has three reference directions (axes) that are parallel to a vertical direction in real space, a horizontal direction perpendicular to the vertical direction, and a front-back direction perpendicular to both the vertical direction and the horizontal direction. The horizontal direction, vertical direction (vertical direction), and longitudinal direction in the real coordinate system are defined as the x-axis, y-axis, and z-axis, respectively. More specifically, in the real coordinate system, the x-axis is parallel to the horizontal direction of real space. The y-axis is parallel to the vertical direction of real space. The z-axis is parallel to the front-back direction of real space.

In some aspects, HMD sensor 410 includes an infrared sensor. When the infrared sensor detects each infrared ray emitted from each light source of HMD 120, the presence of HMD 120 is detected. The HMD sensor 410 further detects the position and inclination (orientation) of the HMD 120 in the real space according to the movement of the user 5 wearing the HMD 120 based on the value of each point (each coordinate value in the real coordinate system). do. More specifically, HMD sensor 410 can detect temporal changes in the position and tilt of HMD 120 using each value detected over time.

Each inclination of HMD 120 detected by HMD sensor 410 corresponds to each inclination of HMD 120 around three axes in the real coordinate system. The HMD sensor 410 sets the uvw visual field coordinate system to the HMD 120 based on the inclination of the HMD 120 in the real coordinate system. The uvw visual field coordinate system set in the HMD 120 corresponds to the viewpoint coordinate system when the user 5 wearing the HMD 120 views an object in the virtual space.

[uvw visual field coordinate system]
The uvw visual field coordinate system will be described with reference to FIG. 3. FIG. 3 is a diagram conceptually representing a UVW visual field coordinate system set in HMD 120 according to an embodiment. HMD sensor 410 detects the position and tilt of HMD 120 in the real coordinate system when HMD 120 is activated. Processor 210 sets the uvw visual field coordinate system in HMD 120 based on the detected value.

As shown in FIG. 3, the HMD 120 sets a three-dimensional UVW visual field coordinate system centered on the head of the user 5 wearing the HMD 120 (origin). More specifically, HMD 120 changes the horizontal direction, vertical direction, and front-back direction (x-axis, y-axis, z-axis) that define the real coordinate system by the inclination around each axis of HMD 120 within the real coordinate system. The three directions newly obtained by tilting around the respective axes are set as the pitch axis (u axis), yaw axis (v axis), and roll axis (w axis) of the uvw visual field coordinate system in HMD 120.

In a certain situation, when the user 5 wearing the HMD 120 is standing upright and viewing the front, the processor 210 sets the uvw visual field coordinate system parallel to the real coordinate system on the HMD 120. In this case, the horizontal direction (x-axis), vertical direction (y-axis), and longitudinal direction (z-axis) in the real coordinate system are the pitch axis (u-axis) and yaw axis (v-axis) of the uvw visual field coordinate system in HMD 120. , and coincides with the roll axis (w axis).

After the uvw visual field coordinate system is set on the HMD 120, the HMD sensor 410 can detect the tilt of the HMD 120 in the set uvw visual field coordinate system based on the movement of the HMD 120. In this case, the HMD sensor 410 detects the pitch angle (θu), yaw angle (θv), and roll angle (θw) of the HMD 120 in the uvw visual field coordinate system as the inclination of the HMD 120, respectively. The pitch angle (θu) represents the tilt angle of the HMD 120 around the pitch axis in the uvw visual field coordinate system. The yaw angle (θv) represents the tilt angle of the HMD 120 around the yaw axis in the uvw visual field coordinate system. The roll angle (θw) represents the tilt angle of the HMD 120 around the roll axis in the uvw visual field coordinate system.

The HMD sensor 410 sets the uvw visual field coordinate system in the HMD 120 after the HMD 120 has moved based on the detected inclination of the HMD 120. The relationship between HMD 120 and the uvw visual field coordinate system of HMD 120 is always constant regardless of the position and inclination of HMD 120. When the position and inclination of HMD 120 change, the position and inclination of the uvw visual field coordinate system of HMD 120 in the real coordinate system change in conjunction with the change in the position and inclination.

In one aspect, the HMD sensor 410 detects the HMD 120 based on the infrared light intensity obtained based on the output from the infrared sensor and the relative positional relationship between a plurality of points (for example, the distance between each point). The position in real space may be specified as a relative position with respect to the HMD sensor 410. The processor 210 may determine the origin of the uvw visual field coordinate system of the HMD 120 in real space (real coordinate system) based on the specified relative position.

[Virtual space]
The virtual space will be further explained with reference to FIG. 4. FIG. 4 is a diagram conceptually representing one aspect of representing the virtual space 11 according to an embodiment. The virtual space 11 has a spherical structure that covers the entire 360 degree direction of the center 12. In FIG. 4, the celestial sphere in the upper half of the virtual space 11 is illustrated in order not to complicate the explanation. In the virtual space 11, each mesh is defined. The position of each mesh is predefined as a coordinate value in an XYZ coordinate system, which is a global coordinate system defined in the virtual space 11. The computer 200 associates each partial image forming the panoramic image 13 (still image, moving image, etc.) that can be developed in the virtual space 11 with each corresponding mesh in the virtual space 11 .

In one aspect, in the virtual space 11, an XYZ coordinate system with the center 12 as the origin is defined. For example, the XYZ coordinate system is parallel to the real coordinate system. The horizontal direction, vertical direction (vertical direction), and front-back direction in the XYZ coordinate system are defined as the X-axis, Y-axis, and Z-axis, respectively. Therefore, the X-axis (horizontal direction) of the XYZ coordinate system is parallel to the x-axis of the real coordinate system, the Y-axis (vertical direction) of the XYZ coordinate system is parallel to the y-axis of the real coordinate system, and the The Z-axis (front-back direction) is parallel to the z-axis of the real coordinate system.

When the HMD 120 is activated, that is, in the initial state of the HMD 120, the virtual camera 14 is placed at the center 12 of the virtual space 11. In one aspect, processor 210 displays an image captured by virtual camera 14 on monitor 130 of HMD 120. The virtual camera 14 similarly moves in the virtual space 11 in conjunction with the movement of the HMD 120 in the real space. Thereby, changes in the position and inclination of HMD 120 in real space can be similarly reproduced in virtual space 11.

As in the case of the HMD 120, the uvw visual field coordinate system is defined for the virtual camera 14. The UVW visual field coordinate system of the virtual camera 14 in the virtual space 11 is defined to be linked to the UVW visual field coordinate system of the HMD 120 in the real space (real coordinate system). Therefore, when the tilt of HMD 120 changes, the tilt of virtual camera 14 also changes accordingly. The virtual camera 14 can also move in the virtual space 11 in conjunction with the movement of the user 5 wearing the HMD 120 in the real space.

The processor 210 of the computer 200 defines the viewing area 15 in the virtual space 11 based on the position and inclination (reference line of sight 16) of the virtual camera 14. The viewing area 15 corresponds to an area of the virtual space 11 that is visible to the user 5 wearing the HMD 120 . In other words, the position of the virtual camera 14 can be said to be the viewpoint of the user 5 in the virtual space 11.

The line of sight of the user 5 detected by the gaze sensor 140 is the direction in the viewpoint coordinate system when the user 5 visually recognizes an object. The uvw visual field coordinate system of the HMD 120 is equal to the viewpoint coordinate system when the user 5 views the monitor 130 . The uvw visual field coordinate system of the virtual camera 14 is linked to the uvw visual field coordinate system of the HMD 120. Therefore, the HMD system 100 according to a certain aspect can consider the line of sight of the user 5 detected by the gaze sensor 140 as the line of sight of the user 5 in the uvw visual field coordinate system of the virtual camera 14.

[User's line of sight]
Determination of the line of sight of the user 5 will be described with reference to FIG. 5 . FIG. 5 is a top view of the head of user 5 wearing HMD 120 according to an embodiment.

In one aspect, the gaze sensor 140 detects each line of sight of the user's 5 right eye and left eye. In a certain situation, when the user 5 is looking nearby, the gaze sensor 140 detects the lines of sight R1 and L1. In another aspect, when the user 5 is looking far away, the gaze sensor 140 detects the lines of sight R2 and L2. In this case, the angle that the lines of sight R2 and L2 make with the roll axis w is smaller than the angle that the lines of sight R1 and L1 make with the roll axis w. Gaze sensor 140 transmits detection results to computer 200.

When the computer 200 receives the detected values of the line of sight R1 and L1 from the gaze sensor 140 as the detection result of the line of sight, it specifies the point of gaze N1, which is the intersection of the line of sight R1 and L1, based on the detected values. On the other hand, when the computer 200 receives the detected values of the lines of sight R2 and L2 from the gaze sensor 140, it specifies the intersection of the lines of sight R2 and L2 as the point of gaze. The computer 200 specifies the line of sight N0 of the user 5 based on the position of the specified gaze point N1. The computer 200 detects, for example, the direction in which a straight line passing through the midpoint of the straight line connecting the right eye R and the left eye L of the user 5 and the gaze point N1 is the line of sight N0. The line of sight N0 is the direction in which the user 5 is actually directing his/her line of sight with both eyes. The line of sight N0 corresponds to the direction in which the user 5 is actually directing his or her line of sight with respect to the viewing area 15.

In another aspect, HMD system 100 may include a television broadcast reception tuner. According to such a configuration, the HMD system 100 can display television programs in the virtual space 11.

In yet another aspect, the HMD system 100 may include a communication circuit for connecting to the Internet or a calling function for connecting to a telephone line.

[Visibility area]
The viewing area 15 will be described with reference to FIGS. 6 and 7. FIG. 6 is a diagram showing a YZ cross section of the viewing area 15 viewed from the X direction in the virtual space 11. FIG. 7 is a diagram showing an XZ cross section of the viewing area 15 viewed from the Y direction in the virtual space 11.

As shown in FIG. 6, the viewing area 15 in the YZ cross section includes a region 18. The region 18 is defined by the position of the virtual camera 14, the reference line of sight 16, and the YZ cross section of the virtual space 11. The processor 210 defines a range including the polar angle α centered on the reference line of sight 16 in the virtual space as the region 18 .

As shown in FIG. 7, the viewing area 15 in the XZ cross section includes a region 19. The region 19 is defined by the position of the virtual camera 14, the reference line of sight 16, and the XZ cross section of the virtual space 11. The processor 210 defines a range including the azimuth angle β centered on the reference line of sight 16 in the virtual space 11 as a region 19 . The polar angles α and β are determined depending on the position of the virtual camera 14 and the inclination (orientation) of the virtual camera 14.

In one aspect, the HMD system 100 provides the user 5 with a view in the virtual space 11 by displaying the view image 17 on the monitor 130 based on a signal from the computer 200. The visibility image 17 is an image corresponding to a portion of the panoramic image 13 that corresponds to the visibility area 15. When the user 5 moves the HMD 120 worn on the head, the virtual camera 14 also moves in conjunction with the movement. As a result, the position of the viewing area 15 in the virtual space 11 changes. Thereby, the visual field image 17 displayed on the monitor 130 is updated to an image of the panoramic image 13 that is superimposed on the visual field 15 in the direction toward which the user 5 faces in the virtual space 11 . The user 5 can visually recognize a desired direction in the virtual space 11.

In this way, the inclination of the virtual camera 14 corresponds to the line of sight (reference line of sight 16) of the user 5 in the virtual space 11, and the position where the virtual camera 14 is placed corresponds to the viewpoint of the user 5 in the virtual space 11. Therefore, by changing the position or inclination of the virtual camera 14, the image displayed on the monitor 130 is updated and the field of view of the user 5 is moved.

While wearing the HMD 120, the user 5 can view only the panoramic image 13 developed in the virtual space 11 without viewing the real world. Therefore, the HMD system 100 can provide the user 5 with a highly immersive feeling in the virtual space 11.

In one aspect, the processor 210 may move the virtual camera 14 in the virtual space 11 in conjunction with the movement of the user 5 wearing the HMD 120 in the real space. In this case, the processor 210 specifies the image area (visual area 15) to be projected on the monitor 130 of the HMD 120 based on the position and tilt of the virtual camera 14 in the virtual space 11.

In some aspects, virtual camera 14 may include two virtual cameras: a virtual camera for providing images for the right eye and a virtual camera for providing images for the left eye. Appropriate parallax is set for the two virtual cameras so that the user 5 can recognize the three-dimensional virtual space 11. In another aspect, the virtual camera 14 may be realized by one virtual camera. In this case, an image for the right eye and an image for the left eye may be generated from an image obtained by one virtual camera. In this embodiment, the virtual camera 14 includes two virtual cameras, and the roll axis (w) generated by combining the roll axes of the two virtual cameras is adapted to the roll axis (w) of the HMD 120. The technical idea according to the present disclosure will be exemplified as being configured so as to.

[controller]
An example of the controller 300 will be described with reference to FIG. 8. FIG. 8 is a diagram showing a schematic configuration of a controller 300 according to an embodiment.

As shown in FIG. 8, in one aspect, the controller 300 may include a right controller 300R and a left controller (not shown). The right controller 300R is operated by the user 5's right hand. The left controller is operated with the user's 5 left hand. In one aspect, the right controller 300R and the left controller are symmetrically configured as separate devices. Therefore, the user 5 can freely move the right hand holding the right controller 300R and the left hand holding the left controller. In another aspect, controller 300 may be an integrated controller that accepts operation with both hands. The right controller 300R will be explained below.

The right controller 300R includes a grip 310, a frame 320, and a top surface 330. The grip 310 is configured to be held by the user's 5 right hand. For example, the grip 310 may be held by the palm and three fingers (middle finger, ring finger, little finger) of the user 5's right hand.

Grip 310 includes buttons 340 and 350 and a motion sensor 420. The button 340 is arranged on the side surface of the grip 310 and accepts operation by the middle finger of the right hand. Button 350 is arranged on the front surface of grip 310 and accepts operation by the index finger of the right hand. In some aspects, buttons 340, 350 are configured as trigger-type buttons. Motion sensor 420 is built into the housing of grip 310. If the motion of the user 5 can be detected from around the user 5 by a camera or other device, the grip 310 may not include the motion sensor 420.

Frame 320 includes a plurality of infrared LEDs 360 arranged along its circumferential direction. The infrared LED 360 emits infrared rays while a program using the controller 300 is being executed in accordance with the progress of the program. The infrared rays emitted from the infrared LED 360 can be used to detect the respective positions and postures (tilt, orientation) of the right controller 300R and the left controller. Although the example shown in FIG. 8 shows infrared LEDs 360 arranged in two rows, the number of arrangements is not limited to that shown in FIG. 8. A single row or an array of three or more rows may be used.

The top surface 330 includes buttons 370 and 380 and an analog stick 390. Buttons 370, 380 are configured as push buttons. The buttons 370 and 380 accept operations by the thumb of the user's 5 right hand. In a certain situation, the analog stick 390 accepts operation in any direction of 360 degrees from the initial position (neutral position). The operation includes, for example, an operation for moving an object placed in the virtual space 11.

In one aspect, the right controller 300R and the left controller include batteries for driving the infrared LEDs 360 and other components. Batteries include, but are not limited to, rechargeable types, button types, dry battery types, and the like. In another aspect, the right controller 300R and the left controller may be connected to a USB interface of the computer 200, for example. In this case, the right controller 300R and the left controller do not require batteries.

As shown in states (A) and (B) of FIG. 8, for example, the yaw, roll, and pitch directions are defined for the right hand of the user 5. When user 5 extends his thumb and index finger, the direction in which the thumb extends is the yaw direction, the direction in which the index finger extends is the roll direction, and the direction perpendicular to the plane defined by the yaw axis and the roll axis is the pitch direction. It is defined as.

[Server hardware configuration]
With reference to FIG. 9, server 600 according to this embodiment will be described. FIG. 9 is a block diagram illustrating an example of the hardware configuration of server 600 according to an embodiment. The server 600 includes a processor 610, a memory 620, a storage 630, an input/output interface 640, and a communication interface 650 as main components. Each component is connected to a bus 660, respectively.

Processor 610 executes a series of instructions included in a program stored in memory 620 or storage 630 based on a signal given to server 600 or based on the fulfillment of a predetermined condition. In some aspects, processor 610 is implemented as a CPU, GPU, MPU, FPGA, or other device.

Memory 620 temporarily stores programs and data. The program is loaded from storage 630, for example. The data includes data input to server 600 and data generated by processor 610. In some aspects, memory 620 is implemented as RAM or other volatile memory.

Storage 630 permanently holds programs and data. The storage 630 is implemented as, for example, a ROM, a hard disk device, a flash memory, or other nonvolatile storage device. The programs stored in the storage 630 may include a program for providing a virtual space in the HMD system 100, a simulation program, a game program, a user authentication program, and a program for realizing communication with the computer 200. The data stored in the storage 630 may include data and objects for defining a virtual space.

In another aspect, storage 630 may be implemented as a removable storage device such as a memory card. In yet another aspect, a configuration may be used in which programs and data stored in an external storage device are used instead of the storage 630 built into the server 600. According to such a configuration, for example, in a scene where a plurality of HMD systems 100 are used, such as in an amusement facility, it becomes possible to update programs and data all at once.

Input/output interface 640 communicates signals with input/output devices. In one aspect, the input/output interface 640 is implemented using a USB, DVI, HDMI (registered trademark), or other terminal. The input/output interface 640 is not limited to those described above.

Communication interface 650 is connected to network 2 and communicates with computer 200 connected to network 2 . In one aspect, the communication interface 650 is implemented as, for example, a LAN or other wired communication interface, or a Wi-Fi, Bluetooth, NFC, or other wireless communication interface. Communication interface 650 is not limited to those described above.

In some aspects, processor 610 accesses storage 630, loads one or more programs stored in storage 630 into memory 620, and executes a series of instructions contained in the programs. The one or more programs may include the operating system of the server 600, an application program for providing a virtual space, game software executable in the virtual space, and the like. Processor 610 may send signals to computer 200 to provide the virtual space via input/output interface 640.

[HMD control device]
A control device for the HMD 120 will be described with reference to FIG. 10. In one embodiment, the control device is implemented by a computer 200 having a known configuration. FIG. 10 is a block diagram illustrating computer 200 as a modular configuration according to an embodiment.

As shown in FIG. 10, the computer 200 includes a control module 510, a rendering module 520, a memory module 530, and a communication control module 540. In some aspects, control module 510 and rendering module 520 are implemented by processor 210. In another aspect, multiple processors 210 may act as control module 510 and rendering module 520. Memory module 530 is realized by memory 220 or storage 230. Communication control module 540 is realized by communication interface 250.

Control module 510 controls virtual space 11 provided to user 5 . Control module 510 defines virtual space 11 in HMD system 100 using virtual space data representing virtual space 11 . The virtual space data is stored in the memory module 530, for example. The control module 510 may generate virtual space data or obtain virtual space data from the server 600 or the like.

The control module 510 places objects in the virtual space 11 using object data representing the objects. Object data is stored in memory module 530, for example. The control module 510 may generate object data or obtain object data from the server 600 or the like. The objects include, for example, an avatar object that is the alter ego of the user 5 (in other words, a character object of the user 5), a character object, an operation object such as a virtual hand operated by the controller 300, and a forest arranged as the story of the game progresses. , landscapes including mountains and the like, cityscapes, animals, etc.

The control module 510 places the avatar object of the user 5 of the other computer 200 connected via the network 2 in the virtual space 11 . In one aspect, the control module 510 places the avatar object of the user 5 in the virtual space 11. In one aspect, the control module 510 places an avatar object imitating the user 5 in the virtual space 11 based on an image including the user 5. In another aspect, the control module 510 arranges in the virtual space 11 an avatar object selected by the user 5 from among multiple types of avatar objects (for example, an object imitating an animal or a deformed human object). do.

Control module 510 identifies the tilt of HMD 120 based on the output of HMD sensor 410. In another aspect, control module 510 identifies the tilt of HMD 120 based on the output of sensor 190 functioning as a motion sensor. The control module 510 detects organs (eg, mouth, eyes, eyebrows) that make up the user's 5 face from the images of the user's 5 face generated by the first camera 150 and the second camera 160. The control module 510 detects the movement (shape) of each detected organ.

Control module 510 detects the line of sight of user 5 in virtual space 11 based on the signal from gaze sensor 140 . The control module 510 detects a viewpoint position (coordinate value in the XYZ coordinate system) where the detected line of sight of the user 5 intersects with the celestial sphere of the virtual space 11. More specifically, the control module 510 detects the viewpoint position based on the line of sight of the user 5 defined by the uvw coordinate system and the position and tilt of the virtual camera 14. Control module 510 transmits the detected viewpoint position to server 600. In another aspect, control module 510 may be configured to send gaze information representative of user's 5 gaze to server 600. In this case, the viewpoint position can be calculated based on the line-of-sight information received by the server 600.

The control module 510 reflects the movement of the HMD 120 detected by the HMD sensor 410 on the avatar object. For example, the control module 510 detects that the HMD 120 is tilted and places the avatar object at a tilt. The control module 510 reflects the detected motions of the facial organs on the face of the avatar object placed in the virtual space 11. The control module 510 receives the gaze information of the other user 5 from the server 600, and reflects the information on the gaze of the avatar object of the other user 5. In one aspect, the control module 510 reflects the movement of the controller 300 on the avatar object and the operation object. In this case, the controller 300 may include a motion sensor, an acceleration sensor, a plurality of light emitting elements (for example, infrared LEDs), etc. for detecting the movement of the controller 300. Further, the movement of the controller 300 may be a movement of the analog stick 390, a movement of the buttons 370, 380, or the like. Then, the control module 510 may move the avatar object within the virtual space 11 based on, for example, the movement of the analog stick 390 (in other words, the user's operation on the analog stick 390).

The control module 510 arranges in the virtual space 11 an operation object for receiving an operation by the user 5 in the virtual space 11 . The user 5 operates, for example, an object arranged in the virtual space 11 by operating the operation object. In one aspect, the operation object may include, for example, a hand object that is a virtual hand corresponding to the user's 5 hand. In one aspect, the control module 510 moves the hand object in the virtual space 11 in conjunction with the movement of the user's 5 hand in the real space based on the output of the motion sensor 420. In one aspect, the manipulation object may correspond to a hand portion of an avatar object.

The control module 510 detects a collision when each of the objects arranged in the virtual space 11 collides with another object. For example, the control module 510 can detect the timing at which the collision area of a certain object touches the collision area of another object, and performs predetermined processing when this detection is performed. The control module 510 can detect the timing when the objects are no longer in contact with each other, and performs predetermined processing when this detection is performed. The control module 510 can detect that objects are touching. For example, when a manipulation object and another object touch, the control module 510 detects the contact between the manipulation object and the other object, and performs a predetermined process.

In one aspect, control module 510 controls image display on monitor 130 of HMD 120. For example, the control module 510 arranges the virtual camera 14 in the virtual space 11. The control module 510 controls the position of the virtual camera 14 in the virtual space 11 and the tilt (orientation) of the virtual camera 14. The control module 510 defines the viewing area 15 according to the tilt of the head of the user 5 wearing the HMD 120 and the position of the virtual camera 14. The rendering module 520 generates the viewing image 17 displayed on the monitor 130 based on the determined viewing area 15. The view image 17 generated by the rendering module 520 is output to the HMD 120 by the communication control module 540.

When control module 510 detects user 5's utterance using microphone 170 from HMD 120, control module 510 identifies computer 200 to which audio data corresponding to the utterance is to be transmitted. The audio data is sent to the specified computer 200 by control module 510. When control module 510 receives audio data from another user's computer 200 via network 2, it outputs audio (speech) corresponding to the audio data from speaker 180.

Memory module 530 holds data used by computer 200 to provide virtual space 11 to user 5 . In some aspects, memory module 530 maintains spatial information, object information, and user information.

The spatial information holds one or more templates defined for providing the virtual space 11.

The object information includes a plurality of panoramic images 13 constituting the virtual space 11 and object data for arranging objects in the virtual space 11. Panoramic image 13 may include still images and moving images. The panoramic image 13 may include an image in a non-real space and an image in a real space. Examples of images in unreal space include images generated by computer graphics.

The user information holds a user ID that identifies the user 5. The user ID is, for example, an IP (Internet Protocol) address or a MAC (Media Access Protocol) address set on the computer 200 used by the user.
Control) address. In another aspect, the user ID may be set by the user. The user information includes a program for causing the computer 200 to function as a control device for the HMD system 100, and the like. Further, the user information includes information managed for each service account (in other words, for each user ID).

Data and programs stored in memory module 530 are input by user 5 of HMD 120. Alternatively, processor 210 downloads a program or data from a computer (for example, server 600) operated by a business providing the content, and stores the downloaded program or data in memory module 530.

The communication control module 540 can communicate with the server 600 and other information communication devices via the network 2.

In certain aspects, control module 510 and rendering module 520 may be implemented using, for example, Unity® provided by Unity Technologies. In another aspect, the control module 510 and the rendering module 520 may be implemented as a combination of circuit elements that implement each process.

Processing in computer 200 is realized by hardware and software executed by processor 210. Such software may be pre-stored in the hard disk or other memory module 530. Software may be stored on a CD-ROM or other computer-readable non-volatile data storage medium and distributed as a program product. Alternatively, the software may be provided as a downloadable program product by an information provider connected to the Internet or other networks. Such software is read from a data recording medium by an optical disk drive or other data reading device, or is downloaded from the server 600 or another computer via the communication control module 540, and then temporarily stored in the memory module 530. Ru. The software is read from memory module 530 by processor 210 and stored in RAM in the form of an executable program. Processor 210 executes the program.

[Control structure of HMD system]
The control structure of the HMD set 110 will be described with reference to FIG. 11. FIG. 11 is a sequence chart showing part of the processing performed in the HMD set 110 according to an embodiment.

As shown in FIG. 11, in step S1110, the processor 210 of the computer 200, as the control module 510, specifies virtual space data and defines the virtual space 11.

In step S1120, processor 210 initializes virtual camera 14. For example, in the work area of the memory, the processor 210 places the virtual camera 14 at a predefined center 12 in the virtual space 11 and directs the line of sight of the virtual camera 14 in the direction in which the user 5 is facing.

In step S1130, processor 210, as rendering module 520, generates view image data for displaying the initial view image. The generated visual field image data is output to the HMD 120 by the communication control module 540.

In step S1132, monitor 130 of HMD 120 displays a visual field image based on the visual field image data received from computer 200. The user 5 wearing the HMD 120 can recognize the virtual space 11 by viewing the visual field image.

In step S1134, HMD sensor 410 detects the position and tilt of HMD 120 based on a plurality of infrared lights emitted from HMD 120. The detection results are output to the computer 200 as motion detection data.

In step S1140, processor 210 identifies the viewing direction of user 5 wearing HMD 120 based on the position and tilt included in the motion detection data of HMD 120.

In step S1150, processor 210 executes the application program and places objects in virtual space 11 based on instructions included in the application program.

In step S1160, controller 300 detects user 5's operation and outputs detection data representing the detected operation to computer 200. In another aspect, the operation by the user 5 may be detected based on images from cameras placed around the user 5.

In step S1170, processor 210 detects the operation of controller 300 by user 5 based on the detection data acquired from controller 300.

In step S1180, processor 210 generates visual field image data based on user 5's operation of controller 300. The generated visual field image data is output to the HMD 120 by the communication control module 540.

In step S1190, HMD 120 updates the view image based on the received view image data, and displays the updated view image on monitor 130.

[Avatar object]
The avatar object according to this embodiment will be described with reference to FIGS. 12(A) and 12(B). The following is a diagram explaining the avatar objects of each user 5 of the HMD sets 110A and 110B. Hereinafter, the user of HMD set 110A will be referred to as user 5A, the user of HMD set 110B will be referred to as user 5B, the user of HMD set 110C will be referred to as user 5C, and the user of HMD set 110D will be referred to as user 5D. A is attached to the reference numeral of each component regarding the HMD set 110A, B is attached to the reference numeral of each component regarding the HMD set 110B, C is attached to the reference numeral of each component regarding the HMD set 110C, and the HMD set D is added to the reference numeral of each component related to 110D. For example, HMD 120A is included in HMD set 110A.

FIG. 12A is a schematic diagram showing a situation in which each HMD 120 provides the virtual space 11 to the user 5 in the network 2. Computers 200A to 200D provide virtual spaces 11A to 11D to users 5A to 5D via HMDs 120A to 120D, respectively. In the example shown in FIG. 12(A), the virtual space 11A and the virtual space 11B are configured by the same data. In other words, the computer 200A and the computer 200B share the same virtual space. In the virtual space 11A and the virtual space 11B, an avatar object 6A of the user 5A and an avatar object 6B of the user 5B exist. The avatar object 6A in the virtual space 11A and the avatar object 6B in the virtual space 11B are each wearing an HMD 120, but this is to make the explanation easier to understand, and in reality, these objects are not wearing the HMD 120. Not yet.

In one aspect, the processor 210A may place the virtual camera 14A that captures the visual field image 17A of the user 5A at the eye position of the avatar object 6A.

FIG. 12(B) is a diagram showing the visual field image 17A of the user 5A in FIG. 12(A). The visual field image 17A is an image displayed on the monitor 130A of the HMD 120A. This visual field image 17A is an image generated by the virtual camera 14A. An avatar object 6B of the user 5B is displayed in the view image 17A. Although not particularly illustrated, the avatar object 6A of the user 5A is also displayed in the visual field image of the user 5B.

In the state shown in FIG. 12(B), the user 5A can communicate with the user 5B through dialogue through the virtual space 11A. More specifically, user 5A's voice acquired by microphone 170A is transmitted to user 5B's HMD 120B via server 600, and output from speaker 180B provided in HMD 120B. User 5B's voice is transmitted to user 5A's HMD 120A via server 600, and output from speaker 180A provided in HMD 120A.

The movements of the user 5B (the movements of the HMD 120B and the movements of the controller 300B) are reflected on the avatar object 6B placed in the virtual space 11A by the processor 210A. Thereby, the avatar object 6B that is operated and moved by the user 5B is visually recognized by the user 5A.

FIG. 13 is a sequence chart showing part of the processing executed in HMD system 100 according to this embodiment. Although the HMD set 110D is not illustrated in FIG. 13, the HMD set 110D also operates in the same manner as the HMD sets 110A, 110B, and 110C. In the following description, A is added to the reference numeral of each component regarding the HMD set 110A, B is added to the reference numeral of each component regarding the HMD set 110B, and C is added to the reference numeral of each component regarding the HMD set 110C. D is added to the reference numeral of each component related to the HMD set 110D.

In step S1310A, processor 210A in HMD set 110A acquires avatar information for determining the behavior of avatar object 6A in virtual space 11A. This avatar information includes information regarding the avatar, such as movement information, face tracking data, and voice data. The motion information includes information indicating temporal changes in the position and tilt of the HMD 120A, information indicating the movement of the user's 5A hand detected by the motion sensor 420A, etc., and information indicating the user's operation detected by the controller 300, etc. Including. Examples of the face tracking data include data that specifies the position and size of each part of the user's 5A face. Examples of the face tracking data include data indicating the movement of each organ that makes up the face of the user 5A and line of sight data. Examples of the audio data include data representing the user's 5A's voice acquired by the microphone 170A of the HMD 120A. The avatar information may include information that specifies the avatar object 6A or the user 5A associated with the avatar object 6A, information that specifies the virtual space 11A in which the avatar object 6A exists, and the like. Information that specifies the avatar object 6A and the user 5A includes a user ID. A room ID can be cited as information for specifying the virtual space 11A in which the avatar object 6A exists. Processor 210A transmits the avatar information acquired as described above to server 600 via network 2.

In step S1310B, processor 210B in HMD set 110B acquires avatar information for determining the behavior of avatar object 6B in virtual space 11B, and transmits it to server 600, similar to the process in step S1310A. Similarly, in step S1310C, processor 210C in HMD set 110C acquires avatar information for determining the behavior of avatar object 6C in virtual space 11C, and transmits it to server 600.

In step S1320, server 600 temporarily stores the avatar information received from each of HMD set 110A, HMD set 110B, and HMD set 110C. The server 600 integrates the avatar information of all users (in this example, users 5A to 5C) associated with the common virtual space 11 based on the user ID, room ID, etc. included in each avatar information. Then, the server 600 transmits the integrated avatar information to all users associated with the virtual space 11 at a predetermined timing. As a result, synchronization processing is executed. Such synchronization processing allows HMD set 110A, HMD set 110B, and HMD set 110C to share each other's avatar information at approximately the same timing.

Subsequently, each HMD set 110A to 110C executes the processes of steps S1330A to S1330C based on the avatar information transmitted from server 600 to each HMD set 110A to 110C. The process in step S1330A corresponds to the process in step S1180 in FIG.

In step S1330A, the processor 210A in the HMD set 110A updates the information of the avatar objects 6B and 6C of the other users 5B and 5C in the virtual space 11A. Specifically, processor 210A updates the position, orientation, etc. of avatar object 6B in virtual space 11 based on movement information included in the avatar information transmitted from HMD set 110B. For example, the processor 210A updates the information (position, orientation, etc.) of the avatar object 6B included in the object information stored in the memory module 530. Similarly, the processor 210A updates the information (position, orientation, etc.) of the avatar object 6C in the virtual space 11 based on the movement information included in the avatar information transmitted from the HMD set 110C.

In step S1330B, processor 210B in HMD set 110B updates information on avatar objects 6A and 6C of users 5A and 5C in virtual space 11B, similar to the process in step S1330A. Similarly, in step S1330C, the processor 210C in the HMD set 110C updates the information of the avatar objects 6A, 6B of the users 5A, 5B in the virtual space 11C.

[Detailed configuration of module]
The details of the module configuration of computer 200 will be described with reference to FIG. 14. FIG. 14 is a block diagram showing a detailed configuration of modules of computer 200 according to an embodiment.

As shown in FIG. 14, the control module 510 includes a virtual camera control module 1421, a viewing area determination module 1422, a reference line of sight identification module 1423, a facial organ detection module 1424, a movement detection module 1425, and a virtual space definition module. It includes a module 1426, a virtual object generation module 1427, an operation object control module 1428, and an avatar control module 1429. Rendering module 520 includes a view image generation module 1438.

The virtual camera control module 1421 arranges the virtual camera 14 in the virtual space 11. The virtual camera control module 1421 controls the arrangement position of the virtual camera 14 in the virtual space 11 and the direction (tilt) of the virtual camera 14. The viewing area determining module 1422 defines the viewing area 15 according to the direction of the head of the user wearing the HMD 120 and the placement position of the virtual camera 14. The visibility image generation module 1438 generates the visibility image 17 to be displayed on the monitor 130 based on the determined visibility area 15.

The reference line of sight specifying module 1423 specifies the line of sight of the user 5 based on the signal from the gaze sensor 140. The facial organ detection module 1424 detects the organs that make up the face of the user 5 (eg, mouth, eyes, eyebrows) from the images of the user's 5 face generated by the first camera 150 and the second camera 160. The motion detection module 1425 detects the motion (shape) of each organ detected by the facial organ detection module 1424.

The virtual space definition module 1426 defines the virtual space 11 in the HMD system 100 by generating virtual space data representing the virtual space 11.

The virtual object generation module 1427 generates objects placed in the virtual space 11. Objects may include, for example, landscapes including forests, mountains, etc., animals, etc. that are placed as the game's story progresses.

The operation object control module 1428 arranges in the virtual space 11 an operation object for receiving a user's operation in the virtual space 11 . The user manipulates, for example, an object placed in the virtual space 11 by manipulating the manipulation object. In one aspect, the operation object may include, for example, a hand object corresponding to the hand of a user wearing HMD 120. In one aspect, the operation object may correspond to a hand portion of an avatar object, which will be described later.

The avatar control module 1429 generates data for placing avatar objects 6 of users of other computers 200 connected via the network 2 in the virtual space 11. In one aspect, the avatar control module 1429 generates data for arranging the avatar object 6 of the user 5 in the virtual space 11. In one aspect, avatar control module 1429 generates avatar object 6 imitating user 5 based on an image including user 5. In another aspect, the avatar control module 1429 selects the avatar object 6 that has been selected by the user 5 from among a plurality of types of avatar objects 6 (for example, an object imitating an animal or a deformed human object) in the virtual space. 11 is generated.

The avatar control module 1429 reflects the movement of the HMD 120 detected by the HMD sensor 410 on the avatar object 6. For example, the avatar control module 1429 detects that the HMD 120 is tilted and generates data for tilting and arranging the avatar object 6. In one aspect, the avatar control module 1429 reflects the movement of the controller 300 on the avatar object 6. In another aspect, the avatar control module 1429 reflects the motion of the facial organs detected by the motion detection module 1425 on the face of the avatar object 6 placed in the virtual space 11. That is, the avatar control module 1429 reflects the facial motion of the user 5A on the avatar object 6. In this way, the avatar object 6 is operated (in other words, moved) by the motion detected by the HMD sensor 410, the controller 300, or the motion detection module 1425.

[Server module configuration]
The module configuration of the server 600 will be described with reference to FIG. 15. FIG. 15 is a block diagram illustrating the configuration of modules of server 600 according to an embodiment. As shown in FIG. 15, the server 600 includes a control module 1610, a memory module 1630, and a communication control module 1640. In some aspects, control module 1610 is implemented by processor 610. Memory module 1630 is implemented by memory 620 or storage 630. Communication control module 1640 is realized by communication interface 650.

The control module 1610 includes a reality information acquisition module 1731.

Memory module 1630 holds data used by computer 200 to provide virtual space 11 to user 5 . In some aspects, memory module 1630 maintains spatial information, object information, and user information. The spatial information, object information, and user information of memory module 1630 may include the spatial information, object information, and user information of memory module 530 described above, respectively. Therefore, the explanation is omitted here.

Communication control module 1640 receives various types of information and various requests from each HMD set 110. As an example, the information that communication control module 1640 receives from each HMD set 110 may include spatial information, object information, user information, and avatar information. Communication control module 1640 transmits various types of information and various requests to each HMD set 110. As an example, the information that communication control module 1640 sends to each HMD set 110 may include spatial information, object information, user information, and avatar information.

Processing in server 600 is realized by hardware and software executed by processor 610. Such software may be pre-stored on a hard disk or other memory module 1630. Software may be stored on a CD-ROM or other computer-readable non-volatile data storage medium and distributed as a program product. Alternatively, the software may be provided as a downloadable program product by an information provider connected to the Internet or other networks. Such software is read from a data recording medium by an optical disk drive or other data reading device, or is downloaded from a predetermined computer via the communication control module 1640, and then temporarily stored in the memory module 1630. The software is read from memory module 1630 by processor 610 and stored in RAM in the form of an executable program. Processor 610 executes the program.

Note that the module configurations of the computer 200 and the server 600 described above are merely examples. Each device, computer 200 and server 600, may include at least some of the modules (in other words, functions) provided by other devices. Further, each device such as the computer 200 and the server 600 does not need to be realized by an integrated device, and may be realized by, for example, a plurality of devices connected via a network or the like.

Furthermore, in this embodiment, the description will be given assuming that the processor 210 of the computer 200 or the processor 610 of the server 600 performs the above-mentioned or later-described processes by executing a program stored in the HMD system 100. However, at least a portion of the processing described above or described below, which is performed by the processor 210, may be performed by a processor other than the processor 210. Further, at least a portion of the processing described above or described below, which is performed by the processor 610, may be performed by a processor other than the processor 610. In other words, the computer that executes the program in this embodiment may be either the computer 200 or the server 600, or may be realized by a combination of multiple devices.

[Processing according to this embodiment]
When the control module 510 of the computer 200 detects an utterance using the microphone 170, it identifies the computer 200 to which audio data corresponding to the utterance is to be transmitted, and transmits the audio data. Further, when receiving audio data from another user's computer 200, the control module 510 outputs audio corresponding to the audio data from the speaker 180. For example, in the state shown in FIG. 12(B), user 5A's voice acquired by microphone 170A is transmitted to user 5B's HMD 120B, and output from speaker 180B provided in HMD 120B. On the other hand, user 5B's voice is transmitted to user 5A's HMD 120A, and output from speaker 180A provided in HMD 120A. In this way, in the virtual space 11, the user 5A and the user 5B are able to interact via the avatar object 6A and the avatar object 6B. The avatar control module 1429 (character control means) included in the control module 510 of each computer 200 causes the avatar object 6 (character) to appear in the virtual space 11 and enables interaction via the avatar object 6. Note that the interaction between the user 5A and the user 5B in the virtual space 11 (in other words, the interaction via the avatar object 6A and the avatar object 6B) may be based on an exchange of text messages.

Hereinafter, as shown in FIG. 12(A), a case where each HMD 120 provides the virtual space 11 to the user 5 will be described. In addition, in the virtual space 11 shown in FIG. 12(A), the avatar object 6B unilaterally follows the avatar object 6A, and the avatar object 6B does not come into contact with the avatar object 6A, but engages in certain actions with the avatar object 6A. It is assumed that the avatar object 6A is causing a nuisance, such as following the avatar object 6A around while maintaining a safe distance. The nuisance act is so-called stalking act. It can be said that the avatar object 6A is a victim of stalking in the virtual space 11.

In this embodiment, in the virtual space 11, when the user 5A operating the avatar object 6A makes predetermined settings for the avatar object 6B, the identifiability of the avatar object 6A to the user 5B operating the avatar object 6B changes. . Distinctiveness (in other words, distinctive characteristics) can be said to be a property (in other words, information that can be identified) that allows each avatar object 6 (in other words, each user 5) to be identified, and specifically includes appearance, identification, etc. There are objects, voices, behaviors, etc.

(Identifiability)
Appearance can also be referred to as appearance, external appearance, etc. If the appearance of the avatar object 6A changes with respect to the user 5B who operates the avatar object 6B based on the user 5A operating the avatar object 6A making predetermined settings for the avatar object 6B, the user 5A operating the avatar object 6B may not be able to operate the avatar object 6B. It becomes difficult for the user 5B to find the avatar object 6A in the virtual space 11. Therefore, it becomes difficult for the avatar object 6B to cause trouble to the avatar object 6A. In other words, the nuisance can be suppressed.
Further, the identification object includes, for example, a handle name display (hereinafter referred to as handle name). In this embodiment, the handle name of each avatar object 6 can be displayed around (for example, above the head of) each avatar object 6 in the virtual space 11. The handle name can be set by the user 5 of each avatar object 6. Note that the handle name may be the same as the user ID. The handle name can also be said to be an identification name that allows each user 5 (in other words, each avatar object 6) to be identified. Note that "identifiable" may mean that the user 5 can be uniquely identified, or may mean that there is a possibility that the user 5 cannot be uniquely identified. In other words, the same handle name may or may not be assigned to a plurality of users 5. Further, the identification name may include a name associated with the avatar object 6 (character). Further, the identification object is not limited to a handle name, but may be a display related to the profile or a status of the user 5 (in other words, the avatar object 6). If the user 5A operating the avatar object 6A has made predetermined settings for the avatar object 6B, and the identification object (for example, handle name) of the avatar object 6A for the user 5B is changed, the user 5B thinks that the avatar object that he thought was the avatar object 6A may not be the avatar object 6A, and there is a high possibility that he will stop bothering the avatar object 6A. In other words, it is possible to suppress acts of nuisance towards the avatar object 6A.

Next, a case where the voice changes as a form of distinctiveness will be explained. When the voice of the avatar object 6A to the user 5B changes, the user 5B hears a voice different from the voice uttered by the user 5A. A change in voice may mean a change in voice quality (for example, pitch), or a change in speaking style such as intonation or dialect. When the voice of the avatar object 6A (user 5A) changes based on the user 5A making predetermined settings, for example from standard Japanese to Kansai dialect, the user 5B feels strange and changes the voice of the avatar object 6A. The user may think that the avatar object that he thought was not the avatar object 6A, and there is a high possibility that he will stop bothering the avatar object 6A. In other words, it is possible to suppress acts of nuisance towards the avatar object 6A.
Next, a case will be described in which the behavior as identifiability changes. Behavior can also be referred to as behavior. Furthermore, behavior may include behavioral habits. When the behavior of the avatar object 6A with respect to the user 5B changes, the behavior of the avatar object 6A that is visually recognized by the user 5B differs in part from the behavior of the user 5A. An example of a change in behavior is when the user 5A moves his right arm while holding the right-hand controller 300, and the left arm of the avatar object 6A is controlled to move in accordance with the movement. Further, even though the user 5A does not perform a predetermined action (for example, combing his hair) with the hand holding the controller 300, the avatar object 6A is controlled to perform the predetermined action at a predetermined frequency. There are things to do. If the behavior of the avatar object 6A changes, for example, if the avatar object 6A, which used to use its right hand more frequently, now uses its left hand more often, the user 5B may feel something strange about the avatar object 6A, which he thought was the avatar object 6A. The user thinks that the object may not be the avatar object 6A, and is more likely to stop bothering the avatar object 6A. In other words, it is possible to suppress acts of nuisance towards the avatar object 6A.
In the present embodiment, a case will be described below in which the appearance and handle name are changed as the distinctiveness of the avatar object 6A based on a predetermined setting by the user 5A. , but is not limited to this.

(Block settings)
The user 5A who operates the avatar object 6A can perform block settings as predetermined settings for the avatar object 6B. For example, the user 5A can operate the controller 300A to display a menu image on the monitor 130A, select "block setting" from the menu image, and input an operation to determine the avatar object 6B as a block target. For example, when "block setting" is selected, avatar objects 6 that exist within a predetermined range centered on the position of the avatar object 6A (or have existed within a predetermined range within a predetermined period) are displayed. The avatar object 6B may be selected from among the avatar objects 6B. Alternatively, it may be possible to select "Block Settings" and input the handle name of the avatar object 6B to search for and select the avatar object 6B. Furthermore, if the avatar object 6A and the avatar object 6B have a friend relationship (described later), the user 5A selects "friend list" from the menu image and selects the avatar object 6B from the displayed friends. However, the avatar object 6B may be set as a block.

As shown in FIG. 13, HMD set 110A (specifically, avatar control module 1429A) acquires avatar information (step S1310A), and transmits the acquired avatar information to server 600. In this embodiment, when the user 5A performs the above-described block setting, the avatar control module 1429A transmits avatar information including block information to the server 600. Note that the block information may be transmitted to the server 600 separately from the avatar information. The block information includes, for example, the ID of the user 5 who made the block setting, and the ID of the user 5 of the avatar object 6 targeted for blocking.

The server 600 temporarily stores the received avatar information, executes the synchronization process (step S1320), and transmits the avatar information to each HMD set 110. However, if the avatar information received from the HMD set 110 does not include block information, If so, the avatar information to be transmitted to the predetermined HMD set 110 is avatar information on which the next display change process (distinguishability change process) is reflected.

(Display change processing)
As shown in FIG. 15, the control module 1610 of the server 600 includes a block control module 1741 (setting reflection means). When the block control module 1741 determines that the avatar information received from the HMD set 110 includes block information, it executes a display change process. FIG. 16 is a flowchart showing the flow of display change processing.

Based on the received block information, the block control module 1741 identifies the user who made the block setting (referred to as the first user) and the user to be blocked (referred to as the second user) (step S1).

Next, the block control module 1741 changes the information (first information) indicating the appearance and handle name of the first user's avatar object 6 in the avatar information transmitted to the second user's HMD set 110 (step S2 ). Here, it is assumed that the block control module 1741 changes information indicating the appearance and handle name of the avatar object 6A of the user 5A in the avatar information transmitted to the HMD set 110B of the user 5B.

Regarding the changed appearance in step S2, changing the appearance means changing the appearance of the avatar object 6 to a different appearance with a different gender, physique, facial structure, race, type of creature, etc. It's okay. For example, if the first user's avatar object 6 has a female appearance, it may be changed to a male appearance, or may be changed to an animal appearance such as a panda. Further, the change in appearance may be, for example, changing part of the appearance or equipment of the current avatar object 6. For example, you may change your hairstyle, clothes, accessories, glasses, etc. Note that changing the appearance may also be referred to as changing the skin.

The change in the appearance of the avatar object 6 in step S2 is preferably such that it is impossible to identify that the avatar object 6 is the same before and after the change. For example, it is preferable to change the gender, facial shape, etc., or change the face so that it is covered with a mask or the like.

It is also preferable that the handle name change in step S2 be such that it is impossible to identify the same avatar object 6 before and after the change. For example, if the gender set for the current avatar object 6 is female, it is preferable that the handle name is changed to a name that reminds the user that the avatar object 6 is male.

In the process of step S2, the block control module 1741 may automatically select the changed appearance of the first user's avatar object 6 or the changed handle name. In other words, the block control module 1741 automatically selects the changed appearance and changed handle name of the first user's avatar object 6 from among one or more candidates stored in the memory module 1630. It's okay.

Furthermore, the changed appearance of the first user's avatar object 6 and the changed handle name may be selectable when the first user sets a block. If selected by the first user, the first user's avatar control module 1429 transmits block information including the selected information to the server 600. In the process of step S2 described above, the block control module 1741 assumes that the appearance and handle name of the first user's avatar object 6 are selected by the first user.
Note that if there are multiple second users that the first user wants to block, the first user can change the changed appearance of his or her avatar object 6 or the changed handle name for each second user. It may also be possible to set the

When the block control module 1741 executes the display change process, it transmits avatar information on which the display change process is reflected to the second user's HMD set 110. In this embodiment, the block control module 1741 transmits avatar information on which the display change process is reflected to the HMD set 110B of the user 5B. Then, in step S1330B of FIG. 13, the avatar control module 1429B in the HMD set 110B uses the changed appearance and changed handle name of the avatar object 6A based on the received avatar information (specifically, the first information). Display. When a block is set, the appearance of the avatar object 6A for the user 5B who has been set to block changes to a different appearance from the appearance before the change (in other words, the appearance for a user who has not been set to block). ing. Note that when changing the appearance (distinguishability), it is sufficient that the appearance (distinguishability) of the avatar object 6A becomes different as a result of display based on the information received by the avatar control module 1429B.
Further, when changing the voice as the distinguishing feature, in the process of step S2, the setting reflecting means sets the changed voice (voice quality, speaking style) of the avatar object 6 of the first user (user 5A). After the setting, the HMD set 110 (avatar control module 1429B) of the second user (user 5B) sets the voice of the avatar object 6A to be the changed voice based on the acquisition of the voice data of the user 5A.
In addition, when changing the behavior as the distinctiveness, in the process of step S2, the setting reflecting means sets the changed behavior (habit) of the avatar object 6 of the first user (user 5A). After the settings, the HMD set 110 (avatar control module 1429B) of the second user (user 5B) controls the behavior of the avatar object 6A so that some of the behaviors are different from those of the user 5A. .

For example, when changing the appearance, the block control module 1741 may change the appearance by including a process of generating an avatar object with a changed appearance; It may also be something that changes the appearance without including it. For example, in the memory module 530 of the HMD set 110 or the memory module 1630 of the server 600, object data indicating the avatar object 6A before change and object data indicating the avatar object 6A after the change are prepared in advance, and settings are made. The reflection means may change the appearance of the avatar object 6A for the user 5B by changing the object data transmitted to the HMD set 110B of the user 5B based on the settings of the user 5A (for example, block settings). There may be. Note that the object data indicating the avatar object 6A before the change and the object data indicating the avatar object 6A after the change are prepared by the user 5A (in other words, they are registered by the user 5A). ).

FIG. 17A shows an example of the visual field image 17B of the user 5B before the user 5A sets the avatar object 6B as a block. The avatar object 6A has the appearance of a female idol, and its handle name is "minamina."
FIG. 17(b) shows an example of the visual field image 17B of the user 5B after the user 5A sets the avatar object 6B as a block. The avatar object 6A has the appearance of a young man and has the handle name "gonzou."

The block control module 1741 controls the appearance and handle name of the avatar object 6A for the user 5 who operates the avatar object 6 other than the avatar object 6B so as not to change. In other words, the appearance and handle name of the avatar object 6A are changed for the user 5B who operates the avatar object 6B for which the user 5A has set a block, but the avatar object (for example, the avatar object 6C) for which the block has not been set by the user 5A is changed. The appearance and handle name of the avatar object 6A for the user operating the avatar object 6A (for example, the user 5C) are not changed. The block control module 1741 transmits the avatar information in which the display change processing is reflected to the HMD set 110B of the user 5B who operates the avatar object 6B to which the block setting has been made by the user 5A. The avatar information is not transmitted to the HMD set 110 of the user 5 who operates the avatar object 6 that is not used.

In this embodiment, it is assumed that the appearance and handle name of the avatar object 6 of the first user who has set the block is changed with respect to the second user who has set the block. In this case, the distinctiveness (appearance and handle name) of the second user's avatar object 6 with respect to the first user may also be changed. In other words, the distinctiveness (appearance and handle name) of the avatar object 6B of the user 5B with respect to the user 5A may also be changed.

In this case, in the process of step S2 in FIG. 16, the block control module 1741 (1) instructs the appearance and handle name of the first user's avatar object 6 in the avatar information transmitted to the second user's HMD set 110; (2) changing the information indicating the appearance and handle name of the second user's avatar object 6 in the avatar information transmitted to the first user's HMD set 110; Then, the second user's avatar control module 1429 displays the first user's avatar object 6 with the changed appearance and the changed handle name based on the received avatar information, and the first user's avatar control module 1429 , based on the received avatar information, the second user's avatar object 6 is displayed with the changed appearance and the changed handle name. In other words, the appearance of the first user's avatar object 6 provided to the second user has changed from the appearance before the block setting, and the appearance of the second user's avatar object 6 provided to the first user has changed. The appearance will be changed from the appearance before the block was set.

The user 5A (first user) who operates the avatar object 6A may have fear, disgust, etc. toward the avatar object 6B (the second user's avatar object 6). Therefore, by changing the appearance and handle name of the avatar object 6B when setting the block, it becomes difficult for the user 5A to recognize the existence of the avatar object 6B after setting the block. Therefore, fear, disgust, etc. toward the avatar object 6B can be reduced.

In this embodiment, the avatar objects 6 can have a friend relationship in the virtual space 11. For example, in the virtual space 11, when the user 5C operating the avatar object 6C makes a friend request for the avatar object 6A, and the user 5A of the avatar object 6A approves the request, the avatar object 6C and the avatar object 6A It becomes a friend relationship. Note that each user 5 can apply for a friend and approve it by operating the controller 300. Furthermore, the fact that the avatar object 6C and the avatar object 6A are friends may also be referred to as the user 5C and the user 5A being friends.

In this embodiment, when the avatar object 6C and the avatar object 6A become friends, at least one of the avatar control module 1429C and the avatar control module 1429A receives information indicating that the avatar object 6C and the avatar object 6A are friends. (Friend information) is sent to the server 600. The friend information includes at least the ID of the user 5 who applied for friending, and the ID of the user 5 who approved the application.

As shown in FIG. 15, the control module 1610 of the server 600 includes a friend management module (friend management means) 1751. The friend management module 1751 creates, for each avatar object 6, a list (specifically, a friend list) of avatar objects 6 that have a friend relationship with the avatar object 6, and stores it in the memory module 1630. The friend management module 1751 updates the corresponding friend list based on receiving friend information.

Here, for the sake of explanation, the appearance of the avatar object 6A shown in FIG. 17(a) is assumed to be a "true appearance", and the appearance of the avatar object 6A shown in FIG. 17(b) is assumed to be a "false appearance". In this embodiment, when the user 5A performs a block setting targeting the avatar object 6B, the false appearance of the avatar object 6A is visually recognized by the user 5B. Note that even if the avatar object 6A and the avatar object 6B are in a friend relationship, the block setting is given priority and the user 5B sees the false appearance of the avatar object 6A.

In addition, in this embodiment, regardless of whether the user 5 of the avatar object 6 has a friend relationship with the avatar object 6A, the user 5 of the avatar object 6 who has not been set to block the user 5A has an avatar The true appearance of object 6A is visible.
Note that the user 5 of the avatar object 6 who is not set to be blocked by the user 5A and who is in a friend relationship with the avatar object 6A cannot see the true appearance of the avatar object 6A. However, the fake appearance of the avatar object 6A is visible to the user 5 of the avatar object 6 who is not blocked by the user 5A and who is not in a friend relationship with the avatar object 6A. It may be possible to do so. In other words, when the user 5A (avatar object 6A) sets the user 5C (avatar object 6C) as a friend, the control module 1610 sets the appearance of the avatar object 6A that is visible to the user 5C as a friend. A previous false appearance may be changed to a true appearance. In other words, each user 5 has an appearance shown to users 5 who have been set as special users 5 (for example, friends), and an appearance shown to users 5 who have not been set as special users 5. It is possible to register a plurality of types of appearances, including the above, and an avatar object 6 having a predetermined appearance may be provided to other users 5 according to the settings.

In this embodiment, an avatar object (herein referred to as an avatar object 6C) that has a friend relationship with the avatar object 6A of the user 5A has a relatively high possibility of acting together with the avatar object 6A in the virtual space 11. In this case, the user 5B of the avatar object 6B who has been set to be blocked by the user 5A may see the avatar object 6C, which has a friend relationship with the avatar object 6A of the user 5A, and indirectly find out the true identity of the avatar object 6A. There is. Figuring out the true identity means that the avatar object 6A (gonzou) having the appearance (fake appearance) shown in FIG. 17(b) is actually the avatar object 6A (minamina) having the appearance (true appearance) shown in FIG. 17(a). It is something to realize that there is. In other words, the user notices that the appearance of the avatar object 6A has changed.

Therefore, in order to prevent identification of the user's true identity via the avatar object 6 that is a friend, the following configuration may be adopted. In the process of step S2 in FIG. 16, the block control module 1741 determines (α) the appearance of the avatar object 6 of the first user (user 5A) and (β) Change the information indicating the handle name, and (β) change the information indicating the appearance and handle name of all the avatar objects 6 (avatar objects 6C) that have a friend relationship with the avatar object 6 of the first user (user 5A). change. The block control module 1741 can identify the avatar object 6 that has a friend relationship with the avatar object 6 of the first user (user 5A) from the friend list of the avatar object 6 of the first user (user 5A) stored in the memory module 1630.

In addition to the appearance and handle name of the avatar object 6A, if the identity (appearance and handle name) of the avatar object 6C that is in a friend relationship with the avatar object 6A is changed, the user 5B can Therefore, it becomes difficult to discern the true identity of the avatar object 6A. Therefore, it becomes even more difficult for the avatar object 6B to cause trouble to the avatar object 6A.

According to this embodiment, the user 5B who operates the avatar object 6B has difficulty finding the avatar object 6A in the virtual space 11, but the user 5 who operates the avatar objects 6 other than the avatar object 6B has difficulty finding the avatar object 6A in the virtual space 11. At 11 the avatar object 6A can be found. Therefore, the user 5A who operates the avatar object 6A can prevent the avatar object 6B from causing trouble to the avatar object 6A, and the user 5 who operates the avatar objects 6 other than the avatar object 6B can Dialogue etc. can be performed via the object 6. Thereby, it is possible to provide the user 5 with a game space (that is, the virtual space 11) in which the possibility that the user 5 feels uncomfortable is reduced.

Furthermore, if the user 5A operating the avatar object 6A makes predetermined settings (for example, block settings) for the avatar object 6B, the user 5B operating the avatar object 6B may move the avatar object 6B into the virtual space. If the punishment is direct and severe when a predetermined setting is made, such as making it impossible for the avatar object 6B to appear in the avatar object 11, it may stimulate the emotions of the user 5B who operates the avatar object 6B, and the user 5B may There is an increased possibility that the person may attempt to cause a nuisance in the form of In this embodiment, the punishment when a predetermined setting is made is to change the appearance of the avatar object 6A for the user 5B, and since the avatar object 6B can be made to appear in the virtual space 11, it is relatively It is not too strict and can reduce the possibility of offending the emotions of the user 5 who operates the avatar object 6B.

(Modified example)
In this embodiment, when the user 5A performs block setting for the avatar object 6B, a display change process is executed and the identifiability of the avatar object 6A to the user 5B who operates the avatar object 6B changes. However, instead of or in addition to the display change process, another process different from the display change process may be executed. A case where dummy placement processing is executed instead of display change processing will be described below.

When the dummy placement process is executed, at least one dummy avatar object 6A' (hereinafter referred to as dummy character 6A') corresponding to the avatar object 6A is placed in the virtual space 11. A dummy can be a ghost, alter ego, substitute, substitute, fake (fake character), or the like. Dummy character 6A' has the same appearance as avatar object 6A. In other words, the dummy character 6A' is a duplicate of the avatar object 6A. Note that the dummy character 6A' may have a different appearance (similar appearance) to the avatar object 6A. Different appearances (similar appearances) refer to, for example, people having the same face but different hairstyles, clothes, and colors of clothes. Further, when a plurality of dummy characters 6A' are arranged, the appearance of each dummy character 6A' may be the same or different (for example, similar). In this embodiment, when arranging a plurality of dummy characters 6A', it is assumed that the dummy characters 6A' are similar to each other, but have different colors of clothes (a part of their appearance), for example.

When the dummy placement process is executed and the dummy character 6A' appears, there is a high possibility that the user 5B will mistake the dummy character 6A' for the avatar object 6A, and the avatar object 6B will follow the dummy character 6A' around. , it becomes difficult for the avatar object 6B to cause trouble to the avatar object 6A. Furthermore, when a plurality of dummy characters 6A' appear, it is better for the user 5B who operates the avatar object 6B to look at the dummy character 6A when the appearances of the dummy characters 6A' are similar than when the appearances of the dummy characters 6A' are the same. It can reduce the possibility of feeling uncomfortable when looking at ´. In other words, it is possible to increase the possibility that the user 5B will follow the dummy character 6A' around without noticing that it is the dummy character 6A'.

Further, the dummy character 6A' may have the same handle name (object for identification) as the avatar object 6A. In this case, there is a higher possibility that the dummy character 6A' is mistaken for the avatar object 6A, and it becomes more difficult for the avatar object 6B to cause trouble to the avatar object 6A.

As will be described later, the dummy character 6A' does not move based on the operation of the user 5A. Furthermore, unlike the avatar object 6A, the dummy character 6A' cannot interact with the dummy character 6A'. Note that although dialogue is not possible in this embodiment, it may be controlled by, for example, AI or the like and may be able to respond. The dummy character 6A' has no substance (in other words, it has no content).

As shown in FIG. 13, HMD set 110A (specifically, avatar control module 1429A) acquires avatar information (step S1310A), and transmits the acquired avatar information to server 600. The server 600 temporarily stores the received avatar information, executes the synchronization process (step S1320), and transmits the avatar information to each HMD set 110. However, if the avatar information received from the HMD set 110 does not include block information, If so, execute dummy placement processing. FIG. 18 is a flowchart showing the flow of dummy placement processing.

(Dummy placement processing)
The block control module 1741 identifies the user who made the block setting (first user) and the user to be blocked (second user) based on the received block information (step S10).

Next, the block control module 1741 instructs the placement of at least one (in other words, one) dummy character 6A' of the first user's avatar object 6 in the avatar information transmitted to the second user's HMD set 110. information (second information) is added (step S20). In addition, in this embodiment, the second information shall be added to the avatar information, but the second information may be transmitted to the second user's HMD set 110 separately from the avatar information. Here, it is assumed that the block control module 1741 transmits information (second information) instructing the placement of the three dummy characters 6A' of the avatar object 6A of the user 5A to the HMD set 110B of the user 5B.

When the block control module 1741 executes the dummy placement process, it transmits avatar information on which the dummy placement process is reflected to the second user's HMD set 110. In this embodiment, the block control module 1741 transmits avatar information on which the dummy placement process is reflected to the HMD set 110B of the user 5B. In step S1330B of FIG. 13, the avatar control module 1429B in the HMD set 110B arranges the three dummy characters 6A' of the avatar object 6A in the virtual space 11B based on the received avatar information. In other words, the block control module 1741 transmits information (second information) instructing the placement of the dummy character 6A' to the avatar control module 1429B, and the avatar control module 1429B controls the virtual space 11B based on the information. A dummy character 6A' is placed at .

FIG. 19 shows an example of the virtual space 11B after the user 5A sets the avatar object 6B as a block. Apart from the avatar object 6A, three dummy characters 6A' appear in the virtual space 11B.

The block control module 1741 controls the dummy character 6A' so that it is not visible to the user 5 operating the avatar object 6 other than the avatar object 6B. In other words, the user 5B of the avatar object 6B that has been set to block can see the dummy character 6A' of the avatar object 6A, but is operating an avatar object that has not been set to block (for example, the avatar object 6C). The dummy character 6A' of the avatar object 6A is not visible to the user (for example, the user 5C). The block control module 1741 transmits avatar information on which the dummy placement process is reflected to the HMD set 110B of the user 5B who operates the avatar object 6B to which the block setting has been made by the user 5A. The avatar information is not transmitted to the HMD set 110 of the user 5 who operates the avatar object 6 that is not used. In other words, for example, the dummy character 6A' is not arranged in the virtual space 11C.

In this modification, regardless of whether or not the user 5 of the avatar object 6 has a friend relationship with the avatar object 6A, the user 5 of the avatar object 6 who has not been set to block the user 5A is assigned the dummy character 6A. ´ is not visible.
Note that the dummy character 6A' is not visible to the user 5 of the avatar object 6 who has not been set to block the user 5A and who is in a friend relationship with the avatar object 6A. The dummy character 6A' may be visible to the user 5 of the avatar object 6 who is not set to be blocked by the avatar object 6 and who is not in a friend relationship with the avatar object 6A. .
Furthermore, although the dummy character 6A' is assumed not to be visible to the user 5A of the avatar object 6A who has set the block, the dummy character 6A' may also be visible to the user 5A.

User 5A can operate avatar object 6A. On the other hand, the user 5A cannot operate the dummy character 6A' of the avatar object 6A. The dummy character 6A' is not operated by the user, but operates based on a program and automatically moves around within the virtual space 11. The block control module 1741 controls the movement of the dummy character 6A' in the virtual space 11. The block control module 1741 controls the dummy character 6A' to move around randomly within the virtual space 11. Moving around randomly is not limited to always moving, but may also mean repeating movement and stopping alternately. Note that when the block control module 1741 arranges a plurality of dummy characters 6A' in the virtual space 11, it is preferable that the dummy characters 6A' are scattered.

The block control module 1741 transmits information instructing the operation of the dummy character 6A' to at least the HMD set 110 (HMD set 110B) of the second user (user 5B). The avatar control module 1429B of the HMD set 110 (HMD set 110B) causes the dummy character 6A' to operate based on the received information.

The block control module 1741 may learn the behavior history of the avatar object 6A to create an AI, and control the motion of the dummy character 6A' in the virtual space 11 based on the AI. In this case, the behavior pattern of the dummy character 6A' becomes closer to the behavior pattern of the avatar object 6A. Therefore, it becomes more difficult to determine whether it is the avatar object 6A or the dummy character 6A'. This makes it even more difficult for the avatar object 6B to cause trouble to the avatar object 6A.

It may be configured such that both the display change process and the dummy placement process are executed when the user 5A performs block settings for the avatar object 6B. In this case, for example, at least one dummy character 6A' corresponding to the avatar object 6A before the change (that is, the true appearance) is arranged, and the appearance of the avatar object 6A for the user 5B who operates the avatar object 6B after the change is (i.e., a false appearance). In this case, since it is difficult to find the avatar object 6A, the possibility of the dummy character 6A' being followed around becomes relatively higher, and it is more difficult for the avatar object 6B to cause trouble to the avatar object 6A. becomes.
Note that both the display change process and the dummy placement process are executed, and the appearance of the avatar object 6A for the user 5B who operates the avatar object 6B becomes the changed appearance, and corresponds to the avatar object 6A having the changed appearance. The control may be such that at least one dummy character 6A' is arranged.

Further, when the user 5A performs block setting for the avatar object 6B, the user 5B (only) operating the avatar object 6B may be unable to visually recognize the avatar object 6A.

According to this modification, there is a high possibility that the user 5B who operates the avatar object 6B will mistake the dummy character 6A' for the avatar object 6A in the virtual space 11, and the avatar object 6B will follow the dummy character 6A' around. Therefore, it becomes difficult for the avatar object 6B to cause trouble to the avatar object 6A. Thereby, it is possible to provide the user 5 with a game space (that is, the virtual space 11) in which the possibility that the user 5 feels uncomfortable is reduced.

(Second embodiment)
Next, a second embodiment of the present invention will be described. Since the HMD system 100 according to this embodiment basically has the same configuration as the HMD system 100 of the first embodiment, the same configuration as the HMD system 100 of the first embodiment is , omit or simplify the explanation.

As shown in FIG. 15, the control module 1610 of the server 600 includes a nuisance determination module 1761 (determination means, detection means). The nuisance determination module 1761 is capable of determining which type of communication between avatar objects 6 in the virtual space 11 is, and one of the types is a predetermined type. There is a type of behavior that causes trouble (so-called stalking behavior) to the avatar object. Note that the type of communication determined by the nuisance determination module 1761 may be only a type in which a predetermined avatar object performs a nuisance act toward a predetermined avatar object. The nuisance behavior determination module 1761 can automatically determine whether or not a nuisance behavior is occurring. The nuisance determination module 1761 can detect behavior of the avatar object in the virtual space 11 that constitutes a nuisance.

Hereinafter, a case will be described in which each HMD set 110 provides the virtual space 11 to the user 5, as shown in FIG. 12(A). Further, similarly to the first embodiment, a case will be described in which the avatar object 6B is causing trouble to the avatar object 6A in the virtual space 11.

As shown in FIG. 13, for example, the HMD set 110B (specifically, the avatar control module 1429B) acquires avatar information (step S1310B), and transmits the acquired avatar information to the server 600. The avatar information that each HMD set 110 transmits to the server 600 includes information (position information) indicating the position in the virtual space 11 of the avatar object 6 operated by the user 5 of the HMD set 110.

Further, in this embodiment, the view image generation module 1438B of the HMD set 110B generates the view image 17B displayed on the monitor 130B based on the view area 15B. The visual field image generation module 1438B transmits information regarding the visual field of the user 5B to the server 600. The information regarding the field of view of the user 5B may be, for example, the field of view image 17B, or information indicating an object displayed in the image presented to the user 5B (namely, the field of view image 17B). Similarly, the other HMD sets 110 transmit information regarding the field of view of the user 5 of the HMD set 110 to the server 600. Hereinafter, information regarding the user's 5 field of view will be referred to as visibility information.

(Judgment based on visibility information)
The nuisance determination module 1761 performs nuisance determination processing based on the visibility information received from the HMD set 110. FIG. 20 is a flowchart showing the flow of the nuisance determination process. In FIG. 20, a nuisance determination process based on visibility information received from the HMD set 110B will be described.

The nuisance determination module 1761 determines whether the visibility information received from the HMD set 110B indicates that the avatar object 6 is included in the field of view (step S101). In step S101, if it is determined that the visibility information does not indicate that the avatar object 6 is included in the field of view (step S101: NO), the nuisance determination module 1761 repeats the process of step S101.

In step S101, if it is determined that the visibility information indicates that the avatar object 6 is included in the field of view (step S101: YES), the nuisance determination module 1761 determines that the avatar object 6 is the user of the HMD set 110B. It is determined whether the avatar object 5B is a friend of the avatar object 6B (step S102). If it is determined in step S102 that the avatar object 6 is a friend of the avatar object 6B (step S102: YES), the process returns to step S101. On the other hand, if it is determined in step S102 that the avatar object 6 is not a friend of the avatar object 6B (step S102: NO), the process proceeds to step S103. Here, it is assumed that the avatar object 6A is included in the visibility information received from the HMD set 110B, and the avatar object 6A and the avatar object 6B are not friends.

The nuisance determination module 1761 starts measuring the time during which the avatar object 6A is included in the field of view (in other words, captured in the field of view) (step S103).

Next, the nuisance determination module 1761 determines whether the avatar object 6A continues to be included in the field of view based on the visibility information received from the HMD set 110B (step S104). In step S104, if it is determined that the avatar object 6A continues to be included in the field of view (step S104: YES), the nuisance determination module 1761 continues measuring time and performs the processing in step S104. repeat. On the other hand, if it is determined in step S104 that the state in which the avatar object 6A is not continuously included in the field of view (step S104: NO), the nuisance determination module 1761 ends the measurement of time, and the measured time is It is determined whether the time is longer than the first predetermined time (step S105). The measured time can be said to be the time during which the avatar object 6A is captured in the field of view of the user 5B. The first predetermined time is, for example, one hour.
Note that step S104 determines that even if the user 5B takes his/her line of sight off the avatar object 6A for a relatively short period of time, the state in which the avatar object 6A is included in the field of vision does not continue (step S104: NO). The configuration may be such that no determination is made and time measurement continues. In other words, if the user 5B averts his/her line of sight from the avatar object 6A for a short period of time (for example, 5 seconds) and then turns his/her line of sight to the avatar object 6A again, the process of step S104 is continued and the process proceeds to step S105. It may be configured so that it does not proceed.

In step S105, if it is determined that the measured time is less than the first predetermined time (step S105: NO), the nuisance behavior determination module 1761 determines that the type of communication of the avatar object 6B to the avatar object 6A is a type of nuisance behavior. It is not determined that there is such a thing (no behavior causing a nuisance is detected), and the process returns to step S101. On the other hand, if it is determined in step S105 that the measured time is equal to or longer than the first predetermined time (step S105: YES), the nuisance determination module 1761 determines that the type of communication of the avatar object 6B to the avatar object 6A is a nuisance. type (in other words, the avatar object 6B is recognized as the avatar object 6 that is causing a nuisance) (in other words, the behavior that constitutes a nuisance is detected), and the nuisance detection information is specified. It is transmitted to the control execution module 1771 (step S106). The nuisance detection information includes at least the ID of the user (user 5B) of the avatar object (avatar object 6B) determined to be causing a nuisance, and the ID of the user (user 5B) of the avatar object (avatar object 6B) determined to be the victim of the nuisance behavior. ID of the user (user 5A) of the object 6A).

Even if it is determined in step S104 that the avatar object 6A continues to be included in the field of view, the position of both the avatar object 6A and the avatar object 6B remains stationary for a certain period of time. Then, time measurement may be temporarily stopped, and time measurement may be restarted when one of the devices starts moving. In other words, the first predetermined time may be configured not to include the predetermined time that elapses while both the avatar object 6A and the avatar object 6B are stopped. Avatar object 6B does not cause any trouble to avatar object 6A, and user 5A of avatar object 6A and user 5B of avatar object 6B simply leave the avatar object 6A alone without performing any operations. If the time measurement in step S104 is continued when both avatar objects 6B are stopped, there is a risk that it will be determined that the avatar object 6B is causing trouble to the avatar object 6B. Therefore, by not including the time during which both are stopped in the first predetermined time period, the possibility of such a determination being made can be reduced.

Further, in step S105, the nuisance determination module 1761 may change the first predetermined time period according to the attribute of the avatar object 6A captured in the field of view. For example, if the avatar object 6A has an attribute that can attract the gaze of the user 5 who operates the other avatar object 6, the first predetermined time may be set longer (for example, 3 hours). ). Examples of such attributes include having a relatively large number of friends and wearing extremely rare items. If the first predetermined time period is set to be long when the user has the attribute, the avatar object 6A is not causing trouble to the avatar object 6A and is gaining popularity in the virtual space 11. It is possible to reduce the possibility that the avatar object 6B present is determined to be causing a nuisance.

The nuisance determination process shown in FIG. 20 determines whether the length of time that the avatar object 6A is continuously seen in the field of view of the user 5B who operates the avatar object 6B is equal to or longer than a first predetermined time. , can be said to be a condition for determining whether or not the avatar object 6B is causing a nuisance.

(Modified example)
The steps after step S105 in FIG. 20 may be configured as follows. As shown in FIG. 21, the nuisance determination module 1761 finishes measuring the time and updates the memory value of the total time stored in the memory module 1630 (step S105'). The total time is the total (in other words, cumulative) time during a predetermined period that a predetermined avatar object (avatar object 6A) is captured in the field of view of the user 5B who operates the predetermined avatar object (avatar object 6B). It's time. The predetermined period may be, for example, one day or one week.

Next, the nuisance determination module 1761 determines whether the total time stored in the memory module 1630 is greater than or equal to the second predetermined time (step S106'). When the predetermined period is one day, the second predetermined time is, for example, three hours. Note that, similar to the first predetermined time, the nuisance determination module 1761 may change the second predetermined time depending on the attribute of the avatar object 6 captured in the field of view.

If it is determined in step S106' that the total time is less than the second predetermined time (step S106': NO), the nuisance behavior determination module 1761 determines that the type of communication of the avatar object 6B to the avatar object 6A is a type of nuisance behavior. It is not determined that the behavior is a nuisance (no behavior that constitutes a nuisance is detected), and the process returns to step S101. On the other hand, if it is determined in step S106' that the total time is equal to or longer than the second predetermined time (step S106': YES), the nuisance determination module 1761 determines that the type of communication between the avatar object 6B and the avatar object 6A is a nuisance. It is determined that the behavior is of the type that causes a nuisance (a behavior that constitutes a nuisance is detected), and nuisance detection information is transmitted to the specific control execution module 1771 (step S107).

In this modification, the avatar object 6A determines whether or not the total time during which the avatar object 6A is seen in the field of view of the user 5B who operates the avatar object 6B is equal to or longer than the second predetermined time period. It can be said that this is a condition for determining whether or not someone is causing a nuisance.

Note that, assuming that the predetermined period is one week, for example, the number of days during which the avatar object 6A is captured in the field of view of the user 5B who operates the avatar object 6B for a predetermined time or more is equal to or longer than a predetermined number of days in the predetermined period. This may be a condition for determining whether or not the avatar object 6B is causing a nuisance. In this case, if the number of days during which the avatar object 6A is seen in the field of view of the user 5B who operates the avatar object 6B is longer than a predetermined period of time, the avatar object 6B becomes a nuisance. It is determined that the act is being carried out.

As described above, the nuisance determining means may determine whether or not the user 5 is causing a nuisance based on the time period during which the avatar object 6 is seen in the user's field of view.

(specific control)
The specific control execution module 1771 (specific control execution means) of the server 600 executes specific control upon receiving the nuisance detection information. As a specific control, for example, the user 5A of the avatar object 6A who has been determined to be the victim of a nuisance act may be notified that the avatar object 6B is committing a nuisance act (or that there is a possibility that the avatar object 6B is committing a nuisance act). There is a control to notify you that there is a For example, the specific control execution module 1771 sends information related to the notification to the control module 510A of the HMD set 110A, and the control module 510A informs the monitor 130A based on the information that the avatar object 6B is causing a nuisance. It may also be possible to display a display (ie, a notification screen) indicating that there is a possibility that there is a problem. In this case, it may be possible to make block settings from the notification screen. In other words, a predetermined operation related to block setting may be accepted on the notification screen.

By receiving the notification, the user 5A can notice that the avatar object 6B may be causing a nuisance. In addition, it is possible to consider whether or not it is necessary to take countermeasures, and if necessary, it is possible to take countermeasures such as setting blocks.

Further, the specific control may be a control that imposes sanctions on the user 5B who operates the avatar object 6B. For example, the user 5B may be in a state in which the avatar object 6B determined to be causing a nuisance cannot appear in the virtual space 11. This control can be said to be control for expelling the avatar object 6B from the virtual space 11, or it can also be said to be control for prohibiting placement of the avatar object 6B in the virtual space 11. When this control is executed, the user 5B cannot make his or her avatar object 6B appear in the virtual space 11. Therefore, the avatar object 6B is in a state where it cannot cause any trouble to the avatar object 6A.

Further, the specific control may be the display change process shown in the first embodiment. In this case, the specific control execution module 1771 instructs the block control module 1741 to change the identity of the avatar object 6A based on the reception of the nuisance detection information. In this case, the display change process is executed without requiring reception of block information based on the block settings of the user 5A.
Further, the specific control may be the dummy placement process shown in the first embodiment. In this case, the specific control execution module 1771 instructs the block control module 1741 to place at least one dummy character 6A' of the avatar object 6A based on the reception of the nuisance detection information. In this case, the dummy placement process is executed without requiring reception of block information based on the block settings of the user 5A.

According to this embodiment, when it is determined that the avatar object 6B is of the type that causes a nuisance to the avatar object 6A (when a behavior that constitutes a nuisance is detected), specific control is automatically executed. Ru. When a specific control is executed based on a predetermined setting by the user 5, the nuisance behavior may be repeated until the user 5 makes the predetermined setting, and the user 5 may feel uncomfortable. According to this configuration, when a behavior that causes a nuisance is detected, specific control is automatically executed, so that the frequency at which the user 5 feels uncomfortable can be reduced. Thereby, it is possible to provide the user 5 with a game space (that is, the virtual space 11) in which the possibility that the user 5 feels uncomfortable is reduced.
In addition, if the user 5 decides for himself whether or not he is being harassed, and takes countermeasures if he determines that he is being harassed, the amount of time it takes to take countermeasures increases the chance of being harassed. There is a high possibility that the user 5 will feel uncomfortable. On the other hand, according to the present embodiment, specific control is automatically executed based on the detection of behavior that constitutes a nuisance. Therefore, countermeasures can be taken in a relatively short time, and the possibility that the user 5 will feel uncomfortable can be reduced. Thereby, it is possible to provide the user with the virtual space 11 in which the possibility that the user 5 will feel uncomfortable is reduced.

(Judgment based on location information)
The nuisance determination module 1761 may perform nuisance determination processing based on the position information received from the HMD set 110. This will be explained in detail below. FIG. 22 is a flowchart showing the flow of the nuisance determination process.

The nuisance determination module 1761 determines the position of the avatar object 6 (referred to as the reference avatar object 6) in the virtual space 11 for each avatar object 6 corresponding to the user 5 of each HMD set 110 based on the received position information. It is determined whether or not another avatar object 6 exists within a predetermined area centered (in other words, within a predetermined range) (step S201). The predetermined area is, for example, an area where the reference avatar object 6 can be visually recognized in the virtual space 11.

If it is determined in step S201 that no other avatar object 6 exists within the predetermined area centered on the position of the reference avatar object 6 (step S201: NO), the process of step S201 is repeated. On the other hand, if it is determined in step S201 that another avatar object 6 exists within a predetermined area centered on the position of the reference avatar object 6 (step S201: YES), the nuisance determination module 1761 It is determined whether the avatar object 6 is a friend of the reference avatar object 6 (step S202). If it is determined in step S202 that the other avatar object 6 is a friend of the reference avatar object 6 (step S202: YES), the process returns to step S201. On the other hand, if it is determined in step S202 that the other avatar object 6 is not a friend of the reference avatar object 6 (step S202: NO), the process proceeds to step S203.

Next, the nuisance determination module 1761 determines whether the reference avatar object 6 is being followed by another avatar object 6 (step S203). The nuisance determination module 1761 determines whether or not another avatar object 6 is being followed, for example, based on whether or not another avatar object 6 is moving in the same direction as the reference avatar object 6 after the reference avatar object 6 moves. If it is determined in step S203 that the reference avatar object 6 is being followed by another avatar object 6 (step S203: YES), the process proceeds to step S204. On the other hand, if it is determined in step S203 that the reference avatar object 6 is not being followed by another avatar object 6 (step S203: NO), the process returns to step S201.

Here, (1) avatar object 6B (or avatar object 6A) exists within a predetermined area centered on the position of avatar object 6A (or avatar object 6B), and (2) avatar object 6A and avatar object 6B are friends. (3) After the movement of the avatar object 6A, the avatar object 6B is moving in the same direction as the avatar object 6A, and after the movement of the avatar object 6B, the avatar object 6A is not moving in the same direction as the avatar object 6B. The flow of processing will be explained using an example where there is no such file.

When the reference avatar object 6 is the avatar object 6A, it is determined in step S201 that the avatar object 6B exists within a predetermined area centered on the position of the avatar object 6A, and in step S202, it is determined that the avatar object 6B is friends with the avatar object 6B. It is determined in step S203 that the avatar object 6A is being followed by the avatar object 6B, and the process proceeds to step S204.
On the other hand, when the reference avatar object 6 is the avatar object 6B, it is determined in step S201 that the avatar object 6A exists within a predetermined area centered on the position of the avatar object 6B, and in step S202, the avatar object 6A is It is determined that the avatar object 6B is not a friend, and in step S203 it is determined that the avatar object 6B is not being followed by the avatar object 6A, and the process returns to step S201.

A case where the reference avatar object 6 is the avatar object 6A will be described below. The nuisance determination module 1761 starts measuring the time that the avatar object 6B exists within a predetermined area centered on the position of the avatar object 6A (step S204).

Next, the nuisance determination module 1761 determines whether the avatar object 6B continues to exist within a predetermined area centered on the position of the avatar object 6A (step S205). In step S205, if it is determined that the avatar object 6B continues to exist within a predetermined area centered on the position of the avatar object 6A (step S205: YES), the nuisance determination module 1761 continues to measure and repeats the process of step S205. On the other hand, if it is determined in step S205 that the avatar object 6B does not continue to exist within the predetermined area centered on the position of the avatar object 6A (step S205: NO), the nuisance determination module 1761 , ends the time measurement, and determines whether the measured time is longer than a third predetermined time (for example, one hour) (step S206).

In step S206, if it is determined that the measured time is less than the third predetermined time (step S206: NO), the nuisance determination module 1761 determines that the type of communication of the avatar object 6B to the avatar object 6A is a type of nuisance. It is not determined that there is such a thing (no behavior that constitutes a nuisance is detected), and the process returns to step S201. On the other hand, if it is determined in step S206 that the measured time is equal to or longer than the third predetermined time (step S206: YES), the nuisance determination module 1761 determines that the type of communication of the avatar object 6B to the avatar object 6A is a nuisance. It is determined that it is of the same type (the behavior that constitutes a nuisance is detected), and the nuisance detection information is transmitted to the specific control execution module 1771 (step S207).

Note that, in step S206, the nuisance determination module 1761 may change the third predetermined time period according to the attribute of the avatar object 6A, similarly to the first predetermined time period. Further, the nuisance determination module 1761 may determine whether or not a nuisance is being committed based on the total time in the predetermined period, similar to the case where the determination is made using the second predetermined time period described above.

Furthermore, in step S205, even if the avatar object 6B continues to exist within a predetermined area centered on the position of the avatar object 6A, the positions of both the avatar object 6A and the avatar object 6B are If the stopped state continues for a certain period of time, time measurement may be temporarily stopped, and time measurement may be restarted when one of the objects starts moving. In other words, the third predetermined time may be configured not to include the predetermined time that elapses while both the avatar object 6A and the avatar object 6B are stopped.

In this example, it is determined whether the avatar object 6B has continuously existed in a predetermined area centered on the position of the avatar object 6A for a third predetermined time or more. It can be said that this is a condition for determining whether or not the practice is being carried out. The nuisance determining means may determine whether or not a nuisance is being committed based on the time that the avatar object 6B exists within a predetermined range from the avatar object 6A.

(Modified example)
In addition, when it is determined in the above-mentioned step S206 that the measured time is equal to or longer than the third predetermined time (step S206: YES), the following configuration may be adopted. As shown in FIG. 23, the nuisance determination module 1761 determines that a predetermined history has been established between the avatar object 6B (other avatar object 6) and the avatar object 6A (reference avatar object 6) during the third predetermined time period. It is determined whether it exists (step S207'). The control module 1610 of the server 600 includes a history management module (history management means) 1781. The history management module 1781 stores history information including the action history of each avatar object 6 in the virtual space 11 in the memory module 1630. The nuisance determination module 1761 performs the process of step S207' based on the history information.

The predetermined history may be a history of a meeting between the avatar object 6B and the avatar object 6A, or a history of a conversation between the user 5B of the avatar object 6B and the user 5A of the avatar object 6A. Note that the history of conversations is not limited to one in which voice data was exchanged in both directions, but may be a history in which voice data was unilaterally transmitted from one side to the other.

In step S207', if it is determined that a predetermined history exists between the avatar object 6B and the avatar object 6A during the third predetermined time period (step S207': YES), the nuisance determination module 1761 It is not determined that the type of communication of the avatar object 6B with respect to the avatar object 6A is a type that causes a nuisance (no behavior that constitutes a nuisance is detected), and the process returns to step S201. On the other hand, in step S207', if it is determined that the predetermined history does not exist between the avatar object 6B and the avatar object 6A during the third predetermined time period (step S207': NO), the nuisance determination module 1761 , determines that the type of communication between the avatar object 6B and the avatar object 6A is a type of nuisance behavior (detects behavior that constitutes a nuisance behavior), and transmits nuisance detection information to the specific control execution module 1771 (step S208).

If the predetermined history exists within the third predetermined time period, there is a possibility that the user 5A of the avatar object 6A does not have fear or aversion toward the avatar object 6B. According to this configuration, it is possible to reduce the possibility that such an avatar object 6B is determined to be causing a nuisance. In this modification, the time during which the avatar object 6B continuously exists within a predetermined area centered on the position of the avatar object 6A is a third predetermined time or more, and during the third predetermined time, the avatar object It can be said that the absence of a predetermined history between the butter object 6B and the avatar object 6A is a condition for determining that the butter object 6B is causing a nuisance.

(Judgment based on the number of times it slips through)
The nuisance determination module 1761 may perform a nuisance determination process based not only on visibility information and position information but also on information on the number of times a user passes through. This will be explained in detail below. In this example, it is assumed that contact (collision) between avatar objects 6 is not detected in the virtual space 11, and one avatar object 6 can pass through the other avatar object 6.

Even if the avatar object 6B walks toward the avatar object 6A and the avatar objects 6B and 6A come into contact, the contact between the two is not detected and the avatar object 6B simply passes through the avatar object 6A. It has become. The avatar object 6B that is causing trouble to the avatar object 6A is walking around the avatar object 6A, and the number of times it passes through the avatar object 6A is relatively higher than other avatar objects 6 that are not causing trouble. There is a high possibility that the number will increase.

The history management module 1781 stores, for each avatar object 6, slip-through number information indicating the number of times the avatar object 6 has passed through other avatar objects 6 in a predetermined period as history information in the memory module 1630. Note that when a plurality of other avatar objects 6 exist, information on the number of times of slipping through can be stored for each of the other avatar objects 6. The predetermined period may be, for example, one day. The nuisance determination module 1761 performs nuisance determination processing based on the slip-through frequency information.

FIG. 24 is a flowchart showing the flow of the nuisance determination process.
The nuisance determination module 1761 determines whether or not there is an avatar object 6 that has slipped through a predetermined number of times (for example, 10 times) within a predetermined period, based on the slip-through number information (step S301).

In step S301, if it is determined that there is no avatar object 6 that has passed through the predetermined number of times (for example, 10 times) (step S301: NO), the process of step S301 is repeated. On the other hand, if it is determined in step S301 that there is an avatar object 6 that has slipped through the predetermined number of times (for example, 10 times) (step S301: YES), the nuisance determination module 1761 determines that the avatar object 6 that has slipped through the Then, the avatar object 6B) and the avatar object 6 that slipped through (here, the avatar object 6A) are identified (step S302).

Next, the nuisance determination module 1761 determines whether the avatar objects identified in step S302, that is, the avatar object 6B and the avatar object 6A, are friends (step S303). In step S303, if it is determined that avatar object 6B and avatar object 6A are friends (step S303: YES), the process returns to step S301. On the other hand, if it is determined in step S303 that the avatar object 6B and the avatar object 6A are not friends (step S303: NO), the nuisance determination module 1761 determines that the avatar object 6B is causing nuisance to the avatar object 6A. (a behavior that constitutes a nuisance is detected), and the nuisance detection information is transmitted to the specific control execution module 1771 (step S304).

In this example, the condition for determining whether or not the avatar object 6B is causing a nuisance is whether or not the number of times the avatar object 6B has passed through the avatar object 6A during a predetermined period has reached a predetermined number of times. be able to.
Note that the predetermined period is, for example, one week, and it is determined whether or not the number of days in which the avatar object 6B has missed the avatar object 6A more than a predetermined number of times in one week is the predetermined number of days or more. It may also be used as a condition for determining whether or not. In this case, if the avatar object 6B passes by the avatar object 6A a predetermined number of times or more during a predetermined period, it is determined that the avatar object 6B is causing a nuisance.

Note that the present invention is not limited to the embodiments described above, and can be implemented with various modifications without departing from the gist thereof. Within the scope of the present invention, each component can be freely combined, any component can be modified, or any component can be omitted. Further, the process flow described in this specification is merely an example, and the order and configuration of each process may be different. Furthermore, some of the processes such as the various determination processes shown in each flowchart may not exist. In other words, the processing flow, specific determination processing, etc. may be different from those exemplified in this specification.

[Additional notes]
The matters described in the above embodiments may also be described as in the following additional notes.

(Additional note 1)
computer,
With respect to a virtual space in which a character corresponding to the user can appear, a character other than the second character is operated based on a predetermined setting made for the second character by the user operating the first character. Setting reflection means (for example, block control module 1741) that changes the identifiability of the first character to the user who operates the second character without changing the identifiability of the first character to the user.
A program that functions as
According to such a configuration, the identifiability of the first character changes based on the predetermined setting, making it difficult for the user who operates the second character to identify the first character in the virtual space. becomes. On the other hand, a user operating a character other than the second character can specify the first character in the virtual space. Therefore, the user operating the first character can prevent the second character from causing trouble to the first character. Furthermore, the user who operates the first character can communicate with users who operate characters other than the second character through the characters in the virtual space. Thereby, it is possible to provide the user with a virtual space in which the possibility that the user feels uncomfortable is reduced.

(Additional note 2)
The program according to supplementary note 1, wherein the distinctiveness is appearance.
According to such a configuration, the appearance of the first character changes based on the predetermined settings, making it difficult for the user operating the second character to find the first character in the virtual space. . This makes it difficult for the second character to cause trouble to the first character, and it is possible to reduce the possibility that the user operating the first character will feel uncomfortable.

(Additional note 3)
The program according to appendix 1 or 2, wherein the distinctiveness is an identification object related to the first character that is displayed together with the first character.
According to such a configuration, since the identification object related to the first character changes based on the predetermined setting, the user who operates the second character can identify the first character in the virtual space. becomes more difficult. This makes it difficult for the second character to cause trouble to the first character, and it is possible to reduce the possibility that the user operating the first character will feel uncomfortable.

(Additional note 4)
The program according to appendix 1 or 2, wherein the setting reflecting means changes the appearance of the second character with respect to the user operating the first character based on the predetermined setting.
According to such a configuration, it becomes difficult for the user operating the first character to recognize the presence of the second character in the virtual space. Therefore, it is possible to reduce fear, disgust, etc. that the user operating the first character has toward the second character.

(Appendix 5)
The program according to Supplementary Note 1 or 2, wherein the setting reflecting means changes the appearance of a friend character of the first character with respect to the user operating the second character based on the predetermined setting.
According to such a configuration, it is possible to reduce the possibility that the user operating the second character will find out the true identity of the first character through the first character's friends in the virtual space. Seeing through the true identity means noticing that the distinctiveness of the first character has changed. This makes it more difficult for the second character to cause trouble to the first character, and it is possible to reduce the possibility that the user operating the first character will feel uncomfortable.

(Appendix 6)
With respect to a virtual space in which a character corresponding to the user can appear, a character other than the second character is operated based on a predetermined setting made for the second character by the user operating the first character. An information processing system comprising a setting reflecting means (for example, a block control module 1741) that changes the identifiability of the first character to a user who operates the second character without changing the identifiability of the first character to the user.
According to such a configuration, the same effects as those of the program described in Supplementary Note 1 can be achieved.

(Appendix 7)
computer,
Regarding a virtual space in which a character corresponding to the user can appear, based on a predetermined setting made for a second character by the user operating the first character, at least one dummy of the first character is set. Setting reflection means (for example, block control module 1741) that causes two dummy characters to appear in the virtual space
A program that functions as
According to such a configuration, since the dummy character appears based on the predetermined settings, the user who operates the second character may mistake the dummy character for the first character, and the second character may The possibility of using dummy characters increases. Therefore, it becomes difficult for the second character to cause trouble to the first character. Thereby, it is possible to provide the user with a virtual space in which the possibility that the user feels uncomfortable is reduced.

(Appendix 8)
The program according to appendix 7, wherein the setting reflecting means causes a plurality of the dummy characters having different appearances to appear in the virtual space based on the predetermined setting.
According to such a configuration, compared to a case where the plurality of dummy characters all have the same appearance, it is possible to reduce the possibility that the user who operates the second character will feel uncomfortable when looking at the dummy character. Therefore, it is possible to increase the possibility that the second character follows the dummy character around without the user noticing that the dummy character is the dummy character. This can reduce the possibility that the user operating the first character will feel uncomfortable.

(Appendix 9)
The program according to appendix 7 or 8, wherein the setting reflecting means changes the distinguishability of the first character with respect to the user who operates the second character based on the predetermined setting.
According to such a configuration, since the identifiability of the first character changes, it becomes difficult to specify the first character in the virtual space, and the possibility of passing around the dummy character becomes relatively higher. Therefore, it becomes more difficult for the second character to cause trouble to the first character, and it is possible to reduce the possibility that the user operating the first character will feel uncomfortable.

(Appendix 10)
Regarding a virtual space in which a character corresponding to the user can appear, based on a predetermined setting made for a second character by the user operating the first character, at least one dummy of the first character is set. An information processing system comprising a setting reflecting means (for example, a block control module 1741) that causes two dummy characters to appear in the virtual space.
According to such a configuration, the same effects as those of the program described in Appendix 7 can be achieved.

(Appendix 11)
computer,
a detection means (for example, a nuisance determination module 1761) capable of detecting behavior that causes a nuisance of a character in a virtual space where characters corresponding to each of a plurality of users can appear;
Specific control execution means (for example, specific control execution module 1771) that executes specific control based on the detection of the behavior that constitutes a nuisance;
A program that functions as
For example, if a user decides for themselves whether or not they are being harassed, and if they decide that they are being harassed, they take countermeasures. There is a high possibility that people will feel uncomfortable. On the other hand, according to this configuration, specific control is automatically executed based on the detection of behavior that constitutes a nuisance. Therefore, countermeasures can be taken in a relatively short time, and the possibility that the user will feel uncomfortable can be reduced. Thereby, it is possible to provide the user with a virtual space in which the possibility that the user feels uncomfortable is reduced.

(Appendix 12)
The specific control execution means is configured to perform the specific control by instructing the user who operates the first character to control the second character based on the detection of the behavior that causes a nuisance of the second character toward the first character. The program set forth in Appendix 11 that notifies that the person is engaging in the above-mentioned nuisance act.
According to such a configuration, the user operating the first character can notice that the second character is causing a nuisance, and can take immediate countermeasures.

(Appendix 13)
The program according to attachment 11 or attachment 12, wherein the detection means detects the behavior that constitutes a nuisance based on the time during which the user has a predetermined character in view.
According to such a configuration, the behavior of a user who has a predetermined character in his or her field of vision for a relatively long time can be determined to be a nuisance.

(Appendix 14)
Detection means capable of detecting behavior that is a nuisance of a character in a virtual space where characters corresponding to each of a plurality of users can appear;
An information processing system comprising: a specific control execution unit (for example, a specific control execution module 1771) that executes specific control based on the detection of the behavior causing a nuisance.
According to such a configuration, the same effects as those of the program described in Appendix 11 can be achieved.

5 user, 6 avatar object, 11 virtual space, 100 HMD system, 110 HMD set, 120 HMD, 130 monitor, 200 computer, 210 processor, 220 memory, 230 storage, 240 input/output interface, 250 communication interface, 300 controller, 410 HMD sensor, 420 motion sensor, 510 control module, 520 rendering module, 530 memory module, 540 communication control module, 600 server, 610 processor, 620 memory, 630 storage, 640 input/output interface, 650 communication interface, 700 external device, 1421 Virtual camera control module, 1422 Viewing area determination module, 1423 Reference line of sight identification module, 1424 Facial organ detection module, 1425 Movement detection module, 1426 Virtual space definition module, 1427 Virtual object generation module, 1428 Operation object control module, 1429 Avatar control module , 1438 View image generation module, 1610 Control module, 1630 Memory module, 1640 Communication control module, 1741 Block control module, 1751 Friend management module, 1761 Nuisance behavior determination module, 1771 Specific control execution module, 1781 History management module

Claims (4)

computer,
Regarding the virtual space in which a character corresponding to the user can appear, based on the setting that the second character is to be blocked by the user who operates the first character, in addition to the first character, the second character A program that functions as a setting reflection means for causing at least one dummy character to appear in the virtual space as a dummy of one character. The program according to claim 1, wherein the setting reflecting means causes a plurality of the dummy characters having different appearances to appear in the virtual space in parallel based on the setting . computer,
Regarding the virtual space in which a character corresponding to the user can appear, based on the user operating the first character making a predetermined setting for the second character, in addition to the first character, the first Functioning as a setting reflection means for causing at least one dummy character to appear in the virtual space as a character dummy;
The setting reflecting means is a program that changes the distinguishability of the first character with respect to a user operating the second character based on the predetermined setting. Regarding the virtual space in which a character corresponding to the user can appear, based on the setting that the second character is to be blocked by the user who operates the first character, in addition to the first character, the second character An information processing system comprising: a setting reflection means for causing at least one dummy character to appear in the virtual space as a dummy for one character.

Images