JP2020110376A - Game program, method, and information processing device - Google Patents

Abstract

To enhance a feeling of absorption in a game world, and improve amusement properties of the game.SOLUTION: A game program (831) causes a processor (810) to execute a step (S203) for arranging a first character and a virtual camera in a virtual space shared by a plurality of user terminals (800) and a character control device (110B); a step (S204) for displaying a visual field image according to the settings of the virtual camera; a step (S206) for progressing a game based on the game program in a first state on the basis of a first operation executed by the first character in response to an operation by a first user of the character control device; a step (S207) for transitioning the progress of the game from the first state to a second state different from the first state on the basis of a predetermined operation executed by the first character in response to the operation by the first user; and a step (S208) for progressing the game in the second state on the basis of a second operation executed by the first character in response to the operation by the first user.SELECTED DRAWING: Figure 21

Description

The present disclosure relates to a game program, a method for executing the game program, and an information processing device.

2. Description of the Related Art Conventionally, a game in which a story progresses with a different ending depending on the option selected by the user is widely known. For example, Non-Patent Document 1 discloses a romance simulation game whose main purpose is to virtually get closer to a girl character. The user proceeds to the story by selecting, from the presented choices, the one that is most suitable for the character, and repeating the reaction of the character to the character.

"Dengeki Online-Game/App/Manga/Anime Information Site", [online], KADOKAWA CORPORATION, [May 8, 2018 Search], Internet (URL: http://dengekionline.com/elem/000/000/ (279/279193/index-2.html)

In the game disclosed in Non-Patent Document 1, the response pattern of the character is prepared in advance. Then, in response to the input operation of the user, the response of the character is determined from the response pattern and output, and the game proceeds. Therefore, the variation of the motion of the character does not extend beyond the content of the data prepared in advance. Therefore, the user cannot feel the reality as if the character were in the real world with respect to the relationship with the character, and there is a problem that the user eventually gets tired. Generally, in a game developed with the intention of allowing the user to play for a long time, how to deal with the problem that the user gets tired of the game is important. Games are always required to provide engaging content that motivates users to play. For example, if the character appearing in the game has a high sense of reality, it is likely that the user will be more likely to immerse himself in the game world, and will find interest in the interaction with the character.

One aspect of the present disclosure aims to enhance the immersive feeling of the game world and improve the fun of the game.

The game program according to the present disclosure is executed by a user terminal including a processor and a memory. The game program is provided to the processor in a first virtual space shared between a first distribution terminal for the first distributor to perform real-time distribution and a plurality of viewing terminals including the viewing terminal. Arranging a corresponding first avatar object and a first virtual camera, displaying a first view image showing a view area of the first virtual camera on a display unit during real-time distribution, and a game program The game in one state is advanced based on the movement of the body of the first distributor and the operation of the viewer for each state of the first game part that progresses while transitioning between a plurality of states included in the game based on. And a step of performing a game process, wherein the game process in each state is performed by the first distribution terminal that detects a motion of the body of the first distributor and generates first motion information indicating a motion of the first avatar object. And a step of updating the first view image so as to include the first avatar object performing a motion indicated by the first motion information in response to the reception of the first motion information. A step of determining whether or not the motion indicated by the first motion information is a motion for causing a state transition in response to the reception of the first motion information; and the motion indicated by the first motion information causes a transition. The movement of the first game part, the step of transitioning the progress of the first game part from the current state to another state.

The method according to the present disclosure is a method in which a computer including a processor and a memory executes a game program. The method includes a processor performing the steps described above.

An information processing device according to the present disclosure includes a memory that stores the above-described game program, and a processor that controls the operation of the information processing device by executing the game program.

According to one aspect of the present disclosure, it is possible to enhance the immersive feeling in the game world and to enhance the interest of the game.

It is a figure showing the outline of the composition of the HMD system according to a certain embodiment. FIG. 16 is a block diagram showing an example of a hardware configuration of a computer according to an embodiment. It is a figure which represents notionally the uvw visual field coordinate system set to HMD according to a certain embodiment. It is a figure which represents notionally one aspect expressing the virtual space according to one embodiment. It is a figure showing from above the head of the user wearing the HMD according to an embodiment. It is a figure showing the YZ section which looked at the field of view from the X direction in virtual space. It is a figure showing the XZ section which looked at the field of view from the Y direction in virtual space. It is a figure showing the schematic structure of the controller according to a certain embodiment. FIG. 7 is a diagram showing an example of yaw, roll, and pitch directions defined for a user's right hand according to an embodiment. It is a block diagram showing an example of the hardware constitutions of the server according to a certain embodiment. FIG. 2 is a block diagram showing a computer according to an embodiment as a module configuration. 7 is a sequence chart showing a part of processing executed in the HMD set according to an embodiment. It is a schematic diagram showing the situation where each HMD provides a virtual space to a user in a network. It is a figure which shows the visual field image of the user 5A in FIG. 12(A). FIG. 7 is a sequence diagram showing a process executed in the HMD system according to an embodiment. It is a block diagram showing a detailed configuration of a module of a computer according to an embodiment. It is a figure showing the outline of the composition of the game system according to a certain embodiment. It is a figure which shows the hardware constitutions of the user terminal according to a certain embodiment. FIG. 7 is a block diagram showing a functional configuration of a user terminal, a server, and a character control device included in the game system according to the embodiment. It is a figure which shows the detailed structure of the virtual space control part contained in the user terminal according to a certain embodiment. It is a figure which shows the detailed structure of the virtual space control part contained in the character control device according to a certain embodiment. 9 is a flowchart showing an outline of a game progress according to an embodiment. It is a flow chart showing a flow of processing in a live game distribution part according to a certain embodiment. FIG. 9A is a diagram showing an example of a first virtual space realized in the character control device according to an embodiment. (B) is a diagram showing an example of a visual field image displayed on the character control device. (A) is a figure showing an example of the 1st virtual space realized in a user terminal according to a certain embodiment. (B) is a figure which shows an example of the visual field image displayed on the said user terminal. It is a flowchart showing a flow of processing in the first mini game executed by the game system according to the embodiment. FIG. 9A is a perspective view showing an example of a first virtual space realized when the first mini game proceeds in the character control device according to the embodiment. (B) is a top view showing an example of the first virtual space realized in the user terminal in synchronization with the first virtual space. FIG. 9A is a diagram showing an example of a visual field image displayed when the first mini game proceeds in the character control device according to an embodiment. FIG. 7B is a diagram showing an example of a game screen displayed on the user terminal when the first mini game proceeds. FIG. 9A is a diagram showing an example of a visual field image displayed when the first mini game proceeds in the character control device according to an embodiment. FIG. 7B is a diagram showing an example of a game screen displayed on the user terminal when the first mini game proceeds. FIG. 9A is a diagram showing an example of a visual field image displayed when the first mini game proceeds in the character control device according to an embodiment. FIG. 7B is a diagram showing an example of a game screen displayed on the user terminal when the first mini game proceeds. FIG. 9A is a diagram showing an example of a visual field image displayed when the first mini game proceeds in the character control device according to an embodiment. FIG. 7B is a diagram showing an example of a game screen displayed on the user terminal when the first mini game proceeds. It is a flowchart showing a flow of processing in a second mini game executed by the game system according to the embodiment. FIG. 9A is a perspective view showing an example of a first virtual space realized when the second mini game proceeds in the character control device according to the embodiment. (B) is a top view showing an example of the first virtual space realized in the user terminal in synchronization with the first virtual space. FIG. 9A is a diagram showing an example of a visual field image displayed when the second mini game proceeds in the character control device according to the embodiment. FIG. 9B is a diagram showing an example of a game screen displayed on the user terminal when the second mini game progresses. It is a figure showing an example of a game screen displayed at the time of progress of the 2nd mini game in a user terminal. FIG. 9A is a diagram showing an example of a visual field image displayed when the second mini game proceeds in the character control device according to the embodiment. FIG. 9B is a diagram showing an example of a game screen displayed on the user terminal when the second mini game progresses. It is a figure showing an example of a game screen displayed at the time of progress of the 2nd mini game in a user terminal. FIG. 9A is a diagram showing an example of a visual field image displayed when the second mini game proceeds in the character control device according to the embodiment. FIG. 9B is a diagram showing an example of a game screen displayed on the user terminal when the second mini game progresses. 9 is a flowchart showing a flow of processing in a live distribution part executed by the game system according to an embodiment. FIG. 9A is a diagram showing an example of a visual field image displayed when moving to the second virtual space in the character control device according to an embodiment. (B) is a figure which shows an example of the game screen displayed when moving to a 2nd virtual space in a user terminal. FIG. 9A is a diagram showing an example of a first virtual space after leaving an avatar object according to an embodiment. (B) is a diagram showing an example of the second virtual space after the appearance of the avatar object. It is a figure which shows an example of the game screen displayed when a user terminal participates in a 2nd virtual space. 7 is a flowchart showing a flow of processing in a chat part executed by the game system according to one embodiment. It is a figure which shows an example of the game screen displayed when a chat part progresses in a user terminal.

Hereinafter, embodiments of this technical idea will be described in detail with reference to the drawings. In the following description, the same parts are designated by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated. In one or more embodiments shown in the present disclosure, elements included in each embodiment can be combined with each other, and the resultant product is also a part of the embodiment shown in the present disclosure.

[Configuration of HMD system]
The configuration of an HMD (Head-Mounted Device) system 100 will be described with reference to FIG. 1. FIG. 1 is a diagram showing a schematic configuration of an HMD system 100 according to the present embodiment. The HMD system 100 is provided as a home system or a business system.

The HMD system 100 includes a server 600, HMD sets 110A, 110B, 110C and 110D, an external device 700, and a network 2. Each of the HMD sets 110A, 110B, 110C, and 110D is configured to be communicable with the server 600 and the external device 700 via the network 2. Hereinafter, the HMD sets 110A, 110B, 110C, and 110D are collectively referred to as the HMD set 110. The number of HMD sets 110 configuring the HMD system 100 is not limited to four, and may be three or less or five or more. The HMD set 110 includes an HMD 120, a computer 200, an HMD sensor 410, a display 430, and a controller 300. The HMD 120 includes a monitor 130, a gaze sensor 140, a first camera 150, a second camera 160, a microphone 170, and a speaker 180. The controller 300 may include a motion sensor 420.

In one aspect, the computer 200 can be connected to the Internet or other network 2 and can communicate with the server 600 or other computers connected to the network 2. Examples of other computers include computers of other HMD sets 110 and external devices 700. In another aspect, HMD 120 may include sensor 190 instead of HMD sensor 410.

The HMD 120 may be mounted on the head of the user 5 and provide the virtual space to the user 5 during operation. More specifically, the HMD 120 displays an image for the right eye and an image for the left eye on the monitor 130, respectively. When each eye of the user 5 visually recognizes each image, the user 5 can recognize the image as a three-dimensional image based on the parallax of both eyes. The HMD 120 can include both a so-called head mounted display including a monitor and a head mounted device to which a terminal having a monitor such as a smartphone can be mounted.

The monitor 130 is realized as, for example, a non-transmissive display device. In one aspect, the monitor 130 is arranged on the main body of the HMD 120 so as to be located in front of both eyes of the user 5. Therefore, the user 5 can immerse himself in the virtual space when viewing the three-dimensional image displayed on the monitor 130. In one aspect, the virtual space includes, for example, a background, an object that the user 5 can operate, and an image of a menu that the user 5 can select. In one aspect, monitor 130 can be realized as a liquid crystal monitor or an organic EL (Electro Luminescence) monitor included in a so-called smartphone or other information display terminal.

In another aspect, the monitor 130 can be realized as a transmissive display device. In this case, the HMD 120 may be an open type such as a spectacle type instead of the closed type that covers the eyes of the user 5 as shown in FIG. 1. The transmissive monitor 130 may be temporarily configurable as a non-transmissive display device by adjusting the transmittance thereof. The monitor 130 may include a configuration for simultaneously displaying a part of an image forming the virtual space and the real space. For example, the monitor 130 may display an image of the real space taken by a camera mounted on the HMD 120, or may make the real space visible by setting a part of the transmittances high.

In one aspect, monitor 130 may include a sub-monitor for displaying an image for the right eye and a sub-monitor for displaying an image for the left eye. In another aspect, the monitor 130 may be configured to integrally display an image for the right eye and an image for the left eye. In this case, the monitor 130 includes a high speed shutter. The high-speed shutter operates so that the image for the right eye and the image for the left eye can be displayed alternately so that the image is recognized by only one of the eyes.

In one aspect, the HMD 120 includes a plurality of light sources (not shown). Each light source is realized by, for example, an LED (Light Emitting Diode) that emits infrared rays. The HMD sensor 410 has a position tracking function for detecting the movement of the HMD 120. More specifically, the HMD sensor 410 reads a plurality of infrared rays emitted by the HMD 120 and detects the position and inclination of the HMD 120 in the physical space.

In another aspect, the HMD sensor 410 may be implemented by a camera. In this case, the HMD sensor 410 can detect the position and inclination of the HMD 120 by executing the image analysis process using the image information of the HMD 120 output from the camera.

In another aspect, the HMD 120 may include a sensor 190 as a position detector instead of the HMD sensor 410 or in addition to the HMD sensor 410. The HMD 120 can detect the position and the tilt of the HMD 120 itself using the sensor 190. For example, when the sensor 190 is an angular velocity sensor, a geomagnetic sensor, or an acceleration sensor, the HMD 120 may use any one of these sensors instead of the HMD sensor 410 to detect its own position and tilt. As an example, when the sensor 190 is an angular velocity sensor, the angular velocity sensor detects the angular velocity around the three axes of the HMD 120 in the physical space over time. The HMD 120 calculates the temporal change of the angle around the three axes of the HMD 120 based on each angular velocity, and further calculates the inclination of the HMD 120 based on the temporal change of the angle.

The gaze sensor 140 detects the directions in which the right and left eyes of the user 5 are directed. That is, the gaze sensor 140 detects the line of sight of the user 5. The detection of the direction of the line of sight is realized by, for example, a known eye tracking function. The gaze sensor 140 is realized by a sensor having the eye tracking function. In one aspect, the gaze sensor 140 preferably includes a sensor for the right eye and a sensor for the left eye. The gaze sensor 140 may be, for example, a sensor that irradiates the right eye and the left eye of the user 5 with infrared light, and receives the reflected light from the cornea and the iris with respect to the irradiated light to detect the rotation angle of each eyeball. .. The gaze sensor 140 can detect the line of sight of the user 5 based on each detected rotation angle.

The first camera 150 photographs the lower part of the face of the user 5. More specifically, the first camera 150 photographs the nose and mouth of the user 5. The second camera 160 captures the eyes and eyebrows of the user 5. The housing of the HMD 120 on the user 5 side is defined as the inside of the HMD 120, and the housing of the HMD 120 opposite to the user 5 is defined as the outside of the HMD 120. In an aspect, the first camera 150 may be arranged outside the HMD 120 and the second camera 160 may be arranged inside the HMD 120. The images generated by the first camera 150 and the second camera 160 are input to the computer 200. In another aspect, the first camera 150 and the second camera 160 may be realized as one camera, and the face of the user 5 may be captured by this one camera.

The microphone 170 converts the utterance of the user 5 into a voice signal (electrical signal) and outputs it to the computer 200. The speaker 180 converts the voice signal into voice and outputs the voice to the user 5. In another aspect, HMD 120 may include earphones instead of speaker 180.

The controller 300 is connected to the computer 200 by wire or wirelessly. The controller 300 receives an instruction input from the user 5 to the computer 200. In one aspect, the controller 300 is configured to be grippable by the user 5. In another aspect, the controller 300 is configured to be attachable to a part of the body or clothing of the user 5. In yet another aspect, the controller 300 may be configured to output at least one of vibration, sound, and light based on a signal transmitted from the computer 200. In still another aspect, the controller 300 receives an operation from the user 5 for controlling the position and movement of the object arranged in the virtual space.

In one aspect, the controller 300 includes multiple light sources. Each light source is realized by, for example, an LED that emits infrared rays. The HMD sensor 410 has a position tracking function. In this case, the HMD sensor 410 reads a plurality of infrared rays emitted by the controller 300 and detects the position and inclination of the controller 300 in the physical space. In another aspect, the HMD sensor 410 may be implemented by a camera. In this case, the HMD sensor 410 can detect the position and inclination of the controller 300 by executing the image analysis process using the image information of the controller 300 output from the camera.

The motion sensor 420 is attached to the hand of the user 5 in one aspect, and detects the movement of the hand of the user 5. For example, the motion sensor 420 detects the rotation speed, the rotation speed, etc. of the hand. The detected signal is sent to the computer 200. The motion sensor 420 is provided in the controller 300, for example. In one aspect, the motion sensor 420 is provided, for example, in the controller 300 that is configured to be held by the user 5. In another aspect, for safety in the real space, the controller 300 is attached to something that does not easily fly by being attached to the hand of the user 5 like a glove type. In yet another aspect, a sensor not attached to the user 5 may detect the movement of the hand of the user 5. For example, a signal of a camera that captures the user 5 may be input to the computer 200 as a signal representing the operation of the user 5. The motion sensor 420 and the computer 200 are wirelessly connected to each other, for example. In the case of wireless, the communication form is not particularly limited, and, for example, Bluetooth (registered trademark) or other known communication methods are used.

The display 430 displays an image similar to the image displayed on the monitor 130. As a result, a user other than the user 5 wearing the HMD 120 can also view the same image as the user 5. The image displayed on the display 430 does not have to be a three-dimensional image, and may be an image for the right eye or an image for the left eye. Examples of the display 430 include a liquid crystal display and an organic EL monitor.

The server 600 can send the program to the computer 200. In another aspect, the server 600 may communicate with other computers 200 for providing virtual reality to the HMD 120 used by other users. For example, when a plurality of users play a participation-type game at an amusement facility, each computer 200 communicates a signal based on the operation of each user with another computer 200 via the server 600 to allow a plurality of users in the same virtual space. Allows users to enjoy common games. Each computer 200 may communicate a signal based on the operation of each user with another computer 200 without passing through the server 600.

The external device 700 may be any device that can communicate with the computer 200. The external device 700 may be, for example, a device capable of communicating with the computer 200 via the network 2, or a device capable of directly communicating with the computer 200 by short-range wireless communication or wired connection. Examples of the external device 700 include, but are not limited to, smart devices, PCs (Personal Computers), and peripheral devices of the computer 200.

[Computer hardware configuration]
The computer 200 according to the present embodiment will be described with reference to FIG. FIG. 2 is a block diagram showing an example of a hardware configuration of computer 200 according to the present embodiment. The computer 200 includes a processor 210, a memory 220, a storage 230, an input/output interface 240, and a communication interface 250 as main components. Each component is connected to the bus 260.

The processor 210 executes a series of instructions included in a program stored in the memory 220 or the storage 230 based on a signal given to the computer 200 or when a predetermined condition is satisfied. In one aspect, the processor 210 is realized as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an MPU (Micro Processor Unit), an FPGA (Field-Programma Ble Gate Array), or other device.

The memory 220 temporarily stores programs and data. The program is loaded from the storage 230, for example. The data includes the data input to the computer 200 and the data generated by the processor 210. In one aspect, the memory 220 is implemented as a RAM (Random Access Memory) or other volatile memory.

The storage 230 permanently holds programs and data. The storage 230 is realized as, for example, a ROM (Read-Only Memory), a hard disk device, a flash memory, or another non-volatile storage device. The programs stored in the storage 230 include a program for providing a virtual space in the HMD system 100, a simulation program, a game program, a user authentication program, and a program for realizing communication with another computer 200. The data stored in the storage 230 includes data and objects for defining the virtual space.

In another aspect, the storage 230 may be realized as a removable storage device such as a memory card. In yet another aspect, instead of storage 230 built in computer 200, a configuration using programs and data stored in an external storage device may be used. With such a configuration, for example, in a situation where a plurality of HMD systems 100 are used, such as in an amusement facility, it is possible to collectively update programs and data.

The input/output interface 240 communicates signals with the HMD 120, the HMD sensor 410, the motion sensor 420, and the display 430. The monitor 130, the gaze sensor 140, the first camera 150, the second camera 160, the microphone 170, and the speaker 180 included in the HMD 120 can communicate with the computer 200 via the input/output interface 240 of the HMD 120. In one aspect, the input/output interface 240 is realized by using a USB (Universal Serial Bus), a DVI (Digital Visual Interface), an HDMI (registered trademark) (High-Definition Multimedia Interface), and other terminals. The input/output interface 240 is not limited to the above.

In certain aspects, the input/output interface 240 may also be in communication with the controller 300. For example, the input/output interface 240 receives input of signals output from the controller 300 and the motion sensor 420. In another aspect, the input/output interface 240 sends the instruction output from the processor 210 to the controller 300. The command instructs the controller 300 to perform vibration, voice output, light emission, or the like. Upon receiving the command, the controller 300 executes any one of vibration, voice output, and light emission according to the command.

The communication interface 250 is connected to the network 2 and communicates with another computer (for example, the server 600) connected to the network 2. In one aspect, the communication interface 250 is realized as, for example, a LAN (Local Area Network) or other wired communication interface, or a WiFi (Wireless Fidelity), Bluetooth (registered trademark), NFC (Near Field Communication), or other wireless communication interface. To be done. The communication interface 250 is not limited to the above.

In one aspect, the processor 210 accesses the storage 230, loads one or more programs stored in the storage 230 into the memory 220, and executes a series of instructions included in the program. The one or more programs may include an operating system of the computer 200, an application program for providing a virtual space, game software executable in the virtual space, and the like. The processor 210 sends a signal for providing a virtual space to the HMD 120 via the input/output interface 240. The HMD 120 displays an image on the monitor 130 based on the signal.

In the example shown in FIG. 2, the computer 200 is shown to be provided outside the HMD 120, but in another aspect, the computer 200 may be built in the HMD 120. As an example, a portable information communication terminal (for example, a smartphone) including the monitor 130 may function as the computer 200.

The computer 200 may be configured to be commonly used by the plurality of HMDs 120. According to such a configuration, for example, the same virtual space can be provided to a plurality of users, so that each user can enjoy the same application as other users in the same virtual space.

In one embodiment, in the HMD system 100, a real coordinate system that is a coordinate system in the real space is preset. The real coordinate system has three reference directions (axes) that are respectively parallel to the vertical direction in the real space, the horizontal direction orthogonal to the vertical direction, and the front-back direction orthogonal to both the vertical direction and the horizontal direction. The horizontal direction, vertical direction (vertical direction), and front-back direction in the real coordinate system are defined as the x-axis, the y-axis, and the z-axis, respectively. More specifically, in the real coordinate system, the x axis is parallel to the horizontal direction of the real space. The y-axis is parallel to the vertical direction of the real space. The z-axis is parallel to the front-back direction in real space.

In one aspect, HMD sensor 410 includes an infrared sensor. When the infrared sensor detects the infrared rays emitted from the respective light sources of the HMD 120, the presence of the HMD 120 is detected. The HMD sensor 410 further detects the position and inclination (direction) of the HMD 120 in the real space according to the movement of the user 5 wearing the HMD 120, based on the value of each point (each coordinate value in the real coordinate system). To do. More specifically, the HMD sensor 410 can detect a temporal change in the position and inclination of the HMD 120 by using each value detected over time.

Each inclination of the HMD 120 detected by the HMD sensor 410 corresponds to each inclination around the three axes of the HMD 120 in the real coordinate system. The HMD sensor 410 sets the uvw visual field coordinate system to the HMD 120 based on the inclination of the HMD 120 in the actual coordinate system. The uvw visual field coordinate system set in the HMD 120 corresponds to the viewpoint coordinate system when the user 5 wearing the HMD 120 views an object in the virtual space.

[Uvw view coordinate system]
The uvw visual field coordinate system will be described with reference to FIG. FIG. 3 is a diagram conceptually showing a uvw visual field coordinate system set in HMD 120 according to an embodiment. The HMD sensor 410 detects the position and inclination of the HMD 120 in the real coordinate system when the HMD 120 is activated. The processor 210 sets the uvw visual field coordinate system in the HMD 120 based on the detected values.

As shown in FIG. 3, the HMD 120 sets a three-dimensional uvw visual field coordinate system with the head of the user 5 wearing the HMD 120 as the center (origin). More specifically, the HMD 120 defines the horizontal direction, the vertical direction, and the front-rear direction (x axis, y axis, z axis) that define the real coordinate system by the inclination around each axis of the HMD 120 within the real coordinate system. Three directions newly obtained by inclining about the axis are set as a pitch axis (u axis), a yaw axis (v axis), and a roll axis (w axis) of the uvw visual field coordinate system in the HMD 120.

In a certain aspect, when the user 5 wearing the HMD 120 stands upright and visually recognizes the front, the processor 210 sets the uvw visual field coordinate system parallel to the real coordinate system in the HMD 120. In this case, the horizontal direction (x axis), the vertical direction (y axis), and the front-back direction (z axis) in the real coordinate system are the pitch axis (u axis) and yaw axis (v axis) of the uvw visual field coordinate system in the HMD 120. , And the roll axis (w axis).

After the uvw visual field coordinate system is set in the HMD 120, the HMD sensor 410 can detect the tilt of the HMD 120 in the set uvw visual field coordinate system based on the movement of the HMD 120. In this case, the HMD sensor 410 detects the pitch angle (θu), the yaw angle (θv), and the roll angle (θw) of the HMD 120 in the uvw visual field coordinate system as the inclination of the HMD 120. The pitch angle (θu) represents the tilt angle of the HMD 120 around the pitch axis in the uvw visual field coordinate system. The yaw angle (θv) represents the tilt angle of the HMD 120 around the yaw axis in the uvw visual field coordinate system. The roll angle (θw) represents the tilt angle of the HMD 120 around the roll axis in the uvw visual field coordinate system.

The HMD sensor 410 sets the uvw visual field coordinate system in the HMD 120 after the HMD 120 has moved to the HMD 120 based on the detected inclination of the HMD 120. The relationship between the HMD 120 and the uvw visual field coordinate system of the HMD 120 is always constant regardless of the position and inclination of the HMD 120. When the position and the inclination of the HMD 120 change, the position and the inclination of the uvw visual field coordinate system of the HMD 120 in the actual coordinate system change in association with the change of the position and the inclination.

In one aspect, the HMD sensor 410 uses the HMD 120 based on the light intensity of infrared light acquired based on the output from the infrared sensor and the relative positional relationship between the points (for example, the distance between the points). The position in the real space may be specified as the relative position with respect to the HMD sensor 410. The processor 210 may determine the origin of the uvw visual field coordinate system of the HMD 120 in the real space (real coordinate system) based on the specified relative position.

[Virtual space]
The virtual space will be further described with reference to FIG. FIG. 4 is a diagram conceptually showing one aspect of expressing virtual space 11 according to an embodiment. The virtual space 11 has a spherical structure that covers the entire center 12 in the 360° direction. In FIG. 4, the celestial sphere in the upper half of the virtual space 11 is illustrated in order not to complicate the description. Each mesh is defined in the virtual space 11. The position of each mesh is defined in advance as coordinate values in the XYZ coordinate system, which is the global coordinate system defined in the virtual space 11. The computer 200 associates each partial image forming the panoramic image 13 (still image, moving image, etc.) expandable in the virtual space 11 with each corresponding mesh in the virtual space 11.

In a certain aspect, in the virtual space 11, an XYZ coordinate system whose origin is the center 12 is defined. The XYZ coordinate system is, for example, parallel to the real coordinate system. The horizontal direction, the vertical direction (vertical direction), and the front-back direction in the XYZ coordinate system are defined as the X axis, the Y axis, and the Z axis, respectively. Therefore, the X axis of the XYZ coordinate system (horizontal direction) is parallel to the x axis of the real coordinate system, the Y axis of the XYZ coordinate system (vertical direction) is parallel to the y axis of the real coordinate system, and the X axis of the XYZ coordinate system is The Z axis (front-back direction) is parallel to the z axis of the real coordinate system.

When the HMD 120 is activated, that is, in the initial state of the HMD 120, the virtual camera 14 is arranged at the center 12 of the virtual space 11. In an aspect, the processor 210 displays the image captured by the virtual camera 14 on the monitor 130 of the HMD 120. The virtual camera 14 similarly moves in the virtual space 11 in association with the movement of the HMD 120 in the physical space. As a result, changes in the position and inclination of the HMD 120 in the physical space can be reproduced in the virtual space 11 as well.

The uvw visual field coordinate system is defined for the virtual camera 14 as in the case of the HMD 120. The uvw visual field coordinate system of the virtual camera 14 in the virtual space 11 is defined so as to be linked to the uvw visual field coordinate system of the HMD 120 in the real space (real coordinate system). Therefore, when the inclination of the HMD 120 changes, the inclination of the virtual camera 14 also changes accordingly. The virtual camera 14 can also move in the virtual space 11 in conjunction with the movement of the user 5 wearing the HMD 120 in the real space.

The processor 210 of the computer 200 defines the view area 15 in the virtual space 11 based on the position and the inclination (reference line of sight 16) of the virtual camera 14. The field-of-view area 15 corresponds to an area of the virtual space 11 that is visually recognized by the user 5 wearing the HMD 120. That is, the position of the virtual camera 14 can be said to be the viewpoint of the user 5 in the virtual space 11.

The line of sight of the user 5 detected by the gaze sensor 140 is the direction in the viewpoint coordinate system when the user 5 visually recognizes an object. The uvw visual field coordinate system of the HMD 120 is equal to the viewpoint coordinate system when the user 5 visually recognizes the monitor 130. The uvw view coordinate system of the virtual camera 14 is linked to the uvw view coordinate system of the HMD 120. Therefore, HMD system 100 according to an aspect can regard the line of sight of user 5 detected by gaze sensor 140 as the line of sight of user 5 in the uvw visual field coordinate system of virtual camera 14.

[User's line of sight]
The determination of the line of sight of the user 5 will be described with reference to FIG. FIG. 5 is a diagram showing the head of user 5 wearing HMD 120 according to an embodiment from above.

In an aspect, the gaze sensor 140 detects each line of sight of the right eye and the left eye of the user 5. In one aspect, when the user 5 is looking close, the gaze sensor 140 detects the sight lines R1 and L1. In another aspect, when the user 5 is looking far away, the gaze sensor 140 detects the sight lines R2 and L2. In this case, the angle formed by the lines of sight R2 and L2 with respect to the roll axis w is smaller than the angle formed by the lines of sight R1 and L1 with respect to the roll axis w. The gaze sensor 140 transmits the detection result to the computer 200.

When the computer 200 receives the detection values of the sight lines R1 and L1 from the gaze sensor 140 as the detection result of the sight line, the computer 200 specifies the attention point N1 which is the intersection of the sight lines R1 and L1 based on the detection value. On the other hand, when the computer 200 receives the detection values of the sight lines R2 and L2 from the gaze sensor 140, the computer 200 specifies the intersection of the sight lines R2 and L2 as the gaze point. The computer 200 specifies the line of sight N0 of the user 5 based on the specified position of the gazing point N1. The computer 200 detects, as the line of sight N0, for example, the extending direction of a straight line passing through the gazing point N1 and the midpoint of the straight line connecting the right eye R and the left eye L of the user 5. The line of sight N0 is the direction in which the user 5 is actually directing his or her eyes with both eyes. The line-of-sight N0 corresponds to the direction in which the user 5 is actually looking at the field of view 15.

In another aspect, the HMD system 100 may include a television broadcast receiving tuner. With such a configuration, the HMD system 100 can display a television program in the virtual space 11.

In still another aspect, the HMD system 100 may include a communication circuit for connecting to the Internet or a call function for connecting to a telephone line.

[Field of view]
The visibility region 15 will be described with reference to FIGS. 6 and 7. FIG. 6 is a view showing a YZ cross section of the view field 15 in the virtual space 11 as seen from the X direction. FIG. 7 is a view showing an XZ cross section of the view field 15 in the virtual space 11 as seen from the Y direction.

As shown in FIG. 6, the view field region 15 in the YZ section includes a region 18. The area 18 is defined by the position of the virtual camera 14, the reference line of sight 16, and the YZ section of the virtual space 11. The processor 210 defines a range including the polar angle α around the reference line of sight 16 in the virtual space as a region 18.

As shown in FIG. 7, the view region 15 in the XZ section includes a region 19. The area 19 is defined by the position of the virtual camera 14, the reference line of sight 16, and the XZ section of the virtual space 11. The processor 210 defines a range including the azimuth angle β around the reference line of sight 16 in the virtual space 11 as a region 19. The polar angles α and β are determined according to the position of the virtual camera 14 and the inclination (direction) of the virtual camera 14.

In one aspect, the HMD system 100 provides the user 5 with a view in the virtual space 11 by displaying the view image 17 on the monitor 130 based on a signal from the computer 200. The visual field image 17 is an image corresponding to a portion of the panoramic image 13 corresponding to the visual field region 15. When the user 5 moves the HMD 120 mounted on the head, the virtual camera 14 also moves in conjunction with the movement. As a result, the position of the view field 15 in the virtual space 11 changes. As a result, the view image 17 displayed on the monitor 130 is updated to the image of the panorama image 13 that is superimposed on the view region 15 in the direction in which the user 5 is facing in the virtual space 11. The user 5 can visually recognize a desired direction in the virtual space 11.

Thus, the inclination of the virtual camera 14 corresponds to the line of sight of the user 5 (reference line of sight 16) in the virtual space 11, and the position where the virtual camera 14 is arranged corresponds to the viewpoint of the user 5 in the virtual space 11. Therefore, by changing the position or inclination of the virtual camera 14, the image displayed on the monitor 130 is updated and the field of view of the user 5 is moved.

While wearing the HMD 120, the user 5 can visually recognize only the panoramic image 13 developed in the virtual space 11 without visually recognizing the real world. Therefore, the HMD system 100 can give the user 5 a high sense of immersion in the virtual space 11.

In an aspect, the processor 210 may move the virtual camera 14 in the virtual space 11 in conjunction with the movement of the user 5 wearing the HMD 120 in the real space. In this case, the processor 210 specifies the image area (visual field area 15) projected on the monitor 130 of the HMD 120 based on the position and inclination of the virtual camera 14 in the virtual space 11.

In one aspect, virtual camera 14 may include two virtual cameras, a virtual camera for providing an image for the right eye and a virtual camera for providing an image for the left eye. Appropriate parallax is set for the two virtual cameras so that the user 5 can recognize the three-dimensional virtual space 11. In another aspect, the virtual camera 14 may be realized by one virtual camera. In this case, the image for the right eye and the image for the left eye may be generated from the image obtained by one virtual camera. In the present embodiment, the virtual camera 14 includes two virtual cameras, and the roll axis (w) generated by combining the roll axes of the two virtual cameras is adapted to the roll axis (w) of the HMD 120. The technical idea according to the present disclosure will be exemplified as one configured as described above.

[controller]
An example of the controller 300 will be described with reference to FIG. FIG. 8 is a diagram showing a schematic configuration of controller 300 according to an embodiment.

As shown in FIG. 8, in one aspect, the controller 300 may include a right controller 300R and a left controller (not shown). The right controller 300R is operated by the right hand of the user 5. The left controller is operated by the left hand of the user 5. In one aspect, the right controller 300R and the left controller are symmetrically configured as separate devices. Therefore, the user 5 can freely move the right hand holding the right controller 300R and the left hand holding the left controller. In another aspect, the controller 300 may be an integrated controller that receives operations of both hands. The right controller 300R will be described below.

The right controller 300R includes a grip 310, a frame 320, and a top surface 330. The grip 310 is configured to be gripped by the right hand of the user 5. For example, the grip 310 may be held by the palm of the user 5's right hand and three fingers (middle finger, ring finger, little finger).

The grip 310 includes buttons 340 and 350 and a motion sensor 420. The button 340 is arranged on the side surface of the grip 310 and receives an operation by the middle finger of the right hand. The button 350 is arranged on the front surface of the grip 310 and receives an operation by the index finger of the right hand. In one aspect, buttons 340 and 350 are configured as trigger-type buttons. The motion sensor 420 is built in the housing of the grip 310. The grip 310 may not include the motion sensor 420 if the motion of the user 5 can be detected from around the user 5 by a camera or other device.

The frame 320 includes a plurality of infrared LEDs 360 arranged along the circumferential direction thereof. The infrared LED 360 emits infrared light according to the progress of the program while the program using the controller 300 is being executed. The infrared rays emitted from the infrared LED 360 can be used to detect each position and posture (tilt, orientation) of the right controller 300R and the left controller. In the example shown in FIG. 8, the infrared LEDs 360 arranged in two rows are shown, but the number of arrays is not limited to that shown in FIG. A single row or an array of three or more rows may be used.

The top surface 330 includes buttons 370 and 380 and an analog stick 390. Buttons 370 and 380 are configured as push buttons. Buttons 370 and 380 accept an operation by the thumb of the right hand of user 5. In a certain aspect, the analog stick 390 receives an operation in an arbitrary direction of 360 degrees from the initial position (neutral position). The operation includes, for example, an operation for moving an object arranged in the virtual sky 11.

In one aspect, the right controller 300R and the left controller include batteries for driving the infrared LEDs 360 and other components. Batteries include, but are not limited to, rechargeable, button type, dry cell type, and the like. In another aspect, the right controller 300R and the left controller may be connected to the USB interface of the computer 200, for example. In this case, the right controller 300R and the left controller do not require batteries.

[Server hardware configuration]
The server 10 according to the present embodiment will be described with reference to FIG. FIG. 9 is a block diagram showing an example of a hardware configuration of server 600 according to an embodiment. The server 600 includes a processor 610, a memory 620, a storage 630, an input/output interface 640, and a communication interface 650 as main components. Each component is connected to the bus 660.

Processor 610 executes a series of instructions included in a program stored in memory 620 or storage 630 based on a signal provided to server 600 or when a predetermined condition is satisfied. In one aspect, the processor 610 is implemented as a CPU, GPU, MPU, FPGA or other device.

The memory 620 temporarily stores programs and data. The program is loaded from the storage 630, for example. The data includes data input to the server 600 and data generated by the processor 610. In one aspect, memory 620 is implemented as RAM or other volatile memory.

The storage 630 permanently holds programs and data. The storage 630 is realized as, for example, a ROM, a hard disk device, a flash memory, or another non-volatile storage device. The programs stored in the storage 630 may include a program for providing a virtual space in the HMD system 100, a simulation program, a game program, a user authentication program, and a program for realizing communication with the computer 200. The data stored in the storage 630 may include data and objects for defining the virtual space.

In another aspect, the storage 630 may be realized as a removable storage device such as a memory card. In yet another aspect, instead of the storage 630 built in the server 600, a configuration using programs and data stored in an external storage device may be used. With such a configuration, for example, in a situation where a plurality of HMD systems 100 are used, such as in an amusement facility, it is possible to collectively update programs and data.

The input/output interface 640 communicates signals with input/output devices. In one aspect, the input/output interface 640 is implemented using a USB, DVI, HDMI, or other terminal. The input/output interface 640 is not limited to the above.

The communication interface 650 is connected to the network 2 and communicates with the computer 200 connected to the network 2. In one aspect, the communication interface 650 is realized as a wired communication interface such as a LAN or a wireless communication interface such as WiFi, Bluetooth, NFC or the like. The communication interface 650 is not limited to the above.

In an aspect, the processor 610 accesses the storage 630, loads one or more programs stored in the storage 630 into the memory 620, and executes the series of instructions included in the program. The one or more programs may include an operating system of the server 610, an application program for providing a virtual space, game software executable in the virtual space, and the like. The processor 610 may send a signal for providing the virtual space to the computer 200 via the input/output interface 640.

[HMD controller]
The control device of the HMD 21 will be described with reference to FIG. 10. In one embodiment, the control device is realized by computer 200 having a well-known configuration. FIG. 10 is a block diagram showing a computer 200 according to an embodiment as a module configuration.

As shown in FIG. 10, the computer 200 includes a control module 510, a rendering module 520, a memory module 530, and a communication control module 540. In one aspect, control module 510 and rendering module 520 are implemented by processor 210. In another aspect, multiple processors 210 may operate as control module 510 and rendering module 520. The memory module 530 is realized by the memory 220 or the storage 230. The communication control module 540 is realized by the communication interface 250.

The control module 510 controls the virtual space 11 provided to the user 5. The control module 510 uses the virtual space data representing the virtual space 11 to define the virtual space 11 in the HMD system 100. The virtual space data is stored in, for example, the memory module 530. The control module 510 may generate virtual space data or acquire virtual space data from the server 600 or the like.

The control module 510 arranges the object in the virtual space 11 by using the object data representing the object. The object data is stored in the memory module 530, for example. The control module 510 may generate the object data or acquire the object data from the server 600 or the like. The objects include, for example, avatar objects that are the alternations of the user 5, character objects, operation objects such as virtual hands operated by the controller 300, landscapes including forests, mountains, etc. arranged according to the progress of the game story, cityscapes, animals. And the like.

The control module 510 arranges the avatar object of the user 5 of the other computer 200 connected via the network 2 in the virtual space 11. In one aspect, the control module 510 arranges the avatar object of the user 5 in the virtual space 11. In one aspect, control module 510 arranges an avatar object imitating user 5 in virtual space 11 based on an image including user 5. In another aspect, the control module 510 arranges in the virtual space 2 an avatar object that has been selected by the user 5 from a plurality of types of avatar objects (for example, an object imitating an animal or an object of a deformed person). To do.

The control module 510 identifies the inclination of the HMD 120 based on the output of the HMD sensor 410. In another aspect, control module 510 identifies the tilt of HMD 120 based on the output of sensor 190 that functions as a motion sensor. The control module 510 detects organs (for example, mouth, eyes, eyebrows) forming the face of the user 5 from the images of the face of the user 5 generated by the first camera 150 and the second camera 160. The control module 510 detects the movement (shape) of each detected organ.

The control module 510 detects the line of sight of the user 5 in the virtual space 11 based on the signal from the gaze sensor 140. The control module 510 detects a viewpoint position (coordinate value in the XYZ coordinate system) at which the detected line of sight of the user 5 and the celestial sphere of the virtual space 11 intersect. More specifically, the control module 510 detects the viewpoint position based on the line of sight of the user 5 defined in the uvw coordinate system and the position and inclination of the virtual camera 14. The control module 510 transmits the detected viewpoint position to the server 600. In another aspect, the control module 510 may be configured to send line-of-sight information representing the line-of-sight of the user 5 to the server 600. In this case, the viewpoint position can be calculated based on the line-of-sight information received by the server 600.

The control module 510 reflects the movement of the HMD 120 detected by the HMD sensor 410 on the avatar object. For example, the control module 510 detects that the HMD 120 is tilted and tilts the avatar object. The control module 510 reflects the detected motion of the facial organ on the face of the avatar object arranged in the virtual space 11. The control module 510 receives the line-of-sight information of the other user 5 from the server 600 and reflects it on the line-of-sight of the avatar object of the other user 5. In one aspect, control module 510 reflects the movement of controller 300 on an avatar object or an operation object. In this case, the controller 300 includes a motion sensor for detecting the movement of the controller 300, an acceleration sensor, or a plurality of light emitting elements (for example, infrared LEDs).

The control module 510 arranges an operation object for receiving the operation of the user 5 in the virtual space 11 in the virtual space 11. The user 5 operates, for example, an object arranged in the virtual space 11 by operating the operation object. In one aspect, the operation object may include, for example, a hand object that is a virtual hand corresponding to the hand of the user 5. In one aspect, the control module 510 moves the hand object in the virtual space 11 based on the output of the motion sensor 420 so as to interlock with the movement of the hand of the user 5 in the real space. In an aspect, the operation object may correspond to the hand part of the avatar object.

When each of the objects arranged in the virtual space 11 collides with another object, the control module 510 detects the collision. The control module 510 can detect, for example, a timing at which a collision area of a certain object and a collision area of another object touch, and when the detection is performed, a predetermined process is performed. The control module 510 can detect the timing when the object is separated from the state in which the objects are in contact with each other, and when the detection is performed, a predetermined process is performed. The control module 510 can detect that the objects are in contact with each other. For example, control module 510, when the operation object touches another object, detects that the operation object touches the other object and performs a predetermined process.

In one aspect, control module 510 controls image display on monitor 130 of HMD 120. For example, the control module 510 arranges the virtual camera 14 in the virtual space 11. The control module 510 controls the position of the virtual camera 14 in the virtual space 11 and the inclination (direction) of the virtual camera 14. The control module 510 defines the field of view 15 according to the tilt of the head of the user 5 wearing the HMD 120 and the position of the virtual camera 14. The rendering module 510 generates the visual field image 17 displayed on the monitor 130 based on the determined visual field area 15. The visual field image 17 generated by the rendering module 520 is output to the HMD 120 by the communication control module 540.

When the control module 510 detects an utterance using the microphone 170 of the user 5 from the HMD 120, the control module 510 identifies the computer 200 to which the voice data corresponding to the utterance is transmitted. The voice data is transmitted to the computer 200 specified by the control module 510. When the control module 510 receives voice data from another user's computer 200 via the network 2, the control module 510 outputs a voice (utterance) corresponding to the voice data from the speaker 180.

The memory module 530 holds data used by the computer 200 to provide the virtual space 11 to the user 5. In one aspect, the memory module 540 holds spatial information, object information, and user information.

The spatial information holds one or more templates defined to provide the virtual space 11.

The object information includes a plurality of panoramic images 13 forming the virtual space 11, and object data for arranging objects in the virtual space 11. The panoramic image 13 may include a still image and a moving image. The panoramic image 13 may include an image in unreal space and an image in real space. Examples of the image in the unreal space include an image generated by computer graphics.

The user information holds a user ID that identifies the user 5. The user ID may be, for example, an IP (Internet Protocol) address or a MAC (Media Access Control) address set in the computer 200 used by the user. In another aspect, the user ID may be set by the user. The user information includes a program or the like for causing the computer 200 to function as a control device of the HMD system 100.

The data and programs stored in the memory module 530 are input by the user 5 of the HMD 120. Alternatively, the processor 210 downloads a program or data from a computer (for example, the server 600) operated by a business operator that provides the content, and stores the downloaded program or data in the memory module 530.

The communication control module 540 can communicate with the server 600 and other information communication devices via the network 2.

In one aspect, the control module 510 and the rendering module 520 can be implemented using Unity (registered trademark) provided by Unity Technologies, Inc., for example. In another aspect, the control module 510 and the rendering module 520 can be implemented as a combination of circuit elements that implement each process.

The processing in the computer 200 is realized by hardware and software executed by the processor 410. Such software may be stored in advance in the memory module 530 such as a hard disk. The software may be stored in a computer-readable non-volatile data recording medium such as a CD-ROM and distributed as a program product. Alternatively, the software may be provided as a downloadable program product by an information provider connected to the Internet or other network. Such software is read from a data recording medium by an optical disk drive or other data reading device, or downloaded from the server 600 or other computer via the communication control module 540, and then temporarily stored in the storage module. .. The software is read from the storage module by the processor 210 and stored in the RAM in the form of an executable program. The processor 210 executes the program.

[Control structure of HMD system]
The control structure of the HMD set 110 will be described with reference to FIG. 11. FIG. 11 is a sequence chart showing a part of the processing executed in HMD set 110 according to an embodiment.

As shown in FIG. 11, in step S1110, the processor 210 of the computer 200, as the control module 510, specifies virtual space data and defines the virtual space 11.

In step S1120, processor 210 initializes virtual camera 14. For example, the processor 210 arranges the virtual camera 14 at a predetermined center point 12 in the virtual space 11 in the work area of the memory, and directs the line of sight of the virtual camera 14 in the direction in which the user 5 is facing.

In step S1130, processor 210, as rendering module 520, generates view field image data for displaying an initial view field image. The generated visual field image data is output to the HMD 120 by the communication control module 540.

In step S1132, monitor 130 of HMD 120 displays the visual field image based on the visual field image data received from computer 200. The user 5 wearing the HMD 120 can recognize the virtual space 11 when viewing the visual field image.

In step S1134, HMD sensor 410 detects the position and inclination of HMD 120 based on the plurality of infrared rays emitted from HMD 120. The detection result is output to the computer 200 as motion detection data.

In step S1140, processor 210 specifies the visual field direction of user 5 wearing HMD 120 based on the position and inclination included in the motion detection data of HMD 120.

In step S1150, processor 210 executes the application program and arranges the object in virtual space 11 based on the instruction included in the application program.

In step S1160, controller 300 detects the operation of user 5 based on the signal output from motion sensor 420, and outputs the detection data representing the detected operation to computer 200. In another aspect, the operation of the controller 300 by the user 5 may be detected based on an image from a camera arranged around the user 5.

In step S1170, processor 210 detects the operation of controller 300 by user 5 based on the detection data acquired from controller 300.

In step S1180, processor 210 generates view image data based on the operation of controller 300 by user 5. The generated visual field image data is output to the HMD 120 by the communication control module 540.

In step S1190, HMD 120 updates the visual field image based on the received visual field image data, and displays the updated visual field image on monitor 130.

[Avatar object]
An avatar object according to the present embodiment will be described with reference to FIGS. 12(A) and 12(B). Hereinafter, it is a figure explaining the avatar object of each user 5 of HMD set 110A, 110B. Hereinafter, a user of the HMD set 110A will be referred to as a user 5A, a user of the HMD set 110B will be referred to as a user 5B, a user of the HMD set 110C will be referred to as a user 5C, and a user of the HMD set 110D will be referred to as a user 5D. The reference numeral of each component related to the HMD set 110A is attached with A, the reference numeral of each component related to the HMD set 110B is attached with B, the reference numeral of each component regarding the HMD set 110C is attached with C, and the HMD set D is attached to the reference numeral of each component related to 110D. For example, the HMD 120A is included in the HMD set 110A.

FIG. 12A is a schematic diagram showing a situation in which each HMD 120 provides the user 5 with the virtual space 11 in the network 2. The computers 200A to 200D provide the virtual spaces 11A to 11D to the users 5A to 5D via the HMDs 120A to 120D, respectively. In the example shown in FIG. 12A, the virtual space 11A and the virtual space 11B are composed of the same data. In other words, the computer 200A and the computer 200B share the same virtual space. The avatar object 6A of the user 5A and the avatar object 6B of the user 5B exist in the virtual space 11A and the virtual space 11B. The avatar object 6A in the virtual space 11A and the avatar object 6B in the virtual space 11B are each equipped with the HMD 120, but this is for the sake of easy understanding of the description. In reality, these objects are equipped with the HMD 120. Not not.

In an aspect, the processor 210A may arrange the virtual camera 14A that captures the visual field image 17A of the user 5A at the eye position of the avatar object 6A.

FIG. 12B is a diagram showing a visual field image 17A of the user 5A in FIG. 12A. The visual field image 17A is an image displayed on the monitor 130A of the HMD 120A. The view image 17A is an image generated by the virtual camera 14A. The avatar object 6B of the user 5B is displayed in the view image 17A. Although not particularly shown, the avatar object 6A of the user 5A is also displayed in the view image of the user 5B.

In the state of FIG. 12B, the user 5A can communicate with the user 5B through the virtual space 11A by dialogue. More specifically, the voice of the user 5A acquired by the microphone 170A is transmitted to the HMD 17120B of the user 5B via the server 600 and output from the speaker 180B provided in the HMD 120B. The voice of the user 5B is transmitted to the HMD 120A of the user 5A via the server 600 and output from the speaker 180A provided in the HMD 120A.

The action of the user 5B (the action of the HMD 120B and the action of the controller 300B) is reflected on the avatar object 6B arranged in the virtual space 11A by the processor 210A. As a result, the user 5A can recognize the action of the user 5B through the avatar object 6B.

FIG. 13 is a sequence chart showing a part of the processing executed in HMD system 100 according to the present embodiment. Although the HMD set 110D is not shown in FIG. 13, the HMD set 110D operates similarly to the HMD sets 110A, 110B, and 110C. Also in the following description, the reference numeral of each component regarding the HMD set 110A is attached with A, the reference numeral of each component regarding the HMD set 110B is attached with B, and the reference numeral of each component regarding the HMD set 110C is attached with C. It is assumed that D is added to the reference symbols of the respective components related to the HMD set 110D.

In step S1310A, the processor 210A in the HMD set 110A acquires avatar information for determining the action of the avatar object 6A in the virtual space 11A. This avatar information includes information about the avatar, such as motion information, face tracking data, and voice data. The movement information includes information indicating a temporal change in the position and inclination of the HMD 120A, information indicating a movement of the hand of the user 5A detected by the motion sensor 420A, and the like. The face tracking data may be data that specifies the position and size of each part of the face of the user 5A. The face tracking data includes data indicating the movement of each organ forming the face of the user 5A and line-of-sight data. The voice data may be data indicating the voice of the user 5A acquired by the microphone 170A of the HMD 120A. The avatar information may include information specifying the avatar object 6A, the user 5A associated with the avatar object 6A, information specifying the virtual space 11A in which the avatar object 6A exists, and the like. User ID is mentioned as information which specifies avatar object 6A and user 5A. Room ID is mentioned as information which identifies 11 A of virtual spaces in which avatar object 6A exists. The processor 210A transmits the avatar information acquired as described above to the server 600 via the network 2.

In step S1310B, the processor 210B in the HMD set 110B acquires avatar information for determining the action of the avatar object 6B in the virtual space 11B, and transmits it to the server 600, as in the process in step S1310A. Similarly, in step S1310C, processor 210B in HMD set 110B acquires avatar information for determining the action of avatar object 6C in virtual space 11C, and transmits it to server 600.

In step S1320, server 600 temporarily stores the player information received from each of HMD set 110A, HMD set 110B, and HMD set 110C. The server 600 integrates the avatar information of all users (in this example, users 5A to 5C) associated with the common virtual space 11 based on the user ID, the room ID, and the like included in each avatar information. Then, the server 600 transmits the integrated avatar information to all users associated with the virtual space 11 at a predetermined timing. As a result, the synchronization process is executed. Through such synchronization processing, the HMD set 110A, the HMD set 110B, and the HMD 11020C can share their avatar information at substantially the same timing.

Then, each HMD set 110A-110C performs the process of step S1330A-S1330C based on the avatar information transmitted from the server 600 to each HMD set 110A-110C. The process of step S1330A corresponds to the process of step S1180 in FIG.

In step S1330A, the processor 210A in the HMD set 110A updates the information of the avatar objects 6B and 6C of the other users 5B and 5C in the virtual space 11A. Specifically, the processor 210A updates the position and orientation of the avatar object 6B in the virtual space 11 based on the motion information included in the avatar information transmitted from the HMD set 110B. For example, the processor 210A updates the information (position, orientation, etc.) of the avatar object 6B included in the object information stored in the memory module 540. Similarly, the processor 210A updates the information (position, orientation, etc.) of the avatar object 6C in the virtual space 11 based on the motion information included in the avatar information transmitted from the HMD set 110C.

In step S1330B, the processor 210B in the HMD set 110B updates the information of the avatar objects 6A and 6C of the users 5A and 5C in the virtual space 11B, similarly to the processing in step S1330A. Similarly, in step S1330C, the processor 210C in the HMD set 110C updates the information of the avatar objects 6A and 6B of the users 5A and 5B in the virtual space 11C.

[Detailed Configuration of Modules of Computer 200]
The module configuration of the computer 200 will be described in detail with reference to FIG. FIG. 14 is a block diagram showing a detailed configuration of modules of computer 200 according to an embodiment. As shown in FIG. 14, the control module 510 includes a virtual object generation module 1421, a virtual camera control module 1422, an operation object control module 1423, an avatar object control module 1424, a motion detection module 1425, a collision detection module 1426, and a virtual object. A control module 1427 is provided.

The virtual object generation module 1421 generates various virtual objects in the virtual space 11. In one aspect, the virtual object may include, for example, a landscape including forests, mountains, and the like arranged according to the progress of the game story, animals, and the like. In one aspect, the virtual object may include an avatar object, an operation object, a stage object, and a UI (User Interface) object.

The virtual camera control module 1422 controls the behavior of the virtual camera 14 in the virtual space 11. The virtual camera control module 1422 controls, for example, the arrangement position of the virtual camera 14 in the virtual space 11 and the orientation (tilt) of the virtual camera 14.

The operation object control module 1423 controls the operation object for receiving the operation of the user 5 in the virtual space 11. The user 5 operates, for example, the virtual object arranged in the virtual space 11 by operating the operation object. In one aspect, the operation object may include, for example, a hand object (virtual hand) corresponding to the hand of the user 5 wearing the HMD 120. In one aspect, the operation object may correspond to the hand portion of the avatar object described below.

The avatar object control module 1424 reflects the movement of the HMD 120 detected by the HMD sensor 410 on the avatar object. For example, the avatar object control module 1424 detects that the HMD 120 is tilted and generates data for tilting and placing the avatar object. In an aspect, the avatar object control module 1424 reflects the movement of the controller 300 on the avatar object. In this case, the controller 300 includes a motion sensor for detecting the movement of the controller 300, an acceleration sensor, or a plurality of light emitting elements (for example, infrared LEDs). For example, the avatar object control module 1424 reflects the motion of the hand detected by the motion detection module 1425 on the motion of the operation object (the virtual hand that is a part of the hand of the avatar object) arranged in the virtual space 11. The avatar object control module 1424 reflects the motion of the facial organs detected by the motion detection module 1425 on the face of the avatar object placed in the virtual space 11. That is, the avatar object control module 1424 reflects the action of the face of the user 5 on the avatar object.

The motion detection module 1425 detects the motion of the user 5. The motion detection module 1425 detects the motion of the hand of the user 5 according to the output of the controller 300, for example. The motion detection module 1425 detects the motion of the body of the user 5, for example, according to the output of the motion sensor attached to the body of the user 5. The motion detection module 1425 can also detect the motion of the facial organs of the user 5.

The collision detection module 1426 detects a collision when each of the virtual objects arranged in the virtual space 11 collides with another virtual object. The collision detection module 1426 can detect the timing at which a certain virtual object touches another virtual object, for example. The collision detection module 1426 can detect the timing when the virtual object and the other virtual object are apart from each other. The collision detection module 1426 can also detect that one virtual object is in contact with another virtual object. The collision detection module 1426 detects, for example, when the operation object touches another virtual object, the operation object touches the other object. The collision detection module 1426 executes a predetermined process based on these detection results.

The virtual object control module 1427 controls the behavior of virtual objects other than the avatar object in the virtual space 11. As an example, the virtual object control module 1427 transforms the virtual object. As another example, the virtual object control module 1427 changes the placement position of the virtual object. As another example, virtual object control module 1427 moves a virtual object.

As still another example, the virtual object control module 1427 detects pressing of a predetermined button included in the controller 300 in a state where the collision detection module 1426 detects that the virtual object and the operation object (virtual hand) are touching. Then, the virtual object and the operation object are associated with each other. At this time, the avatar object control module 1424 may control the operation object (virtual hand) to perform an operation targeting the virtual object. As an example of the target motion, for example, a motion of holding, grasping, grasping, placing on the palm, pinching, or the like may be performed.

In addition, when the motion detection module 1425 detects the movement of the hand of the user 5 in a state where the virtual object and the operation object are associated with each other, the virtual object control module 1427 moves the operation object and moves the virtual object. Control to

Further, the virtual object control module 1427 releases the association between the virtual object and the operation object when the pressing of the predetermined button included in the controller 300 is detected in the state where the virtual object and the operation object are associated with each other. At this time, the avatar object control module 1424 may control the operation object (virtual hand) to release the virtual object.

[Game system configuration]
FIG. 15 is a diagram showing a schematic configuration of a game system 1500 according to the present embodiment. The game system 1500 is a system for providing a game to a plurality of users. The game system 1500 includes a server 600, an HMD set 110B, user terminals 800A, 800C and 800D, and a network 2. The HMD set 110B and each of the user terminals 800A, 800C, 800D are configured to be communicable with the server 600 via the network 2. Hereinafter, the user terminals 800A, 800C, and 800D are also collectively referred to as the user terminal 800. The number of user terminals 800 configuring the game system 1500 is not limited to three, and may be two or less or four or more.

The HMD set 110B constitutes an embodiment of the character control device in the present invention. Hereinafter, the HMD set 110B will also be referred to as a character control device 110B. At least one character control device 110B is provided in the game system 1500. A plurality of character control devices 110B may be provided according to the number of user terminals 800 that use the service provided by the server 600. One character control device 110B may be provided for one user terminal 800. One character control device 110B may be provided for a plurality of user terminals 800.

In the present embodiment, an example in which the same character control device 110B is applied as the first character control device and the second character control device in the present invention will be described.

Hereinafter, a user of the HMD set 110B will be referred to as a user 5B. Further, the user of the user terminal 800A is referred to as a user 8A, the user of the user terminal 800C is referred to as a user 8C, and the user of the user terminal 800D is referred to as a user 8D. The users 8A, 8C, and 8D are also collectively referred to as the user 8. The reference numeral of each constituent element regarding the user terminal 800A is attached with A, the reference numeral of each constituent element regarding the HMD set 110B is attached with B, the reference numeral of each constituent element regarding the user terminal 800C is attached with C, and the user terminal D is attached to the reference numeral of each component related to 800D.

The game system 1500 is a system for streaming delivery of a program performed by the avatar object 6B associated with the user 5B in the virtual space 11B from the character control device 110B to each user terminal 800. The avatar object 6B is an example of the first character and the second character in the present invention. The virtual space 11B is shared between the character control device 110B and a plurality of user terminals 800. In other words, the virtual spaces 11A, 11C and 11D are synchronized with the virtual space 11B, respectively. Hereinafter, if there is no particular need to distinguish between the virtual spaces 11A, B, C, and D, they are also simply referred to as "virtual space 11". The user 5B advances the program of the avatar object 6B by controlling the avatar object 6B in the character control device 110B. The user 8A views the delivered program of the avatar object 6B through the user terminal 800A. The user 8C views the delivered program of the avatar object 6B through the user terminal 800C. The user 8D views the delivered program of the avatar object 6B through the user terminal 800D.

<Hardware configuration of each device constituting game system 1500>
The hardware configuration of each device constituting the game system 1500 will be described. The server 600 (computer, information processing device) may be a general-purpose computer such as a workstation or a personal computer. The hardware configuration of the server 600 is as described with reference to FIG.

The character control device 110B may be a server, a desktop personal computer, a notebook personal computer, a computer such as a tablet, or a computer group combining these. The hardware configuration of the character control device 110B is as described with reference to FIG. The network 2 to which the respective devices constituting the game system 1500 are connected is composed of the Internet and various mobile communication systems constructed by a wireless base station (not shown). Examples of this mobile communication system include so-called 3G and 4G mobile communication systems, LTE (Long Term Evolution), and a wireless network (for example, Wi-Fi (registered trademark)) that can be connected to the Internet by a predetermined access point. To be

FIG. 16 is a diagram showing a hardware configuration of the user terminal 800. The user terminal 800 (computer, information processing device) may be a mobile terminal such as a smartphone, a feature phone, a PDA (Personal Digital Assistant), or a tablet computer. The user terminal 800 may be a game device suitable for game play. As illustrated, the user terminal 800 includes a processor 810, a memory 820, a storage 830, a communication interface (IF) 13, an input/output IF 840, a touch screen 870 (display unit), a camera 860, and a distance measuring sensor 880. With. These components included in the user terminal 800 are electrically connected to each other by a communication bus. Note that the user terminal 800 may include an input/output IF 840 to which a display (display unit) configured separately from the main body of the user terminal 800 can be connected instead of or in addition to the touch screen 870.

Further, as shown in FIG. 16, the user terminal 800 may be configured to be able to communicate with one or more controllers 1020. The controller 1020 establishes communication with the user terminal 800 according to a communication standard such as Bluetooth (registered trademark). The controller 1020 may have one or more buttons and the like, and sends an output value based on the input operation of the user 8 to the buttons and the like to the user terminal 800. Further, the controller 1020 may have various sensors such as an acceleration sensor and an angular velocity sensor, and outputs output values of the various sensors to the user terminal 800.

Note that instead of or in addition to the user terminal 800 including the camera 860 and the distance measuring sensor 880, the controller 1020 may include the camera 860 and the distance measuring sensor 880.

The user terminal 800 preferably causes the user 8 who uses the controller 1020 to input user identification information such as the name or login ID of the user 8 via the controller 1020 at the start of the game, for example. As a result, the user terminal 800 can associate the controller 1020 with the user 8 and determine which user 8 the output value belongs to based on the transmission source (controller 1020) of the received output value. Can be specified.

When the user terminal 800 communicates with a plurality of controllers 1020, each user 10 holds each controller 1020 so that the user terminal 800 does not communicate with other devices such as the server 600 via the network 2 and the one user terminal Multiplay can be realized with 800. In addition, each user terminal 800 communicates with each other according to a wireless standard such as a wireless LAN (Local Area Network) standard (communicatively connects without passing through the server 600), thereby realizing multi-play locally by a plurality of user terminals 800. You can also do it. When the above-described multiplay is locally realized by one user terminal 800, the user terminal 800 may further include at least a part of various functions of the server 600, which will be described later. When the above-described multiplay is locally realized by the plurality of user terminals 800, the plurality of user terminals 800 may be provided with various functions of the server 600, which will be described later, in a distributed manner.

Note that the user terminal 800 may communicate with the server 600 even when the above-described multiplay is realized locally. For example, information indicating a play result such as a score or a win or loss in a game and user identification information may be associated with each other and transmitted to the server 600.

Moreover, the controller 1020 may be configured to be attachable to and detachable from the user terminal 800. In this case, a coupling part with the controller 1020 may be provided on at least one surface of the housing of the user terminal 800. When the user terminal 800 and the controller 1020 are connected by wire via the connecting unit, the user terminal 800 and the controller 1020 send and receive signals via wire.

As shown in FIG. 16, the user terminal 800 may accept mounting of the storage medium 1030 such as an external memory card via the input/output IF 840. As a result, the user terminal 800 can read the program and data recorded in the storage medium 1030. The program recorded in the storage medium 1030 is, for example, a game program.

The user terminal 800 may store the game program acquired by communicating with an external device such as the server 600 in the memory 820 of the user terminal 800, or the game program acquired by reading from the storage medium 1030 in the memory 820. May be stored in.

As described above, the user terminal 800 includes the communication IF 850, the input/output IF 840, the touch screen 870, the camera 860, the distance measuring sensor 880, and the speaker 890 as an example of a mechanism for inputting information to the user terminal 800. Prepare Each of the above-described units as a mechanism for inputting can be regarded as an operation unit configured to receive the input operation of the user 8.

For example, when the operation unit is configured by at least one of the camera 860 and the distance measuring sensor 880, the operation unit detects the object 1010 near the user terminal 800 and performs an input operation from the detection result of the object. Identify. As an example, the hand of the user 8 as the object 1010, a marker having a predetermined shape, or the like is detected, and the input operation is specified based on the color, shape, movement, or type of the object 1010 obtained as a detection result. To be done. More specifically, when the hand of the user 8 is detected from the captured image of the camera 860, the user terminal 800 uses the gesture (a series of movements of the hand of the user 8) detected based on the captured image to the user 8 Is specified and accepted as the input operation of. The captured image may be a still image or a moving image.

Alternatively, when the operation unit includes the touch screen 870, the user terminal 800 identifies and accepts the operation of the user 8 performed on the input unit 8701 of the touch screen 870 as the input operation of the user 8. Alternatively, when the operation unit is configured by the communication IF 850, the user terminal 800 identifies and receives a signal (for example, output value) transmitted from the controller 1020 as the input operation of the user 8. Alternatively, when the operation unit is configured by the input/output IF 840, a signal output from an input device (not shown) different from the controller 1020 connected to the input/output IF 840 is specified and accepted as the input operation of the user 8. ..

<Hardware components of each device>
The processor 810 controls the operation of the entire user terminal 800. The processor 610 controls the operation of the entire server 600. The processor 210 controls the operation of the entire character control device 110B. Processors 810, 20 and 30 include a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a GPU (Graphics Processing Unit).

The processor 810 reads the program from the storage 830 and loads it in the memory 820. The processor 610 reads the program from the storage 630 and loads it in the memory 620. The processor 210 reads a program from a storage 230 described later and loads it on a memory 220 described later. The processor 810, the processor 610, and the processor 210 execute the expanded program.

Memories 820, 620 and 220 are the main storage devices. The memories 820, 620 and 220 are configured by storage devices such as ROM (Read Only Memory) and RAM (Random Access Memory). The memory 820 provides a work area to the processor 810 by temporarily storing a program and various data read by the processor 810 from a storage 830 described later. The memory 820 also temporarily stores various data generated while the processor 810 operates according to the program. The memory 620 provides a work area to the processor 610 by temporarily storing various programs and data read by the processor 610 from the storage 630. The memory 620 also temporarily stores various data generated while the processor 610 operates according to a program. The memory 220 provides a work area to the processor 210 by temporarily storing various programs and data read by the processor 210 from the storage 230. The memory 220 also temporarily stores various data generated while the processor 210 operates according to a program.

In the present embodiment, the program may be a game program for realizing the game on the user terminal 800. Alternatively, the program may be a game program for realizing the game by cooperation between the user terminal 800 and the server 600.
Alternatively, the program may be a game program for realizing the game in cooperation with the user terminal 800, the server 600, and the character control device 110B. The game realized by the cooperation of the user terminal 800 and the server 600 and the game realized by the cooperation of the user terminal 800, the server 600, and the character control device 110B are started in the user terminal 800 as an example. It may be a game executed on a browser. Alternatively, the program may be a game program for realizing the game by cooperation of a plurality of user terminals 800. In addition, the various data includes data relating to the game such as user information and game information, and instructions or notifications transmitted and received between the respective devices of the game system 1500.

The storages 830, 630 and 230 are auxiliary storage devices. The storages 830, 630, and 230 are configured by a storage device such as a flash memory or an HDD (Hard Disk Drive). Various data relating to the game are stored in the storages 830, 630 and 230.

The communication IF 850 controls transmission/reception of various data in the user terminal 800. The communication IF 650 controls transmission/reception of various data in the server 600. The communication IF 250 controls transmission/reception of various data in the character control device 110B. The communication IFs 850, 650, and 250 control communication using, for example, communication via a wireless LAN (Local Area Network), wired LAN, wireless LAN, or Internet communication via a mobile phone network, and communication using short-range wireless communication. To do.

The input/output IF 840 is an interface for the user terminal 800 to receive data input, and is an interface for the user terminal 800 to output data. The input/output IF 840 may input/output data via a USB (Universal Serial Bus) or the like. The input/output IF 840 may include, for example, a physical button of the user terminal 800, a camera, a microphone, a speaker, or the like. The input/output IF 640 of the server 600 is an interface for the server 600 to receive data input, and is an interface for the server 600 to output data. The input/output IF 640 may include, for example, an input unit that is an information input device such as a mouse or a keyboard, and a display unit that is a device that displays and outputs an image. The input/output IF 240 of the character control device 110B is an interface for the character control device 110B to receive data input, and is an interface for the character control device 110B to output data. The input/output IF 240 is, for example, an information input device such as a mouse, a keyboard, a stick, and a lever, a display 430 that displays and outputs an image such as a liquid crystal display, and peripheral devices (HMD 120, HMD sensor 410, motion sensor 420, controller 300, etc.). ) And a connection for sending and receiving data to and from.

In the touch screen 870 of the user terminal 800, the input unit 8701, which is an electronic component that combines the input unit 8701 and the display unit 8702, is, for example, a touch-sensitive device, and is configured by, for example, a touch pad. The display unit 8702 is composed of, for example, a liquid crystal display, an organic EL (Electro-Luminescence) display, or the like.

The input unit 8701 detects a position at which an operation of the user 8 (mainly a physical contact operation such as a touch operation, a slide operation, a swipe operation, and a tap operation) on the input surface is detected, and information indicating the position is displayed. It has a function of transmitting as an input signal. The input unit 8701 may include a touch sensing unit (not shown). The touch sensing unit may be of any type such as a capacitance type or a resistance film type.

Like the touch screen 870 of the user terminal 800, the display 430 of the character control device 110B may be configured by a touch screen in which an input unit and a display unit are combined.

Although not shown, the user terminal 800 may include one or more sensors for specifying the holding posture of the user terminal 800. This sensor may be, for example, an acceleration sensor or an angular velocity sensor. When the user terminal 800 includes a sensor, the processor 810 can also specify the holding posture of the user terminal 800 from the output of the sensor and perform processing according to the holding posture. For example, the processor 810 may perform vertical screen display in which a vertically long image is displayed on the display unit 8702 when the user terminal 800 is held vertically. On the other hand, when the user terminal 800 is held in the horizontal orientation, a horizontally long image may be displayed on the display unit. As described above, the processor 810 may be capable of switching between the vertical screen display and the horizontal screen display according to the holding posture of the user terminal 800.

The speaker 890 outputs a sound based on the sound data. The camera 860 includes an image sensor and the like, and converts the incident light incident from the lens into an electric signal to generate a captured image.

The distance measuring sensor 880 is a sensor that measures the distance to the measurement target. The distance measuring sensor 880 includes, for example, a light source that emits pulse-converted light and a light receiving element that receives the light. The distance measuring sensor 880 measures the distance to the measurement object by the light emission timing from the light source and the light reception timing of the reflected light generated by the light emitted from the light source reflected on the measurement object. The distance measuring sensor 880 may include a light source that emits light having directivity.

Here, an example in which the user terminal 800 receives the detection result of detecting the object 1010 near the user terminal 800 using the camera 860 and the distance measuring sensor 880 as the input operation of the user 8 will be further described. The camera 860 and the distance measuring sensor 880 may be provided, for example, on the side surface of the housing of the user terminal 800. A distance measuring sensor 880 may be provided near the camera 860. As the camera 860, for example, an infrared camera can be used. In this case, the camera 860 may be provided with a lighting device that emits infrared rays, a filter that blocks visible light, and the like. Thereby, the detection accuracy of the object based on the image captured by the camera 860 can be further improved regardless of whether it is outdoors or indoors.

The processor 810 may perform one or more processes of the following processes (1) to (5) on the captured image of the camera 860. (1) The processor 810 identifies whether or not the hand of the user 8 is included in the captured image by performing image recognition processing on the captured image of the camera 860. The processor 810 may use, for example, a technique such as pattern matching as the analysis technique used in the above-described image recognition processing. (2) Further, the processor 810 detects the gesture of the user 8 from the shape of the hand of the user 8. The processor 810 specifies the number of fingers of the user 8 (the number of extended fingers) from the shape of the hand of the user 8 detected from the captured image, for example. The processor 810 further specifies the gesture made by the user 8 from the specified number of fingers. For example, when the number of fingers is 5, the processor 810 determines that the user 8 has made a “par” gesture. Further, when the number of fingers is 0 (no finger is detected), the processor 810 determines that the user 8 has made a gesture of “goo”. Further, when the number of fingers is two, the processor 810 determines that the user 8 has made a gesture of “cutting”. (3) The processor 810 performs the image recognition process on the image captured by the camera 860 to determine whether the user's 8 finger is in the state where only the forefinger is raised or the user's 8 finger is repelled. To detect. (4) The processor 810, based on at least one of the image recognition result of the image captured by the camera 860, the output value of the distance measuring sensor 880, and the like, the object 1010 near the user terminal 800 (the hand of the user 8). Etc.) and the user terminal 800. For example, the processor 810 determines whether the hand of the user 8 is near the user terminal 800 (for example, a distance less than a predetermined value) or far (for example, depending on the size of the shape of the hand of the user 8 specified from the image captured by the camera 860). It is detected whether the distance is a predetermined value or more). If the captured image is a moving image, the processor 810 may detect whether the hand of the user 8 is approaching or moving away from the user terminal 800. (5) When it is determined that the distance between the user terminal 800 and the hand of the user 8 changes while the hand of the user 8 is detected based on the image recognition result of the image captured by the camera 860 and the like. The processor 810 recognizes that the user 8 is shaking his hand in the shooting direction of the camera 860. When an object is detected or not detected by the distance measuring sensor 880, which has a higher directivity than the shooting range of the camera 860, the processor 810 causes the user 8 to shake his hand in a direction orthogonal to the shooting direction of the camera. Recognize that.

In this way, the processor 810 recognizes whether the user 8 is clasping his/her hand based on the image recognition of the image captured by the camera 860 (whether the gesture is “goo” or another gesture (for example, “par”)). If there is). The processor 810 also detects the shape of the hand of the user 8 and how the user 8 is moving this hand. Further, the processor 810 detects whether the user 8 is moving this hand close to or away from the user terminal 800. Such an operation can correspond to an operation using a pointing device such as a mouse or a touch panel. The user terminal 800, for example, moves the pointer on the touch screen 870 according to the movement of the hand of the user 8 to detect the gesture “goo” of the user 8. In this case, the user terminal 800 recognizes that the user 8 is continuing the selection operation. The continuation of the selection operation corresponds to, for example, maintaining the state where the mouse is clicked and pressed, or maintaining the state where the touch panel is touched after the touchdown operation. Further, when the user 8 further moves the hand while the gesture “goo” of the user 8 is detected, the user terminal 800 responds to such a series of gestures with a swipe operation (or a drag operation). It can also be recognized as an operation. Further, when the user terminal 8 detects a gesture such that the user 8 repels a finger based on the detection result of the hand of the user 8 by the image captured by the camera 860, the user terminal 800 transmits the gesture to a mouse click or a touch panel. It may be recognized as an operation corresponding to the tap operation.

<Game outline>
The game executed by the game system 1500 according to the first embodiment (hereinafter, this game) includes, as an example, a game part in which one or more characters are made to appear and at least one of the characters is operated. The character appearing in the game may be a player character (hereinafter, PC) or a non-player character (hereinafter, NPC). The PC is a character that the user 8 can directly operate. The NPC is a character that operates according to the game program and the operation instruction data, that is, the character that the user 8 cannot directly operate. In the following, when there is no particular need to distinguish between the two, "character" is used as a generic name.

This game includes a live distribution part in which the virtual space 11 is shared between the character control device 110B and the plurality of user terminals 800, and the avatar object 6B operates according to the operation of the user 5B of the character control device 110B. Here, the operation of the user 5B includes information input via the HMD 120, the HMD sensor 410, the motion sensor 420, and the controller 300 according to the movement of the user 5B. Further, the operation of the user 5B includes an operation performed by the user 5B on a button included in the controller 300. In addition, the operation of the user 5B may include an operation input via various input devices (not shown) connected to the computer 200.

In this game, as the virtual space 11, the first virtual space 11_1 and the second virtual space 11_2 different from the first virtual space 11_1 are provided. Details of the first virtual space 11_1 and the second virtual space 11_2 will be described later.

In the live distribution part, for example, motion instruction data representing the motion of the character is generated by the character control device 110B according to the operation of the user 5B. Further, the operation instruction data is supplied to the user terminal 800 that is executing the present game from any device other than the user terminal 800 at an arbitrary timing. The motion instruction data may be supplied to the user terminal 800 from the character control device 110B via the server 600, for example. The user terminal 800 analyzes (renders) the operation instruction data by using the reception of the operation instruction data as a trigger. The live distribution part is a part in which the user terminal 800 presents to the user 8 a character that operates according to the analyzed operation instruction data in real time. As a result, the user 8 can have a sense of reality as if the character really exists, and can further immerse himself in the game world and enjoy the present game.

This game may be composed of a plurality of play parts. In this case, one character may be a PC in one part and an NPC in another part, and the character characteristics may be different for each part.

The genre of the game executed in the game system 1500 is not limited to a particular genre. The game system 1500 can execute games of all genres. For example, games on sports such as tennis, table tennis, dodge ball, baseball, soccer and hockey, puzzle games, quiz games, RPG (Role-Playing Game), adventure games, shooting games, simulation games, training games, and It may be an action game or the like.

Further, the play mode of this game is not limited to a specific play mode. The game system 1500 can execute a game of any play form. For example, a single-player game by a single user 8, a multi-player game by a plurality of users 8, or, among the multi-player games, a competitive game in which a plurality of users 8 compete, and a cooperative play in which a plurality of users 8 cooperate. It may be a game or the like.

In the following description, the distribution terminal, the first distribution terminal, and the second distribution terminal according to the present invention are also described as a character control device, a first character control device, and a second character control device, as an example. Further, the user 5B is applied as an example of the distributor in the present invention, and the first distributor and the second distributor in the present invention are also described as the first user and the second user as an example. The viewing terminal in the present invention is also described as a user terminal as an example. Moreover, the user 8 is applied as an example of the viewer in the present invention. In addition, the avatar object, the first avatar object, and the second avatar object in the present invention are also described as a character, a first character, and a second character, as an example. Further, the first game part in the present invention is also described as a live game progression part, as an example. Further, the second game part in the present invention is also described as a third play part as an example. Further, the third game part in the present invention is also described as a live distribution part as an example. The motion information in the present invention is also described as motion instruction data as an example. In addition, motion instruction data transmitted and received in the live game progression part is applied as an example of the first motion information in the present invention. Further, the operation indicated by the first operation information in the present invention is also described as a first operation or a second operation as an example. In addition, motion instruction data transmitted and received in the live distribution part is applied as an example of the second motion information in the present invention. Further, as an example of the “first condition associated with the current state” in the present invention, the first condition in the first mini game and the second condition in the second mini game are applied. Further, as an example of the “second condition based on the operation by the viewer” in the present invention, the third problem in the third play part is applied. Further, as an example of “a combination of the first UI objects in a state in which the selection can be recognized” is a first condition in the present invention, a condition that a bingo is established in a bingo game as an example of a second mini game. Has been applied. In addition, as an example of “a combination of the specified option object and the first UI object selected by the viewer is a second condition” in the present invention, it is possible to win or lose a rock-paper-scissors game in a rock-paper-scissors game as an example of the first mini game. The relevant conditions are applied. Further, the first object in the present invention is also described as a predetermined object or a progress button object as an example. In addition, the second object in the present invention is also described as “first object according to first state” or “second object according to second state” as an example. Further, the third object in the present invention is also described as a third object or a fourth object as an example.

<Functional configuration of game system 1500>
FIG. 17 is a block diagram showing the functional configurations of the user terminal 800, the server 600, and the character control device 110B included in the game system 1500. Each of the user terminal 800, the server 600, and the character control device 110B has a functional configuration required to function as a general computer (not shown) and a functional configuration required to realize a known function in a game. Can be included.

The user terminal 800 has a function as an input device that receives an input operation of the user 8 and a function as an output device that outputs a game image or sound. The user terminal 800 functions as the control unit 8100 and the storage unit 8200 in cooperation with the processor 810, the memory 820, the storage 830, the communication IF 850, the input/output IF 840, and the like.

The server 600 has a function of communicating with each user terminal 800 to assist the user terminal 800 in proceeding with a game. For example, when the user terminal 800 downloads the application relating to the present game for the first time, the user terminal 800 is provided with data to be stored in the user terminal 800 at the start of the first game. For example, the server 600 transmits motion instruction data for moving the character to the user terminal 800. The motion instruction data may include motion capture data that captures the motion of an actor such as a model in advance, may include voice data in which the voice of an actor such as a voice actor is recorded, and may cause a character to move. May include operation history data indicating a history of input operations, and may include a motion command group in which commands associated with the series of input operations described above are arranged in time series. When the present game is a multi-play game, the server 600 may have a function of communicating with each user terminal 800 participating in the game to mediate exchanges between the user terminals 800 and a synchronization control function. Further, the server 600 has a function of mediating between the user terminal 800 and the character control device 110B. As a result, the character control device 110B can supply the action instruction data to the user terminal 800 or a group of a plurality of user terminals 800 in a timely manner and without mistaking the destination. The server 600 functions as the control unit 6100 and the storage unit 6200 in cooperation with the processor 610, the memory 620, the storage 630, the communication IF 650, the input/output IF 640, and the like.

The character control device 110B has a function of generating motion instruction data for instructing the motion of the character in the user terminal 800 and supplying the motion command data to the user terminal 800. The character control device 110B serves as a control unit 2100 (control module 510, rendering module 520) and a storage unit 2200 (memory module 530) in cooperation with the processor 210, memory 220, storage 230, communication IF 250, input/output IF 240, and the like. Function.

The storage unit 8200 stores a game program 831. The storage unit 6200 stores the game program 631. The storage unit 2200 stores the game program 231. The game program 831 is a game program executed by the user terminal 800. The game program 631 is a game program executed by the server 600. The game program 231 is a game program executed by the character control device 110B. The devices according to the game programs 831, 631, and 231 operate in cooperation with each other to realize the game according to the game system 1500. The game programs 831 and 231 may be stored in the storage unit 6200 and downloaded to the storage units 8200 and 2200, respectively.

The storage units 8200, 6200, and 2200 store game information 132 and user information 133, respectively. The game information 132 is data that the control units 8100, 6100, and 2100 refer to when executing the game programs 831, 631, 231 respectively. The user information 133 is data regarding the account of the user 8. In the storage units 6200 and 2200, the game information 132 and the user information 133 are stored for each user terminal 800.

(Functional configuration of server 600)
The control unit 6100 controls the server 600 by executing the game program 631 stored in the storage unit 6200. For example, the control unit 6100 transmits various data and programs to the user terminal 800. The control unit 6100 receives part or all of the game information or the user information from the user terminal 800. When the game is a multi-play game, the control unit 6100 may receive a request for multi-play synchronization from the user terminal 800 and transmit data for synchronization to the user terminal 800. In addition, the control unit 6100 communicates with the user terminal 800 and the character control device 110B as necessary to send and receive information.

The control unit 6100 functions as the progress support unit 6101 and the sharing support unit 6102 according to the description of the game program 631. The control unit 6100 can also function as other functional blocks (not shown) in order to support the progress of the game on the user terminal 800, depending on the nature of the game to be executed.

The progress support unit 6101 communicates with the user terminal 800 and provides support for the user terminal 800 to progress various parts included in the present game. For example, the progress support unit 6101 provides the user terminal 800 with information necessary for proceeding the game when the user terminal 800 progresses the game.

The sharing support unit 6102 communicates with the plurality of user terminals 800 and the character control device 110B, and supports the progress of the game and sharing of the virtual space 11 between these devices. The sharing support unit 6102 may have a function of matching the online user terminal 800 and the character control device 110B. As a result, information can be transmitted and received smoothly between the user terminal 800 and the character control device 110B.

(Functional configuration of user terminal 800)
The control unit 8100 centrally controls the user terminal 800 by executing the game program 831 stored in the storage unit 8200. For example, the control unit 8100 advances the game according to the operations of the game program 831 and the user 8. In addition, the control unit 8100 communicates with the server 600 and the character control device 110B as necessary to transmit and receive information during the progress of the game.

The control unit 8100, according to the description of the game program 831, the operation receiving unit 8101, the display control unit 8102, the user interface (hereinafter, UI) control unit 113, the animation generation unit 8104, the game progress unit 8105, the virtual space control unit 8106. , And the progress information generation unit 8107. The control unit 8100 can also function as other functional blocks (not shown) in order to progress the game depending on the nature of the game to be executed.

The operation receiving unit 8101 detects and receives the input operation of the user 8 on the input unit 8701. The operation reception unit 8101 determines what input operation has been performed based on the action exerted by the user 8 on the console via the touch screen 870 and other input/output IFs 14, and outputs the result to each element of the control unit 8100. Output.

For example, the operation receiving unit 8101 receives an input operation on the input unit 8701, detects the coordinates of the input position of the input operation, and specifies the type of the input operation. The operation reception unit 8101 identifies, as the type of input operation, for example, a touch operation, a slide operation, a swipe operation, a tap operation, or the like. Further, the operation receiving unit 8101 detects that the contact input is released from the touch screen 870 when the input that is continuously detected is interrupted.

The UI control unit 8103 controls a UI object displayed on the display unit 8702 to construct a UI. The UI object is a tool for the user 8 to make an input necessary for the progress of the game to the user terminal 800, or a tool for obtaining information output during the progress of the game from the user terminal 800. The UI object is, for example, but not limited to, an icon, a button, a list, a menu screen, or the like.

The animation generation unit 8104 generates animations showing the motions of various objects based on the control modes of various objects. For example, the animation generation unit 8104 may generate an animation or the like that represents a character moving as if it were there, moving its mouth, or changing its facial expression.

The display control unit 8102 outputs, to the display unit 8702 of the touch screen 870, the game screen on which the processing result executed by each element described above is reflected. The display control unit 8102 may display a game screen including the animation generated by the animation generating unit 8104 on the display unit 8702. Further, the display control unit 8102 may superimpose and draw the above-mentioned UI object controlled by the UI control unit 8103 on the game screen.

The virtual space control unit 8106 performs various controls for sharing the virtual space 11 with the character control device 110B. FIG. 18 is a diagram showing a detailed configuration of the virtual space control unit 8106. As shown in FIG. 18, the virtual space control unit 8106 includes a virtual object generation module 8121, a virtual camera control module 8122, an avatar object control module 8124, and a virtual object control module 8127.

The virtual object generation module 8121 has the same configuration as the virtual object generation module 1421 described with reference to FIG. Here, the virtual object generated in the user terminal 800 is synchronized with the virtual object generated in the character control device 110B. However, whether or not to synchronize each virtual object may be determined for each virtual object. For example, among the virtual objects generated by the character control device 110B, some may not be generated by the user terminal 800. Further, among the virtual objects generated by the character control device 110B, there may be virtual objects generated in different display forms on the user terminal 800. The display form may be, for example, color, transparency, etc., but is not limited to these.

The virtual camera control module 8122 has the same configuration as the virtual camera control module 1422 described with reference to FIG. Here, the virtual cameras 14A, 14C and 14D arranged in the virtual spaces 11A, 11C and 11D realized in the user terminals 800A, 800C and 800D are virtual arranged in the virtual space 11B realized in the character control device 110B. It is different from the camera 14B. That is, the view image of the virtual space 11 shared between the character control device 110B and the user terminal 800 may be different between the character control device 110B and the user terminal 800.

In the user terminals 800A, 800C, 800D, the positions and directions of the virtual cameras 14A, 14C, 14D may be operated by the users 8A, 8C, 8D, or may not be operated. When the operation is impossible, the positions and directions of the virtual cameras 14A, 14C, 14D may be the same. In this case, the plurality of users 8 will view the common view image through each user terminal 800. That is, in this case, the plurality of users 8 can view the view image (live image) captured by the common virtual camera. In the present embodiment, the positions and directions of the virtual cameras 14A, 14C, and 14D cannot be operated by the users 8A, 8C, and 8D, and a description will be centered on an example in which a plurality of users 8 view a common view image.

Hereinafter, the virtual camera 14 arranged in the first virtual space 11_1 in the character control device 110B will be referred to as a virtual camera 14_B1, and the virtual camera 14 arranged in the second virtual space 11_2 will be referred to as a virtual camera 14_B2. Further, in the user terminals 800A, 800C, 800D, the virtual camera 14 arranged in the first virtual space 11_1 is described as a virtual camera 14_A1, and the virtual camera 14 arranged in the second virtual space 11_2 is described as a virtual camera 14_A2. To do. The visual field images based on the virtual cameras 14_B1, 14_B2, 14_A1, and 14_A2 are referred to as visual field images 1400_B1, 1400_B2, 1400_A1, 1400_A2.

The avatar object control module 8124 operates according to the motion instruction data, instead of operating according to the movement of the HMD 120 or the movement of the controller 300 with respect to the avatar object control module 1424 described with reference to FIG. The configuration is similar except that the points are different. The action instruction data is generated in the character control device 110B in accordance with the movement of the HMD 120 or the movement of the controller 300, and is distributed to the user terminal 800.

The virtual object control module 8127 has the same configuration as the virtual object control module 1427 described with reference to FIG. 14 except that it operates according to the operation instruction data. The action instruction data is generated in the character control device 110B so as to represent the behavior of the virtual object in the character control device 110B, and is distributed to the user terminal 800.

In this way, the virtual space control unit 8106 analyzes (renders) the motion instruction data and operates the avatar object 6B (first character, second character) or the virtual object based on the analysis result. In the present embodiment, the virtual space control unit 8106 uses the action instruction data supplied by the character control device 110B as a trigger to receive the action instruction data via the communication IF 33, and sets the avatar object 6B or the virtual object based on the data. Make it work. As a result, it is possible to show the character and the virtual object moving in real time to the user 8.

In addition, the virtual space control unit 8106, in the first virtual space 11_1 shared between the plurality of user terminals 800 including the user terminal 800 and the character control device 110B, the avatar object 6B, the virtual camera 14_A1, and the plurality of virtual cameras 14_A1. Place virtual objects that represent choices.

In addition, the virtual space control unit 8106 includes the avatar object 6B and the virtual camera in the second virtual space 11_2 that is shared between the user terminal 800 that satisfies a predetermined condition among the plurality of user terminals 800 and the character control device 110B. 14_A2. In the present embodiment, an example in which the same avatar object 6B as the first character arranged in the first space is applied as an example of the second character arranged in the second virtual space 11_2 in the present invention will be described. The user terminal 800 shares the second virtual space 11_2 when the predetermined condition is satisfied in the user terminal 800. Details of the predetermined condition will be described later.

Further, the virtual space control unit 8106 displays the visual field image 1400_A1 (first visual field image) according to the setting of the virtual camera 14_A1 arranged in the first virtual space 11_1 on the display unit 8702.

In addition, the virtual space control unit 8106 displays the visual field image 1400_A2 (second visual field image) according to the setting of the virtual camera 14_A2 arranged in the second virtual space 11_2 on the display unit 8702. The second visual field image is displayed on the user terminal 800 when the predetermined condition is satisfied on the user terminal 800.

That is, the first view image of the first virtual space 11_1 can be viewed by the user 8 of the plurality of user terminals 800, while the second view image of the second virtual space 11_2 satisfies the predetermined condition. It can be viewed only by the user 8 of the user terminal 800.

Hereinafter, viewing the first view image of the first virtual space 11_1 is also referred to as participating in the first virtual space 11_1. Further, viewing the second view image of the second virtual space 11_2 is also referred to as participating in the second virtual space 11_2.

In addition, the virtual space control unit 8106 causes the avatar object 6B to perform an action of selecting any of the plurality of options based on the operation of the user 5B of the character control device 110B.

The game progressing unit 8105 progresses the game in response to either or both of the input operation of the user 8 input via the operation receiving unit 8101 and the input operation of the user 5B of the character control device 110B. The game progressing unit 8105 causes one or more characters to appear during the progress of the game and causes the characters to move. The game progression unit 8105 may move the character according to the game program 831 or may be operated according to the input operation of the user 8, or may be generated according to the operation of the user 5B of the character control device 110B. It may be operated according to the operation instruction data.

In addition, the game progression unit 8105 advances the game in the first state based on the first action performed by the avatar object 6B in the first virtual space 11_1 in response to the operation of the user 5B of the character control device 110B. Details of the first operation and the first state will be described later.

In addition, the game progressing unit 8105 changes the progress of the game from the first state to a first state different from the first state based on a predetermined action performed by the avatar object 6B in the first virtual space 11_1 according to the operation of the user 5B. Transition to 2 states. Details of the second state will be described later.

The predetermined motion may be, for example, a motion targeting a predetermined virtual object. The target motion may be, for example, a push, a holding, a pointing, a turning, or a stepping on a foot, but is not limited to this. The predetermined virtual object may be, for example, a button-type object that is a target of a pushing operation, a dial-type object that is a target of a turning operation, or the like, but is not limited to these.

Further, the game progression unit 8105 advances the game in the second state based on the second action performed by the avatar object 6B in the first virtual space 11_1 in response to the operation of the user 5B. Details of the second operation will be described later.

Here, the first state and the second state may be, for example, states according to the type of game in progress. For example, the first state and the second state may include the state of the virtual space 11, the state of the display unit 8702, the application state of the game process to be executed, and the like. Specifically, the virtual space 11 has a state in which virtual objects according to the type of game are arranged. The display unit 8702 has a state in which UI objects corresponding to the type of game are displayed. In addition, there is a state in which game processing according to the type of game is applied according to the operation of the avatar object 6B or the operation on the UI object. In addition, the first state and the second state may include other states related to the progress of the game.

Hereinafter, the game progressing in the first state is also referred to as a first mini game, and the game progressing in the second state is also referred to as a second mini game. Note that the types of states in which this game can progress are not limited to two, and may be three or more. In other words, this game may include a mini game in addition to the first mini game and the second mini game. The second mini game corresponds to an example of the first play part in the present invention. The first mini game corresponds to an example of the second play part in the present invention.

The first condition that can be satisfied by the play of the user 8 is set in the first mini game. The first condition corresponds to an example of the second problem in the present invention. In the second mini game, a second condition that can be satisfied by the play of the user 8 is set. The second condition corresponds to an example of the first problem in the present invention. Each mini game proceeds according to the operation of the avatar object 6B and the operation of the user 8.

Further, the first motion and the second motion may be, for example, the motion of the avatar object 6B according to the type of game that is in progress. For example, if the first mini game is a rock-paper-scissors game, the first action may be an action in which the avatar object 6B selects a virtual object corresponding to goo choki per. If the second mini game is a bingo game, the second action may be an action in which the avatar object 6B selects a virtual object corresponding to the bingo ball.

Further, in the first state, there may be a plurality of types of the first action according to the type of game. For example, in the first state, the action of the avatar object 6B that selects the virtual object and the action of the avatar object 6B that pushes the virtual button for determining the winning or losing may be regarded as the first action. Further, also in the second state, there may be a plurality of types of second motions depending on the type of game. Further, any one of the first operation and any of the second operations may be the same operation or may be different operations. For example, in the first state, there may be a first action of pressing a virtual button for determining the outcome of a game, and in the second state, there may be a second action of pushing a virtual button for determining the outcome of a game. ..

In addition, the game progressing unit 8105 executes a game process according to the selection result of the selected action in response to the action of the avatar object 6B selecting one of the plurality of options. Here, virtual objects representing a plurality of options are arranged in the virtual space 11.

In addition, the game progression unit 8105, when the above-described predetermined condition is not satisfied, information input by the user 8 of the user terminal 800 and the user 8 of another user terminal 800 that does not meet the predetermined condition. The information input by is displayed on the display unit 8702 together with the first visual field image. Hereinafter, the part in which the information input by each user 8 of the user terminal 800 that does not satisfy the predetermined condition is displayed is also referred to as a chat part.

The progress information generation unit 8017 generates progress information indicating the progress of the game executed by the game progress unit 8105, and transmits the progress information to the server 600 or the character control device 110B in a timely manner. The progress information may include, for example, information designating the currently displayed game screen, or may include a progress log showing the progress of the game in time series by characters and symbols. In the game system 1500, in an embodiment in which the server 600 and the character control device 110B do not need progress information, the progress information generation unit 8017 may be omitted.

(Functional configuration of character control device 110B)
The control unit 2100 executes the game program 231 stored in the storage unit 2200 to integrally control the character control device 110B. For example, the control unit 2100 generates operation instruction data according to the operations of the game program 231 and the user 5B, and supplies the operation instruction data to the user terminal 800. The control unit 2100 may further execute the game program 231 if necessary. Further, the control unit 2100 communicates with the server 600 and the user terminal 800 that is executing the present game to send and receive information.

The control unit 2100, according to the description of the game program 231, the operation reception unit 2101, the display control unit 2102, the UI control unit 2103, the animation generation unit 2104, the game progress unit 2105, the virtual space control unit 2106, and the reaction processing unit 2107. Function as. The control unit 2100 can also function as other functional blocks (not shown) in order to control the characters that appear in the game according to the nature of the game executed in the game system 1500.

The operation receiving unit 2101 detects and receives the input operation of the user 5B. The input operation is, for example, an input unit integrally formed with the display unit when the HMD 120, the motion sensor 420, and the display 430 are touch screens, a mouse (not shown), a keyboard (not shown), and the like. It is input from the device through the input/output IF 34. The operation receiving unit 2101 discriminates what input operation is performed from the action exerted by the user 5B, and outputs the result to each element of the control unit 2100. Details of the function of the operation receiving unit 2101 are almost the same as those of the operation receiving unit 8101 in the user terminal 800.

The UI control unit 2103 controls the UI object displayed on the HMD 120 or the display 430.

The animation generation unit 2104 generates animations showing the motions of various objects based on the control modes of various objects. For example, the animation generation unit 2104 may generate an animation or the like that reproduces the game screen actually displayed on the user terminal 800 that is the communication partner.

The display control unit 2102 outputs, to the HMD 120 or the display 430, an image in which the processing result executed by each of the above-described elements is reflected.

The game progressing unit 2105 progresses the game based on the avatar object 6B in response to the input operation by the user 5B and the operation of the user 8 of the user terminal 800. For example, the game progression unit 2105 performs a predetermined game process when the avatar object 6B performs a predetermined action. Further, for example, the game progression unit 2105 receives information indicating the operation of the user 8 on the user terminal 800, and performs the game process based on the operation of the user 8.

In addition, the game progress unit 2105 grasps the progress of the game on the user terminal 800 based on the progress information indicating the progress of the game received from the user terminal 800. Then, the game progression unit 2105 presents the progress of the user terminal 800 to the user 5B by imitating the behavior of the user terminal 800 in the character control device 110B.

For example, the game progression unit 2105 may display a reproduction of the game screen displayed on the user terminal 800 on the display unit 352 of the own device. Further, the game progression unit 2105 may display the progress of the game on the display unit 352 as the above-mentioned progress log in the user terminal 800.

Further, a part of the function of the game progression unit 2105 may be realized by the control unit 2100 executing the game program 231. For example, the game progress unit 2105 first grasps the progress of the game of the user terminal 800 based on the progress information. Then, the game progression unit 2105 may reproduce the game screen currently displayed on the user terminal 800 based on the game program 831 on the display unit 352 of its own device completely or in a simplified form. Alternatively, the game progress unit 2105 may grasp the progress of the game at the present time, predict the game progress after the present time based on the game program 231, and output the prediction result to the display unit 352.

The virtual space control unit 2106 includes the modules 1421 to 1427 of the control module 510 described with reference to FIG. These details are as already described.

The virtual space control unit 2106 performs various controls for sharing the virtual space 11 with the user terminal 800. FIG. 19 is a diagram showing a detailed configuration of the virtual space control unit 2106. As shown in FIG. 19, the virtual space control unit 2106 includes the modules 1421 to 1427 described with reference to FIG. The virtual space control unit 2106 controls the behavior of the virtual object by operating the avatar object 6B in the character control device 110B by the operations of these modules 1421 to 1427. The virtual space control unit 2106 also generates operation instruction data for operating the avatar object 6B in the user terminal 800 and operation instruction data for controlling the behavior of each virtual object based on the operations of the modules 1421 to 1427. To do. These operation instruction data are supplied to the user terminal 800 via the server 600. For example, the virtual space control unit 2106 generates action instruction data for instructing the character to be controlled to speak based on the voice data input by the user 5B (voice actor, etc.) via the microphone 170. The operation instruction data thus generated includes at least the above-mentioned voice data. In addition, for example, the user 5B (model or the like) generates motion instruction data instructing the character to be controlled to perform a motion based on the motion capture data input via the motion sensor 420. The motion instruction data thus generated includes at least the above-mentioned motion capture data. In addition, for example, the user 5B operates the character to be controlled based on the history of input operations input by the user 5B via an input mechanism such as the controller 300 or an operation unit such as a touch screen, a mouse, and a keyboard, that is, operation history data. The motion instruction data for instructing the action is generated. The motion instruction data thus generated includes at least the above-mentioned operation history data. The operation history data is, for example, information in which an operation log indicating which button of the controller 300 is pressed by the user 5B when the screen is displayed on the HMD 120 is arranged in chronological order. .. Alternatively, the virtual space control unit 2106 identifies a command instructing the action of the character, which is associated with the input operation input by the user 5B via the input mechanism or the operation unit described above. Then, the virtual space control unit 2106 arranges the commands in the input order to generate a motion command group indicating a series of motions of the character, and generates motion instruction data instructing to move the character according to the motion command group. You may. The motion command data thus generated includes at least the above-mentioned motion command group.

The reaction processing unit 2107 receives feedback about the reaction of the user 8 from the user terminal 800 and outputs it to the user 5B of the character control device 110B. In the present embodiment, for example, the user terminal 800 allows the user 8 to create a comment for the character while the character is moving in accordance with the above-described motion instruction data. The reaction processing unit 2107 receives the comment data of the comment and outputs it. The reaction processing unit 2107 may display the text data corresponding to the comment of the user 8 on the display unit 352, or may output the voice data corresponding to the comment of the user 8 from a speaker (not shown).

The functions of the user terminal 800, the server 600, and the character control device 110B shown in FIG. 17 are merely examples. Each of the user terminal 800, the server 600, and the character control device 110B may have at least a part of the functions of another device. Furthermore, another device other than the user terminal 800, the server 600, and the character control device 110B may be a constituent element of the game system 1500, and the other device may execute a part of the processing in the game system 1500. That is, the computer that executes the game program in this embodiment may be any of the user terminal 800, the server 600, the character control device 110B, and another device other than the user terminal 800, and is realized by a combination of these devices. May be done.

Note that the functions of the server 600 and the user terminal 800 shown in FIG. 17 are merely examples. The server 600 may include at least a part of the functions included in the user terminal 800. In addition, the user terminal 800 may include at least a part of the functions included in the server 600. Further, a device other than the user terminal 800 and the server 600 may be a constituent element of the game system 1500, and the other device may be caused to execute a part of the processing in the game system 1500. That is, the computer that executes the game program in the present embodiment may be any of the user terminal 800, the server 600, and other devices, or may be realized by a combination of a plurality of these devices.

<Game structure>
FIG. 20 is a flowchart showing an example of basic game progress of this game. The game includes three parts. As an example, the first is a live game progression part that can be participated by a plurality of user terminals 800. The second is a live distribution part that is distributed to the user terminals 800 satisfying a predetermined condition among the user terminals 800 participating in the live game progression part. The third is a chat part provided to a user terminal 800 that does not satisfy a predetermined condition among the user terminals 800 that participated in the live game progression part. The live game progression part includes the above-described first mini game (game that progresses in the first state) and second mini game (game that progresses in the second state).

In step S1, the game system 1500 advances the live game progression part. The live game progression part proceeds in the first virtual space 11_1.

In step S2, the game system 1500 progresses the first mini game in the live game progress part. The first mini game proceeds based on the operation of the avatar object 6B in the first virtual space 11_1 and the operation of the user 8 of the user terminal 800. In the first mini game, based on the first action in which the avatar object 6B selects one of the plurality of options represented by the virtual object, game processing is executed according to the result of the selection in the first action.

In step S3, when the avatar object 6B performs a predetermined action in the first virtual space 11_1, the game system 1500 ends the first mini game and starts the second mini game. That is, the progress of the game transits from the first state to the second state.

In step S4, the game system 1500 progresses the second mini game in the live game progress part. The second mini game proceeds based on the operation of the avatar object 6B in the first virtual space 11_1 and the operation of the user 8 of the user terminal 800. In the second mini game, based on the second action in which the avatar object 6B selects one of the plurality of options represented by the virtual object, game processing is executed according to the result of the selection by the second action.

In step S5, the game system 1500 determines whether or not a predetermined condition is satisfied in the live game progression part. Specifically, the game progression unit 8105 may determine whether or not the second condition is satisfied in the second mini game.

If Yes in step S5, in step S6, the game system 1500 advances the live distribution part in the second virtual space 11_2. An avatar object 6B (second character) is placed in the second virtual space 11_2. In other words, the avatar object 6B that operates according to the operation of the user 5B moves from the first virtual space 11_1 to the second virtual space 11_2 and operates.

In the case of No in step S5, in step S7, the game system 1500 advances the chat part in the user terminal 800. The chat part is a part that proceeds without being operated by the user 5B of the character control device 110B. In other words, the chat part proceeds regardless of the action of the avatar object 6B. The chat part may proceed with the first view image of the first virtual space 11_1 displayed on the user terminal 800.

<Outline of processing>
In the present embodiment, the user terminal 800 is configured to execute the following steps based on the game program 831. Specifically, the first distributor (user 5B) shares the information between the first distribution terminal (character control device 110B) for real-time distribution and a plurality of viewing terminals including the viewing terminal (user terminal 800). Arranging a first avatar object corresponding to a first distributor and a first virtual camera in a first virtual space (first virtual space 11_1), and a first virtual camera while real-time distribution is performed. Displaying a first visual field image showing the visual field area on the display unit, and for each state of the first game part (live game progression part) included in the game based on the game program and progressing while transitioning between a plurality of states. A step of performing a game process of advancing the game in one state (first mini game, second mini game) based on the movement of the body of the first distributor and the operation of the viewer. In the game processing in, the step of receiving the first motion information indicating the motion of the first avatar object, which is generated by the first distribution terminal by detecting the movement of the body of the first distributor, and the first motion information is received. In response to receiving the first motion information, the step of updating the first field-of-view image to include the first avatar object performing the motion indicated by the first motion information, and in response to receiving the first motion information. , A step of determining whether or not the motion indicated by the first motion information is a motion for transitioning the state, and if the motion indicated by the first motion information is a motion for transition, Transitioning the progress from the current state to another state.

As a result, in the first game part in which each state progresses based on the movement of the body of the first distributor and the operation of the viewer, the viewer performs an action for transition of the first avatar object that advances the game, It can be recognized that the state of the first game part is switched. Therefore, the sense of presence is increased, and the sense of immersion in the game is enhanced.

Further, the user terminal 800 is shared between a distribution terminal (character control device 110B) for a distributor (user 5B) to perform real-time distribution, and a plurality of viewing terminals including a viewing terminal (user terminal 800). A step of arranging an avatar object (avatar object 6B) corresponding to a distributor, a virtual camera, and a plurality of option objects in a virtual space, and a view image showing a view area of the virtual camera during real-time distribution ( A step of displaying the field-of-view image 1400_A1) on the display unit, a step of displaying a user interface (UI) object that accepts an input operation of the viewer on the display unit while the real-time distribution is performed, and the distribution terminal operates the distributor. Generated by detecting, selecting a part from the plurality of choice objects arranged in the virtual space, receiving motion information indicating the motion of the avatar object, and responding to the motion information being received, Updating the view image to include the action of the avatar object selecting a portion from the multiple choice objects located in the virtual space, and selected by the avatar object in response to the action information being received. The step of specifying the choice object and the step of changing the state of the UI object according to the specified choice object are executed.

As a result, the state of the UI object for the viewer changes based on the options selected by the avatar object, so that the interaction between the avatar object and the viewer can give a sense of the progress of the game, giving a realistic feeling. Increase. Therefore, the sense of presence is increased, and the sense of immersion in the game is enhanced.

The user terminal 800 may execute the step of changing the state of the UI object displayed on the display unit of the viewing terminal according to the action of the distributor. Changing the state of the UI object may be, for example, changing the UI object between the operable state and the inoperable state, or deleting the displayed UI object from the display unit. Alternatively, the UI object that has not been displayed may be newly displayed on the display unit.

<Processing flow of live game distribution part>
FIG. 21 is a flowchart showing the flow of processing in the live game progression part, which is executed by each device constituting the game system 1500. In the flowcharts shown in the drawings below, the left diagram represents the operation of the character control device 110B, the right diagram represents the operation of the user terminal 800A, and the broken arrow connecting the left and right represents the data flow. The data transmitted and received between the character control device 110B and the user terminal 800A is assumed to be transmitted via the server 600. Hereinafter, the fact that the character control device 110B transmits information to the user terminal 800A via the server 600 will be simply referred to as the character control device 110B transmitting information to the user terminal 800A. Further, the fact that the user terminal 800A transmits information to the character control device 110B via the server 600 is simply described as the user terminal 800A transmits information to the character control device 110B. Further, in the following description, the description related to the user terminals 800C and 800D will be similarly described by replacing A at the end of each reference symbol with C and D in the description related to the user terminal 800A.

In step S101, the virtual space control unit 2106 of the character control device 110B defines the first virtual space 11_1B based on the first virtual space data representing the first virtual space 11_1. The first virtual space 11_1B is a virtual space realized by the character control device 110B. The first virtual space data is supplied from the server 600, for example. Step S101 may be executed after a predetermined distribution start time in response to an operation instructing the distribution start by the user 5B.

In step S201, the virtual space control unit 8106 of the user terminal 800A defines the first virtual space 11_1A based on the first virtual space data representing the first virtual space 11_1. The first virtual space 11_1A is a virtual space realized in the user terminal 800A. The first virtual space data is supplied from the server 600, for example. Step S201 may be executed at a predetermined distribution start time based on the operation of instructing the user 8A to start viewing.

The predetermined distribution start time may be notified to the character control device 110B and the user terminal 800A from the server 600 in advance. Further, the predetermined distribution start time may be notified from the character control device 110B to the user terminal 800A.

In step S102, the virtual space control unit 2106 arranges the virtual camera 14_B1 in the first virtual space 11_1B. A visual field image 1400_B1 by the virtual camera 14_B1 is a visual field image of the first virtual space 11_1B viewed from the avatar object 6B.

In step S202, the virtual space control unit 8106 arranges the virtual camera 14_A1 in the first virtual space 11_1A. The virtual camera 14</b>_A<b>1 is arranged in a position and a direction in which a region where the avatar object 6</b>B can exist in the first virtual space 11_1</b>A can be visually recognized. For example, assuming that the avatar object 6B may exist in the area defined as the front in the first virtual space 11_1, the virtual camera 14_A1 may be arranged behind the first virtual space 11_1 in a direction toward the front. The visual field image 1400_A1 by the virtual camera 14_A1 is a visual field image suitable for viewing the motion of the avatar object 6B in the first virtual space 11_1A, and corresponds to an example of the first visual field image in the present invention.

In step S103, the virtual space control unit 2106 arranges the avatar object 6B and a predetermined object (progress button object) in the first virtual space 11_1B. The progress button object is an example of the predetermined virtual object described above.

In step S104, the virtual space control unit 2106 transmits information indicating the avatar object 6B and the progress button object to the user terminal 800A.

In step S203, the virtual space control unit 8106 arranges the avatar object 6B and a predetermined virtual object in the first virtual space 11_1A based on the received information. Here, it is assumed that a progress button object that is the target of the pressing operation is arranged as the predetermined virtual object. As a result, the avatar object 6B and the progress button object are synchronously arranged between the user terminal 800A and the character control device 110B.

In step S105, the virtual space control unit 2106 displays and updates the visual field image 1400_B1 based on the position and direction of the virtual camera 14_B1 that changes according to the movement of the HMD 120. In the subsequent steps, it is assumed that the view image 1400_B1 is continuously updated according to the movement of the HMD 120.

In step S204, the virtual space control unit 8106 displays and updates the visual field image 1400_A1 based on the position and direction of the virtual camera 14_A1. Also in the subsequent steps, it is assumed that the view image 1400_A1 is continuously updated according to the motion of the avatar object 6B and the behavior of each virtual object.

In step S106, the virtual space control unit 2106 controls the motion of the avatar object 6B according to the motion of the user 5B.

In step S107, the virtual space control unit 2106 transmits the motion instruction data representing the motion of the avatar object 6B to the user terminal 800A.

In step S205, the virtual space control unit 8106 controls the motion of the avatar object 6B based on the received motion instruction data.

Note that the series of processes of steps S106, S107, and S205 are continuously executed in the subsequent steps.

In step S108 and step S206, the game progressing unit 2105 and the game progressing unit 8105 progress the first mini game while transmitting and receiving necessary information to and from each other. Details of the flow of processing for advancing the first mini game will be described later.

In step S109, the game progression unit 2105 determines whether or not the avatar object 6B has performed a predetermined action according to the movement of the user 5B. As described above, the predetermined motion is a motion targeting the virtual object, and here is a motion of bringing the operation object into contact with the progress button object. For example, it may be an operation of pressing the progress button object. The step is repeatedly executed until it is determined Yes. If Yes in step S109, the process of the next step S110 is executed.

In step S110, the virtual space control unit 2106 transmits motion instruction data representing a predetermined motion to the user terminal 800A.

In step S207, the virtual space control unit 8106 causes the avatar object 6B to perform a predetermined motion based on the received motion instruction data. Here, the avatar object 6B performs an operation of pushing the progress button object.

As a result, the progress of the game transitions from the first mini-game to the second mini-game according to a predetermined action of the avatar object 6B.

In steps S112 and S208, the game progressing unit 2105 and the game progressing unit 8105 progress the second mini game while transmitting and receiving necessary information to and from each other. Details of the flow of processing for advancing the second mini game will be described later.

<First virtual space 11_1B in character control device 110B>
FIG. 22 is a diagram schematically showing an example of the first virtual space 11_1B and the visual field image 1400_B1 realized by executing steps S101 to S107 in the character control device 110B. In FIG. 22A, at least an avatar object 6B, a virtual camera 14_B1, and a progress button object 2241 are arranged in the first virtual space 11_1. User 5B (first user) wears HMD 120 on the head. The user 5B holds the right controller 300R with the right hand (first part) forming a part of the right side of the user 5B's body, and left with the left hand (second part) forming a part of the left side of the user 5B's body. Holds the controller 300L.

The HMD 120 includes a sensor 190 that functions as a motion sensor. The right controller 300RB and the left controller 300LB include a motion sensor 420. User 5B further wears motion sensors 1841 to 1843. The motion sensor 1841 is attached to the waist of the user 5B by the belt 1844. The motion sensor 1842 is attached to the instep of the right foot of the user 5B. The motion sensor 1843 is attached to the instep of the left foot of the user 5B of the user 5B. The motion sensors 1841 to 1843 are connected to the computer 200 by wire or wirelessly.

In one aspect, the motion sensor attached to the user 5B detects the arrival time and the angle of the signal (for example, infrared laser) emitted from the base station (not shown). The processor 210 of the computer 200 (hereinafter simply referred to as the processor 210) detects the position of the motion sensor with respect to the base station based on the detection result of the motion sensor. The processor 210 may further standardize the position of the motion sensor with respect to the base station with reference to a predetermined point (for example, the position of the sensor 190 mounted on the head).

The avatar object 6B includes a virtual right hand 1831RB and a virtual left hand 1831LB. The virtual right hand 1831RB is a kind of operation object, and can move in the first virtual space 11_1B according to the movement of the right hand of the user 5B. The virtual left hand 1831LB is a kind of operation object, and can move in the first virtual space 11_1B according to the movement of the left hand of the user 5B.

The first virtual space 11_1 shown in FIG. 22(A) is constructed by playing back program content on the computer 200. The user 5B moves his/her body in order to cause the avatar object 6B to perform a performance. The computer 200 detects the movement of the user 5B based on the outputs of various motion sensors attached to the user 5B. In the first virtual space 11_1, the avatar object 6B executes a performance in which the movement of the user 5B in the real space is reflected according to the specified movement of the user 5B. When the avatar object 6B executes a performance corresponding to the movement of the user 5B in the first virtual space 11_1, the avatar object 6B synchronously executes the same performance in the first virtual space 11_1A realized in the user terminal 800A. In this way, the user 5B has a role as a distributor who distributes the live game progress part to the users 8A of the plurality of user terminals 800A as a program by the avatar object 6B.

In FIG. 22A, the virtual camera 14_B1 is arranged on the head of the avatar object 6B. The virtual camera 14_B1 defines the view field 15_11 according to the position and orientation of the virtual camera 14_B1. The virtual camera 14_B1 generates a visual field image 1400_B1 corresponding to the visual field area 15B and displays it on the HMD 120 as shown in FIG. The user 5B visually recognizes a part of the first virtual space 11_1B from the viewpoint of the avatar object 6B by visually recognizing the visual field image 1400_B1. Thereby, the user 5B can obtain a virtual experience as if the user 5B were the avatar object 6B. The view image 1400_B1 includes a progress button object 2241. Therefore, the user 5B can visually recognize the progress button object required to advance the progress in the live game progress part.

<First virtual space 11_1A in user terminal 800>
FIG. 23 is a diagram schematically showing an example of the first virtual space 11_1A and the visual field image 1400_A1 realized by executing steps S201 to S205 in the user terminal 800. In FIG. 23A, at least an avatar object 6B, a virtual camera 14_A1, and a progress button object 2241 are arranged in the first virtual space 11_1A. The first virtual space 11_1A is synchronized with the first virtual space 11_1B shown in FIG. The user 8A holds the user terminal 800A in the left hand. In the example of FIG. 23, the user 8 views the live game progress part, which is a program demonstrated by the avatar object 6B, while visually checking the screen of the user terminal 800A.

The first virtual space 11_1A shown in FIG. 23A is constructed by reproducing the program content on the user terminal 800A. In the first virtual space 11_1A, the avatar object 6B executes a performance as a distributor of the live game progress part based on the movement of the user 5B. The user 8A views the performance of the avatar object 6B as a participant of the live game progression part through the screen of the user terminal 800A. In addition, the users 8C and 8D view the performance of the avatar object 6B as participants of the live game progression part through the screens of the user terminals 800C and 800D. In this way, the live game progression part progressed by the avatar object 6B is simultaneously streamed to a plurality of users 8.

In FIG. 23A, a virtual camera 14_A1 is arranged behind the first virtual space 11_1A. The virtual camera 14_A1 defines a view area 15A according to the position and orientation of the virtual camera 14_A1. The processor 810 generates a visual field image 1400_A1 corresponding to the visual field region 15A and displays it on the display unit 8702 of the touch screen 870, as illustrated in FIG. The view image 1400_A1 includes at least the avatar object 6B that executes the performance. The user 8A visually recognizes the view image 1400_A1 to visually recognize the avatar object 6B and a part of the first virtual space 11_1A in which the avatar object 6B appears. As a result, the user 8A can obtain a virtual experience as if the avatar object 6B is an actual distributor.

<Processing flow of the first mini game>
FIG. 24 is a flowchart showing the flow of processing in the first mini game, which is executed by each device constituting the game system 1500. The processing flow shown in FIG. 24 corresponds to details of the processing in steps S108 and S206 shown in FIG.

Here, it is assumed that the rock-paper-scissors game is applied as the first mini game. Further, in the first mini game, the first condition is set. The first condition is that the user 8A wins the avatar object 6B with a rock-paper-scissors game.

As described below, the first mini game (second play part) in which the first condition (second task) is set is before the start of the second mini game (first play part) described later. , A character avatar object 6B moves according to the operation of the user 5B of the character control device 110B.

In step S301, the virtual space control unit 2106 of the character control device 110B arranges the third object of the present invention and virtual objects representing a plurality of options in the first virtual space 11_1B.

Here, the third object is a virtual object that represents information about the first state. For example, the third object may be a virtual panel capable of displaying the information, a virtual display, a virtual blackboard, a virtual electronic bulletin board, or the like, but is not limited to these. In the present embodiment, a panel object representing a virtual panel is arranged as the third object.

On the panel object, information about the first mini game is displayed as information about the first state. The information on the first mini game may include, for example, information on the rule of the first mini game, the first condition set in the first mini game, information about a reward obtained by satisfying the first condition, and the like. However, it is not limited to these.

Also, here, virtual objects representing each of the options are arranged as virtual objects representing the plurality of options. Specifically, the choice objects corresponding to each of the rock-paper-scissors goo, choki, and par are arranged. In this example, one option object corresponds to one option, and as many option objects as the number of options (here, three) are arranged.

In step S302, the virtual space control unit 2106 transmits the information of the panel object and the three option objects to the user terminal 800A.

In step S401, the virtual space control unit 8106 of the user terminal 800A sets the first object (here, the panel object and the three selection objects) corresponding to the first state to the first virtual space based on the received information. It is placed at 11_1A. In addition, the virtual space control unit 8106 applies the action of the avatar object 6B that targets the first object as the first action.

In step S402, the game progressing unit 8105 displays the first UI object on the display unit 8702.

Here, the first UI object is a UI object corresponding to the first state. Specifically, the first UI object is a user interface object that receives an operation from the user 8A in the first mini game. Here, three UI buttons corresponding to each of the choices of rock-paper-scissors goo choki par are displayed. The three UI buttons have an operable state and an inoperable state, and may be inoperable in the initial state. However, the first UI object is not limited to the UI button and may be another UI object.

In step S303, the virtual space control unit 2106 causes the avatar object 6B to perform an operation of prompting the user 8A to operate the first UI object according to the movement of the user 5B.

In step S304, the virtual space control unit 2106 transmits the motion instruction data representing the motion of the avatar object 6B to the user terminal 800A.

In step S403, the virtual space control unit 8106 causes the avatar object 6B to perform the relevant motion based on the received motion instruction data.

In step S404, the game progressing unit 8105 executes a first process for satisfying at least a part of the first condition based on the first action of the avatar object 6B.

Here, the game progressing unit 8105 executes a first process of bringing the first UI object into an operable state according to the action urged by the avatar object 6B. In order to satisfy the first condition of winning the rock-paper-scissors, at least three UI buttons, which are the first UI objects, need to be operable, so the first processing is executed. Then, the game progressing unit 8105 receives a user operation on the first UI object. Here, an operation on any of the three UI buttons is accepted.

In step S305, the virtual space control unit 2106 causes the avatar object 6B to perform an operation of selecting any of the three option objects according to the movement of the user 5B. The action to be selected may be, for example, an action having a choice object, pointing, rotating, or a combination of various actions including these actions.

In step S306, the virtual space control unit 2106 transmits the motion instruction data representing the motion of the avatar object 6B to the user terminal 800A.

In step S405, the virtual space control unit 8106 causes the avatar object 6B to perform the selecting operation based on the received operation instruction data.

In step S406, the game progressing unit 8105 executes a process of making the first UI object inoperable according to the action selected by the avatar object 6B. This step is a process for not accepting an operation that is "later".

In step S407, the game progression unit 8105 determines the result of the first mini game. In the present embodiment, the game progression unit 8105 determines whether the rock-paper-scissors wins or loses between the user 5B and the user 8. Here, the option indicated by the UI button operated by the user 8 and the option indicated by the option object selected by the avatar object 6B are compared to determine the win or loss.

When it is determined in step S407 that the player has lost, the process of the next step S408 is omitted and the process of step S409 described below is executed. If it is determined in step S407 that the player has won, the process of step S408 is performed.

In step S408, the game progressing unit 8105 determines that the higher the achievement level of the first condition (second task) in the first mini game (second play part) is, the second mini game (first play part) is executed as the game process. A more advantageous process is executed to satisfy the second condition (first problem) in.

Specifically, the game progression unit 8105 causes the memory 820 to store information indicating the right to advance the second mini game in an advantageous manner in association with the user.

In step S409, the game progression unit 8105 transmits the result of the first mini game to the character control device 110B. Here, the rock-paper-scissors result is transmitted.

In step S307, the game progression unit 2105 of the character control device 110B displays information indicating the result of the first mini game of each user 8A on the HMD 120. The information indicating the result may be displayed by being superimposed on the visual field image 1400_B1, for example. Here, the information showing the result of the rock-paper-scissors of each user is displayed. The information indicating the rock-paper-scissors result of each user may be aggregate information indicating the number of people who have won, the number of people who have lost, the number of icons, or the like, or may be list information of the results of each user. Moreover, the information indicating the result of the rock-paper-scissors may include the user name.

<First virtual spaces 11_1A and 11B while the first mini game is in progress>
FIG. 25 is a diagram schematically showing the first virtual space 11_1 during the progress of the first mini game. FIG. 25A is a perspective view of the first virtual space 11_1B realized in the character control device 110B. FIG. 25B is a top view of the first virtual space 11_1A realized in the user terminal 800A. These first virtual spaces 11_1A and 11_B are synchronized.

In FIGS. 25A and 25B, an avatar object 6B, a progress button object 2241, a panel object 2551, and choice objects 2552_1 to 2552_3 are arranged in the first virtual spaces 11_1A and 11_B. A virtual camera 14_B1 is arranged in the first virtual space 11_B. A virtual camera 14_A1 is arranged in the first virtual space 11_A.

The virtual camera 14_B1 is placed on the head of the avatar object 6B in the direction in which the avatar object 6B is facing. Since the virtual camera 14_B1 is not arranged in the first virtual space 11_1A, it is not included in the visual field image 1400_A1 displayed on the user terminal 800A.

The virtual camera 14_A1 is arranged behind the first virtual space 11_1 toward the front area. Here, the front side means the left direction on the paper surface. Since the virtual camera 14_A1 is not arranged in the first virtual space 11_1B, it is not included in the visual field image 1400_B1 displayed on the character control device 110B.

The avatar object 6B is arranged facing the same direction as the virtual camera 14_B1. The direction of the virtual camera 14_B1 is the direction in which the user 8A virtually exists. Hereinafter, the direction of the virtual camera 14_B1 will be simply referred to as the direction of the user 8A.

The three option objects 2552_1 to 2552_3 have an illustration showing goo choki per on one surface of the disk portion at the tip. No illustration is drawn on the other side of these discs. The three option objects 2552_1 to 2552_3 are arranged so that the surface on which the illustration is not drawn faces the user 8A.

The panel object 2551 is arranged in a region in front of the avatar object 6B, in other words, behind the avatar object 6B. Here, the panel object 2551 is plate-shaped, and information regarding the first mini game is displayed on the surface on the direction side of the user 8A.

It is assumed that the advance button object 2241 is arranged in the area on the left side with respect to the direction in which the avatar object 6B is directed (the area on the paper surface in the first virtual space 11_1A shown in FIG. 25B).

<Field-of-view image and game screen during the first mini game>
26 to 29 are diagrams showing transition examples of the visual field image and the game screen during the progress of the first mini game. 26 to 29, (A) represents a visual field image 1400_B1 displayed on the HMD 120 of the character control device 110B by executing steps S301 to S307. (B) represents a game screen displayed on the display unit 8702 of the user terminal 800A by executing S401 to 410.

FIG. 26 is a diagram showing a view image and a game screen at the start of the rock-paper-scissors game.

In FIG. 26A, the visual field image 1400_B1 in the character control device 110B is a progress button object 2241, option objects 2552_1 to 2552_3, a virtual right hand 1831RB and a virtual left hand 1831LB that are the hand parts of the avatar object 6B, and information 2651. Including and Since the panel object 2551 is behind the avatar object 6B, it is not included in the view image 1400_B1.

The progress button object 2241 is arranged on the left side when viewed from the avatar object 6B.

The option objects 2552_1 to 2552_3 are arranged so that the surface on which the illustration of Goo Choki Par is drawn can be seen when viewed from the avatar object 6B.

Further, the virtual right hand 1831RB and the virtual left hand 1831LB of the avatar object 6B itself are included in the visual field image 1400_B1. Accordingly, the user 5B can easily move so that the virtual right hand 1831RB and the virtual left hand 1831LB select the choice objects 2552_1 to 2552_3 or the progress button object 2241.

The information 2651 is information displayed by being superimposed on the visual field image 1400_B1. The information 2651 represents the number of users 8 who are currently viewing the live game progress part including the first mini game. Here, the virtual space 11_1 is shared by 132 user terminals 800 including the user terminals 800A, 800C, and 800D, and it is displayed that the number of users 8 who are watching (the number of viewers) is 132. ing. The information 2651 may be displayed in a superimposed manner by being transmitted from the server 600, for example.

Here, when the user 5B performs an operation of uttering a voice 2671 "I will start a rock-paper-scissors game" as an operation of notifying the user 8A that the first mini game will be started, the avatar object 6B outputs the voice. 2671 is controlled. The operation of notifying that the first mini game is started is not limited to utterance, and may be a hand gesture or a combination of various operations.

Further, when the user 5B performs an operation of uttering a voice 2672 such as "Press any of the goo choki per button" as an operation for prompting the user 8A to operate the first UI object, the avatar object 6B Are controlled to produce voice 2672. The action for prompting the operation of the first UI object is not limited to vocalization, and may be a hand gesture or a combination of various actions.

In FIG. 26B, the game screen displayed on the display portion 8702 of the user terminal 800A includes a visual field image 1400_A1 and a UI area 2661. The view image 1400_A1 includes an avatar object 6B, a progress button object 2241, option objects 2552_1 to 2552_3, and a panel object 2551.

Since the avatar object 6B faces the user 8A, it faces the front. Further, the virtual right hand 1831RB and the virtual left hand 1831LB of the avatar object 6B are included in the visual field image 1400_A1. This allows the user 8A to pay attention to which option object the virtual right hand 1831RB and the virtual left hand 1831LB perform.

The progress button object 2241 is arranged on the left side of the avatar object 6B, that is, on the right side of the visual field image 1400_A1.

The option objects 2552_1 to 2552_3 are arranged so that the surface on which the illustration of Goo Choki Par is drawn cannot be seen from the user 8A.

The panel object 2551 is arranged behind the avatar object 6B with the surface on which information is displayed facing the user 8A. Here, on the panel object 2551, information indicating a description of the rock-paper-scissors game is displayed as information regarding the first mini game. In addition, the information may include information indicating a reward when winning the rock-paper-scissors game (here, a right to be advantageous in the second mini game is granted).

The UI area 2661 includes UI buttons 2661_1 to 2661_3. Each of the UI buttons 2661_1 to 2661_3 includes an illustration of goo choki per. When the operation on the UI button 2661_1 is accepted, a process in which the selection of goo is stored in the memory 820 is executed. When the operation on the UI button 2661_2 is accepted, the process of storing the selection of the par in the memory 820 is executed. When the operation on the UI button 2661_1 is accepted, a process in which the selection of the cutout is stored in the memory 820 is executed. The UI buttons 2661_1 to 2661_3 are inoperable in the initial state.

Here, when the action instruction data representing the action of uttering the voices 2671 and 2672 is received from the character control device 110B, the avatar object 6B included in the view image 1400_A1 is controlled to utter the voices 2671 and 2672. ..

As a result, the user 8A recognizes that the rock-paper-scissors game starts in response to the action of the avatar object 6B uttering the voice 2671. In addition, the user 8A can recognize the rules of the rock-paper-scissors game, rewards, and the like by visually recognizing the panel object 2551.

Further, the UI buttons 2661_1 to 2661_3 change from the inoperable state to the operable state in response to the action of the avatar object 6B uttering the voice 2672. The UI buttons 2661_1 to 2661_3 may be displayed in a mode that indicates that the UI buttons 2661_1 to 2661_3 have changed to the operable state. The user 8A is motivated to press any one of the UI buttons 2661_1 to 2661_3 according to the action of the avatar object 6B uttering the voice 2672.

In addition, a new UI button may be displayed or the displayed UI button may be hidden according to the action of the avatar object 6B uttering the voice 2672.

FIG. 27 is a diagram showing a visual field image and a game screen after any of the buttons 2661_1 to 2661_3 is pressed by the user 8A.

In FIG. 27A, the visual field image 1400_B1 in the character control device 110B has a point that information 2751 and information 2752 are superposed on the visual field image 1400_B1 shown in FIG. 26A instead of the information 2651. different.

Like the information 2651, the information 2751 represents the number of users 8 who are currently viewing the live game progress part including the first mini game. Here, after the start of the first mini game, the number of viewers has increased by 2 to 134. Thus, the information 2751 is updated in real time based on the information transmitted from the user terminal 800A.

The information 2752 represents the number of users 8 who have operated the first UI object among the users 8 of the plurality of user terminals 800 sharing the first virtual space 11_1. Here, 132 people including the user 8A have performed the operation. The information 2752 is updated in real time based on the information transmitted from the user terminal 800A.

In FIG. 27B, the game screen displayed on the display unit 8702 of the user terminal 800A includes the field-of-view image 1400_A1 and the UI area 2661 similarly to the game screen shown in FIG. However, the mode of the UI button 2661_1 in the UI area 2661 has changed. The UI button 2661_1 is displayed in a mode showing a state in which an operation on the button has been accepted. The mode may be, for example, a mode in which the color of the UI button is changed or a mode in which the UI button is pressed, but is not limited thereto.

FIG. 28 is a diagram showing a view image and a game screen after any of the choice objects 2552_1 to 2552_3 is selected by the avatar object 6B.

In FIG. 28A, the visual field image 1400_B1 in the character control device 110B includes a point including a state in which the avatar object 6B selects the choice object 2552_3 with respect to the visual field image 1400_B1 shown in FIG. The difference is that information 2852 is superimposed instead of.

The avatar object 6B selects the option object 2552_3 with the virtual right hand 1831RB according to the operation of the user 5B. Specifically, here, it is assumed that the virtual space control unit 2106 determines that the virtual right hand 1831RB and the option object 2552_3 are in contact with each other, and detects that the predetermined button included in the controller 300 is pressed. As a result, the virtual right hand 1831RB is associated with the option object 2552_3, and the option object 2552_3 moves together with the virtual right hand 1831RB. Here, the avatar object 6B grips the selection object 2552_3, lifts it, and rotates the selection object 2552_3 so that the surface on which the illustration of “choki” which is one of the selections is drawn is directed to the user 8. It is assumed that The rotation of the option object 2552_3 in the gripped state may be set to be executed in response to, for example, pressing of another predetermined button included in the controller 300. Further, the rotation of the option object 2552_3 in the gripped state may be set to be performed according to the predetermined movement set for the virtual right hand 1831RB.

Further, the user 5B performs an operation for causing the avatar object 6B to select any one of the choice objects 2552_1 to 2552_3, and also performs an operation for uttering a voice 2871 “Jean Kempon”. As a result, the avatar object 6B is controlled so as to select the choice object 2552_3 and speak the voice 2871.

The information 2852 represents the operation result for the first UI object in the plurality of user terminals 800. Here, the information 2852 represents the number of users 8 who have operated each of the UI buttons 2661_1 to 2661_3. According to the information 2852, 43 people including the user 8A press the UI button 2661_1 to select goo, and other 46 users 8 press the UI button 2661_2 to select par and the other 45 users User 8 selects 2600 by pressing 2661_1. The information 2852 is preferably displayed based on the information transmitted from the user terminal 800A after the avatar object 6B performs the operation of selecting the option object 2552_3.

The details of the operation result on the plurality of user terminals 800 may be displayed by accepting the operation of the avatar object 6B for the area including the information 2852. For example, the details of the operation result may be list information of identification information of the user 8 who has selected each option. For example, when an operation of contacting the virtual right hand 1831RB or the virtual left hand 1831LB with respect to the display area of “Goo's person: 43 people” is accepted, “User A, User X, User Y...” that selects Goo is displayed. The list information may be displayed in a superimposed manner. However, the details of the operation result are not limited to this.

Further, the information 2852 is not limited to the totalized information for each option as illustrated. For example, the information 2852 may include identification information of each of the plurality of users 8 and identification information of any of the UI buttons 2661_1 to 2661_3 pressed by the user 8 as a list. In this case, the information 2852 may be scrollably displayed. In this case, the information 2852 includes the identification information of the user 8 and the pressed UI buttons 2661_1 to 2661_3 every time any one of the UI buttons 2661_1 to 2661_3 is pressed on any of the plurality of user terminals 800. It may be updated to additionally include any of the identification information. In this case, the information 2852 may be automatically scrolled and displayed.

In addition, the user 5B causes the avatar object 6B to select the option object 2552_3, and then performs an operation according to the result of the rock-paper-scissors game on the plurality of user terminals 800. Here, for example, a voice 2872 such as “Congratulations on 43 people who won by Goo, congratulations!” is uttered. As a result, the avatar object 6B is controlled to speak the voice 2872.

28B, the game screen displayed on the display portion 8702 of the user terminal 800A includes a visual field image 1400_A1, a UI area 2661, and information 2865.

Here, when the action instruction data representing the action of uttering the voice 2871 is received from the character control device 110B, the avatar object 6B included in the view image 1400_A1 is controlled to utter the voice 2871. With the utterance of this voice 2871, the user 8A can pay attention to the action of the avatar object 6B while hoping that “choki” that can be won by the “goo” selected by himself/herself is selected. The sense of presence increases.

Further, based on the action instruction data transmitted from the character control device 110B, the avatar object 6B included in the visual field image 1400_A1 is controlled to perform the action of selecting the option object 2552_3. Here, as for the option object 2552_3 selected by the avatar object 6B, the surface on which the “cut” is drawn is directed toward the user 8A. As a result, the user 8A can recognize that the avatar object 6B has selected “choki” that he/she will win.

Further, when the operation instruction data indicating the operation of uttering voice 2872 is received from character control device 110B, avatar object 6B included in visual field image 1400_A1 is controlled to utter voice 2872. As a result, the user 8A can recognize that there are 43 people who have won the rock-paper-scissors game like himself, and the sense of presence of participating in the rock-paper-scissors game together with other users 8 increases.

The information 2865 represents the result of the rock-paper-scissors game by the user 8A. Here, the information 2865 is displayed in the form of a balloon including the content of “win”. However, the content and mode of the information 2865 are not limited to this.

FIG. 29 is a diagram showing an example of a field-of-view image and a game screen when the rock-paper-scissors game ends and a transition is made to the next mini game.

In FIG. 29A, the visual field image 1400_B1 in the character control device 110B includes a progress button object 2241, option objects 2552_1 to 2552_3, a virtual left hand 1831LB, and information 2751.

Here, it is assumed that the avatar object 6B has moved closer to the progress button object 2241 on the left side of itself. Therefore, the progress button object 2241 is included in the area near the center of the visual field image 1400_B1. In addition, the option objects 2552_1 to 2552_3 are included in the lower right area of the view image 1400_B1.

Here, when the user 5B performs an operation of uttering a voice 2971 saying "I will go to the next game" as an operation of notifying the transition from the first mini game to the second mini game, the avatar object 6B is displayed. , Voice 2971 is controlled. Note that the operation of notifying the transition to the second mini game is not limited to utterance, and may be a hand gesture or a combination of various operations.

Further, when the user 5B speaks the voice 2971 and at the same time performs an operation of causing the avatar object 6B to press the progress button object 2241, the avatar object 6B is controlled to perform the operation.

In FIG. 29B, the game screen displayed on the display portion 8702 of the user terminal 800A includes a visual field image 1400_A1, a UI area 2661, and information 2981.

Here, when the motion instruction data representing the motion of uttering the voice 2971 is received from the character control device 110B, the avatar object 6B included in the view image 1400_A1 is controlled to utter the voice 2971.

Further, when the operation instruction data representing the operation of pressing the progress button object 2241 is received from the character control device 110B, the avatar object 6B included in the view image 1400_A1 is controlled to perform the operation.

Thus, the user 8A recognizes that the avatar object 6B makes a transition to the second mini game in response to the operation of speaking the voice 2971 and pressing the progress button object 2241. In this way, the user 8A can recognize with realism that the mini game is switched according to the operation of the avatar object 6B.

The information 2981 indicates that the right to advance the next second mini game is granted by winning the rock-paper-scissors game. This allows the user to recognize that the next second mini game has become advantageous.

<Processing flow of the second mini game>
FIG. 30 is a flow chart showing the flow of processing in the second mini game, which is executed by each device constituting the game system 1500. The processing flow shown in FIG. 30 corresponds to the details of the processing in steps S112 and S208 shown in FIG.

Here, it is assumed that a bingo game is applied as the second mini game. The second condition is set in the second mini game. The second condition is that bingo is established in the bingo card given to the user 8A.

In step S501, the virtual space control unit 2106 of the character control device 110B arranges the fourth object of the present invention and virtual objects representing a plurality of options in the first virtual space 11_1B.

Here, the fourth object is a virtual object that represents information regarding the second state. For example, the fourth object may be a virtual panel capable of displaying the information, a virtual display, a virtual blackboard, a virtual electronic bulletin board, or the like, but is not limited thereto. In the present embodiment, a panel object representing a virtual panel similar to the third object is arranged as the fourth object. In other words, when the first mini game is switched to the second mini game, the information displayed on the panel object changes.

Also, a bingo ball box object is arranged as a virtual object representing a plurality of options. The bingo ball box object includes a plurality of bingo ball objects each having a number drawn on its surface, and has an opening on the surface through which the bingo ball object can be taken out.

In step S502, the virtual space control unit 2106 transmits the information about the panel object and the box object to the user terminal 800A.

In step S601, the virtual space control unit 8106 of the user terminal 800A arranges the second object (here, the panel object and the box object) corresponding to the second state in the first virtual space 11_1A based on the received information. To do. In addition, the virtual space control unit 8106 applies the action of the avatar object 6B that targets the second object as the second action. The first object corresponding to the first state is deleted from the first virtual space 11_1A.

In step S602, the game progressing unit 8105 determines whether or not information indicating the right to advance the second mini game in an advantageous manner is stored in the memory 820. If No in step S602, the process of step S603 is executed. If Yes in step S602, the process of step S604 is executed.

In step S603 or S604, the game progression unit 8105 displays the second UI object on the display unit 8702.

Here, the second UI object is a UI object corresponding to the second state. Specifically, the second UI object is a user interface object that receives an operation from the user 8A in the second mini game.

In step S603, a normal bingo card is displayed as the second UI object. In step S604, an advantageous bingo card is displayed on the display unit 8702 as the second UI object.

The normal bingo card and the advantageous bingo card are cards each including an arbitrary numerical value in each of the squares arranged in a plurality of columns and a plurality of rows. Further, each mass has an operable state and an inoperable state, and is inoperable in the initial state. Further, each mass has an open state and an unopened state, and is in an unopened state in the initial state. It is assumed that each mass is brought into an open state by being operated when it is in an operable state. The establishment of bingo, which is the second condition described above, means that a predetermined number of open squares are arranged vertically, horizontally, or diagonally.

Here, the advantageous bingo card is a card that can advance the second mini game more favorably than the normal bingo card. For example, the advantageous bingo card may include a plurality of normal bingo cards. In addition, for example, when the normal bingo card and the advantageous bingo card can include a plurality of numerical values in one square, the advantageous bingo card is the number of numerical values included in one square than the normal bingo card. It may be many. However, the second UI object is not limited to the bingo card as described above, and may be another UI object.

In addition, the numerical value included in each box of the bingo card displayed in this step is determined for each of the plurality of user terminals 800 sharing the first virtual space 11_1. For example, when executing this step, the numerical value drawn on each square may be randomly selected from the numerical values in an arbitrary range and determined.

In step S504, the virtual space control unit 2106 causes the avatar object 6B to perform an operation of selecting a bingo ball object according to the movement of the user 5B. Different identification information (number or the like) is associated with each of the plurality of bingo ball objects.

In step S505, the virtual space control unit 2106 transmits the motion instruction data representing the motion of the avatar object 6B to the user terminal 800A.

In step S605, the virtual space control unit 8106 causes the avatar object 6B to perform an operation of selecting a bingo ball object based on the received operation instruction data.

In step S606, the game progression unit 8105 determines whether or not the numerical value drawn on the bingo ball object selected by the avatar object 6B is a numerical value included in any of the squares of the displayed bingo card. .. If No in step S606, step S612 described below is executed. If Yes in step S606, the next step S607 is executed.

In step S607, the game progressing unit 8105 executes a second process for satisfying at least a part of the second condition based on the second action of the avatar object 6B. In other words, the game progression unit 8105 executes a game process for achieving at least a part of the second condition (first problem of the present invention) according to the action selected by the avatar object 6B.

Here, the game progression unit 8105 executes a second process of making a square including a corresponding numerical value operable according to the action selected by the avatar object 6B. In order to satisfy the second condition that the bingo is established, at least the square that is a part of the second UI object needs to be operable, so the second process is executed.

In step S608, the game progressing unit 8105 receives an operation for a square including the corresponding numerical value and changes the square to the open state.

In step S609, the game progression unit 8105 notifies the character control device 110B that a square including the corresponding numerical value has been opened.

In step S505, the game progression unit 2105 displays, on the HMD 120, information regarding the user 8A whose square including the corresponding numerical value is opened.

In step S610, the game progression unit 8105 of the user terminal 800A determines whether or not a bingo has been established due to the opening of the square in step S608. If No in step S610, step S612 described below is executed. If Yes in step S610, the next step S611 is executed.

In step S611, the game progression unit 8105 notifies the character control device 110B that the bingo has been established.

In step S506, the game progression unit 2105 displays on the HMD 120 the information regarding the user 8A having the bingo established.

In step S507, the game progression unit 2105 determines whether the game has ended. For example, the ending condition of the game may be that the bingo ball object is selected by the avatar object 6B a predetermined number of times. Further, for example, the game ending condition may be that a predetermined time has elapsed from the selection of the first bingo ball object. In addition, for example, the game ending condition may be that the number of users 8 having bingo established exceeds a predetermined number. However, the conditions for ending the game are not limited to these.

If No in step S507, the processes from step S503 are executed. If Yes in step S507, the process of step S508 is executed.

In step S508, the game progression unit 2105 notifies the user terminal 800A that the game has ended.

In step S612, the game progression unit 8105 determines whether or not the end of the game has been notified. This step is also executed in the case of No in step S606 or S610. If No in step S612, the processes from step S605 are executed. If Yes in step S612, the process of step S613 is executed.

In step S613, the game progressing unit 8105 causes the display unit 8702 to display information indicating the result of the bingo game.

<First virtual spaces 11_1A and 11B while the second mini game is in progress>
FIG. 31 is a diagram schematically showing the first virtual space 11_1 during the progress of the second mini game. FIG. 31A is a perspective view of the first virtual space 11_1B realized in the character control device 110B. FIG. 31B is a top view of the first virtual space 11_1A realized in the user terminal 800A. These first virtual spaces 11_1A and 11_B are synchronized.

31A and 31B, an avatar object 6B, a special button object 3141, a panel object 3151, and a box object 3142 are arranged in the first virtual spaces 11_1A and 11_B. A virtual camera 14_B1 is arranged in the first virtual space 11_1B. A virtual camera 14_A1 is arranged in the first virtual space 11_1A. The avatar object 6B and the virtual cameras 14_A1 and 14_B1 are as described with reference to FIG.

The box object 3142 is a rectangular parallelepiped object in which a plurality of bingo ball objects 3144 are arranged. The upper surface of the box object 3142 includes an opening 3143. The avatar object 6B can perform an operation of selecting any of the plurality of bingo ball objects 3144 by inserting the virtual right hand 1831RB or the virtual left hand 1831LB into the box object 3142 from the opening 3143.

Each of the plurality of bingo ball objects 3144 is a spherical object, and a numerical value is drawn on the surface. It is desirable that the numerical value be any of the numerical values in the range that can be displayed on the square of the bingo card as the second UI object. However, there may be a bingo ball object in which a numerical value outside the range is drawn. Further, the numerical values drawn on the plurality of bingo ball objects may be different from each other. Alternatively, the numerical values drawn on some of the plurality of bingo ball objects may be the same.

The panel object 3151 has the same configuration as the panel object 2551 except that the information on the second mini game is displayed on the surface on the direction side of the user 8A instead of the information on the first mini game.

The special button object 3141 is assumed to be arranged in the area on the left side with respect to the direction in which the avatar object 6B is directed. The "left side" is as described in FIG.

<Field-of-view image and game screen during the second mini game>
32 to 36 are diagrams showing an example of the visual field image and the display unit during the progress of the second mini game. In each of FIGS. 32 to 36, (A) represents a visual field image 1400_B1 displayed on the HMD 120 of the character control device 110B by executing steps S501 to S508. (B) represents a game screen displayed on the display unit 8702 of the user terminal 800A by executing S601 to 613.

FIG. 32 is a diagram showing a visual field image and a game screen at the start of the bingo game.

In FIG. 32A, the visual field image 1400_B1 in the character control device 110B.
Includes a special button object 3141, a box object 3142, a virtual right hand 1831RB and a virtual left hand 1831LB, and information 2751. The information 2751 is as described with reference to FIG.

The special button object 3141 is arranged on the left side when viewed from the avatar object 6B.

The box object 3142 is arranged so that the opening 3143 can be seen when viewed from the avatar object 6B.

Further, the virtual right hand 1831RB and the virtual left hand 1831LB of the avatar object 6B itself are included in the visual field image 1400_B1. Accordingly, the user 5B can easily operate so that the virtual right hand 1831RB and the virtual left hand 1831LB can select any of the plurality of bingo ball objects 3144 through the opening 3143 of the box object 3142.

Here, when the user 5B performs an operation of uttering a voice 3271 "Start a bingo game" as an operation of notifying that the second mini game is started, the avatar object 6B utters a voice 3271. Controlled. Note that the operation of notifying that the second mini game is started is not limited to vocalization, and may be a hand gesture or a combination of various operations.

In FIG. 32B, the game screen displayed on the display portion 8702 of the user terminal 800A includes a visual field image 1400_A1 and a UI area 3261. The view image 1400_A1 includes an avatar object 6B, a special button object 3141, a box object 3142, and a panel object 3151.

Since the avatar object 6B faces the user 8A, it faces the front. Further, the virtual right hand 1831RB and the virtual left hand 1831LB of the avatar object 6B are included in the visual field image 1400_A1. This allows the user 8A to pay attention to which bingo ball object 3144 the virtual right hand 1831RB and the virtual left hand 1831LB perform.

The special button object 3141 is arranged on the left side of the avatar object 6B, that is, on the right side of the visual field image 1400_A1.

In this example, the box object 3142 is displayed such that each surface is opaque and the bingo ball object inside is displayed invisible. However, the positions in the first virtual space 11_1A are set for the plurality of bingo ball objects, respectively. This makes it possible to determine the contact between the bingo ball object 3144 and the virtual right hand 1831RB or the virtual left hand 1831LB, and the bingo ball object 3144 can be selected.

The panel object 3251 is arranged behind the avatar object 6B with the surface on which information is displayed facing the user 8A. Here, on the panel object 2551, information representing the description of the bingo game is displayed as information regarding the second mini game. In addition, the information includes information indicating the rules of the bingo game, the set second condition (for example, the establishment of bingo), and the reward when the bingo is established (in this case, being able to participate in the second virtual space). It may be included, but is not limited to these.

The UI area 3261 includes a normal bingo card 3262. A typical bingo card 3262 includes nine squares 3263 arranged in three columns and three rows. Each cell 3263 includes an integer in the range of 1 to 100. When an operation on each cell 3263 is accepted, a process is executed in which information indicating that the cell 3263 is open is stored in the memory 820. Each of the cells 3263 is inoperable in the initial state and is in an unopened state.

Here, when the motion instruction data representing the motion of uttering the voice 3271 is received from the character control device 110B, the avatar object 6B included in the view image 1400_A1 is controlled to utter the voice 3271.

Accordingly, the user 8A recognizes that the bingo game starts in response to the action of the avatar object 6B uttering the voice 3271.

FIG. 33 is a diagram showing another example of the game screen at the start of the bingo game.

In FIG. 33, the game screen displayed on the display unit 8702 of the user terminal 800A includes an advantageous bingo card 3362 instead of the normal bingo card 3262 in the game screen shown in FIG. 32B. Is different. An advantageous bingo card 3362 includes nine masses 3363 arranged in three columns and three rows. Each square 3363 includes two integers in the range of 1 to 100. Since the mass 3363 includes two integers each, the mass 3363 is more easily opened than the mass 3263, which is advantageous. Other than that, each square 3363 is configured similarly to each square 3263 of the normal bingo card 3262. In the description below, it is assumed that a normal bingo card 3262 is displayed on the user terminal 800A.

FIG. 34 is a diagram showing a field-of-view image and a game screen when the avatar object 6B performs an operation of selecting any of the plurality of bingo ball objects 3144 according to the operation of the user 5B.

In FIG. 34(A), the visual field image 1400_B1 in the character control device 110B includes a point in which the avatar object 6B selects one bingo ball object 3144 with respect to the visual field image 1400_B1 shown in FIG. 32(A). , Information 3452 and 3453 are further superimposed.

The avatar object 6B performs an operation of selecting any one of the plurality of bingo ball objects 3144 from the inside of the box object 3142 by the virtual right hand 1831RB in accordance with the operation of the user 5B. Since the bingo ball object 3144 inside the box object 3142 is not displayed in the visual field image 1400_B1, it is impossible for the user 5B to recognize which numerical value of the bingo ball object 3144 is selected until it is taken out. To do.

Here, it is assumed that "54" is drawn on the bingo ball object 3144 selected by the virtual right hand 1831RB. Hereinafter, the bingo ball object 3144 depicted as “54” is also referred to as the bingo ball object 3144 “54”. The details of the process of selecting the bingo ball object 3144 “54” by the virtual right hand 1831RB are the same as the process of selecting the selection object 2552_3 by the virtual right hand 1831RB in FIG. Do not repeat. Note that the virtual space control unit 2106 may output information indicating that the virtual right hand 1831RB and the bingo ball object 3144 are in contact with each other to an output device such as the HMD 120 when it is determined that the virtual right hand 1831RB is in contact with the bingo ball object 3144. In this case, the user 5B may press a predetermined button included in the controller 300 according to the output. Accordingly, the user 5B can easily perform the operation of causing the virtual right hand 1831RB to select the bingo ball object 3144 even if the bingo ball object 3144 is not displayed in the view image 1400_B1.

Alternatively, in the view image 1400_B1, a plurality of bingo ball objects 3144 inside the box object 3142 may be displayed in a mode that does not include the numerical value of the surface. For example, the bingo ball object 3144 may be displayed only in the outline. In this case, since the user 5B can visually recognize the position of each bingo ball object 3144, the user 5B can easily perform the operation of causing the virtual right hand 1831RB to select the bingo ball object 3144.

Further, the user 5B performs an operation for causing the avatar object 6B to select any of the plurality of bingo ball objects 3144, and then utters the numerical value “54” drawn on the selected bingo ball object 3144 as the voice 3471. Take action. As a result, the avatar object 6B is controlled so as to select the bingo ball object 3144 “54” and produce a voice 3471.

Information 3452 represents the progress status of the second mini game. Here, the information 3452 represents the number of times the avatar object 6B has selected the bingo ball object 3144. The information 3452 is updated each time the avatar object 6B performs an operation of selecting any of the plurality of bingo ball objects 3144.

The information 3453 includes operation results for the second UI objects in the plurality of user terminals 800. Here, the information 3453 includes the number of users 8 whose bins on the bingo card have been opened by the bingo ball object 3144 most recently selected by the avatar object 6B, and the number of users 8 who have achieved bingo. According to the information 3453, 14 users 8 perform the operation of opening the cell 3263 or 3363 of “54”, and one user 8 establishes a bingo.

It should be noted that detailed information of the operation result for the second UI object may be displayed in accordance with the action of the avatar object 6B for the area including the information 3453. For example, the details of the operation result may be list information of the identification information of the user 8 who has the cell 3263 or 3363 opened. For example, when the operation of contacting the virtual right hand 1831RB or the virtual left hand 1831LB with respect to the display area of "the person who has opened the cell of 54: 14 persons" is accepted, "the user A, the user X, the user who has opened the cell of 54" are received. The list information such as “Y...” may be displayed in a superimposed manner. However, the details of the operation result are not limited to this.

Further, the information 3453 is not limited to the tabulation information of the operation result as illustrated. For example, the information 3453 may include, as a list, the identification information of the user 8 of each of the plurality of user terminals 800 in the state where the square 3263 or 3363 is opened. In this case, the information 3453 may be scrollably displayed. Further, in this case, the information 3453 may be updated so as to additionally include the identification information of the user 8 every time the cell 3263 or 3363 is opened in any of the plurality of user terminals 800. In this case, the information 3453 may be automatically scrolled and displayed.

In addition, the user 5B performs an action on the avatar object 6B according to the display of the information 3453, according to the operation result of the plurality of user terminals 800. Here, for example, the voice 3472 such as "One person from bingo has come out" is uttered. When the information 3453 includes the identification information of the user 8 who has established the bingo, the voice 3472 further includes the identification information of the user 8 and has a content such as “Mr. User 8A became bingo!”. It may be. As a result, the avatar object 6B is controlled to speak the voice 3472.

In FIG. 34B, the game screen displayed on the display portion 8702 of the user terminal 800A includes a visual field image 1400_A1 and a UI area 3261.

Here, when the motion instruction data representing the motion of selecting the bingo ball object 3144 “54” is received from the character control device 110B, the avatar object 6B included in the view image 1400_A1 changes the bingo ball object 3144 “54”. Perform the selected action. Here, the bingo ball object 3144 “54” selected by the avatar object 6B has the surface with “54” drawn toward the user 8.

Further, when the operation instruction data representing the operation of uttering voice 3471 is received from character control device 110B, avatar object 6B included in visual field image 1400_A1 is controlled to utter voice 3471.

As a result, the user 8A can recognize that the bingo ball object 3144 in which the numerical value “54” included in the bingo card of the user 8A is drawn is selected by the avatar object 6B.

Further, in response to the avatar object 6B selecting the bingo ball object 3144 “54”, the square 3263 including “54” changes to an operable state. The mass 3263 changed to the operable state may be changed and displayed in a mode indicating that it is operable. The user 8A can open the square 3263 by operating the square 3263 including "54". As a result, the cell 3263 including "54" is displayed in an open state. The open state may be, for example, a color changed to represent the open state, a state in which a hole is opened in the area of the mass, or the like, but is not limited thereto.

In this way, the user 8A can pay attention to the action of the avatar object 6B while hoping that the bingo ball object 3144 in which the numerical value included in the bingo card 3262 of his/herself is drawn is selected, and the user 8A has a realistic feeling. Increase.

Further, when motion instruction data representing a motion of uttering voice 3472 is received from character control device 110B, avatar object 6B included in visual field image 1400_A1 is controlled to utter voice 3472. As a result, the user 8A can recognize that the other user 8 has established the bingo, and the sense of presence of participating in the bingo game together with the other user 8 increases. Further, when the voice 3472 includes the identification information of the user 8A as the user 8 who has established the bingo or the user 8 who has opened the square, the user 8A gives a special feeling among the plurality of users 8. I can feel it.

FIG. 35 is a diagram showing another example of the game screen when the avatar object 6B selects one of the plurality of bingo ball objects 3144 according to the operation of the user 5B.

The game screen shown in FIG. 35 differs from the game screen shown in FIG. 34B in that a plurality of bingo ball objects 3144 are displayed in the view image 1400_A1. Also in this case, it is assumed that the plurality of bingo ball objects 3144 are not displayed in the view image 1400_B1 displayed on the character control device 110B, as in FIG. 34A. Accordingly, the user 8A can recognize the position of the bingo ball object 3144 in which the numerical value included in the bingo card 3262 of the user 8A is drawn. As a result, whether or not the bingo ball object 3144 is selected by the avatar object 6B can be noticed with a more realistic feeling.

FIG. 36 is a diagram showing a field-of-view image and a game screen when a bingo is established according to the action selected by the avatar object 6B according to the action of the user 5B.

In FIG. 36(A), the visual field image 1400_B1 in the character control device 110B differs from the visual field image 1400_B1 shown in FIG. 34(A) in the numerical value drawn on the bingo ball object 3144 selected by the avatar object 6B. Further, the information 3652 and 3653 are displayed instead of the information 3452 and 3453.

It is assumed that the avatar object 6B selects and takes out the bingo ball object 3144 “25” from the inside of the box object 3142 according to the operation of the user 5B. The details of the operation of selecting the bingo ball object 3144 “25” are the same as the operation of selecting the bingo ball object 3144 “54” described with reference to FIG.

In addition, the user 5B performs an operation of uttering the numerical value “25” drawn on the selected bingo ball object 3144 as the sound 3671. As a result, the avatar object 6B is controlled so as to select the bingo ball object 3144 “25” and produce a voice 3471.

Information 3652 is updated with the progress status of the second mini game indicated by information 3452. According to the information 3652, the bingo ball object 3144 is selected 30 times. Here, it is assumed that the number of times of selection reaches 30 times as a condition for ending the bingo game.

The information 3653 is updated with the operation result for the second UI object in the plurality of user terminals 800 shown in the information 3453. According to the information 3653, 14 users 8 have opened the cell 3263 or 3363 of “25”, and a total of 9 users 8 have a bingo.

In addition, the user 5B causes the avatar object 6B to perform an operation in accordance with the progress status of the second mini game and the operation result on the plurality of user terminals 800 in response to the display of the information 3452 and 3453. Here, for example, a voice 3672 such as "End of bingo! There are nine people in bingo. Congratulations" is issued. Further, the case where the information 3653 includes the identification information of the user 8 who has established the bingo is the same as the case where the information 3453 is described.

In FIG. 36B, the game screen displayed on the display portion 8702 of the user terminal 800A includes a visual field image 1400_A1, a UI area 3261, and information 3665.

Here, based on the action instruction data transmitted from the character control device 110B, the avatar object 6B included in the visual field image 1400_A1 performs the action of selecting the bingo ball object 3144 “25” and also produces the voice 3671. Controlled.

Further, in response to the avatar object 6B selecting the bingo ball object 3144 “25”, the square 3263 including “25” changes to an operable state. The user 8A can operate the square 3263 including “25” to open the square 3263 and establish a bingo.

The information 3655 is information indicating that bingo has been established in the user terminal 800A. The information 3655 is displayed according to the fact that the square 3263 including “25” has changed to the open state.

Thereby, the user 8A can recognize that he/she has established the bingo.

Further, when the motion instruction data representing the motion of uttering voice 3672 is received from character control device 110B, avatar object 6B included in visual field image 1400_A1 is controlled to utter voice 3672. As a result, the user 8A can recognize that the nine users 8 including himself/herself have established bingo, and can feel a special feeling among the plurality of users 8. When the voice 3672 includes the identification information of the user 8A who has established the bingo, the user 8A can feel a further special feeling.

<Processing flow of live distribution part>
FIG. 37 is a flowchart showing the flow of processing in the live distribution part, which is executed by each device constituting the game system 1500. The processing flow shown in FIG. 37 is executed subsequent to the processing shown in FIG.

In step S701, the virtual space control unit 2106 of the character control device 110B receives an operation of moving to the second virtual space 11_2. The operation of moving to the second virtual space 11_2 may be an operation targeting a predetermined virtual object (for example, a special button object) by the avatar object 6B. The operation of moving to the second virtual space 11_2 may be moving of the avatar object 6B within a predetermined area of the first virtual space 11_1B (for example, near a virtual door). Further, the operation of moving to the second virtual space 11_2 may be an operation on a predetermined button included in the controller 300. However, the operation of moving to the second virtual space 11_2B is not limited to these.

In step S702, the virtual space control unit 2106 defines the second virtual space 11_2B based on the second virtual space data representing the second virtual space 11_2. The second virtual space data is supplied from the server 600, for example.

In step S703, the virtual space control unit 2106 arranges the virtual camera 14_B2 in the second virtual space 11_2B. The visual field image of the virtual camera 14_B2 is the visual field image of the second virtual space 11_2B viewed from the avatar object 6B.

In step S704, the virtual space control unit 2106 arranges the avatar object 6B in the second virtual space 11_2B. Also, the avatar object 6B is deleted from the first virtual space 11_1B.

On the other hand, in step S801, the game progression unit 8105 of the user terminal 800A determines whether or not a predetermined condition is satisfied. The predetermined condition may be whether or not the second condition set in the second mini game is satisfied. If Yes in step S801, step S804 described below is executed. If No in step S801, the process of the next step S802 is executed.

In step S802, the game progression unit 8105 determines whether or not the user 8A has paid the consideration or the user operation for instructing to consume the valuable item given to the user 8A based on the payment of the consideration. The valuable item may be, for example, virtual currency.

The case of No in step S802 will be described later. If Yes in step S802, the process of next step S803 is executed.

In step S803, the game progression unit 8105 executes a charging process or a process of consuming a valuable item, and then executes a process of step S804.

In step S804, the game progression unit 8105 receives an operation for instructing participation in the second virtual space 11_2A, and determines whether or not the operation has been received. If No in step S804, the process in the user terminal 800A ends. If Yes in step S804, the process of next step S805 is executed.

In step S805, the game progression unit 8105 defines the second virtual space 11_2A based on the second virtual space data representing the second virtual space 11_2. The second virtual space data is supplied from the server 600, for example.

In step S806, the virtual space control unit 8106 arranges the virtual camera 14_A2 in the second virtual space 11_2A. Here, the position and direction of the virtual camera 14_A2 cannot be operated by the user 8A, and the plurality of users 8 view the common view image 1400_A2.

In step S705, the virtual space control unit 2106 transmits information indicating the avatar object 6B to the user terminal 800A.

In step S807, the virtual space control unit 8106 arranges the avatar object 6B in the second virtual space 11_2A based on the received information. Also, the avatar object 6B is deleted from the first virtual space 11_1A.

As a result, the second virtual space 11_2 is shared between the user terminal 800A satisfying the predetermined condition and the character control device 110B. In addition, the avatar object 6B is arranged in synchronization between the user terminal 800A and the character control device 110B.

In step S706, the virtual space control unit 2106 displays and updates the visual field image 1400_B2 based on the position and direction of the virtual camera 14_B2 that changes according to the movement of the HMD 120.

In step S808, the virtual space control unit 8106 displays and updates the visual field image 1400_A2 based on the position and direction of the virtual camera 14_A2.

In step S707, the virtual space control unit 2106 controls the motion of the avatar object 6B according to the motion of the user 5B.

In step S708, the virtual space control unit 2106 transmits the motion instruction data representing the motion of the avatar object 6B to the user terminal 800A.

In step S809, the virtual space control unit 8106 controls the movement of the avatar object 6B based on the received motion instruction data.

Thereby, in the user terminal 800A, the avatar object 6B operates in the second virtual space 11_2A based on the operation of the user 5B. Further, the avatar object 6B disappears from the first virtual space 11_1A.

In the live distribution part, the operation of the avatar object 6B may be distributed for the purpose of having the user 8 enjoy it. For example, the movement of the avatar object 6B may include, but is not limited to, singing, dancing, sports techniques, and the like.

As described above, the live distribution part can be viewed only by the user 8 who satisfies the predetermined condition, here, the user 8 who satisfies the second condition in the second mini game. Therefore, the motivation of the user 8 for satisfying the second condition in the second mini game is improved. In addition, for the user 8 who satisfies the second condition, the special feeling of being able to view the live distribution part is increased.

<Example of visual field image and game screen when moving from first virtual space to second virtual space>
38 to 40 are diagrams showing transition examples of the visual field image and the game screen until the live distribution part is started by moving from the first virtual space 11_1 to the second virtual space 11_2.

FIG. 38A is a diagram showing an example of a visual field image 1400_B1 displayed on the HMD 120 of the character control device 110B when moving from the first virtual space 11_1 to the second virtual space 11_2. In FIG. 38(A), the visual field image 1400_B1 in the character control device 110B includes a special button object 3141, a virtual left hand 1831LB, and information 2751. The information 2751 is as described above.

Here, it is assumed that the avatar object 6B has moved closer to the special button object 3141 existing on the left side of itself. Therefore, the special button object 3141 is included in the area near the center of the visual field image 1400_B1. The box object 3142 is not included in the visual field image 1400_B1.

Here, when the user 5B performs an operation of uttering the voice 3871 notifying that the user can join the second virtual space 11_2 only for the user 8A having the bingo, the avatar object 6B outputs the voice 3871. Controlled to speak. Here, the voice 3871 has the content of “inviting nine bingo people to a special stage”. Note that the operation of notifying that the second virtual space 11_2 can participate is not limited to utterance, and may be a hand gesture or a combination of various operations.

Further, when the user 5B speaks the voice 3871 and at the same time performs an operation of causing the avatar object 6B to press the special button object 3141, the avatar object 6B is controlled to perform the operation. Further, the avatar object 6B is controlled to disappear from the first virtual space 11_1B according to the action.

In FIG. 38B, the game screen displayed on the display portion 8702 of the user terminal 800A includes a visual field image 1400_A1 and a UI area 3861. The UI area 3861 differs from the UI area 3261 in that it includes a special UI button 3864.

Here, when the motion instruction data representing the motion of uttering the voice 3871 is received from the character control device 110B, the avatar object 6B included in the view image 1400_A1 is controlled to utter the voice 3871.

Further, when the operation instruction data representing the operation of pressing the special button object 3141 is received from the character control device 110B, the avatar object 6B included in the view image 1400_A1 performs the operation and then the first virtual space 11_1. Controlled to disappear from.

Accordingly, the user 8A recognizes that the avatar object 6B has moved from the first virtual space 11_1 to the second virtual space 11_2 in response to the action of the avatar object 6B uttering the voice 3871 and pressing the special button object 3141. To do.

The special UI button 3864 is an example of a UI object displayed when bingo is established. When the operation on the special UI button 3864 is accepted, the field-of-view image 1400_A2 representing the second virtual space 11_2A is displayed.

FIG. 39 is a diagram schematically showing the first virtual space 11_1 and the second virtual space 11_2 after the avatar object 6B has moved.

FIG. 39(A) is a perspective view of the first virtual space 11_1 shared between the user terminal 800A and the character control device 110B. The avatar object 6B is controlled to disappear from the first virtual space 11_1 after the operation targeting the special button object 3141 is performed. Therefore, the first virtual space 11_1 shown in FIG. 39A includes the panel object 3151, the special button object 3141, and the box object 3142, but does not include the avatar object 6B.

FIG. 39B is a perspective view of the second virtual space 11_2 shared between the user terminal 800A and the character control device 110B. The second virtual space 11_2 includes an avatar object 6B. As shown, the avatar object 6B may be displayed in a different mode from when it was present in the first virtual space 11_1. The different mode may be, for example, different clothes and belongings (here, microphone), but is not limited thereto. By displaying the avatar object 6B in a different manner, the special feeling of the live distribution part proceeding in the second virtual space 11_2 is improved.

FIG. 40 is a diagram showing an example of a visual field image 1400_A2 displayed on the user terminal 800A. In FIG. 40, the visual field image 1400_A2 includes the avatar object 6B operating in the second virtual space 11_2A. The view image 1400_A2 can be viewed only by the user 8 who has established bingo.

<Process flow of chat part>
FIG. 41 is a flowchart showing the flow of processing in the chat part, which is executed by the user terminal 800A. The process flow shown in FIG. 41 is executed when the result in step S802 in FIG. 37 is No.

In step S901, the game progression unit 8105 displays the UI object corresponding to the chat part on the display unit 8702. The UI object includes a field for accepting a comment input and a button for accepting an operation for instructing the comment transmission.

In step S902, the game progressing unit 8105 acquires information indicating the comment input by the user 8A.

In step S903, the game progressing unit 8105 transmits the information indicating the acquired comment to the server 600 in response to the operation of the user 8A that instructs to transmit the comment.

In step S904, the game progressing unit 8105 superimposes and displays the information representing the acquired comment on the view image 1400_A1.

In step S905, the game progression unit 8105 acquires information representing a comment input by the other user 8 who has not satisfied the predetermined condition on the other user terminal 800. The information can be acquired by being transmitted from the server 600.

In step S906, the game progression unit 8105 displays the information indicating the comment input by the other user 8 so as to be superimposed on the view image 1400_A1.

In step S907, the game progression unit 8105 determines whether or not to end the chat part. For example, the chat part may determine to end the chat part when an operation for instructing the end of the chat part is received by the user 8A.

If No in step S907, the processes from step S902 are executed. If Yes in step S907, the process ends.

In this way, the game progression unit 8105 superimposes and displays the information indicating the comment input by the user 8A and the information indicating the comment input by the other user 8 on the view image 1400_A1.

<Game screen example in chat part>
FIG. 42 is a diagram showing an example of a game screen example in the chat part. The display unit 8702 includes a view image 1400_A1, a UI area 3261, a field 4231, a send button 4232, and comments 4251 to 4254. The field-of-view image 1400_A1 and the UI area 3261 are configured in the same manner as at the time of transition from the first mini game to the second mini game (FIG. 38B).

The field 4231 and the send button 4232 are examples of UI objects corresponding to chat parts. The field 4231 is a UI object that receives a comment input. The send button 4232 is a UI object that receives an instruction to send a comment. The information input in the field 4231 is transmitted to the server 600 when the operation of the send button 4232 is accepted.

The comments 4251 and 4254 are information representing comments input by the other user 8 in the other user terminal 800. These pieces of information are received from another user terminal 800 via the server 600. The comments 4252 and 4253 are information representing comments input by the user 8A on the user terminal 800A. These pieces of information are transmitted to another user terminal 800 via the server 600. The comments 4251 and 4254 are superimposed and displayed on the visual field image 1400_A1 in the order received by the user terminal 800. The information input by the other user 8 is displayed on the left side of the game screen, and the information input by the user 8A is displayed on the right side of the game screen.

In this way, by superimposing the comments 4251 to 4254 on the view image 1400_A1, it is unfortunate that the users 8 who cannot satisfy the predetermined condition cannot participate in the second virtual space 11_2A and are left behind in the first virtual space 11_1A. You can share the feeling.

[Modification]
In the present embodiment, in the game according to the game system 1500, the third task is set before the first play part (second mini game) is started, regardless of the operation of the first user (user 5B). The played third play part may be advanced. In this case, the higher the achievement level of the third task in the third play part, the higher the achievement level of the third task in the third play part, the more the game progressing unit 8105 of the user terminal 800 selects the avatar object 6B in the first play part. Accordingly, a more advantageous process for achieving the first subject may be executed. The third play part may be, for example, a role playing game, a quiz game, an action game, a game using position information, or a game of another game genre. The third task in the third play part is, for example, a game task set by the player in the game play, in which the player obtains a predetermined game item and a clear condition for a predetermined play unit (so-called quest, etc.). This includes satisfying, achieving a certain score, and other challenges. In the description of the above embodiment, the better the result of the first mini game (janken), the more advantageous the player is to the second mini game (bingo game). However, the better the play result of the third play part, the more advantageous the player is in the progress of the first mini-game or the second mini-game. You may do it. Further, by accepting an operation for charging processing from the player during or before the progress of the first mini game or the second mini game, the first mini game or the second mini game is made advantageous to the player. May be. Here, the operation of the player for the billing process may be an operation of purchasing paid virtual currency in the game program, or the player consumes the paid virtual currency (or obtains the game object by the charging process). Then, an operation (so-called coin change) for instructing the game object to be placed in the virtual space in which the avatar object 6B is placed may be used.

In other words, the third play part is provided as a part that proceeds regardless of the operation of the avatar object 6B in response to the operation of the user 5B. Also, the third play part can be played before the live game progress part including the second mini game starts. The third play part may be a single play part that progresses based on the operation of the user 8 of the user terminal 800. Further, the third play part may be a multi-play part that progresses based on the operation of the user 8 of the plurality of user terminals 800. Further, the third play part may proceed using a character that operates without being operated by the user 5B.

Further, in the present embodiment, the positions and directions of the virtual cameras 14_A1 and 14_A2 arranged in the first virtual space 11_1A and the second virtual space 11_2A realized in the user terminal 800 may be operable by the user 8. .. For example, the user may be able to control the positions and directions of the virtual cameras 14_A1 and 14_A2 by operating the display unit 8702 on which the visual field images 1400_A1 and 1400_A2 are displayed. In this case, the virtual cameras 14_A1 and 14_A2 may be controlled so that they cannot be placed in a predetermined area including the avatar object 6B. For example, the virtual stage area in which the avatar object 6B exists is defined as the predetermined area. In this case, the user 8 can have a virtual experience of moving to any position in the area other than the stage and viewing the avatar object 6.

Further, in the present embodiment, the option objects in the first mini game and the second mini game may represent a plurality of options with one option object. For example, in the case of a dice object, one dice object represents six options 1 to 6. If it is a coin object, one coin object represents two options, front and back. In this case, as an operation for the avatar object 6B to select one of the options, an operation according to the option object such as an operation of rolling a dice object or an operation of throwing a coin object is applied. In this case, as the behavior when the operation of the predetermined button included in the controller 300 is detected in the state in which it is determined that the virtual object is in contact with the option object, the option object is a rolling behavior, a behavior of falling after rising, or the like. The behavior according to is defined.

Further, in the present embodiment, after the bingo ball object is selected by the avatar object 6B, the numerical value of the bingo ball object may be determined. In this case, the determined numerical value may be displayed on the surface of the selected bingo ball object. Further, in this case, at least one bingo ball object may be arranged in the box object 3142.

Further, in the present embodiment, the degree to which the bingo game, which is the second mini game, becomes advantageous may be determined stepwise according to the degree to which the first condition is satisfied in the first mini game. For example, in the rock-paper-scissors game, when the rock-paper-scissors game is performed a plurality of times, the number of bingo cards corresponding to the number of times the user 8 has won may be given to the user in the bingo game.

Further, in the present embodiment, the user terminal 800 may not have the virtual space control unit 8106. In this case, the character control device 110B may generate a visual field image for the user terminal 800 in addition to the visual field image displayed on the HMD 120 in the device. Further, in this case, the view image for the user terminal 800 is reproduced by being distributed to the user terminal 800. For example, in this case, the virtual camera 14_A1 is arranged in addition to the virtual camera 14_B1 in the first virtual space 11_1B realized in the character control device 110B. The character control device 110B may generate a view image 1400_A1 based on the virtual camera 14_A1 and deliver it to the user terminal 800A. The field-of-view image 1400_A2 corresponding to the second virtual space 11_2B is similarly generated in the character control device 110B and can be distributed. In this case, the field-of-view images 1400_A1 and 1400_A2 may be generated in real time according to the operation of the user 5B of the character control device 110B, and may be delivered in real time. Further, each of the visual field images 1400_A1 and 1400_A2 may be delivered as a pre-generated recorded image.

Further, in the present embodiment, different characters may be applied as the first character and the second character in the present invention. In this case, the user 8 who satisfies the predetermined condition in the live game progress part can view the live distribution part by the avatar object different from the avatar object 6B that has progressed the live game progress part.

Further, in the present embodiment, different character control devices may be applied as the first character control device and the second character control device in the present invention. For example, the HMD set 110C may be applied as the second character control device. In this case, the live distribution part proceeds based on the operation of the user 5C (second user).

In the above-described embodiment, the virtual space (VR space) in which the user is immersed by the HMD has been described as an example, but a transparent HMD may be adopted as the HMD. In this case, by outputting a view image in which a part of an image forming a virtual space is combined with a real space visually recognized by a user via a transmissive HMD, an augmented reality (AR) space or a mixed reality (AR) is output. The user may be provided with a virtual experience in MR (Mixed Reality) space. In this case, instead of the operation object, the action on the target object in the virtual space may be generated based on the movement of the user's hand. Specifically, the processor may specify the coordinate information of the position of the user's hand in the physical space, and define the position of the target object in the virtual space in relation to the coordinate information in the physical space. As a result, the processor can grasp the positional relationship between the user's hand in the real space and the target object in the virtual space, and can execute the process corresponding to the above-described collision control or the like between the user's hand and the target object. .. As a result, it becomes possible to act on the target object based on the movement of the user's hand.

[Example of software implementation]
A control block of the control unit 2100 (in particular, an operation reception unit 2101, a display control unit 2102, a UI control unit 2103, an animation generation unit 2104, a game progression unit 2105, a virtual space control unit 2106, and a reaction processing unit 2107), a control unit 6100. Control blocks (in particular, the progress support unit 6101 and the sharing support unit 6102) and control blocks of the control unit 8100 (in particular, the operation reception unit 8101, the display control unit 8102, the UI control unit 113, the animation generation unit 8104, the game progress). The unit 8105, the virtual space control unit 8106, and the progress information generation unit 8107) may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or a CPU (Central Processing Unit). It may be used and realized by software.

In the latter case, the information processing apparatus including a part or all of the control unit 2100, the control unit 6100, and the control unit 8100 has a CPU that executes instructions of a program that is software that realizes each function, the program, and various data. A ROM (Read Only Memory) or a storage device (these are referred to as a "recording medium") recorded so that they can be read by a computer (or a CPU), a RAM (Random Access Memory) for expanding the program, and the like are provided. Then, the computer (or CPU) reads the program from the recording medium and executes the program to achieve the object of the present invention. As the recording medium, a "non-transitory tangible medium", for example, a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. The program may be supplied to the computer via any transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. Note that one aspect of the present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

The present invention is not limited to the above-described embodiments, but various modifications can be made within the scope of the claims, and embodiments obtained by appropriately combining the technical means disclosed in the different embodiments Is also included in the technical scope of the present invention.

[Appendix]
The contents according to one aspect of the present invention are listed below.

(Item 1) The game program (831) executed by the computer (user terminal 800) including the processor (810) and the memory (820) has been described. According to an aspect of the present disclosure, a game program is a game program that includes a processor and a memory and that is executed by a computer as a viewing terminal for a viewer to view real-time distribution performed by the distributor. The game program is provided to the processor in a first virtual space shared between a first distribution terminal for the first distributor to perform real-time distribution and a plurality of viewing terminals including the viewing terminal. Arranging a corresponding first avatar object and a first virtual camera, displaying a first view image showing a view area of the first virtual camera on a display unit during real-time distribution, and a game program The game in one state is advanced based on the movement of the body of the first distributor and the operation of the viewer for each state of the first game part that progresses while transitioning between a plurality of states included in the game based on. And a step of performing a game process, wherein the game process in each state is performed by the first distribution terminal that detects a motion of the body of the first distributor and generates first motion information indicating a motion of the first avatar object. And a step of updating the first view image so as to include the first avatar object performing a motion indicated by the first motion information in response to the reception of the first motion information. A step of determining whether or not the motion indicated by the first motion information is a motion for causing a state transition in response to the reception of the first motion information; and the motion indicated by the first motion information causes a transition. The movement of the first game part, the step of transitioning the progress of the first game part from the current state to another state. According to the above configuration, in the first game part in which each state progresses based on the movement of the body of the first distributor and the operation of the viewer, the viewer transitions the first avatar object that advances the game. By this operation, it is possible to recognize that the state of the first game part is switched, so that the sense of presence is increased and the immersive feeling in the game is enhanced.

(Item 2) In (Item 1), the step of disposing the first avatar object and the first virtual camera further disposes the first object in the first virtual space, and the step of determining is the movement indicated by the first motion information. May be determined to be a movement for making a transition when the movement is for causing the first avatar object to operate the first object in the first virtual space. Thereby, the viewer can recognize that the state of the first game part is switched according to the operation of the first avatar object operating the first object.

(Item 3) In (Item 2), the step of arranging the first avatar object and the first virtual camera includes arranging an operation object representing the hand of the first avatar object as at least a part of the first avatar object, and determining. The step of performing may determine that the movement indicated by the first motion information is a movement for making a transition when the movement of the operation object to contact the first object in the first virtual space. Accordingly, the viewer can recognize that the state of the first game part is switched when the hand of the first avatar object contacts the first object.

(Item 4) In any one of (Item 1) to (Item 3), the game process in each state is a user interface (UI) object for accepting an operation by a viewer, and is in a state before transition. The method may further include deleting the associated UI object from the display unit and displaying the UI object associated with the current state on the display unit together with the first visual field image. As a result, the viewer can recognize that the state of the first game part has transitioned by switching the operability (UI object) as a result of the action for transition by the first avatar object.

(Item 5) In any one of (Item 1) to (Item 4), in the game process in each state, the second object associated with the state before the transition is deleted from the first virtual space, and the transition is performed. The method may further include the step of arranging a second object associated with the current state, which has a different aspect from the second object associated with the previous state, in the first virtual space. As a result, the viewer is notified that the state of the first game part has transitioned due to the switching of the second object arranged in the first virtual space as a result of the action for transition by the first avatar object. Can be recognized.

(Item 6) In the item of any one of (Item 1) to (Item 5), the step of arranging the first avatar object and the first virtual camera may include a third object for displaying information indicating each state. The game process in each state further arranged in one virtual space may further include a step of changing the information displayed on the third object from the information indicating the state before the transition to the information indicating the current state. As a result, the viewer performs the transition operation by the first avatar object, and as a result, the display of the third object arranged in the first virtual space is switched, and the state of the first game part is transited. I can recognize that.

(Item 7) In any one of (Item 1) to (Item 6), at least one of the game processes in each state is stored in the memory in association with at least one state before the transition to the current state. The achievement level obtained from the memory, and the higher the achievement level obtained from the memory, the better the achievement of the first condition associated with the current state, and the step of executing a process advantageous to the viewer. , Determining the degree of achievement of the first condition based on the movement of the body of the first distributor and the operation of the viewer, and storing the degree of achievement in a memory in association with the current state. It may further include. Accordingly, in order to increase the achievement level of the first condition in a certain state, it is effective to increase the achievement level of the first condition in at least one state before the transition to the state. As a result, the motivation of the viewer to improve the achievement level of the first condition in each state is improved.

(Item 8) In any one of (Item 1) to (Item 7), the game program causes the processor to operate a second game part, which is included in the game and is different from the first game part, by a viewer's operation. A step of responding to progress, a step of determining the degree of achievement of the second condition based on an operation by a viewer in the second game part, and a step of storing the degree of achievement of the second condition in a memory. Furthermore, at least one of the game processes in each state of the first game part is associated with the current state as the step of acquiring the achievement level of the second condition from the memory, and the higher the achievement level of the second condition. In order to improve the achievement level of the first condition, a step of executing a process advantageous to the viewer may be further included. Thus, in order to more advantageously achieve the first condition in at least one of the states of the first game part, it is effective to increase the degree of achievement of the second condition in the second game part before that. As a result, the viewer's motivation for playing the second game part and increasing the degree of achievement of the second condition before watching the first game part is improved.

(Item 9) In (Item 7) or (Item 8), at least one of the game processes in each state of the first game part includes a step of further arranging a plurality of option objects in the first virtual space, A step of displaying a plurality of first UI objects, which are associated with a part or all of the option objects and accepts a viewer's operation, on the display unit together with the first visual field image; A step of determining whether or not the first avatar object is a motion for causing the first avatar object to select a part of the plurality of choice objects in one virtual space, and a case where the motion indicated by the first motion information is a motion for selecting , A step of identifying the choice object selected by the first avatar object and at least one of the first conditions associated with the current state in response to a viewer operation on the first UI object associated with the identified choice object. And a step of executing an advantageous process, wherein the step of displaying the first UI object includes a step of increasing the number of the first UI objects displayed on the display unit. You may execute. As a result, if more first UI objects are displayed on the display unit, the selection object associated with any of the first UI objects is more likely to be selected by the first avatar object. There is an advantageous effect that the first condition is easily achieved.

(Item 10) In any one of (Item 1) to (Item 9), it is determined whether a specific game process among the game processes in each state of the first game part is a movement for transition. The step of determining whether a specific condition associated with the current state is satisfied, instead of the step of performing the transition and the step of causing the transition, and when the specific condition is satisfied, the game is executed in the first game part. From the first game part to a third game part different from the first game part, the game program causes the processor to satisfy a specific condition among the plurality of viewing terminals in the third game part. A second virtual space different from the first virtual space, which is shared between the plurality of viewing terminals and the second distribution terminal for the second distributor to perform the real-time distribution, corresponds to the second distributor. The step of arranging the 2 avatar object and the second virtual camera, and the second visual field image showing the visual field area of the second virtual camera is replaced with the first visual field image during real-time distribution in the third game part. Displaying on the display unit, receiving second motion information indicating a motion of the second avatar object, which is generated by the second distribution terminal by detecting the movement of the body of the second distributor, and second motion information The step of updating the second visual field image so as to include the second avatar object performing the movement indicated by the second motion information in response to the reception of the. Thereby, in order to view the real-time distribution in the second virtual space different from the first virtual space, the motivation for advancing a specific state is improved so that a specific condition is satisfied in the first game part. ..

(Item 11) In (Item 10), the game program obtains the first input information input by the viewer when the processor does not meet the specific condition, and the first input information is obtained. In response to the request, the step of transmitting the first input information to another viewing terminal among the plurality of viewing terminals that does not meet the specific condition, and the other viewing terminal that does not meet the specific condition. The step of receiving the second input information input by the viewer of the terminal and the step of displaying the first input information and the second input information on the display unit together with the first view image may be further executed. As a result, it is possible to provide a place where viewers who do not meet the specific condition can share the disappointment that they could not participate in the second virtual space.

(Item 12) In (Item 10) or (Item 11), the game program causes the processor to pay the consideration by the viewer when the specific condition is not satisfied, or to the viewer based on the payment of the consideration. A step of accepting an operation for instructing consumption of the given item, and in response to the operation for instructing, performing a fee charging process or a process of consuming an item, and moving the game from the first game part to the third game part. The step of shifting may be further executed. As a result, it is possible to provide the viewer with means capable of participating in the third game part even if the specific condition is not satisfied.

(Item 13) The method of executing the game program has been described. According to an aspect of the present disclosure, a game program is executed by a computer including a processor and a memory. The method is a method in which the processor executes the steps described in (Item 1). The method according to (Item 13) has the same effects as the game program according to (Item 1).

(Item 14) The information processing device has been described. According to an aspect of the present disclosure, the information processing apparatus operates the information processing apparatus (user terminal 800) by executing a memory (820) that stores a game program according to (Item 1) and the game program. And a processor (810) for controlling the. The information processing device according to (Item 14) has the same effects as the game program according to (Item 1).

2 network, 5, 5A, 5B, 5C, 5D user, 6, 6A, 6B, 6C, 6D avatar object, 11, 11A, 11B, 11C, 11D virtual space, 12 center, 13 panoramic image, 14, 14A, 14B Virtual camera, 15, 15A, 15B, 15C view area, 16 reference line of sight, 17, 17A, 17B view image, 18, 19 area, 100 HMD system, 110, 110A, 110B, 110C, 110D HMD set, 120, 120A, 120B, 120C, HMD, 130, 130A, 130B, 130C monitor, 140 gaze sensor, 150 first camera, 160 second camera, 170, 170A, 170B microphone, 180, 180A, 180B speaker, 190 sensor, 200, 200A, 200B computer, 210, 210A, 210B, 210C, 210D, 610, 810, 810A processor, 220, 620, 820 memory, 230, 230A, 230B, 630, 830 storage, 240, 640, 840 input/output interface, 250, 650 , 850 communication interface, 260,660 bus, 300,300B controller, 300R right controller, 300L left controller, 310 grip, 320 frame, 330 top surface, 340,340,350,370,380 button, 360 infrared LED, 390 analog Stick, 410 HMD sensor, 420, 420A motion sensor, 430, 430A display, 510 control module, 520 rendering module, 530 memory module, 540 communication control module, 600 server, 1421, 8121 virtual object generation module, 1422, 8122 virtual camera Control Module, 1423 Operation Object Control Module, 1424, 8124 Avatar Object Control Module, 1425 Motion Detection Module, 1426 Collision Detection Module, 1427, 8127 Virtual Object Control Module, 1500 Game System, 810 Processor, 820 Memory, 830 Storage, 840 Input Output IF, 850 Communication IF, 870 Touch screen (display unit, operation unit), 860 Camera (operation unit), 880 Distance measuring sensor (Operation unit), 800 User terminal (information processing device ) 2100, 6100, 8100 control unit, 2200, 6200, 8200 storage unit 2101, 8101 operation reception unit, 2102, 8102 display control unit, 2103, 8103 UI control unit, 2104, 8104 animation generation unit, 2105, 8105 virtual space Control unit, 2106, 8106 virtual space control unit, 2107 reaction processing unit, 8107 progress information generation unit, 6101 progress support unit, 6102 sharing support unit, 231, 631, 831 game program, 132 game information, 133 user information, 151 input Unit (operation unit), 152 display unit, 200 server, 1010 object, 1020 controller (operation unit), 1030 storage medium

Claims (14)

A game program including a processor and a memory, which is executed by a computer as a viewing terminal for a viewer to view real-time distribution performed by the distributor,
The game program, in the processor,
A first avatar corresponding to the first distributor in a first virtual space shared between a first distribution terminal for the first distributor to perform real-time distribution and a plurality of viewing terminals including the viewing terminal. Arranging the object and the first virtual camera,
Displaying a first visual field image showing a visual field area of the first virtual camera on a display unit while the real-time distribution is performed;
For each of the states of the first game part that progresses while transitioning between a plurality of states, which is included in the game based on the game program, the game in one state is defined as the body movement of the first distributor and the viewer. And a step of performing a game process to be advanced based on the operation of
The game process in each state is
Receiving first motion information indicating a motion of the first avatar object, which is generated by the first distribution terminal by detecting a movement of the body of the first distributor, and
Updating the first field-of-view image to include the first avatar object performing a motion indicated by the first motion information in response to receiving the first motion information;
Responsive to receiving the first motion information, determining whether the motion indicated by the first motion information is a motion for transitioning the state,
And a step of transitioning the progress of the first game part from the current state to another state when the movement indicated by the first motion information is the movement for the transition. Arranging the first avatar object and the first virtual camera, further disposing a first object in the first virtual space,
In the determining step, when the motion indicated by the first motion information is a motion that causes the first avatar object to operate the first object in the first virtual space, the motion is determined to be the transition. The game program according to claim 1. Arranging the first avatar object and the first virtual camera, arranging an operation object representing the hand of the first avatar object as at least a part of the first avatar object,
In the determining step, when the motion indicated by the first motion information is a motion for bringing the operation object into contact with the first object in the first virtual space, it is determined that the motion is for the transition. The game program according to claim 2. The game process in each state is
A user interface (UI) object for receiving the operation of the viewer, wherein the UI object associated with the state before the transition is deleted from the display unit, and the UI object associated with the current state is displayed. The game program according to any one of claims 1 to 3, further comprising a step of displaying the first visual field image on the display unit. The game process in each state is
The second object associated with the state before transition is deleted from the first virtual space, and the second object associated with the current state, which has a different mode from the second object associated with the state before transition, The game program according to claim 1, further comprising arranging two objects in the first virtual space. In the step of arranging the first avatar object and the first virtual camera, a third object for displaying information indicating each state is further arranged in the first virtual space,
The game process in each state is
The game program according to claim 1, further comprising a step of changing information displayed on the third object from information indicating a state before transition to information indicating a current state. At least one of the game processes in each state is
Obtaining from the memory the degree of achievement stored in the memory in association with at least one state prior to transitioning to the current state;
The higher the degree of achievement obtained from the memory, the more advantageous the processing for the viewer in increasing the degree of achievement of the first condition associated with the current state;
Determining the degree of achievement of the first condition based on the movement of the body of the first distributor and the operation of the viewer;
The game program according to any one of claims 1 to 6, further comprising: storing the achievement level in the memory in association with a current state. The game program, in the processor,
Advancing a second game part included in the game, which is different from the first game part, in response to an operation by the viewer,
In the second game part, determining the degree of achievement of the second condition being achieved based on the operation of the viewer;
Storing the degree of achievement of the second condition in the memory,
At least one of the game processes in each of the states of the first game part is
Obtaining the degree of achievement of the second condition from the memory,
The higher the achievement level of the second condition is, the higher the achievement level of the first condition associated with the current state is, and the step of performing a process advantageous to the viewer is further included. 7. The game program according to any one of 7. At least one of the game processes in each of the states of the first game part is
Further arranging a plurality of choice objects in the first virtual space,
Displaying a plurality of first UI objects, which are associated with each of a part or all of the plurality of option objects and accept the operation of the viewer, on the display unit together with the first view image.
Determining whether the motion indicated by the first motion information is a motion for causing the first avatar object to select a part of the plurality of choice objects in the first virtual space,
Specifying the option object selected by the first avatar object when the motion indicated by the first motion information is the motion for selecting;
Performing a process of achieving at least a part of the first condition associated with the current state in response to the operation of the viewer with respect to the first UI object associated with the identified option object. ,
9. The game program according to claim 7, wherein the step of executing the advantageous process executes a process of increasing the number of the first UI objects displayed on the display unit in the step of displaying the first UI object. .. Of the game processing in each state of the first game part, a specific game processing is
In place of the step of determining whether or not it is a movement for making the transition, and the step of making the transition,
Determining whether a particular condition associated with the current state has been met,
Transitioning the game from the first game part to a third game part different from the first game part when the specific condition is satisfied,
The game program, in the processor,
In the third game part, among the plurality of viewing terminals, a plurality of viewing terminals satisfying the specific condition and a second distribution terminal for the second distributor to perform real-time distribution are shared. Arranging a second avatar object corresponding to the second distributor and a second virtual camera in a second virtual space different from the first virtual space,
A step of displaying a second visual field image showing a visual field area of the second virtual camera on the display unit in place of the first visual field image while the real-time distribution is performed in the third game part;
Receiving second motion information indicating a motion of the second avatar object, which is generated by the second distribution terminal by detecting the movement of the body of the second distributor, and
Updating the second visual field image so that the second avatar object performs a motion indicated by the second motion information in response to the reception of the second motion information. The game program according to claim 1, wherein the game program is executed. The game program, in the processor,
Acquiring first input information input by the viewer when the specific condition is not satisfied;
Transmitting the first input information to another viewing terminal that does not meet the specific condition among the plurality of viewing terminals in response to the acquisition of the first input information;
Receiving second input information input by a viewer of another viewing terminal whose specific condition is not satisfied;
The game program according to claim 10, further comprising: displaying the first input information and the second input information on the display unit together with the first visual field image. The game program, in the processor,
When the specific condition is not satisfied, payment of the consideration by the viewer, or a step of accepting an operation for instructing consumption of an item given to the viewer based on the payment of the consideration,
In response to the instructed operation, performing the fee charging process or the item consuming process, and further shifting the game from the first game part to the third game part. The game program according to claim 10 or 11. A method for a computer to execute a game program, the method comprising:
The computer includes a processor and memory,
A method by which the processor performs the steps of claim 1. An information processing device,
The information processing device,
A memory for storing the game program according to claim 1;
An information processing apparatus, comprising: a processor that controls the operation of the information processing apparatus by executing the game program.

Images