JP7366821B2 - Video display program - Google Patents

Description

The present invention relates to a video display program .

In recent years, an increasing number of users are using terminal devices such as smartphones to view videos distributed via the Internet. When viewing such a video on a terminal device, a gap (delay) occurs in the video played between the terminal devices due to a sudden or temporary failure in communication between the terminal devices. In order to prevent such discrepancies in videos between terminal devices, when a video corresponding to a certain chapter is being played, when a predetermined execution time has elapsed from the start of playback, the video corresponding to the chapter is There is a device that attempts synchronization between terminal devices by forcibly starting the playback of a video corresponding to the next chapter even if the video is in the middle (until it has finished) (Patent Document 1).

JP2019-75606A

However, in the distribution system described in Patent Document 1, a video ends abruptly in the middle of a terminal device that is delayed, and the next chapter video suddenly starts. For this reason, there is a risk that the user may feel that the delivery quality is poor or that the content of the delivery is strange, and there is room for improvement.

The present invention was devised in view of the above circumstances, and its purpose is to provide a video display program that can minimize the impression of distribution quality and prevent the distribution content from causing a sense of discomfort as much as possible. It is to provide.

According to an aspect of an embodiment disclosed in the present disclosure, there is provided a video display program executed on a terminal device including a processor, a memory, and a display unit, the video display program being configured to cause the processor to operate in real time according to the movements of a performer. A step of receiving live-distributed data for displaying an image of a virtual space including a character, and a step of displaying an image of the virtual space based on the received data, and the data is transmitted live. The step of displaying the first data corresponding to the end of the first part constituting the distribution and the second data corresponding to the start of the second part following the first part is based on the first data. When the first part is finished, the virtual space image of the first part is switched to a predetermined specific image, and when the second data is received while the specific image is being displayed, the specific image is displayed. Regardless of the display time, the specific video is switched to the second part of the virtual space video and displayed.

According to the present invention, it is possible to prevent as much as possible the impression of distribution quality from being degraded, and to prevent the distribution contents from causing a feeling of discomfort as much as possible.

FIG. 1 is a diagram illustrating an example of an overview of a system according to an embodiment. FIG. 2 is a block diagram illustrating an example of the hardware configuration of a user terminal according to an embodiment. FIG. 2 is a block diagram illustrating an example of the hardware configuration of a server according to an embodiment. 1 is a block diagram illustrating an example of a hardware configuration of a game play terminal according to an embodiment. FIG. FIG. 2 is a block diagram illustrating an example of the hardware configuration of a distribution terminal according to an embodiment. FIG. 2 is a block diagram illustrating an example of a functional configuration of a user terminal, a server, and an HMD set according to an embodiment. FIG. 2 is a block diagram illustrating an example of a functional configuration of a distribution terminal according to an embodiment. 3 is a flowchart illustrating a portion of processing performed at a user terminal and a game play terminal according to an embodiment. FIG. 2 is a diagram illustrating an example of a virtual space provided to a player and a visual field image visually recognized by the player, according to an embodiment. FIG. 2 is a diagram illustrating an example of a virtual space provided to a user of a user terminal and a visual field image viewed by the user, according to an embodiment. FIG. 7 is a diagram illustrating another example of a visual field image visually recognized by a user of a user terminal. FIG. 7 is a diagram illustrating yet another example of a visual field image visually recognized by a user of a user terminal. 3 is a flowchart illustrating a portion of processing performed at a game play terminal according to an embodiment. 3 is a flowchart illustrating a portion of processing executed in a user terminal according to an embodiment. 3 is a flowchart illustrating a portion of processing executed in a server according to an embodiment. FIG. 3 is a diagram illustrating a specific example of a list of users who have participated in a game, according to an embodiment. It is a flowchart showing a part of processing performed in a distribution terminal according to a certain embodiment. FIG. 3 is a diagram showing a specific example of a screen displayed on a distribution terminal according to an embodiment. FIG. 7 is a diagram showing another specific example of a screen displayed on a distribution terminal according to an embodiment. FIG. 3 is a diagram illustrating a specific example of audio input by a player, according to an embodiment. FIG. 7 is a diagram illustrating still another specific example of a screen displayed on a distribution terminal and an overview of distribution of operation instruction data according to an embodiment. FIG. 7 is a diagram illustrating another specific example of audio input by a player, according to an embodiment. FIG. 7 is a diagram illustrating still another specific example of a screen displayed on a distribution terminal and an overview of distribution of operation instruction data according to an embodiment. FIG. 3 is a diagram illustrating an overview of transmission of game progress information from a game play terminal to a user terminal according to an embodiment. 3 is a flowchart illustrating a portion of processing executed in a user terminal according to an embodiment. FIG. 3 is a diagram showing a specific example of video playback. FIG. 7 is a diagram showing another specific example of video playback. FIG. 3 is a diagram for explaining a display example on a touch screen of a user terminal during live distribution. 3 is a timing chart for explaining a shift in video between a plurality of user terminals. (A) is a diagram for explaining an example of a predetermined schedule regarding the progress of the game from the start of one scene (for example, the first round) of the rock-paper-scissors game part to the notification of the result, and (B) to FIG. 30E is a diagram for explaining a display example on the touch screen of the user terminal when viewing the live broadcast from the middle. (A) is a diagram showing an example of a configuration of game progress information, and (B) is a diagram showing an example of input/output timing of audio data, motion data, and virtual space control data. (A) is a diagram showing another example of the input/output timing of audio data, motion data, and virtual space control data, and (B) is another example of the input/output timing of audio data, motion data, and virtual space control data. FIG.

A system according to the present disclosure is a system for providing a game to multiple users. The system will be described below with reference to the drawings. It should be noted that the present invention is not limited to these examples, but is indicated by the scope of the claims, and it is intended that all changes within the meaning and range equivalent to the scope of the claims are included in the present invention. Ru. In the following description, the same elements are given the same reference numerals in the description of the drawings, and overlapping description will not be repeated.

<Operation overview of system 1>
FIG. 1 is a diagram showing an overview of a system 1 according to this embodiment. The system 1 includes a plurality of user terminals 100 (computers), a server 200, a game play terminal 300 (external device, second external device), and a distribution terminal 400 (external, first external device). Note that although FIG. 1 shows user terminals 100A to 100C, in other words, three user terminals 100, as an example of the plurality of user terminals 100, the number of user terminals 100 is not limited to the illustrated example. . Furthermore, in this embodiment, when there is no need to distinguish between the user terminals 100A to 100C, they are referred to as "user terminal 100." User terminal 100, game play terminal 300, and distribution terminal 400 are connected to server 200 via network 2. The network 2 includes the Internet and various mobile communication systems constructed by wireless base stations (not shown). Examples of this mobile communication system include so-called 3G, 4G mobile communication systems, LTE (Long Term Evolution), and wireless networks (for example, Wi-Fi (registered trademark)) that can be connected to the Internet through predetermined access points. It will be done.

(Game overview)
In this embodiment, a game mainly played by a user of the game play terminal 300 will be described as an example of a game provided by the system 1 (hereinafter referred to as the main game). Hereinafter, the user of the game play terminal 300 will be referred to as a "player". For example, the player (performer) advances the game by operating a character that appears in the game. Furthermore, in this game, the user of the user terminal 100 plays a role of supporting the player's progress in the game. Details of this game will be described later. Note that the game provided by the system 1 may be any game in which a plurality of users participate, and is not limited to this example.

(Game play terminal 300)
Game play terminal 300 advances the game in response to input operations by the player. Furthermore, the game play terminal 300 sequentially delivers information generated by the player's game play (hereinafter referred to as game progress information) to the server 200 in real time.

(Server 200)
The server 200 transmits game progress information (second data) received in real time from the game play terminal 300 to the user terminal 100. Further, the server 200 mediates transmission and reception of various information between the user terminal 100, the game play terminal 300, and the distribution terminal 400.

(Distribution terminal 400)
The distribution terminal 400 generates motion instruction data (first data) in response to an input operation by the user of the distribution terminal 400, and distributes the motion instruction data to the user terminal 100 via the server 200. The motion instruction data is data for playing a video on the user terminal 100, and specifically, it is data for causing a character appearing in the video to move.

In this embodiment, as an example, the user of the distribution terminal 400 is a player of the present game. Further, as an example, the video played on the user terminal 100 based on the motion instruction data is a video in which a character operated by a player in a game moves. "Action" refers to moving at least part of the character's body, and includes speech. Therefore, the motion instruction data according to the present embodiment includes, for example, audio data for making the character speak and motion data for moving the character's body.

As an example, the action instruction data is transmitted to the user terminal 100 after the game ends. Details of the motion instruction data and the moving image played based on the motion instruction data will be described later.

(User terminal 100)
The user terminal 100 receives game progress information in real time, and uses the information to generate and display a game screen. In other words, the user terminal 100 reproduces the game screen of the game being played by the player by real-time rendering. This allows the user of the user terminal 100 to view the same game screen that the player is viewing while playing the game, at approximately the same timing as the player.

Furthermore, the user terminal 100 generates information for supporting the player's progress in the game in response to input operations by the user, and transmits the information to the game play terminal 300 via the server 200. Details of this information will be described later.

Further, the user terminal 100 receives motion instruction data from the distribution terminal 400, uses the motion instruction data to generate and reproduce a moving image (video). In other words, the user terminal 100 renders and reproduces the motion instruction data.

<Hardware configuration of system 1>
FIG. 2 is a diagram showing the hardware configuration of the user terminal 100. FIG. 3 is a diagram showing the hardware configuration of the server 200. FIG. 4 is a diagram showing the hardware configuration of the game play terminal 300. FIG. 5 is a diagram showing the hardware configuration of distribution terminal 400.

(User terminal 100)
In this embodiment, an example in which the user terminal 100 is implemented as a smartphone will be described, but the user terminal 100 is not limited to a smartphone. For example, the user terminal 100 may be implemented as a feature phone, a tablet computer, a laptop computer (so-called notebook computer), a desktop computer, or the like. Further, the user terminal 100 may be a game device suitable for playing games.

As shown in FIG. 2, the user terminal 100 includes a processor 10, a memory 11, a storage 12, a communication interface (IF) 13, an input/output IF 14, a touch screen 15 (display section), a camera 17, and a measurement device. A distance sensor 18 is provided. These components included in the user terminal 100 are electrically connected to each other by a communication bus. Note that, instead of or in addition to the touch screen 15, the user terminal 100 may include an input/output IF 14 to which a display (display unit) configured separately from the user terminal 100 body can be connected.

Further, as shown in FIG. 2, the user terminal 100 may be configured to be able to communicate with one or more controllers 1020. The controller 1020 establishes communication with the user terminal 100, for example, according to a communication standard such as Bluetooth (registered trademark). The controller 1020 may have one or more buttons or the like, and transmits an output value to the user terminal 100 based on a user's input operation on the button or the like. Further, the controller 1020 may include various sensors such as an acceleration sensor and an angular velocity sensor, and transmits output values of the various sensors to the user terminal 100.

Note that instead of or in addition to the user terminal 100 including the camera 17 and the distance measurement sensor 18, the controller 1020 may include the camera 17 and the distance measurement sensor 18.

It is desirable that the user terminal 100 allows the user using the controller 1020 to input user identification information such as the user's name or login ID via the controller 1020, for example, when starting the game. This makes it possible for the user terminal 100 to link the controller 1020 and the user, and identify which user the received output value belongs to based on the source (controller 1020) of the received output value. be able to.

When the user terminal 100 communicates with a plurality of controllers 1020, by each user holding each controller 1020, the user terminal 100 can communicate with the one user terminal 100 without communicating with other devices such as the server 200 via the network 2. Multiplayer can be realized. In addition, each user terminal 100 communicates with each other using a wireless standard such as a wireless LAN (Local Area Network) standard (communicates without going through the server 200), thereby realizing local multiplayer with multiple user terminals 100. You can also. When the above-described multiplayer is locally implemented using one user terminal 100, the user terminal 100 may further include at least some of the various functions provided by the server 200, which will be described later. Moreover, when the above-mentioned multiplayer is realized locally by a plurality of user terminals 100, the plurality of user terminals 100 may have various functions provided in the server 200, which will be described later, in a distributed manner.

Note that even when the above-described multiplayer is implemented locally, the user terminal 100 may communicate with the server 200. For example, information indicating a play result such as a score or a win or loss in a certain game and user identification information may be associated with each other and transmitted to the server 200.

Further, the controller 1020 may be configured to be detachable from the user terminal 100. In this case, a coupling portion with the controller 1020 may be provided on at least one surface of the casing of the user terminal 100. When the user terminal 100 and the controller 1020 are connected via a wire via the coupling section, the user terminal 100 and the controller 1020 transmit and receive signals via the wire.

As shown in FIG. 2, the user terminal 100 may accept attachment of a storage medium 1030 such as an external memory card via the input/output IF 14. This allows the user terminal 100 to read programs and data recorded on the storage medium 1030. The program recorded on the storage medium 1030 is, for example, a game program.

The user terminal 100 may store a game program obtained by communicating with an external device such as the server 200 in the memory 11 of the user terminal 100, or may store a game program obtained by reading from the storage medium 1030 into the memory 11. It may be stored in

As explained above, the user terminal 100 includes the communication IF 13, the input/output IF 14, the touch screen 15, the camera 17, and the distance measurement sensor 18 as an example of a mechanism for inputting information to the user terminal 100. Each of the above-mentioned units serving as an input mechanism can be regarded as an operation unit configured to accept user input operations.

For example, when the operation unit is configured with at least one of the camera 17 and the distance measurement sensor 18, the operation unit detects an object 1010 near the user terminal 100 and performs an input operation based on the detection result of the object. Identify. As an example, a user's hand, a marker with a predetermined shape, or the like is detected as the object 1010, and an input operation is specified based on the color, shape, movement, type, etc. of the object 1010 obtained as a detection result. Ru. More specifically, when the user's hand is detected from the captured image of the camera 17, the user terminal 100 converts the gesture (a series of movements of the user's hand) detected based on the captured image into the user's input operation. be identified and accepted as such. Note that the photographed image may be a still image or a moving image.

Alternatively, when the operation unit is configured with the touch screen 15, the user terminal 100 identifies and accepts the user's operation performed on the input unit 151 of the touch screen 15 as the user's input operation. Alternatively, when the operation unit is configured with the communication IF 13, the user terminal 100 identifies and receives a signal (for example, an output value) transmitted from the controller 1020 as a user's input operation. Alternatively, when the operation unit is configured with the input/output IF 14, a signal output from an input device (not shown) connected to the input/output IF 14 and different from the controller 1020 is identified and accepted as the user's input operation.

(Server 200)
Server 200 may be, by way of example, a general purpose computer such as a workstation or a personal computer. The server 200 includes a processor 20, a memory 21, a storage 22, a communication IF 23, and an input/output IF 24. These components included in the server 200 are electrically connected to each other by a communication bus.

(Game play terminal 300)
Game play terminal 300 may be, for example, a general-purpose computer such as a personal computer. The game play terminal 300 includes a processor 30, a memory 31, a storage 32, a communication IF 33, and an input/output IF 34. These components included in the game play terminal 300 are electrically connected to each other by a communication bus.

As shown in FIG. 4, the game play terminal 300 according to the present embodiment is included in an HMD (Head Mounted Display) set 1000, as an example. That is, it can be expressed that the HMD set 1000 is included in the system 1, and it can also be expressed that the player plays a game using the HMD set 1000. Note that the device for the player to play the game is not limited to the HMD set 1000. As an example, the device may be any device that allows the player to experience the game virtually. Further, the device may be realized as a smartphone, a feature phone, a tablet computer, a laptop computer (so-called notebook computer), a desktop computer, or the like. Further, the device may be a game device suitable for playing games.

HMD set 1000 includes, in addition to game play terminal 300, HMD 500, HMD sensor 510, motion sensor 520, display 530, and controller 540. HMD 500 includes a monitor 51, a gaze sensor 52, a first camera 53, a second camera 54, a microphone 55, and a speaker 56. Controller 540 may include motion sensor 520.

The HMD 500 is attached to the player's head and can provide the player with a virtual space during operation. More specifically, HMD 500 displays a right eye image and a left eye image on monitor 51, respectively. When each of the player's eyes views each image, the player can recognize the image as a three-dimensional image based on the parallax between the eyes. The HMD 500 may include either a so-called head-mounted display including a monitor, or a head-mounted device to which a smartphone or other terminal having a monitor can be attached.

The monitor 51 is realized, for example, as a non-transmissive display device. In one aspect, the monitor 51 is arranged on the main body of the HMD 500 so as to be located in front of both eyes of the player. Therefore, when the player views the three-dimensional image displayed on the monitor 51, he or she can immerse himself in the virtual space. In one aspect, the virtual space includes, for example, a background, an object that can be operated by the player, and an image of a menu that can be selected by the player. In one aspect, the monitor 51 can be implemented as a liquid crystal monitor or an organic EL (Electro Luminescence) monitor included in a so-called smartphone or other information display terminal.

In another aspect, the monitor 51 can be realized as a transmissive display device. In this case, the HMD 500 is not a closed type that covers the player's eyes as shown in FIG. 1, but may be an open type like a pair of glasses. The transmissive monitor 51 may be temporarily configured as a non-transmissive display device by adjusting its transmittance. The monitor 51 may include a configuration that simultaneously displays a part of the image forming the virtual space and the real space. For example, the monitor 51 may display an image of the real space taken with a camera mounted on the HMD 500, or may make the real space visible by setting the transmittance of a part to be high.

In one aspect, monitor 51 may include a sub-monitor for displaying an image for the right eye and a sub-monitor for displaying an image for the left eye. In another aspect, the monitor 51 may be configured to display the right eye image and the left eye image as one unit. In this case, monitor 51 includes a high speed shutter. The high-speed shutter operates so that images for the right eye and images for the left eye can be alternately displayed so that the images are only perceived by one eye.

In one aspect, HMD 500 includes a plurality of light sources (not shown). Each light source is realized by, for example, an LED (Light Emitting Diode) that emits infrared rays. HMD sensor 510 has a position tracking function for detecting movement of HMD 500. More specifically, HMD sensor 510 reads a plurality of infrared rays emitted by HMD 500 and detects the position and tilt of HMD 500 in real space.

In another aspect, HMD sensor 510 may be realized by a camera. In this case, the HMD sensor 510 can detect the position and tilt of the HMD 500 by performing image analysis processing using image information of the HMD 500 output from the camera.

In another aspect, HMD 500 may include a sensor (not shown) as a position detector instead of or in addition to HMD sensor 510. HMD 500 can detect its own position and inclination using the sensor. For example, when the sensor is an angular velocity sensor, a geomagnetic sensor, or an acceleration sensor, HMD 500 can detect its own position and inclination using any of these sensors instead of HMD sensor 510. As an example, when the sensor provided in HMD 500 is an angular velocity sensor, the angular velocity sensor detects the angular velocity around three axes of HMD 500 in real space over time. The HMD 500 calculates temporal changes in angles around the three axes of the HMD 500 based on each angular velocity, and further calculates the inclination of the HMD 500 based on the temporal changes in angles.

The gaze sensor 52 detects the direction in which the player's right and left eyes are directed. In other words, the gaze sensor 52 detects the player's line of sight. Detection of the direction of the line of sight is realized, for example, by a known eye tracking function. The gaze sensor 52 is realized by a sensor having the eye tracking function. In one aspect, gaze sensor 52 preferably includes a right eye sensor and a left eye sensor. The gaze sensor 52 may be, for example, a sensor that irradiates the player's right and left eyes with infrared light and detects the rotation angle of each eyeball by receiving reflected light from the cornea and iris of the irradiated light. The gaze sensor 52 can detect the player's line of sight based on each detected rotation angle.

The first camera 53 photographs the lower part of the player's face. More specifically, the first camera 53 photographs the player's nose, mouth, and the like. The second camera 54 photographs the player's eyes, eyebrows, and the like. The player-side casing of the HMD 500 is defined as the inside of the HMD 500, and the casing of the HMD 500 on the opposite side from the player is defined as the outside of the HMD 500. In one aspect, the first camera 53 may be placed outside the HMD 500, and the second camera 54 may be placed inside the HMD 500. Images generated by the first camera 53 and the second camera 54 are input to the game play terminal 300. In another aspect, the first camera 53 and the second camera 54 may be implemented as one camera, and the player's face may be photographed with this one camera.

The microphone 55 converts the player's speech into an audio signal (electrical signal) and outputs it to the game play terminal 300. The speaker 56 converts the audio signal into audio and outputs it to the player. In another aspect, HMD 500 may include earphones instead of speaker 56.

Controller 540 is connected to game play terminal 300 by wire or wirelessly. Controller 540 accepts command input from the player to game play terminal 300 . In one aspect, controller 540 is configured to be graspable by the player. In another aspect, the controller 540 is configured to be attachable to the player's body or part of his clothing. In yet another aspect, controller 540 may be configured to output at least one of vibration, sound, and light based on a signal transmitted from game play terminal 300. In yet another aspect, the controller 540 receives operations from the player for controlling the position and movement of objects placed in the virtual space.

In some aspects, controller 540 includes multiple light sources. Each light source is realized by, for example, an LED that emits infrared rays. HMD sensor 510 has a position tracking function. In this case, the HMD sensor 510 reads a plurality of infrared rays emitted by the controller 540 and detects the position and tilt of the controller 540 in real space. In another aspect, HMD sensor 510 may be realized by a camera. In this case, the HMD sensor 510 can detect the position and tilt of the controller 540 by executing image analysis processing using image information of the controller 540 output from the camera.

In one aspect, the motion sensor 520 is attached to the player's hand and detects the movement of the player's hand. For example, the motion sensor 520 detects the rotation speed, number of rotations, etc. of the hand. The detected signal is sent to game play terminal 300. Motion sensor 520 is provided in controller 540, for example. In one aspect, the motion sensor 520 is provided, for example, in a controller 540 that is configured to be held by the player. In another aspect, for safety in real space, the controller 540 is attached to the player's hand, such as a glove, so that it cannot easily fly away. In yet another aspect, a sensor not attached to the player may detect the player's hand movements. For example, a signal from a camera that photographs the player may be input to the game play terminal 300 as a signal representing the player's actions. The motion sensor 520 and the game play terminal 300 are, for example, connected to each other wirelessly. In the case of wireless communication, the communication form is not particularly limited, and for example, Bluetooth or other known communication methods may be used.

Display 530 displays an image similar to the image displayed on monitor 51. This allows users other than the player wearing the HMD 500 to view the same images as the player. The image displayed on the display 530 does not need to be a three-dimensional image, and may be an image for the right eye or an image for the left eye. Examples of the display 530 include a liquid crystal display and an organic EL monitor.

The game play terminal 300 moves the character to be operated by the player based on various information acquired from each part of the HMD 500, the controller 540, and the motion sensor 520, and advances the game. "Actions" here include moving parts of the body, changing posture, changing facial expressions, moving, speaking, touching and moving objects placed in virtual space, and the character's movements. This includes the use of grasping weapons, tools, etc. That is, in this game, when the player moves each part of his body, the character also moves each part of his body in the same way as the player. Furthermore, in this game, the character speaks what the player has said. In other words, in this game, the character is an avatar object that acts as the player's alter ego. As an example, at least some of the character's actions may be performed by input to the controller 540 by the player.

In this embodiment, the motion sensors 520 are attached to, for example, both hands of the player, both feet of the player, the waist of the player, and the head of the player. The motion sensors 520 attached to both hands of the player may be provided on the controller 540, as described above. Moreover, the motion sensor 520 attached to the player's head may be provided in the HMD 500. Motion sensor 520 may also be attached to both elbows or knees of the user. By increasing the number of motion sensors 520 attached to the player, the player's movements can be more accurately reflected on the character.
Further, instead of attaching the motion sensors 520 to various parts of the body, the player may wear a suit to which one or more motion sensors 520 are attached. In other words, the motion capture method is not limited to the example using the motion sensor 520.

(Distribution terminal 400)
Distribution terminal 400 may be a mobile terminal such as a smartphone, a PDA (Personal Digital Assistant), or a tablet computer. Furthermore, the distribution terminal 400 may be a so-called stationary terminal such as a desktop computer.

As shown in FIG. 5, the distribution terminal 400 includes a processor 40, a memory 41, a storage 42, a communication IF 43, an input/output IF 44, and a touch screen 45. Note that, instead of or in addition to the touch screen 45, the distribution terminal 400 may include an input/output IF 44 to which a display (display unit) configured separately from the distribution terminal 400 main body can be connected.

Controller 1021 may include physical input mechanisms such as one or more buttons, levers, sticks, wheels, etc. The controller 1021 transmits to the distribution terminal 400 an output value based on an input operation input to the input mechanism by the operator of the distribution terminal 400 (player in this embodiment). Further, the controller 1021 may include various sensors such as an acceleration sensor and an angular velocity sensor, and may transmit output values of the various sensors to the distribution terminal 400. The above output value is received by the distribution terminal 400 via the communication IF 43.

Distribution terminal 400 may include a camera and a distance measurement sensor (both not shown). In place of or in addition to the distribution terminal 400, the controller 1021 may include a camera and a distance measuring sensor.

As described above, the distribution terminal 400 includes the communication IF 43, the input/output IF 44, and the touch screen 45 as an example of a mechanism for inputting information to the distribution terminal 400. Each of the above-mentioned units serving as an input mechanism can be regarded as an operation unit configured to accept user input operations.

When the operation unit is configured with the touch screen 45, the distribution terminal 400 identifies and accepts the user's operation performed on the input unit 451 of the touch screen 45 as the user's input operation. Alternatively, when the operation unit is configured with the communication IF 43, the distribution terminal 400 identifies and receives a signal (for example, an output value) transmitted from the controller 1021 as a user's input operation. Alternatively, when the operation unit is configured with the input/output IF 44, the distribution terminal 400 identifies and receives a signal output from an input device (not shown) connected to the input/output IF 44 as the user's input operation.

<Hardware components of each device>
Processors 10, 20, 30, and 40 control the overall operation of user terminal 100, server 200, game play terminal 300, and distribution terminal 400, respectively. The processors 10, 20, 30, and 40 include a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a GPU (Graphics Processing Unit). Processors 10, 20, 30, and 40 read programs from storages 12, 22, 32, and 42, respectively, which will be described later. Then, the processors 10, 20, 30, and 40 expand the read programs into memories 11, 21, 31, and 41, which will be described later. The processors 10, 20, and 30 execute the expanded program.

Memories 11, 21, 31, and 41 are main storage devices. The memories 11, 21, 31, and 41 are composed of storage devices such as ROM (Read Only Memory) and RAM (Random Access Memory). The memory 11 provides a work area for the processor 10 by temporarily storing programs and various data that the processor 10 reads from a storage 12, which will be described later. The memory 11 also temporarily stores various data generated while the processor 10 is operating according to a program. The memory 21 provides a work area for the processor 20 by temporarily storing various programs and data that the processor 20 reads from a storage 22, which will be described later. The memory 21 also temporarily stores various data generated while the processor 20 is operating according to a program. The memory 31 provides a work area for the processor 30 by temporarily storing various programs and data that the processor 30 reads from a storage 32, which will be described later. The memory 31 also temporarily stores various data generated while the processor 30 is operating according to a program. The memory 41 provides a work area for the processor 40 by temporarily storing programs and various data that the processor 40 reads from a storage 42, which will be described later. The memory 41 also temporarily stores various data generated while the processor 40 is operating according to a program.

In this embodiment, the program executed by the processors 10 and 30 may be a game program of the present game. In this embodiment, the program executed by the processor 40 may be a distribution program for realizing distribution of operation instruction data. Further, the processor 10 may further execute a viewing program for realizing playback of a moving image.

In this embodiment, the program executed by the processor 20 may be at least one of the above-described game program, distribution program, and viewing program. The processor 20 executes at least one of a game program, a distribution program, and a viewing program in response to a request from at least one of the user terminal 100, the game play terminal 300, and the distribution terminal 400. Note that the distribution program and the viewing program may be executed in parallel.

That is, the game program may be a program that realizes a game through cooperation between the user terminal 100, the server 200, and the game play terminal 300. The distribution program may be a program that realizes distribution of operation instruction data through cooperation between the server 200 and the distribution terminal 400. The viewing program may be a program that realizes video playback through cooperation between the user terminal 100 and the server 200.

The storages 12, 22, 32, and 42 are auxiliary storage devices. The storages 12, 22, 32, and 42 are composed of storage devices such as flash memory or HDD (Hard Disk Drive). The storages 12 and 32 store, for example, various data related to games. The storage 42 stores various data related to distribution of operation instruction data. Further, the storage 12 stores various data related to playback of moving images. The storage 22 may store at least a portion of various data related to games, distribution of motion instruction data, and playback of videos.

Communication IFs 13, 23, 33, and 43 control transmission and reception of various data in user terminal 100, server 200, game play terminal 300, and distribution terminal 400, respectively. The communication IFs 13, 23, 33, and 43 are capable of, for example, communication via a wireless LAN (Local Area Network), wired LAN, wireless LAN, Internet communication via a mobile phone network, and communication using short-range wireless communication. Control.

The input/output IFs 14, 24, 34, and 44 are interfaces through which the user terminal 100, server 200, game play terminal 300, and distribution terminal 400 receive data input and output data, respectively. The input/output IFs 14, 24, 34, and 44 may input/output data via a USB (Universal Serial Bus) or the like. The input/output IFs 14, 24, 34, 44 may include physical buttons, cameras, microphones, speakers, mice, keyboards, displays, sticks, levers, and the like. Furthermore, the input/output IFs 14, 24, 34, and 44 may include connections for transmitting and receiving data with peripheral devices.

The touch screen 15 is an electronic component that combines an input section 151 and a display section 152 (display). The touch screen 45 is an electronic component that combines an input section 451 and a display section 452. The input units 151 and 451 are, for example, touch-sensitive devices, and are configured by, for example, touch pads. The display units 152 and 452 are configured by, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display.

The input units 151 and 451 detect the position where a user's operation (mainly physical contact operations such as touch operation, slide operation, swipe operation, and tap operation) is input on the input screen, and generate information indicating the position. It has a function to transmit as an input signal. The input units 151 and 451 may include a touch sensing unit (not shown). The touch sensing section may employ any method such as a capacitance method or a resistive film method.

Although not illustrated, the user terminal 100 and the distribution terminal 400 may each include one or more sensors for identifying the holding posture of the user terminal 100 and the distribution terminal 400. This sensor may be, for example, an acceleration sensor or an angular velocity sensor.

When the user terminal 100 and the distribution terminal 400 are equipped with sensors, the processors 10 and 40 respectively identify the holding postures of the user terminal 100 and the distribution terminal 400 from the outputs of the sensors, and perform processing according to the holding postures. It also becomes possible. For example, when the user terminal 100 and the distribution terminal 400 are held in portrait orientation, the processors 10 and 40 may display portrait images on the display units 152 and 452, respectively. On the other hand, when the user terminal 100 and the distribution terminal 400 are held horizontally, a horizontal screen display may be used in which a horizontally long image is displayed on the display unit. In this way, processors 10 and 40 may be able to switch between vertical screen display and horizontal screen display depending on the holding posture of user terminal 100 and distribution terminal 400, respectively.

<Functional configuration of system 1>
FIG. 6 is a block diagram showing the functional configuration of the user terminal 100, server 200, and HMD set 1000 included in the system 1. FIG. 7 is a block diagram showing the functional configuration of distribution terminal 400 shown in FIG. 6.

The user terminal 100 has a function as an input device that receives input operations from a user, and a function as an output device that outputs images and sounds of the game. The user terminal 100 functions as a control unit 110 and a storage unit 120 through cooperation of the processor 10, memory 11, storage 12, communication IF 13, input/output IF 14, touch screen 15, and the like.

The server 200 has a function of mediating the transmission and reception of various information between the user terminal 100, the HMD set 1000, and the distribution terminal 400. The server 200 functions as a control unit 210 and a storage unit 220 through cooperation of the processor 20, memory 21, storage 22, communication IF 23, input/output IF 24, and the like.

The HMD set 1000 (game play terminal 300) functions as an input device that receives input operations from the player, functions as an output device that outputs game images and sounds, and transmits game progress information to the user via the server 200. It has a function of transmitting to the terminal 100 in real time. The HMD set 1000 controls the control unit 310 by the cooperation of the processor 30, memory 31, storage 32, communication IF 33, input/output IF 34 of the game play terminal 300, as well as the HMD 500, HMD sensor 510, motion sensor 520, controller 540, etc. and functions as a storage unit 320.

The distribution terminal 400 has a function of generating action instruction data and transmitting the action instruction data to the user terminal 100 via the server 200. Distribution terminal 400 functions as control section 410 and storage section 420 through cooperation of processor 40, memory 41, storage 42, communication IF 43, input/output IF 44, touch screen 45, and the like.

(Data stored in the storage unit of each device)
The storage unit 120 stores a game program 131 (program), game information 132, and user information 133. Storage unit 220 stores game program 231, game information 232, user information 233, and user list 234. Storage unit 320 stores game program 331, game information 332, and user information 333. The storage unit 420 stores a user list 421, a motion list 422, and a distribution program 423 (program, second program).

Game programs 131, 231, and 331 are game programs executed by user terminal 100, server 200, and HMD set 1000, respectively. This game is realized by each device working together based on the game programs 131, 231, and 331. Note that the game programs 131 and 331 may be stored in the storage unit 220 and downloaded to the user terminal 100 and the HMD set 1000, respectively. In this embodiment, it is assumed that the user terminal 100 renders data received from the distribution terminal 400 based on the game program 131, and plays back a video. In other words, the game program 131 is also a program for reproducing a video using video instruction data distributed from the distribution terminal 400. The program for playing the video may be different from the game program 131. In this case, the storage unit 120 stores a program for playing the moving image separately from the game program 131.

The game information 132, 232, and 332 are data that the user terminal 100, the server 200, and the HMD set 1000 refer to when executing the game program, respectively. The user information 133, 233, and 333 is data related to the account of the user of the user terminal 100. The game information 232 is the game information 132 of each user terminal 100 and the game information 332 of the HMD set 1000. The user information 233 is the player's user information included in the user information 133 of each user terminal 100 and the user information 333. The user information 333 is the user information 133 of each user terminal 100 and the user information of the player.

The user list 234 and the user list 421 are lists of users who have participated in the game. The user list 234 and the user list 421 may include a list of users who participated in the most recent game play by the player, as well as a list of users who participated in each game play before the game play. The motion list 422 is a list of a plurality of motion data created in advance. The motion list 422 is, for example, a list in which motion data is associated with each piece of information that identifies each motion (eg, motion name). The distribution program 423 is a program for realizing distribution of operation instruction data for playing a moving image on the user terminal 100 to the user terminal 100.

(Functional configuration of server 200)
The control unit 210 centrally controls the server 200 by executing the game program 231 stored in the storage unit 220. For example, the control unit 210 mediates transmission and reception of various information between the user terminal 100, the HMD set 1000, and the distribution terminal 400.

The control unit 210 functions as a communication intermediary unit 211, a log generation unit 212, and a list generation unit 213 according to the description of the game program 231. The control unit 210 can also function as other functional blocks (not shown) in order to mediate the transmission and reception of various information related to game play and distribution of action instruction data, and to support the progress of the game.

The communication intermediary unit 211 mediates the transmission and reception of various information between the user terminal 100, the HMD set 1000, and the distribution terminal 400. For example, the communication intermediary unit 211 transmits game progress information received from the HMD set 1000 to the user terminal 100. The game progress information includes data indicating the movements of the character operated by the player, parameters of the character, information on items and weapons possessed by the character, enemy characters, and the like. The server 200 transmits game progress information to the user terminals 100 of all users participating in the game. In other words, the server 200 transmits common game progress information to the user terminals 100 of all users participating in the game. As a result, the game progresses in the same manner as in the HMD set 1000 on each of the user terminals 100 of all users participating in the game.

Further, for example, the communication intermediary unit 211 transmits information received from one of the user terminals 100 and used to support the progress of the game by the player to the other user terminals 100 and the HMD set 1000. The information may be, for example, item information indicating an item to be provided to the player (character), which is an item for the player to progress through the game advantageously. The item information includes information indicating the user who provided the item (user name, user ID, etc.). Further, the communication intermediary unit 211 may mediate the distribution of operation instruction data from the distribution terminal 400 to the user terminal 100.

The log generation unit 212 generates a game progress log based on the game progress information received from the HMD set 1000. The list generation unit 213 generates a user list 234 after the game play ends. Although details will be described later, each user in the user list 234 is associated with a tag indicating the content of the support provided to the player by that user. The list generation unit 213 generates a tag based on the game progress log generated by the log generation unit 212, and associates the tag with the corresponding user. Note that the list generation unit 213 may associate, as a tag, the content of support for the player provided by each user, which is input by a game operator or the like using a terminal device such as a personal computer, with the corresponding user. . This makes the details of the support provided by each user more detailed. Note that when the user participates in the game, the user terminal 100 transmits information indicating the user to the server 200 based on the user's operation. For example, the user terminal 100 transmits the user ID input by the user to the server 200. That is, the server 200 holds information indicating each user for all users participating in the game. The list generation unit 213 may generate the user list 234 using this information.

(Functional configuration of HMD set 1000)
The control unit 310 comprehensively controls the HMD set 1000 by executing the game program 331 stored in the storage unit 320. For example, the control unit 310 advances the game according to the game program 331 and the player's operations. Further, while the game is progressing, the control unit 310 communicates with the server 200 to send and receive information as necessary. The control unit 310 may directly transmit and receive information to and from the user terminal 100 without going through the server 200.

The control unit 310 includes an operation reception unit 311, a display control unit 312, a UI control unit 313, an animation generation unit 314, a game progression unit 315, a virtual space control unit 316, and a reaction processing unit 317 according to the description of the game program 331. functions as The control unit 310 can also function as other functional blocks (not shown) in order to control characters appearing in the game depending on the nature of the game being executed.

The operation reception unit 311 detects and receives input operations from the player. The operation reception unit 311 receives signals input from the HMD 500, motion sensor 520, controller 540, etc., determines what input operation has been performed, and outputs the results to each element of the control unit 310.

The UI control unit 313 controls a user interface (hereinafter referred to as UI) image displayed on the monitor 51, display 530, and the like. The UI image is a tool for the player to input information necessary for the progress of the game to the HMD set 1000, or a tool for obtaining information outputted from the HMD set 1000 during the progress of the game. UI images include, but are not limited to, icons, buttons, lists, menu screens, etc., for example.

The animation generation unit 314 generates animations showing motions of various objects based on control modes of the various objects. For example, the animation generation unit 314 may generate an animation that shows an object (for example, a player's avatar object) moving, moving its mouth, or changing its expression as if it were there. .

The game progression unit 315 advances the game based on the game program 331, input operations by the player, and actions of the avatar object in response to the input operations. For example, the game progression unit 315 performs predetermined game processing when the avatar object performs a predetermined action. Further, for example, the game progression unit 315 may receive information representing a user's operation on the user terminal 100, and perform game processing based on the user's operation. Further, the game progress unit 315 generates game progress information according to the progress of the game, and transmits it to the server 200. The game progress information is transmitted to the user terminal 100 via the server 200. Thereby, the progress of the game on the HMD set 1000 is shared on the user terminal 100. In other words, the progress of the game on HMD set 1000 and the progress of the game on user terminal 100 are synchronized.

The virtual space control unit 316 performs various controls regarding the virtual space provided to the player in accordance with the progress of the game. As an example, the virtual space control unit 316 generates various objects and places them in the virtual space. Further, the virtual space control unit 316 arranges a virtual camera in the virtual space. Further, the virtual space control unit 316 operates various objects arranged in the virtual space according to the progress of the game. Further, the virtual space control unit 316 controls the position and tilt of a virtual camera placed in the virtual space according to the progress of the game.

The display control unit 312 outputs, to the monitor 51 and display 530, a game screen that reflects the processing results executed by each of the above-mentioned elements. The display control unit 312 may display an image based on the field of view from a virtual camera arranged in the virtual space on the monitor 51 and the display 530 as a game screen. Furthermore, the display control unit 312 may include the animation generated by the animation generation unit 314 in the game screen. Further, the display control unit 312 may draw the above-mentioned UI image controlled by the UI control unit 313 in a superimposed manner on the game screen.

The reaction processing unit 317 receives feedback from the user of the user terminal 100 regarding reactions to the player's game play, and outputs the feedback to the player. In this embodiment, for example, the user terminal 100 can create a comment (message) addressed to the avatar object based on the user's input operation. The reaction processing unit 317 receives comment data of the comment and outputs it. The reaction processing unit 317 may display text data corresponding to the user's comment on the monitor 51 and display 530, or may output audio data corresponding to the user's comment from a speaker (not shown). In the former case, the reaction processing unit 317 may draw an image corresponding to the text data (that is, an image including the content of the comment) superimposed on the game screen.

(Functional configuration of user terminal 100)
The control unit 110 comprehensively controls the user terminal 100 by executing the game program 131 stored in the storage unit 120. For example, the control unit 110 advances the game according to the game program 131 and the user's operations. Further, while the game is progressing, the control unit 110 communicates with the server 200 to send and receive information as necessary. The control unit 110 may directly transmit and receive information to and from the HMD set 1000 without going through the server 200.

The control unit 110 includes an operation reception unit 111, a display control unit 112, a UI control unit 113, an animation generation unit 114, a game progression unit 115, a virtual space control unit 116, and a video playback unit 117 according to the description of the game program 131. functions as The control unit 110 can also function as other functional blocks (not shown) in order to advance the game depending on the nature of the game being executed.

The operation reception unit 111 detects and accepts a user's input operation on the input unit 151. The operation reception unit 111 determines what input operation has been performed from the user's actions on the console via the touch screen 15 and other input/output IF 14, and outputs the result to each element of the control unit 110. do.

For example, the operation reception unit 111 receives an input operation on the input unit 151, detects the coordinates of the input position of the input operation, and identifies the type of the input operation. The operation reception unit 111 specifies, for example, a touch operation, a slide operation, a swipe operation, a tap operation, etc. as the type of input operation. Further, when the continuously detected input is interrupted, the operation reception unit 111 detects that the contact input from the touch screen 15 is released.

The UI control unit 113 controls a UI image to be displayed on the display unit 152 in order to construct a UI, depending on at least one of the user's input operation and the received game progress information. The UI image is a tool for the user to input information necessary for the progress of the game to the user terminal 100, or a tool for obtaining information outputted from the user terminal 100 during the progress of the game. UI images include, but are not limited to, icons, buttons, lists, menu screens, etc., for example.

The animation generation unit 114 generates animations showing motions of various objects based on control modes of the various objects.

The game progression unit 115 advances the game based on the game program 131, received game progression information, input operations by the user, and the like. When the game progress unit 115 performs a predetermined game process based on an input operation by the user, the game progress unit 115 transmits information regarding the game process to the HMD set 1000 via the server 200 . Thereby, the predetermined game processing is shared among the HMD sets 1000. In other words, the progress of the game on HMD set 1000 and the progress of the game on user terminal 100 are synchronized. The predetermined game process is, for example, a process of providing an item to an avatar object, and in this example, the information regarding the game process is the item information described above.

The virtual space control unit 116 performs various controls regarding the virtual space provided to the user in accordance with the progress of the game. As an example, the virtual space control unit 116 generates various objects and places them in the virtual space. Further, the virtual space control unit 116 arranges a virtual camera in the virtual space. Further, the virtual space control unit 116 operates various objects arranged in the virtual space according to the progress of the game, specifically, the received game progress information. Further, the virtual space control unit 316 controls the position and tilt of the virtual camera placed in the virtual space according to the progress of the game, specifically, the received game progress information.

The display control unit 112 outputs, to the display unit 152, a game screen on which the results of processing performed by each of the above-mentioned elements are reflected. The display control unit 112 may display, as a game screen, an image based on the field of view from a virtual camera arranged in a virtual space provided to the user on the display unit 152. Furthermore, the display control unit 112 may include the animation generated by the animation generation unit 114 in the game screen. Further, the display control unit 112 may draw the above-mentioned UI image controlled by the UI control unit 113 in a superimposed manner on the game screen. In any case, the game screen displayed on display unit 152 is the same game screen as the game screen displayed on other user terminals 100 and HMD set 1000.

The video playback unit 117 analyzes (render) the motion instruction data received from the distribution terminal 400 and plays back the video.

(Functional configuration of distribution terminal 400)
The control unit 410 comprehensively controls the distribution terminal 400 by executing a program (not shown) stored in the storage unit 420. For example, the control unit 410 generates motion instruction data and distributes it to the user terminal 100 according to the program and the operation of the user (in this embodiment, the player) of the distribution terminal 400 . Further, the control unit 410 communicates with the server 200 to send and receive information as necessary. The control unit 410 may directly transmit and receive information to and from the user terminal 100 without going through the server 200.

The control unit 410 functions as a communication control unit 411, a display control unit 412, an operation reception unit 413, a voice reception unit 414, a motion identification unit 415, and a motion instruction data generation unit 416 according to the program description. The control unit 410 can also function as other functional blocks (not shown) in order to generate and distribute operation instruction data.

The communication control unit 411 controls transmission and reception of information with the server 200 or the user terminal 100 via the server 200. The communication control unit 411 receives the user list 421 from the server 200, for example. Furthermore, the communication control unit 411 transmits operation instruction data to the user terminal 100, as an example.

The display control unit 412 outputs various screens on which the results of processing executed by each element are reflected to the display unit 452. The display control unit 412 displays a screen including the received user list 234, as an example. Further, the display control unit 412 displays, for example, a screen including a motion list 422 for allowing the player to select motion data for making the avatar object move, which is included in the movement instruction data to be distributed.

The operation reception unit 413 detects and accepts a player's input operation on the input unit 151. The operation reception unit 111 determines what input operation has been performed based on the action exerted by the player on the console via the touch screen 45 and other input/output IF 44, and outputs the result to each element of the control unit 410. do.

For example, the operation reception unit 413 receives an input operation on the input unit 451, detects the coordinates of the input position of the input operation, and specifies the type of the input operation. The operation reception unit 413 specifies, for example, a touch operation, a slide operation, a swipe operation, a tap operation, etc. as the type of input operation. Further, when the continuously detected input is interrupted, the operation reception unit 413 detects that the contact input from the touch screen 45 is released.

The audio reception unit 414 receives audio generated around the distribution terminal 400, and generates audio data of the audio. For example, the audio reception unit 414 receives audio uttered by the player and generates audio data of the audio.

The motion specifying unit 415 specifies motion data selected by the player from the motion list 422 in response to the player's input operation.

The motion instruction data generation unit 416 generates motion instruction data. As an example, the motion instruction data generation unit 416 generates motion instruction data including generated audio data and specified motion data.

Note that the functions of HMD set 1000, server 200, and user terminal 100 shown in FIG. 6, and the functions of distribution terminal 400 shown in FIG. 7 are only examples. Each of the HMD set 1000, server 200, user terminal 100, and distribution terminal 400 may have at least some of the functions of other devices. Furthermore, another device other than the HMD set 1000, the server 200, the user terminal 100, and the distribution terminal 400 may be used as a component of the system 1, and a part of the processing in the system 1 may be executed by the other device. That is, the computer that executes the game program in this embodiment may be any of the HMD set 1000, the server 200, the user terminal 100, the distribution terminal 400, and other devices, or a plurality of these devices. It may be realized by a combination of devices.

<Virtual space control processing>
FIG. 8 is a flowchart illustrating an example of the flow of control processing for the virtual space provided to the player and the virtual space provided to the user of the user terminal 100. FIG. 9 is a diagram illustrating a virtual space 600A provided to the player and a visual field image visually recognized by the player, according to an embodiment. FIG. 10 is a diagram illustrating a virtual space 600B provided to the user of the user terminal 100 and a visual field image viewed by the user, according to an embodiment. Note that hereinafter, when there is no need to distinguish between the virtual spaces 600A and 600B, they will be referred to as "virtual space 600."

In step S1, the processor 30 defines a virtual space 600A shown in FIG. 9 as the virtual space control unit 316. Processor 30 defines virtual space 600A using virtual space data (not shown). The virtual space data may be stored in the game play terminal 300, may be generated by the processor 30 based on the game program 331, or may be obtained by the processor 30 from an external device such as the server 200. Good too.

For example, the virtual space 600 has a spherical structure that covers the entire 360 degrees of a point defined as the center. In FIGS. 9 and 10, the celestial sphere in the upper half of the virtual space 600 is illustrated in order not to complicate the explanation.

In step S2, the processor 30, as the virtual space control unit 316, places the avatar object 610 (character) in the virtual space 600A. The avatar object 610 is an avatar object associated with the player, and operates according to the player's input operation.

In step S3, the processor 30, as the virtual space control unit 316, places other objects in the virtual space 600A. In the example of FIG. 9, processor 30 places objects 631-634. Other objects include, for example, character objects (so-called non-player characters, NPCs) that operate according to the game program 331, operation objects such as virtual hands, animals, plants, artificial objects, natural objects, etc. that are arranged as the game progresses. May include objects, etc.

In step S4, the processor 30, as the virtual space control unit 316, arranges the virtual camera 620A in the virtual space 600A. For example, the processor 30 places the virtual camera 620A at the head position of the avatar object 610.

In step S5, processor 30 displays visual field image 650 on monitor 51 and display 530. Processor 30 defines viewing area 640A, which is the field of view from virtual camera 620A in virtual space 600A, according to the initial position and tilt of virtual camera 620A. The processor 30 then defines a view image 650 corresponding to the view area 640A. Processor 30 displays visual field image 650 on HMD 500 and display 530 by outputting visual field image 650 to monitor 51 and display 530.

In the example of FIG. 9, since a part of the object 634 is included in the viewing area 640A as shown in FIG. 9(A), the viewing image 650 is a portion of the object 634 as shown in FIG. 9(B). Including.

In step S6, the processor 30 transmits the initial arrangement information to the user terminal 100 via the server 200. The initial placement information is information indicating the initial placement positions of various objects in the virtual space 600A. In the example of FIG. 9, the initial placement information includes information on the initial placement positions of the avatar object 610 and objects 631 to 634. The initial arrangement information can also be expressed as one type of game progress information.

In step S7, the processor 30, as the virtual space control unit 316, controls the virtual camera 620A according to the movement of the HMD 500. Specifically, processor 30 controls the direction and tilt of virtual camera 620A according to the movement of HMD 500, that is, the posture of the player's head. As will be described later, when the player moves his head (changes the posture of his head), processor 30 moves the head of avatar object 610 in accordance with this movement. For example, the processor 30 controls the direction and tilt of the virtual camera 620A so that the direction of the line of sight of the avatar object 610 and the direction of the line of sight of the virtual camera 620A match. In step S8, the processor 30 updates the view image 650 in response to the change in the orientation and inclination of the virtual camera 620A.

In step S9, the processor 30, as the virtual space control unit 316, moves the avatar object 610 according to the player's movement. As an example, the processor 30 moves the avatar object 610 in the virtual space 600A in response to the player's movement in the real space. Furthermore, the processor 30 moves the head of the avatar object 610 in the virtual space 600A in response to the player moving his head in the real space.

In step S10, the processor 30, as the virtual space control unit 316, moves the virtual camera 620A to follow the avatar object 610. In other words, virtual camera 620A is always located at the head of avatar object 610 even if avatar object 610 moves.

Processor 30 updates field of view image 650 in response to movement of virtual camera 620A. That is, the processor 30 updates the viewing area 640A according to the posture of the player's head and the position of the virtual camera 620A in the virtual space 600A. As a result, the field of view image 650 is updated.

In step S11, processor 30 transmits motion instruction data for avatar object 610 to user terminal 100 via server 200. The motion instruction data here includes motion data capturing the player's motions, audio data of the voice uttered by the player, and operation data indicating the content of input operations to the controller 540 during the virtual experience (for example, during game play). Contains at least one of them. When the player is playing the game, the action instruction data is transmitted to the user terminal 100 as, for example, game progress information.

The processes of steps S7 to S11 are continuously and repeatedly executed while the player is playing the game.

In step S21, the processor 10 of the user terminal 100 of the user 3 defines the virtual space 600B shown in FIG. 10 as the virtual space control unit 116. Processor 10 defines virtual space 600B using virtual space data (not shown). The virtual space data may be stored in the user terminal 100, may be generated by the processor 10 based on the game program 131, or may be obtained by the processor 10 from an external device such as the server 200. good.

In step S22, the processor 10 receives initial placement information. In step S23, the processor 10, as the virtual space control unit 116, arranges various objects in the virtual space 600B according to the initial arrangement information. In the example of FIG. 10, the various objects are an avatar object 610 and objects 631 to 634.

In step S24, the processor 10, as the virtual space control unit 116, arranges the virtual camera 620B in the virtual space 600B. As an example, the processor 10 places the virtual camera 620B at the position shown in FIG. 10(A).

In step S25, the processor 10 displays the visual field image 660 on the display unit 152. Processor 10 defines viewing area 640B, which is the viewing area from virtual camera 620B in virtual space 600B, according to the initial position and tilt of virtual camera 620B. The processor 10 then defines a view image 660 corresponding to the view area 640B. The processor 10 displays the visual field image 660 on the display unit 152 by outputting the visual field image 660 to the display unit 152.

In the example of FIG. 10, since the avatar object 610 and the object 631 are included in the viewing area 640B as shown in FIG. 10(A), the viewing image 660 is and object 631.

In step S26, processor 10 receives operation instruction data. In step S27, the processor 10, as the virtual space control unit 116, moves the avatar object 610 in the virtual space 600B according to the motion instruction data. In other words, the processor 10 reproduces the video of the avatar object 610 operating by real-time rendering.

In step S28, the processor 10, as the virtual space control unit 116, controls the virtual camera 620B according to the user's operation received by the operation reception unit 111. In step S29, the processor 10 updates the view image 660 in response to the change in the position of the virtual camera 620B in the virtual space 600B, and the orientation and tilt of the virtual camera 620B. Note that in step S28, the processor 10 may automatically control the virtual camera 620B according to the movement of the avatar object 610, for example, the movement of the avatar object 610 or a change in direction. For example, the processor 10 may automatically move the virtual camera 620B or change the orientation and inclination so as to always photograph the avatar object 610 from the front. Further, as an example, the processor 10 may automatically move the virtual camera 620B so as to always photograph the avatar object 610 from behind, or change the direction and tilt according to the movement of the avatar object 610. good.

In this way, in the virtual space 600A, the avatar object 610 moves according to the movement of the player. Operation instruction data indicating this operation is transmitted to the user terminal 100. In the virtual space 600B, the avatar object 610 operates according to the received motion instruction data. As a result, the avatar object 610 performs similar operations in the virtual space 600A and the virtual space 600B. In other words, the user 3 can use the user terminal 100 to visually recognize the actions of the avatar object 610 in accordance with the actions of the player.

<Game Overview>
FIG. 11 is a diagram showing another example of the view image displayed on the user terminal 100. Specifically, it is a diagram showing an example of a game screen of a game (main game) executed by the system 1, which is being played by a player.

As an example, in this game, an avatar object 610 that operates a weapon such as a gun or a knife, and a plurality of enemy objects 671 that are NPCs appear in a virtual space 600, and the avatar object 610 has a battle with the enemy objects 671. It is a game that allows you to Various game parameters such as the physical strength of the avatar object 610, the number of usable magazines, the number of bullets remaining in the gun, and the number of remaining enemy objects 671 are updated as the game progresses.

This game includes a plurality of stages, and the player can clear each stage by satisfying predetermined achievement conditions associated with each stage. The predetermined achievement conditions include, for example, defeating all the enemy objects 671 that appear, defeating a boss object among the enemy objects 671 that appear, acquiring a predetermined item, reaching a predetermined position, etc. It may also include conditions for The achievement conditions are defined within the game program 131. In addition, in accordance with the content of the game, the player clears the stage when the achievement condition is satisfied. In other words, the player clears the stage when the avatar object 610 defeats the enemy object 671 (the avatar object 610 and the enemy object 671 (win or loss) will be determined. On the other hand, for example, when the game executed by the system 1 is a racing game or the like, the ranking of the avatar object 610 is determined when the condition of reaching the goal is satisfied.

In this game, in order to share the virtual space between the HMD set 1000 and the plurality of user terminals 100, game progress information is live distributed to the plurality of user terminals 100 at predetermined time intervals. As a result, the view image of the view area defined by the virtual camera 620B corresponding to the user terminal 100 is displayed on the touch screen 15 of the user terminal 100 that is viewing the game. In addition, parameter images representing the physical strength of the avatar object 610, the number of usable magazines, the number of remaining bullets of the gun, the number of remaining enemy objects 671, etc. are displayed in a superimposed manner in the upper right and upper left rows of the field of view image. . This visual field image can also be expressed as a game screen.

As described above, the game progress information includes motion data capturing the player's actions, audio data of the voice uttered by the player, and operation data indicating the content of input operations on the controller 540. These data include information for specifying the position, posture, direction, etc. of the avatar object 610, information specifying the position, posture, direction, etc. of the enemy object 671, and other objects (for example, obstacle objects 672, 673). This is information that specifies the location, etc. The processor 10 specifies the position, posture, orientation, etc. of each object by analyzing (rendering) the game progress information.

The game information 132 includes data on various objects such as an avatar object 610, an enemy object 671, and obstacle objects 672 and 673. The processor 10 uses the data and the analysis result of the game progress information to update the position, posture, orientation, etc. of each object. As a result, the game progresses, and each object in the virtual space 600B moves in the same way as each object in the virtual space 600A. Specifically, in the virtual space 600B, each object including the avatar object 610 operates based on the game progress information regardless of whether or not the user operates the user terminal 100.

On the touch screen 15 of the user terminal 100, as an example, UI images 701 and 702 are displayed superimposed on the view image. The UI image 701 is a UI image that accepts an operation for displaying on the touch screen 15 a UI image 711 that accepts an item insertion operation from the user 3 to support the avatar object 610 . The UI image 702 accepts an operation for displaying on the touch screen 15 a UI image (described later) that accepts an operation for inputting and transmitting a comment for the avatar object 610 (in other words, the player 4) from the user 3. This is a UI image. The operation accepted by the UI images 701 and 702 may be, for example, an operation of tapping the UI images 701 and 702.

When the UI image 701 is tapped, the UI image 711 is displayed superimposed on the view image. The UI images 711 include, for example, a UI image 711A with a magazine icon drawn, a UI image 711B with a first aid kit icon, a UI image 711C with a triangular cone icon, and a UI with a barricade icon. Contains image 711D. The item insertion operation corresponds to, for example, an operation of tapping any UI image.

As an example, when the UI image 711A is tapped, the number of remaining bullets of the gun used by the avatar object 610 increases. When the UI image 711B is tapped, the physical strength of the avatar object 610 is restored. When the UI images 711C and 711D are tapped, obstacle objects 672 and 673 that obstruct the movement of the enemy object 671 are placed in the virtual space. One of the obstacle objects 672 and 673 may obstruct movement of the enemy object 671 more than the other.

The processor 10 transmits item insertion information indicating that an item insertion operation has been performed to the server 200. The item insertion information includes at least information for specifying the type of item specified by the item insertion operation. The item insertion information may include other information regarding the item, such as information indicating the position where the item is placed. The item insertion information is transmitted to other user terminals 100 and HMD set 1000 via server 200.

FIG. 12 is a diagram showing another example of the view image displayed on the user terminal 100. Specifically, it is a diagram showing an example of a game screen of the present game, and is a diagram for explaining communication between the player and the user terminal 100 during game play.

In the example of FIG. 12A, the user terminal 100 causes the avatar object 610 to make an utterance 691. Specifically, the user terminal 100 causes the avatar object 610 to perform the utterance 691 according to the audio data included in the game progress information. The content of the utterance 691 is ``I don't have any bullets!'' uttered by the player 4. That is, the content of the utterance 691 is to inform each user that the magazine is now 0 and the number of bullets loaded in the gun is 1, so that the user is likely to lose the means to attack the enemy object 671.

Although speech bubbles are used in FIG. 12A to visually indicate the speech of the avatar object 610, in reality, the speech is output from the speaker of the user terminal 100. In addition to the audio output, a speech bubble shown in FIG. 12(A) (that is, a speech bubble containing text of the audio content) may be displayed in the view image. This also applies to utterance 692, which will be described later.

Upon receiving a tap operation on the UI image 702, the user terminal 100 displays UI images 705 and 706 (message UI) superimposed on the view image, as shown in FIG. 12(B). The UI image 705 is a UI image that displays a comment for the avatar object 610 (in other words, the player). The UI image 706 is a UI image that accepts a comment sending operation from the user 3 in order to send the input comment.

For example, when the user terminal 100 receives a tap operation on the UI image 705, the user terminal 100 displays a UI image imitating a keyboard (not shown, hereinafter simply referred to as "keyboard") on the touch screen 15. The user terminal 100 causes the UI image 705 to display text according to the user's input operation on the keyboard. In the example of FIG. 12B, the text “I will send you a magazine” is displayed on the UI image 705.

For example, when the user terminal 100 receives a tap operation on the UI image 706 after inputting text, it sends comment information including information indicating the input content (text content) and information indicating the user to the server 200. do. Comment information is transmitted to other user terminals 100 and HMD set 1000 via server 200.

The UI image 703A is a UI image that shows the user name of the user who sent the comment, and the UI image 704A is a UI image that shows the content of the comment that the user sent. In the example of FIG. 12(B), the user whose user name is "BBBBB" uses his user terminal 100 to send comment information with the content "Dangerous!" is displayed. UI image 703A and UI image 704A are displayed on touch screens 15 of all user terminals 100 participating in this game and on monitor 51 of HMD 500. Note that the UI images 703A and 704A may be one UI image. That is, one UI image may include the user name and the content of the comment.

In the example of FIG. 12C, the user with the user name "AAAAAA" who is the user of the user terminal 100 shown in FIG. 703B and 704B are displayed. The UI image 703B includes the user name "AAAA", and the UI image 704B includes the comment "I'll send you a magazine!" that was input in the example of FIG. 12(B).

In the example of FIG. 12C, the user "AAAA" further inputs a tap operation on the UI image 701, displays the UI image 711 on the touch screen 15, and inputs a tap operation on the UI image 711A. This is a rear view image 611. In other words, as a result of the item insertion information indicating the magazine being sent from the user terminal 100 of the user "AAAAAA" to the other user terminals 100 and the HMD set 1000, the user terminal 100 and the HMD set 1000 receive the production object 674 (described later). are placed in the virtual space 600. As an example, the user terminal 100 and the HMD set 1000 perform processing related to the performance object 674 and activate the effect of the item object after the elapsed time indicated by the item insertion information has elapsed.

In the example of FIG. 12(D), the number of magazines increases from 0 to 1 by executing the process of activating the effect of the item object. As a result, the player utters "Thank you!" to the user "AAAA", and the audio data of the utterance is transmitted to each user terminal 100. As a result, each user terminal 100 outputs the voice "Thank you!" as the utterance 692 of the avatar object 610.

As described above, in this game, communication between the user and the avatar object 610 is realized by outputting the voice uttered by the avatar object 610 based on the player's utterance and inputting comments by each user.

(Game progress processing on game play terminal 300)
FIG. 13 is a flowchart illustrating an example of the flow of game progress processing executed by the game play terminal 300.

In step S31, the processor 30, as the game progression unit 315, advances the game based on the game program 331 and the player's movements. In step S32, the processor 30 generates game progress information and distributes it to the user terminal 100. Specifically, the processor 30 transmits the generated game progress information to each user terminal 100 via the server 200.

In step S33, when the processor 30 receives the item insertion information (YES in S33), in step S34, it arranges the item object in the virtual space 600A based on the item insertion information. As an example, the processor 30 places the rendering object 674 in the virtual space 600A before placing the item object (see FIG. 11C). The effect object 674 may be, for example, an object imitating a present box. For example, the processor 30 may execute the effect regarding the effect object 674 after the elapsed time indicated by the item input information has elapsed. The effect may be, for example, an animation in which the lid of a present box opens. After executing the animation, the processor 30 executes a process of activating the effect of the item object. For example, in the example of FIG. 11(D), an obstacle object 673 is placed.

After executing the animation, the processor 30 may place the item object corresponding to the tapped UI image in the virtual space 600A. For example, when a tap operation is performed on the UI image 711A, the processor 30 places a magazine object representing a magazine in the virtual space 600A after executing the animation. Further, when a tap operation is performed on the UI image 711B, the processor 30 executes the animation and then arranges a first aid kit object representing a first aid kit in the virtual space 600A. For example, when the avatar object 610 moves to the position of the magazine object or the first aid kit object, the processor 30 may perform a process of activating the effect of the magazine object or the first aid kit object.

The processor 30 continues and repeats the processing of steps S31 to S34 until the game ends. When the game ends, for example, when the player inputs a predetermined input operation to end the game (YES in step S35), the process shown in FIG. 13 ends.

(Game progress processing on user terminal 100)
FIG. 14 is a flowchart illustrating an example of the flow of game progress processing executed by the user terminal 100.

In step S41, the processor 10 receives game progress information. In step S42, the processor 10, as the game progress unit 115, advances the game based on the game progress information.

In step S43, when the processor 10 accepts the item insertion operation by the user 3 (YES in step S43), in step S44, the processor 10 consumes virtual currency and places the performance object 674 in the virtual space 600B. Here, the virtual currency is purchased (charged for the game) when the user 3 performs a predetermined operation on the processor 10 before or during participation in the game. Alternatively, it may be given to the user 3 when a predetermined condition is met. The predetermined condition may be something that requires participation in this game, such as clearing a quest in this game, or may be something that does not require participation in this game, such as answering a questionnaire. The amount of virtual currency (amount of virtual currency owned) is stored in the user terminal 100 as game information 132, for example.

In step S45, the processor 10 transmits item input information to the server 200. The item insertion information is transmitted to the game play terminal 300 via the server 200.

The processor 10 arranges the item object in the virtual space 600A after a predetermined period of time has elapsed after placing the presentation object 674. In the example of FIG. 11, an obstacle object 673 is placed. That is, when the user 3 inputs a tap operation on the UI image 711C, a predetermined amount of virtual currency is consumed and the obstacle object 673 is placed.

The processor 10 continues and repeats the processing of steps S41 to S45 until the game ends. When the game ends, for example, when the player performs a predetermined input operation to end the game, or when the user 3 performs a predetermined input operation to leave the game midway through (step S46: YES) ), the process shown in FIG. 14 ends.

(Game progress processing in server 200)
FIG. 15 is a flowchart showing an example of the flow of game progress processing executed by the server 200. As shown in FIG.

In step S51, the processor 20 receives game progress information from the game play terminal 300. In step S52, the processor 20, as the log generation unit 212, updates a game progress log (hereinafter referred to as a play log). Note that the play log is generated by the processor 20, for example, when initial arrangement information is received from the game play terminal 300.

In step S53, the processor 20 transmits the received game progress information to each user terminal 100.

In step S54, when item input information is received from any user terminal 100 (YES in step S54), the processor 20 updates the play log as the log generation unit 212 in step S55. In step S56, the processor 20 transmits the received item insertion information to the game play terminal 300.

The processor 20 continues and repeats the processing of steps S51 to S56 until the game ends. When the game ends, for example, when information indicating that the game ends is received from the game play terminal 300 (YES at step S57), in step S58, the processor 20, as the list generation unit 213, extracts the play log from the play log. A list of users who have participated in the game (user list 234) is generated. The processor 20 stores the generated user list 234 in the server 200.

FIG. 16 is a diagram showing a specific example of the user list 234. The "user" column stores information (for example, user name) indicating each user who participated in the game. The "tag" column stores information (tags) generated based on the support provided by each user to the player. In the example of FIG. 16, among the tags stored in the "tag" column, those without square brackets are information automatically generated by the processor 20, and those with square brackets are information generated by the game operator. The information was entered manually.

In the example of FIG. 16, the user "AAAAA" is associated with a magazine, 10F, a boss, and information "win against the boss by giving the magazine as a gift." This indicates that, for example, in the boss battle on the stage 10F, the user "AAAA" inserted a magazine, and the avatar object 610 defeated the boss with the bullets from the inserted magazine.

In addition, the user "BBBBB" is associated with the information "first aid kit, 3F, Zako, and "Recovery on the verge of game over." "BBBBB" threw in a first aid kit, and as a result, the avatar object 610's physical strength recovered just before it became 0 (game over).

Further, the user "CCCCC" is associated with information such as barricade, 5F, Zako, and "stopping two zombies with barricade." This means that, for example, in the battle with the most powerful enemies on the stage 5F, the user "CCCCC" threw a barricade (obstacle object 672 in FIG. 11), and as a result, succeeded in stopping the two most powerful enemies. It shows.

In the example of FIG. 16, one support provided is associated with the user name of each user 3, but the user name of user 3 who provided support multiple times has tags for each of the multiple times of support. Can be matched. In the user list 234, the respective tags are preferably distinguished. Thereby, after the game ends, the player who refers to the user list 421 using the distribution terminal 400 can accurately understand the contents of each support.

<Distribution of movement instruction data>
(Distribution processing at distribution terminal 400)
FIG. 17 is a flowchart illustrating an example of the flow of distribution processing executed by the distribution terminal 400. FIG. 18 is a diagram showing a specific example of a screen displayed on distribution terminal 400. FIG. 19 is a diagram showing another specific example of the screen displayed on the distribution terminal.

In step S61, the processor 40, as the operation receiving unit 413, receives a first operation for displaying a list of users who have participated in the game (user list 234). A download screen 721 shown in FIG. 18A is a screen for downloading the user list 234 from the server 200 and displaying it on the display unit 452. The download screen 721 is, for example, a screen that is displayed immediately after an operation to start an application that executes the distribution process shown in FIG. 17 is input to the distribution terminal 400.

Download screen 721 includes UI images 722 and 723, as an example. The UI image 722 accepts the operation for downloading the user list 234, that is, the first operation described above. The first operation may be, for example, an operation of tapping the UI image 722. The UI image 723 accepts an operation to terminate the application. The operation may be, for example, an operation of tapping the UI image 723.

Upon receiving the tap operation on the UI image 722, the processor 40, as the communication control unit 411, acquires (receives) the user list 234 from the server 200 in step S62. In step S63, the processor 40, as the display control unit 412, causes the display unit 452 to display the user list 234. Specifically, the processor 40 causes the display unit 452 to display a user list screen generated based on the user list 234. The user list screen may be, for example, a user list screen 731 shown in FIG. 18(B). The user list screen 731 consists of record images corresponding to each record in the user list 234. In the example of FIG. 18(B), record images 732A to 732C are described as record images, but the number of record images is not limited to three. In the example of FIG. 18(B), if the number of records in the user list 234 is greater than 3 (that is, the number of users who have participated in the game is greater than 3), the player may perform an operation such as scrolling the screen (for example, By inputting a drag operation or a flick operation on the touch screen 45, other record images can be displayed on the display section 452.

As an example, record images 732A-732C each include user names 733A-733C, tag information 734A-734C, and icons 735A-735C. Hereinafter, if there is no need to distinguish record images 732A to 732C, user names 733A to 733C, tag information 734A to 734C, and icons 735A to 735C, they will be referred to as "record image 732," "user name 733," and ", respectively. "tag information 734" and "icon 735".

The user name 733 is information stored in the "user" column in the user list 234 and indicating each user who participated in the game. The tag information 734 is information indicating a tag associated with each piece of information indicating each user who participated in the game in the user list 234. For example, the record image 732A includes "AAAAA" as the user name 733A. Therefore, the record image 732A includes, as tag information 734A, "Magazine, 10F, Boss, "Win the Boss by Presenting the Magazine", which is associated with "AAAAAA" in the user list 234. The icon 735 is, for example, an image set in advance by the user.

Note that the processor 40 may store the received user list in the distribution terminal 400 (user list 421 in FIG. 7). The download screen 721 may include a UI image (not shown) for displaying the user list 421 on the display unit 452. In this example, when the UI image is tapped, the processor 40 does not download the user list 234, reads the user list 421, generates a user list screen from the user list 421, and displays it on the display unit 452. .

In step S64, the processor 40, as the operation accepting unit 413, accepts a second operation for selecting any of the users included in the user list screen 731. The second operation may be, for example, an operation of tapping any of the record images 732 on the user list screen 731. In the example of FIG. 18(B), the player inputs a tap operation to the record image 732A. That is, the player has selected the user "AAAA" as the user to whom the action instruction data will be distributed.

Upon receiving the tap operation on the record image 732, the processor 40, as the display control unit 412, causes the display unit 452 to display the motion list 422 in step S65. Specifically, the processor 40 causes the display unit 452 to display a motion list screen generated based on the motion list 422. The motion list screen may be a motion list screen 741 shown in FIG. 19, for example. The motion list screen 741 consists of record images corresponding to each record in the motion list 422. In the example of FIG. 19, record images 742A to 742C are described as record images, but the number of record images is not limited to three. In the example of FIG. 19, if the number of records in the motion list 422 is greater than 4, the player can scroll other records by inputting an operation to scroll the screen (for example, a drag operation or a flick operation) on the touch screen 45. The image can be displayed on the display section 452.

As an example, record images 742A-742C each include motion names 743A-743C, motion images 744A-744C, and UI images 745A-745C. Hereinafter, when there is no need to distinguish record images 742A to 742C, motion names 743A to 743C, motion images 744A to 744C, and UI images 745A to 745C, they will be referred to as "record image 7432", "motion name 743", and "motion name 743", respectively. They are described as "motion image 744" and "UI image 745."

The motion name 743 is information that identifies a motion stored in the motion list 422. The motion image 744 is an image generated from motion data associated with each motion name in the motion list 422. For example, the processor 40 includes an image of the avatar object 610 taking the first posture in each motion data in the record image 742 as a motion image 744. The motion image 744 may be a UI image that accepts a predetermined operation by the player (for example, a tap operation on the motion image 744). When the processor 40 receives the predetermined operation, it may play a motion video in which the avatar object 610 moves based on the motion data. The processor 40 may automatically redisplay the motion list screen 741 when the motion video ends.

Note that, instead of the motion image 744, the record image 742 may include, for example, a UI image including the text "motion playback".

In step S66, the processor 40, as the operation receiving unit 413, receives a third operation for selecting a motion. The third operation may be, for example, a tap operation on the UI image 745. That is, the UI image 745 accepts an operation to select motion data corresponding to each record image 742. Upon receiving the third operation, the processor 40, as the motion specifying unit 415, specifies the motion data selected by the player.

In step S67, the processor 40, as the display control unit 412 and the audio reception unit 414, receives audio input from the player while playing back a motion video in which the avatar object 610 operates based on the selected motion data.

FIG. 20 is a diagram showing a specific example of voice input by the player 4. As shown in FIG. 20, player 4 is inputting speech sound 820A while playing motion video 810A. This uttered voice 820A is addressed to user 3 (hereinafter referred to as user 3A) whose user name is “AAAAAA”. That is, in the example of FIG. 20, the player 4 selects the user 3A (first user) and creates motion instruction data addressed to the user 3A in step S64. Note that it is assumed that the user terminal 100 used by the user 3A is the user terminal 100A.

Since the uttered voice 820A is addressed to the user 3A, the uttered voice is based on the content of the support provided by the user 3A to the avatar object 610 (in other words, the player 4). Specifically, the user 3A inserts a magazine into the boss battle on the stage 10F, and the avatar object 610 defeats the boss with the bullets from the inserted magazine. Therefore, the uttered voice 820A has the content, "Thank you for giving me the magazine during the boss battle! The timing was perfect! Thanks to Mr. AAAAAA, I was able to clear the game!" In this way, it is preferable that the uttered voice includes the content of the support provided by the user 3 in the game and gratitude to the user 3.

In one aspect, the player 4 creates speech content addressed to the user 3 before starting voice input, that is, before inputting the third operation to the distribution terminal 400. In another aspect, the content of the utterance addressed to the user 3 may be automatically generated by the processor 40. Further, the processor 40 may display the tag associated with the user 3 selected by the second operation in a superimposed manner on the motion video 810A.

Processor 40 converts the received voice into voice data. In step S68, the processor 40, as the motion instruction data generation unit 416, generates motion instruction data including the audio data and motion data of the selected motion.

In step S69, the processor 40, as the communication control unit 411, distributes the generated motion instruction data to the user terminal 100 (first computer) of the selected user 3 (user 3A in the example of FIG. 20). FIG. 21 is a diagram showing yet another specific example of the screen displayed on the distribution terminal 400. After executing step S68, the processor 40, as the display control unit 412, causes the display unit 452 to display the distribution screen. The distribution screen may be, for example, the distribution screen 751 shown in FIG. 21(A). Distribution screen 751 includes a UI image 752 and motion image 753A. Further, the distribution screen 751 may include information indicating the user to whom the operation instruction data is to be distributed, as shown in FIG. 21(A).

The UI image 752 accepts an operation for distributing motion instruction data to the selected user 3. The operation may be, for example, a tap operation on the UI image 752. The motion image 753A is a UI image that accepts an operation for reproducing a video based on the generated motion instruction data, that is, a video based on the motion instruction data generated for the user 3A. The operation may be, for example, a tap operation on the motion image 753A. Note that the UI image that accepts the operation for playing back the generated video is not limited to the motion image 753A. For example, it may be a UI image that includes the text "Play Video". The processor 40 may automatically redisplay the distribution screen 751 when the video ends.

It is preferable that the distribution screen 751 further includes a UI image that accepts an operation to return to accepting voice input. The operation may be, for example, a tap operation on the UI image. By including the UI image in the distribution screen 751, the player 4 can perform voice input again when voice input fails, for example, when the player 4 makes a mistake in what is being uttered. Note that the UI image may be a UI image that accepts an operation to return to selection of motion data.

Upon receiving the tap operation on the UI image 752, the processor 40 transmits operation instruction data to the server 200 along with information indicating the user 3A. The server 200 specifies the user terminal 100 to which the movement instruction data is to be transmitted based on the information indicating the user 3A, and transmits the movement instruction data to the specified user terminal 100 (namely, the user terminal 100A).

When the transmission of the operation instruction data is completed, the processor 40 may display, for example, a distribution completion screen 761 shown in FIG. 21(B) on the display unit 452. Distribution completion screen 761 includes UI images 762 and 763, as an example. Further, the distribution completion screen 761 may include text indicating that the transmission of the operation instruction data has been completed, as shown in FIG. 21(B).

The UI image 762 accepts an operation to start creating action instruction data addressed to another user 3. The operation may be, for example, an operation of tapping the UI image 762. When the processor 40 receives the tap operation, the processor 40 causes the display unit 452 to display the user list screen again. That is, if the tap operation is accepted, the distribution process returns to step S63. At this time, the processor 40 may generate a user list screen based on the user list 421 stored in the distribution terminal 400 and display it on the display unit 452. The UI image 763 accepts an operation to terminate the application. The operation may be, for example, an operation of tapping the UI image 763. When the operation is accepted, the distribution process ends.

In the example described with reference to FIG. 20 and FIG. 21, as shown in FIG. , is transmitted only to the user terminal 100A.

FIG. 22 is a diagram showing another specific example of voice input by the player 4. As shown in FIG. 22, player 4 is inputting speech sound 820B while playing motion video 810B. This uttered voice 820B is a uttered voice addressed to user 3 (hereinafter referred to as user 3B) whose user name is “BBBBB”. That is, in the example of FIG. 22, the player 4 inputs a tap operation on the record image 732B corresponding to the user 3B in step S64, and creates motion instruction data addressed to the user 3B. Note that it is assumed that the user terminal 100 used by the user 3B is the user terminal 100B.

Since the uttered voice 820B is addressed to the user 3B, the uttered voice is based on the content of the support provided by the user 3B to the avatar object 610 (in other words, the player 4). Specifically, user 3B is on the verge of losing the physical strength of avatar object 610 to 0 (game over) when user "BBBBB" throws in a first aid kit during the battle with the worst enemy on stage 3F. My physical strength is recovering. Therefore, the uttered voice 820B has the content, ``Thanks to the first aid kit BBBBB gave me, I didn't have to get game over on the third floor. Thank you so much!''

FIG. 23 is a diagram showing still another specific example of the screen displayed on the distribution terminal 400. The distribution screen 751 shown in FIG. 23(A) includes a UI image 752 and a motion image 753B. When the motion image 753B receives a tap operation, it plays a moving image based on the motion instruction data generated for the user 3B.

Upon receiving the tap operation on the UI image 752, the processor 40 transmits operation instruction data to the server 200 along with information indicating the user 3B. The server 200 identifies the user terminal 100 to which the motion instruction data is to be transmitted based on the information indicating the user 3B, and transmits the motion instruction data to the identified user terminal 100 (ie, the user terminal 100B).

In the example described with reference to FIG. 22 and FIG. 23, as shown in FIG. , is transmitted only to the user terminal 100B.

As described above, the content of the audio based on the audio data included in the action instruction data is based on the content of the support provided by the user 3 to the player 4 in participating in the most recent game. Since the content of the support is different for each user 3, the content of the voice is different for each user 3. That is, after the game ends, motion instruction data including voices with different contents is transmitted to at least some of the user terminals 100 of the users 3 who participated in the game.

Furthermore, the motion of the avatar object 610 in the example of FIG. 22 is different from the motion in the example of FIG. 20. That is, when generating the motion instruction data addressed to the user 3B, the player 4 selects different motion data from when generating the motion instruction data addressed to the user 3A. Specifically, in step S66, the player 4 inputs a tap operation on the UI image 745B to select motion data corresponding to the record image 742B. In this way, the player 4 can vary the motion data included in the action instruction data for each user 3.

Then, motion instruction data for each user 3, including audio data with different contents for each user 3 and motion data selected for each user 3, is transmitted only to the user terminal 100 of each user 3. In other words, unique operation instruction data for each user terminal 100 is transmitted to each of the user terminals 100 of the selected users 3.

FIG. 24 is a diagram showing an overview of transmission of game progress information from game play terminal 300 to user terminal 100. While the operation instruction data for video playback on the user terminal 100 is unique for each user terminal 100, as shown in FIG. The game progress information transmitted to the user terminals 100 is common to each user terminal 100. That is, the action instruction data included in the game progress information is also common among each user terminal 100. In this way, it can be said that the motion instruction data for playing a video and the motion instruction data for advancing the game are different data in terms of the sameness between user terminals 100 and the destination of transmission.

(Video playback processing in user terminal 100)
FIG. 25 is a flowchart illustrating an example of the flow of video playback processing executed by the user terminal 100.

In step S71, the processor 10, as the video playback unit 117, receives the motion instruction data. In step S72, the processor 10, as the video playback unit 117, notifies the user 3 of receiving the motion instruction data. For example, the processor 10 displays a notification image on the display section 152, plays a notification sound from a speaker (not shown), lights up a lighting section (not shown) including an LED (light-emitting diode), or the like. The reception of the operation instruction data is notified to the user 3 by at least one of blinking.

In step S73, the processor 10, as the operation receiving unit 111, receives a first playback operation for playing back the video. The first reproduction operation may be, for example, an operation of tapping a notification image. In step S74, the processor 10, as the video playback unit 117, renders the motion instruction data and plays the video. For example, the processor 10 may start an application for playing the present game and play the video, or may start an application for video playback that is separate from the application and play the video. good. Hereinafter, this video will be referred to as a "thank you video."

FIG. 26 is a diagram showing a specific example of playing a thank you video. Specifically, it is a diagram showing an example of reproduction of a thank you video on the user terminal 100 of the user 3A. In the thank you video 910A played on the user terminal 100, the avatar object 610 is uttering a voice 920A while performing a certain motion. In other words, the processor 10 outputs the audio 920A from a speaker (not shown) while reproducing the thank you video 910A including the avatar object 610 that performs a certain motion.

The motion in the thank you video 910A is based on the motion data selected by the player 4 in generating action instruction data addressed to the user 3A, and the audio 920A is based on the uttered audio input by the player 4 in generating the action instruction data. It is based on audio data generated from 820A. In other words, the voice 920A is a voice that includes the details of the support provided by the user 3A in the game and gratitude for the support. In this way, by inputting the first playback operation, the user 3A can view a thank you video in which the avatar object 610 speaks about the support provided by the user in the game and gratitude for the support.

As an example, the user terminal 100 may display at least one UI image on the touch screen 15 after the thank you video 910A finishes playing. The UI image may be, for example, a UI image that accepts an operation to play the thank you video 910A again, a UI image that accepts an operation to transition to another screen, or a UI image that accepts an operation to transition to another screen, or a UI image that accepts an operation to play the thank you video 910A again. It may also be a UI image that accepts an operation to end the process.

Further, as an example, the user terminal 100 may display at least one UI image on the touch screen 15 while the thank you video 910A is being played. The UI image may be a plurality of UI images that respectively accept operations such as temporarily stopping or ending the currently playing thank you video 910A, or changing the scene to be played.

Note that these UI images displayed while the thank you video 910A is being played and after the thank you video 910A is being played do not include a UI image for responding to the avatar object 610. That is, the thank you video 910A according to this embodiment is not provided with a means for responding to the avatar object 610.

FIG. 27 is a diagram showing another specific example of playing a thank you video. Specifically, it is a diagram showing an example of reproduction of a thank you video on the user terminal 100 of the user 3B. In the thank you video 910B played on the user terminal 100, the avatar object 610 is uttering a voice 920B while performing a certain motion. In other words, the processor 10 outputs the audio 920B from a speaker (not shown) while reproducing the thank you video 910B that includes the avatar object 610 that performs a certain motion.

The motion in the thank you video 910B is based on the motion data selected by the player 4 in generating the action instruction data addressed to the user 3B, and the audio 920B is based on the uttered audio input by the player 4 in generating the action instruction data. It is based on audio data generated from 820B. Therefore, the motion performed by avatar object 610 in the example of FIG. 27 is different from the motion in the example of FIG. 26. Furthermore, the voice 920B is a voice that includes the details of the support provided by the user 3B in the game and gratitude for the support. Therefore, in the example of FIG. 27, the content of audio 920B is different from the content of audio 920A in the example of FIG. 26.

In this way, the thank you video that is received by at least some of the user terminals 100 of the users 3 who participated in the game after the game ends is a video in which the utterance content of the avatar object 610 differs for each user 3.

Note that the processor 10 may display a UI image 930 that includes content encouraging participation in the next game, superimposed on the video 910. The UI image 930 may be distributed together with the action instruction data, or may be stored by the user terminal 100 as the game information 132.

In the live distribution game, the same virtual space 600B as the virtual space 600A defined by the game play terminal 300 is defined by the user terminal (viewing terminal, terminal device) 100. The game play terminal 300 distributes game progress information including movement instruction data that can identify the movements of the performer and the voice uttered by the performer to the plurality of user terminals 100 via the server 200. The memory 11 of the user terminal 100 stores in advance multiple types of object data for displaying objects including the avatar object 610. When an operation is performed to start a live broadcast game, the user terminal 100 identifies motion instruction data from the game progress information distributed from the game play terminal 300 via the server 200, and analyzes the motion instruction data ( (rendering) to identify the type, position, orientation, etc. of the object. Furthermore, based on the analysis result, the user terminal 100 uses the object data stored in the memory 11 to arrange objects including the avatar object 610 in the virtual space 600B.

On the monitor 51 of the HMD 500, a view image 650 of the view area 640A is displayed according to the position, orientation, and inclination of the virtual camera 620A arranged in the virtual space 600A. Furthermore, the touch screen 15 of the user terminal 100 displays a visual field image 660 of the visual field 640B according to the position, orientation, and tilt of the virtual camera 620B arranged in the virtual space 600B. A speaker (not shown) included in the user terminal 100 outputs the voice uttered by the performer based on the action instruction data specified from the game progress information.

<Game Overview>
The game that can be provided by the system 1 according to the present embodiment includes, for example, a live distribution part performed during a live distribution period allocated before and after the live game period, and a live game part performed during the live game period. The live distribution part is a part in which interaction such as dialogue can take place between the user and the avatar object 610. On the other hand, the live game part is a part where a predetermined game can be played between the user and the performer. The game that can be provided by the system 1 according to the present embodiment is composed of multiple types of parts including a live distribution part and a live game part, and the parts are arranged in a predetermined order on the distribution side and according to the progress of the performer. The process progresses by being migrated. During live streaming, the user (viewer) should check the status of the avatar object 610 and insert items, input comments, etc. at appropriate times during both the live streaming part and the live game part. I can do it. For example, in response to a touch operation from the user, an object of an item (hereinafter also referred to as a tipping item) specified by the user can be placed or displayed (reflected) in the virtual space where the avatar object 610 is placed. Thereby, the user's presence, contribution, etc. can be appealed to the performer and other users, and the interest of the game can be improved.

The game that can be provided by the system 1 is a video in which a virtual space is generated on the user terminal 100 side based on game progress information distributed all at once from the game play terminal 300 side, and the avatar object 610 is moved according to the movement of the performer. Display. Even if the same game progress information is distributed all at once from the game play terminals 300, live video may be displayed between the user terminals 100 depending on the communication status of the user terminals 100 (sudden or temporary malfunctions in communication), etc. This causes a lag (delay) in the display timing. If a delay occurs, the live video, etc. will be out of sync with the performers and other users, making it impossible to insert items or input comments at the appropriate time, making it impossible to make an effective appeal. It becomes impossible. Therefore, in this embodiment, when transitioning from one part (first part) of the game to the next part (second part), a conversion screen (specific video) is displayed for a certain period of time as a transition. Distribute game progress information for display, and absorb (reset) the deviation (delay) by allowing the user terminal 100 that is delayed while displaying the specific video to reach the end timing of the current part. This allows all user terminals 100 viewing the next part to start viewing the next part at approximately the same timing. Thereby, as described above, the user can insert items, input comments, etc. at appropriate timings depending on the situation of the avatar object 610, etc., and can make an appeal to the performer and other users. A specific example will be described below using FIGS. 28 and 29. FIG. 28 shows an example of a display on the touch screen 15 of the user terminal 100 during live distribution. FIG. 29 is a timing chart for explaining the video misalignment between multiple user terminals 100.

(Live distribution part: example of the first part)
When the performer performs a live distribution start operation using the controller 540, distribution of game progress information (live distribution) is started. Here, an example will be described in which the live distribution part is started when the live distribution starts. In the live distribution part, the game play terminal 300 defines the virtual space 600A as the live distribution space, and the game play terminal 300 defines the virtual space 600A as the live distribution space, and displays an avatar object 610 imitating a girl wearing a dress, a banner with the word "banquet" written on it, and an image imitating a mountain. A virtual object including a decoration object (background object) is placed in the live distribution space (see FIG. 28A, which is a display example on the user terminal 100 side, etc.). The game play terminal 300 detects the performer's body movement based on the output from each of the HMD 500 and the motion sensor 520 (that is, captures the performer's motion), and based on the motion data etc. generated thereby, Virtual objects including the avatar object 610 are arranged, operated, etc. within the live distribution space. In addition, the game play terminal 300 generates motion instruction data for specifying the placement, motion, utterance, etc. of virtual objects including the avatar object 610 and decoration objects based on the motion data, audio data, etc. Game progress information including the action instruction data is distributed to the plurality of user terminals 100.

In the live distribution part, each of the plurality of user terminals 100 defines the virtual space 600B as the live distribution space, and each of the plurality of user terminals 100 defines the virtual space 600B as the live distribution space, and displays the virtual space at a predetermined position in the live distribution space (for example, a position where the avatar object 610 can be seen from the front). A camera 620B is arranged. The user terminal 100 arranges virtual objects including the avatar object 610 and decorative objects in the live distribution space based on model data stored in advance in the memory 11 and game progress information distributed from the game play terminal 300. , or operate a virtual object that is already in place. Further, the user terminal 100 defines a viewing area 640B according to the current position and orientation of the virtual camera 620B, and displays a viewing image 660 corresponding to the viewing area 640B on the touch screen 15 as a live distribution video (FIG. 28 (A) and FIG. 28(B)). Game progress information distributed from the game play terminal 300 side is distributed to a plurality of user terminals 100 all at once at predetermined time intervals. This allows the game play terminal 300 and each of the plurality of user terminals 100 to share at least the movements of the avatar object 610 according to the movements of the performer, the contents of the performer's utterances, etc., and are updated and output at predetermined time intervals.

On the other hand, the actual timing of displaying images based on game progress information on the touch screen 15 may vary between multiple user terminals 100 due to the communication status of each user terminal 100 (for example, communication speed, communication stability, etc.). may differ in terms of As a result, even if game progress information is distributed all at once from the game play terminal 300 side, the timing at which images based on the game progress information are displayed on the touch screen 15 will differ between the plurality of user terminals 100. (Differences may occur).

FIG. 29(A) illustrates an example of the timing when, after the game progress information is distributed from the game play terminal 300 side, a video corresponding to the game progress information is displayed on each of the user terminals 100 owned by users A to C. This is a diagram for From the top of FIG. 29(A), game progress information distributed from the game play terminal 300 (distribution side) at predetermined time intervals is shown (downward arrows "↓" and "..."), and each of the users A to C at the bottom The timing at which the video based on the game progress information received in response to the above is displayed on each of the user terminals 100 owned by users A to C is shown. Note that "..." indicates that game progress information is distributed at predetermined intervals. Further, the circled numbers in the lower column of FIG. 29(A) indicate timings 1 to 13. Note that after the game progress information is distributed all at once until the user terminal 100 receives the game progress information and displays the corresponding video, there is a time when the communication status is normal (if there is a sudden or temporary malfunction after the start of viewing). Even if the information is received in the case where the information is not available (for example, if the information is not available), there will be a slight delay because it takes time for reception and image processing in the user terminal 100. However, in the present embodiment including FIG. 29, the slight delay that occurs when receiving in a normal communication state will be ignored for convenience, and when the distributed game progress information is received in a normal communication state, The video based on the game progress information shall be displayed at the distributed timing.

FIG. 29(A) shows an example in which a delay occurs due to the communication status of each of the user terminals 100 owned by users A to C (such as when a sudden or temporary problem occurs after the start of viewing). There is. For example, a video based on game progress information α distributed from the game play terminal 300 at timing 1 is displayed on user A's user terminal 100 at timing 2 (delay a from timing 1), and is displayed on user B's user terminal 100 at timing 2 (delay a from timing 1). 100, it is displayed at timing 3 (delay b from timing 1), and in user C's user terminal 100, it is displayed at timing 4 (delay c from timing 1). Similarly, the video based on the game progress information β distributed from the game play terminal 300 at timing 5 is displayed on user A's user terminal 100 at timing 6 (delay a from timing 5), and user B The user terminal 100 of user C is displayed at timing 7 (delay b from timing 5), and the user terminal 100 of user C is displayed at timing 8 (delay c from timing 5).

During the live streaming period, the performer may ask the viewer (user), for example, to have a predetermined conversation (such as a greeting, report on a familiar event, etc.), or a preview or explanation of the next part (for example, when a live game has started). There will also be an overview of the live game, etc.). When the performer completes the explanation etc. in the live distribution part, the performer performs a part transition operation (such as a predetermined button operation) on the controller 540 to move to the next part. When the game play terminal 300 detects a part transition operation, it outputs game progress information including part end information for ending the current part and moving to preparation for the next part.

On the user terminal 100 side, when receiving game progress information including part end information and starting display based on the game progress information, regardless of the position and orientation of the virtual camera 620B (that is, the viewing area 640B for each user), First, a predetermined conversion screen is displayed over the entire display area of the touch screen 15 (see FIG. 28(C)). Thereby, it is possible to notify the user in an easy-to-understand manner that the process for migrating the part is being performed. Display data for displaying the conversion screen is stored in advance in the memory 11 of the user terminal 100 when the game program is downloaded or at an update timing. In FIG. 28C, as a transition screen, a predetermined pattern is displayed as a background, and a still image in which three characters are each taking a predetermined action is illustrated. Note that the conversion screen is not limited to a still image, and may be a predetermined moving image. Further, the type of conversion screen can be set in advance for each user terminal 100, and the type of conversion screen set for each user terminal 100 may be displayed.

Timing 9 in FIG. 29(A) indicates the timing at which game progress information including part end information is distributed from the game play terminal 300. Timings 10 to 12 indicate the timings at which the conversion screen was displayed on the user terminals 100 owned by users A to C, respectively. In this way, the timing at which the display of the conversion screen starts differs among the plurality of user terminals 100 depending on the deviation (delay) that has occurred before that time (see timings 10 to 12). On the other hand, in the user terminals 100 owned by users A to C that were experiencing delays, the conversion screen could be displayed under normal communication conditions (specifically, until timings 9 to 13 are reached), Game progress information including end information Outputs the video and speech content of the undisplayed part of the live broadcast part based on the game progress information received previously and stored in the buffer, and outputs until the end of the live broadcast part. A conversion screen will then be displayed.

After performing a part transition operation, the performer prepares for live streaming the next part and reconfirms the arrangements and progress. Here, an example will be described in which a live game part is performed as the next part. A rock, paper, scissors game is exemplified as a type of game. When preparations for the next part are complete, the performer performs a part start operation (operating a predetermined button, etc.) on the controller 540 to start the next part. When the game play terminal 300 detects a part start operation, it simultaneously distributes game progress information including part start information for ending the conversion screen display and starting the next part. The user terminal 100 that has received the game progress information including the part start information switches and displays the video based on the game progress information from the conversion screen at the received timing, regardless of the display time during which the conversion screen was displayed. .

At timing 13 in FIG. 29(A), game progress information including part start information is distributed from the game play terminal 300, and each of the user terminals 100 owned by users A to C receives the game progress information in a normal communication state. This shows the timing at which the video/utterance content based on the above was output. When the game progress information is distributed at timing 13, if the game progress information can be received under normal communication conditions (such as when no sudden or temporary problems have occurred since the start of viewing), the user terminal Video and speech content based on the game progress information are output simultaneously (or almost simultaneously, taking into account the difference in processing ability and performance of the terminals) for 100 games. As a result, even for the user terminal 100 that has been delayed until the conversion screen is displayed, the video of the next part can be displayed at approximately the same timing as other user terminals 100. In other words, even if a delay occurs, the delay can be absorbed (reset) while the conversion screen is being displayed without damaging the distribution content, and game progress information including part start information can be transmitted to each of the plurality of user terminals 100. The video of the next part can be displayed at the timing of reception. Specifically, while making it possible to output live-distributed videos, speech content, etc. to the end without forcibly ending it midway through, multiple user terminals 100 including the user terminals 100 owned by users A to C that were experiencing delays. Display images and speech content can be synchronized. This allows the user to input items, input comments, etc. at appropriate timings in accordance with the status of the avatar object 610 during the live game part, and to appeal to performers and other users.

In addition, as mentioned above, between performing part transition operations and performing part start operations, the performers will need some time to prepare for the live streaming of the next part and reconfirm the arrangements. It is expected that it will take some time. In this embodiment, by effectively utilizing the time that occurs before moving to the next part, delays at multiple user terminals 100 are absorbed while displaying the conversion screen without damaging the distribution content, and The video of the next part can be displayed (synchronized) between the plurality of user terminals 100 at approximately the same timing.

(Live game part: example of the second part)
In the live game part of the rock-paper-scissors game, an avatar object 610 and choice objects 281 to 283 are placed in a virtual space 600A defined on the game play terminal 300 side, and a wreath object reminiscent of a wreath is placed as a background. (See FIG. 28(D), which is a display example on the user terminal 100 side, etc.). Other objects arranged in the virtual space 600A include objects that decorate the periphery of the avatar object 610, viewer objects corresponding to viewers, background objects, floor objects, and the like. Even in the live game part, the avatar object 610 moves according to the motion of the performer and utters the voice of the performer.

Further, each of the option objects 281 to 283 is composed of a rod portion and a disk portion at the tip. On the surface of the disk part, illustrations representing Gu, Choki, and Pa are drawn. Each of the option objects 281 to 283 can grab a stick in response to a grabbing motion by the player. The operation (motion) in which the player grasps the option objects 281 to 283, etc. is a motion that is attached to the player's hand and detects the movement of the hand (for example, the position of each finger, the shape and direction of the grip, etc.) The sensor 520 is detected by the controller 540 provided with the sensor 520.The player grabs the option object corresponding to the final move (goo, chop, pa) among the option objects 281 to 283 and holds it up. The move made by the object 610 is determined.

During the live game part, the game play terminal 300 provides information and audio data for specifying the position, posture, orientation, etc. of objects (avatar object 610, choice objects 281 to 283, etc.) arranged in the virtual space 600A. (movement instruction data), etc., is live-distributed to a plurality of user terminals 100 at predetermined time intervals. Note that the game progress information distributed from the game play terminal 300 to the plurality of user terminals 100 includes not only action instruction data but also initial arrangement information, game information 132, distribution time, and the moves determined by the avatar object 610. It includes information for specifying the type of option object selected (goo, scissor, par), the current part/scene/period, etc. As a result, on the user terminal 100 side, similarly to the virtual space 600A, the virtual space 600B in which objects such as the avatar object 610, the option objects 281 to 283, the wreath object 284, the viewer object, and the background object are arranged is displayed. stipulated. In addition, on the user terminal 100 side, the avatar object 610 or the like is operated in the virtual space 600B based on the game progress information, and the avatar object 610 or the like is operated in the virtual space 600B to capture images from the virtual camera 620B placed at a position corresponding to the input operation from the user. A visual field image is displayed on the touch screen 15. Note that the objects shared between the game play terminal 300 and the plurality of user terminals 100 include at least the avatar object 610 that is performed live by the performer among the objects arranged in the virtual space 600A. It may be only the avatar object 610, or it may include objects other than the avatar object 610 and the option objects 281 to 283.

On the user terminal 100 side, when game progress information including part start information is received, the conversion screen is switched to a display corresponding to the next part based on the game progress information. At this time, the position and orientation of the virtual camera 620B (that is, the viewing area 640B for each user) are the position where the avatar object 610 is viewed from the front (predetermined position) and The direction (predetermined direction) is set. Note that the position and orientation of the virtual camera 620B when starting a new part are not limited to this, and may inherit the position and orientation of the virtual camera 620B before displaying the conversion screen.

FIGS. 28(D) to 28(F) are diagrams showing display examples of the touch screen 15 of the user terminal 100 while the rock, paper, scissors game is in progress. In the examples of FIGS. 28(D) to 28(F), it is assumed that the virtual camera 620B is placed at a position where the avatar object 610 is viewed from the front. Therefore, as illustrated in FIG. 28(D), the avatar object 610 is displayed in an upright position at approximately the center of the touch screen 15 as a view image from the virtual camera 620B, and the avatar object 610 as a reference, option objects 281 to 283 are displayed on the right front of the avatar object 610, and a wreath object 284 and the like are displayed behind it. The avatar object 610 in FIG. 28(E) shows a scene in which, for example, the avatar object 610 is thinking, "What should I say? Which should I use?" and a voice is being output along with a motion. .

Further, while the rock, paper, scissors game is in progress, various UI images, images related to game information, etc. are displayed on the touch screen 15. In FIG. 28(D), for example, a selection UI 180, a rule explanation display 181, an item insertion icon 185, a comment input icon 186, etc. are displayed superimposed on the view image in the virtual space. Further, at the lower left of the touch screen 15, a comment display section 184 is provided for displaying the contents of comments input to the avatar object 610.

The rule explanation display 181 is an image for explaining the rules for the rock, paper, scissors game. In FIG. 28(D), for example, messages such as "・3 rounds in total!", "・Press the button within the time limit!", "Win 50 points", "Aim for 10 points", "Loss 0 points", etc. are displayed. Note that a message such as "1st round" is displayed at the upper left of the touch screen 15 as information for specifying which scene (for example, what round) in the live game part it is.

The selection UI 180 is a UI that includes a button icon for the user to select a move to make, and for accepting an input operation (tap operation) to the button icon. In other words, the selection UI 180 can be said to be a UI image for accepting operations related to the results of the rock, paper, scissors game. In the example of FIG. 28(D), the selection UI 180 is displayed with priority over the visual field image including the avatar object 610, the comment display section 184, the item input icon 185, the comment input icon 186, and the like. In Figure 28 (D), a message such as "Choose your move!" is displayed, and below it, from the left, button icons corresponding to goo, button icons corresponding to scissors, and button icons corresponding to par are displayed. has been done. Thereby, the user can intuitively recognize that he or she can select the hand to be used by touching the button icon. In addition, on the left side of the message "Choose your move!", a time limit (also referred to as the reception period; here "10") for selecting a move is displayed, and on the right side of the message (in the selection UI 180) In the upper right corner), an "x" button for erasing the display of the selection UI 180 itself is displayed. Note that the Rock, Paper, Scissors game starts with the selection UI 180 displayed, but if the user's initial settings are set to hide the selection UI 180 at the start of the game, etc. The process may be started in a state where the selection UI 180 is not displayed (such as the screen illustrated in FIG. 28(I)). The time limit displayed in the selection UI 180 starts counting (starts subtracting) after the reception of the operation by the performer starts.

Even if the user has not finished selecting the hand to be played (input operation to the button icon corresponding to Goo, Choki, or Pa), the user can select the "X" button (touch operation on the "X" button, etc.). UI 180 can be deleted. This is because when the selection UI 180 is displayed and the avatar object 610 (character) is hidden in the selection UI 180, as shown in FIG. This is because it is not visible (the overlapping portion is not visible). Additionally, when the selection UI 180 is displayed, the comment display area 184, item insertion icon 185, etc. are hidden, as shown in FIG. This is because the overlapping portion is not visible) and is difficult to operate (or cannot be operated). By erasing the selection UI 180 according to the user's own operation, the user can view images that were not visible using the selection UI 180.

The item input icon 185 is an icon that triggers input of an item and giving it to the avatar object 610. When an input operation to the item input icon 185 is accepted, a window for selecting and inputting items from a plurality of types is displayed. As described above, the selection UI 180 is displayed with priority over the item input icon 185. Therefore, when the selection UI 180 is displayed superimposed on the item insertion icon 185, input operations to the item insertion icon 185 are not effectively accepted unless the selection UI 180 is hidden. Good too. Furthermore, even when the selection UI 180 is displayed superimposed on the item insertion icon 185, input operations to the item insertion icon 185 are enabled while the selection UI 180 remains displayed. It may also be accepted.

The comment input icon 186 is an icon that provides an opportunity to input a comment to the avatar object 610. When an input operation to the comment input icon 186 is accepted, a window for inputting and transmitting a comment is displayed. Note that the comment input icon 186 is displayed in a position where the selection UI 180 is not displayed in an overlapping manner, but as with the item input icon 185, unless the selection UI 180 is deleted, An input operation to the comment input icon 18 may not be effectively accepted. Further, even when the selection UI 180 is displayed, an input operation to the comment input icon 18 may be effectively accepted while the selection UI 180 remains displayed. In the comment display section 184, a predetermined number of comments input to the avatar object 610 are displayed in order from the newest to the newest, together with the user name.

The item insertion icon 185 and the comment input icon 186 are icons that each accept an input operation for the user to interact with the avatar object 610, and can also be said to be interaction images. Furthermore, the comment display section 184 can be said to be an interaction image because it displays the result of the interaction performed by the user with the avatar object 610.

FIG. 28(E) is an example of a display when the user selects a move to make. FIG. 28(E) shows an example in which "Goo" is selected. When "Goo" is selected, the "Goo" button icon in the selection UI 180 is updated to a selected state (enclosed in a double circle), as shown in FIG. 28(E). . Note that in FIG. 28(E), since the time limit is "7", the user has decided to make a move with 7 seconds remaining. On the other hand, the avatar object 610 in FIG. 28(E) seems to be grasping the option objects 281 to 283 and thinking, "Which one should I do? Should I go for a shortcut or a shortcut?" (motion) Also shown is a scene in which audio is being output. Note that the move selected by the user in the rock-paper-scissors game is stored in a predetermined area of the memory 11, and thereafter, the move to be played can be changed by operating the button icon again within the time limit. Furthermore, even if the selection UI 180 is deleted with the hand to be played selected, information that allows identification of the selected hand is maintained in the memory 11.

In the rock-paper-scissors game, after the time limit has passed, there is a predetermined period in which the performer can make a move (hereinafter also referred to as a period in which the performer can make a move). ” and the player (avatar object 610) raises one of the option objects 281 to 283 upward, thereby announcing the game result.

FIG. 28(F) is a display example when the avatar object 610 moves after the time limit has passed. In FIG. 28(F), the avatar object 610 is holding up the option object 282 of "paperwork". This notifies the user that the avatar object 610 has issued a "chop". When the avatar object 610 makes a move, information for specifying the type of option object determined as the move made by the avatar object 610 (goo, scissor, par) is received. Based on the information, the processor 10 compares the hand made by the user to determine whether the user has won or lost. In FIG. 28, since the user selected "Goo", it is determined that the user has won. As a result, a message such as "Win" is displayed above the avatar object 610. Note that in the rock-paper-scissors game, the moves made by the avatar object 610 are the same for all users, but the moves selected by the users differ for each user. Therefore, the actions of the avatar object 610 and the like played by the player will be the same among the plurality of user terminals, but the results of the rock-paper-scissors game will differ among the plurality of user terminals depending on the user's input operation.

When a performer wants to move to the next scene at the end of a certain scene (one round) even during the same part, the performer can send a scene to the controller 540 to move to the next scene (next round). Perform a transition operation (predetermined button operation, etc.). When the game play terminal 300 detects a scene transition operation, it outputs game progress information including scene end information for moving to preparation for the next scene in the current part.

On the user terminal 100 side, by receiving the game progress information including the scene end information, the position and orientation of the virtual camera 620B (i.e., the user The conversion screen is displayed over the entire display area of the touch screen 15 regardless of the viewing area 640B) (see FIG. 28(G)). Thereby, it is possible to notify the user in an easy-to-understand manner that the process for changing the scene is being performed. In addition, even if there is a lag (delay) on any of the user terminals 100 during the first round of the rock-paper-scissors game, the part ends while the conversion screen is displayed, similar to when changing parts. Game progress information including information Outputs the video and speech content of the undisplayed part of the live broadcast part based on the game progress information received previously and stored in the buffer, and outputs until the end of the first round of the rock-paper-scissors game. After that, the conversion screen will be displayed.

The performer performs the scene transition operation in the same way as when changing parts, and then prepares for live distribution of the next scene and reconfirms the arrangements and progress. When preparations for the next scene are completed, a scene start operation (such as a predetermined button operation) is performed on the controller 540 to start the next scene. When the game play terminal 300 detects a scene start operation, it simultaneously distributes game progress information including scene start information for ending the switching screen display and starting the next scene. The user terminal 100 that has received the game progress information including the scene start information switches and displays the video based on the game progress information from the conversion screen at the received timing, regardless of the display time of the conversion screen. Specifically, the performer performs a scene transition operation at the end of the first round of the rock-paper-scissors game, and then prepares for the second round and reconfirms the arrangements and progress. When preparations for the second round are completed, the performer performs a scene start operation on the controller 540 to start the second round. As a result, even for the user terminal 100 that has been delayed until the conversion screen is displayed, the video of the second round, which is the next scene, can be displayed at approximately the same timing as the other user terminals 100. In other words, even if a delay occurs during the first round, the delay can be absorbed (reset) while the conversion screen is displayed, and each of the plurality of user terminals 100 receives game progress information including scene start information. At this timing, the next scene, the video of the second round, can be displayed. Thereby, in the second round, the user can insert items, input comments, etc. at appropriate timings depending on the situation of the avatar object 610, etc., and can appeal to the performers and other users.

FIG. 28(H) shows a display example on the touch screen 15 when the second round of the Rock, Paper, Scissors game is started based on game progress information including scene start information. It is expected that a certain amount of time will be required between performing the scene transition operation and performing the scene start operation, as the performer will prepare for the next scene's live distribution and reconfirm the arrangements. In this embodiment, by effectively utilizing the time that occurs before transitioning to the next scene in the same part, the delay at multiple user terminals 100 is absorbed while displaying the transition screen, and the video of the next scene is can be displayed (synchronized) at approximately the same timing between a plurality of user terminals 100. Note that the position and orientation of the virtual camera 620B when a new scene starts in the same part (that is, the viewing area 640B for each user) is different from that at the time of part transition, and is the same as the position of the virtual camera 620B before the transition screen is displayed. However, the present invention is not limited to this, and the front view position (predetermined position) and direction (predetermined direction) of the avatar object 610 are set regardless of the position and direction of the virtual camera 620B before the conversion screen is displayed. It may be.

FIG. 28(I) is a diagram showing a display example of the touch screen 15 when an input operation to the "X" button in the selection UI 180 is accepted. Similar to FIG. 28(D), FIG. 28(I) shows a state in which the remaining time limit immediately after the start of the rock-paper-scissors game is "10" seconds and the user has not yet selected a move.

In rock-paper-scissors games, performers tend to use motions and speech to enliven the game. In addition, especially immediately after the start of the rock-paper-scissors game, since there is a high percentage of other users selecting moves to play, it is better for the performer to respond (motions) than to operate the item insertion icon 185 or input comments. and speech) is relatively high. However, in the example of FIG. 28(D), the selection UI 180 is displayed superimposed on the avatar object 610, and some movements of the avatar object 610 cannot be visually recognized, so even if there is a response, it may not be visible. There is sex. Note that depending on the position and orientation of the virtual camera 600B, the selection UI 180 may not be superimposed on the avatar object 610, but on the contrary, the selection UI 180 may be superimposed on the face, body, or the entire avatar object 610. There may also be cases where this happens. In addition, in the rock-paper-scissors game, the selection UI 180 is displayed superimposed on the item insertion icon 185 and the comment input icon 186, as shown in FIG. 28(D), regardless of the position of the virtual camera 600B. This creates a situation in which it is difficult for the user to interact with the avatar object 610 and it is difficult to check the response.

Considering this situation, in the game of this embodiment, even when operations related to the result of the game are not accepted, an input operation to the "x" button in the selection UI 180 allows the selection UI 180 to be can be erased. FIG. 28(I) shows a display example of the touch screen 15 when the "x" button in the selection UI 180 is touched and the selection UI 180 is erased in a situation such as in FIG. 28(D). There is. As a result, the avatar object 610, the item input icon 185, and the comment input icon 186, which were not visible due to the selection UI 180, become visible. As a result, by the user's own operation, it is possible to check the movement (including responses) of the avatar object 610, and it is possible to change the situation to a situation where it is easy to perform interaction and check the response from the performer.

Furthermore, when the selection UI 180 is deleted, an icon 187 (such as a message icon such as "Return to the game") for displaying the selection UI 180 is displayed on the touch screen 15 again, as shown in FIG. 28(I). be done. The icon 187 has a smaller display size than the selection UI 180, a display position closer to the edge than the selection UI 180, and is displayed in a position that does not overlap with any other icons. Therefore, it is possible to prevent inconveniences such as the image including the avatar object 610 becoming invisible due to the icon 187. Note that the remaining time limit may be displayed in a predetermined area of the touch screen 15 (for example, the area within the icon 187) even when the selection UI 180 is erased.

After the selection UI 180 is deleted, the selection UI 180 is displayed again in response to the user's operation on the icon 187, and the user can select or change the move he or she wants to make. Furthermore, even if the selection UI 180 is erased, the processor 10 automatically displays the selection UI 180 again according to the remaining time limit, and forces the user to take action. It is possible to confirm the selected or already selected moves. If no move has been selected yet, the selection UI 180 is redisplayed in such a manner that no move is selected. On the other hand, when the move to be put out has already been selected, the selection UI 180 is redisplayed in the manner in which the move is selected. Note that if the user does not select a move within the time limit, the user's move may be automatically and randomly determined. In this case, for example, when the selection UI 180 is redisplayed, The selection UI 180 may be redisplayed in such a manner that the automatically determined hand is selected, and when the time limit expires, the automatically determined hand is selected. The selection UI 180 may be redisplayed.

28 and 29(A), the time required for the performer to prepare for the next part, reconfirm the arrangements, etc. at the time of part transition or scene transition (for example, at least 5 seconds, or 10 seconds). Display the conversion screen after outputting the current part or scene to the end, and switch from the conversion screen to the video based on the game progress information at the timing when part start information or scene start information is received. We have explained an example of outputting.

However, the time required for the performer to prepare for the next part, reconfirm the arrangements, etc., that is, the time required from the part transition operation to the part start operation, and the time required from the scene transition operation to the scene start operation. If the delay exceeds the average time required, there is a high possibility that the delay cannot be absorbed while displaying the conversion screen. For this reason, a time shorter than the average time required from a part transition operation to a part start operation or from a scene transition operation to a scene start operation is determined as a specific time (for example, 5 seconds), and the user terminal 100 When viewing a live broadcast, it is determined whether or not the time required from the distribution of game progress information to the display of the video based on the game progress information is a specific time or more, and the specified If it is determined that the situation requires more than a certain amount of time, a configuration may be provided that forcibly displays a conversion screen to absorb a delay that exceeds a certain amount of time. In this case, the controller 540 may not effectively accept the part start operation or the scene start operation until at least a specific time (5 seconds) has elapsed after accepting the part change operation or the scene change operation. The timing at which the game progress information is distributed is specified, for example, from the distribution time included in the game progress information. Such an example will be explained with reference to FIG. 29(B).

FIG. 29(B) shows an example of the timing at which a video corresponding to the game progress information is displayed on the user terminal 100 owned by user X after the game progress information is delivered from the game play terminal 300 during live distribution. It is a figure for explaining. FIG. 29B shows a timing chart when a delay occurs in the user terminal 100 owned by user X and is not resolved, and a timing chart when the delay is resolved and the delay is resolved.

The unresolved delay column in FIG. 29(B) indicates that the user terminal 100 owned by user An example is shown. In the delay unresolved column, for example, the video based on the game progress information θ distributed from the game play terminal 300 at timing 1 is displayed at the user terminal 100 of user X at timing 2 (delay x from timing 1). An example is shown below. Here, the delay x is a time longer than the above-mentioned specific time (for example, 5 seconds). In this case, after the game progress information including part end information is distributed at timing 4, the timing at which the conversion screen is displayed based on the game progress information is as shown in timing 6 in the When Delay Not Resolved column. This is later than timing 5 at which game progress information including start information can be distributed, and there is a possibility that the delay cannot be absorbed while the conversion screen is being displayed.

In this way, when the user terminal 100 side determines that the delay x is longer than the specific time, the user terminal 100 forces the conversion screen to appear, as shown at timing 2 in the delay cancellation column of FIG. 29(B). indicate. Further, the user terminal 100 switches to and displays the live distribution video based on the game progress information received at timing 3 after a predetermined time (for example, 1 second, etc.) has elapsed since the conversion screen display. Thereby, the delay in the user terminal 100 owned by user X can be eliminated during live distribution. In this way, the user terminal 100 performs processing to eliminate the delay each time the delay time exceeds a specific time. As a result, even if game progress information including part end information is distributed as shown at timing 4, the user terminal 100 displays the conversion screen from the distribution to at least timing 5. can. Therefore, by making it possible to display the video of the next part at timing 5 when the game progress information including the part start information is distributed, it is possible to synchronize with other user terminals 100. By providing a configuration for displaying a conversion screen, the following effects are achieved.

The user terminal 100 in the above embodiment switches to a conversion screen when ending the current part or scene based on game progress information including part end information and scene end information, as shown in FIG. 29(A) etc. (Refer to timings 10 to 12), and when game progress information including part start information and scene start information is received, the time during which the conversion screen was displayed (or the time when game progress information including part end information was delivered) is displayed. Regardless of the time required from the time until the game progress information including the part start information is distributed, the video/utterance content based on the game progress information is output (see timing 13). As a result, even if there is a delay, the current part or scene can be output to the end while the conversion screen is displayed on other user terminals, and the game progresses when a new part or scene starts. From the moment the information is received, the next part or scene's video and utterances can be output. As a result, it is possible to avoid as much as possible the current part or scene ending midway and the next part or scene suddenly starting, thereby minimizing the impression of the delivery quality and causing a sense of discomfort in the delivery content. This can be prevented as much as possible. Further, the conversion screen is a screen on which a predetermined image is displayed over the entire display area of the touch screen 15, as shown in FIG. 28(C) and the like. Therefore, the processing load for displaying the conversion screen can be reduced.

Further, game progress information including part end information and scene end information is distributed all at once in response to a part change operation or a scene change operation on the controller 540. Further, game progress information including part start information and scene start information is distributed all at once in response to a part start operation and a scene start operation on the controller 540. On the other hand, on the user terminal 100 side, the time during which the conversion screen is displayed is after the game progress information including part end information and scene end information is distributed on the game play terminal 300 side. Even if the time required for the game progress information including the game progress information to be delivered (i.e. the time at which the conversion screen can be displayed) has not yet been reached, when the game progress information including the part start information or scene start information is received, the relevant information will be sent from the conversion screen. Outputs video and speech content based on game progress information. As a result, there is no need to manage the time during which the conversion screen is displayed on the user terminal 100 side, and the processing when game progress information including part start information and scene start information is received can be uniform. can be simplified.

In addition, as shown in FIG. 29(B), when the delay time exceeds a specific time during live streaming, after displaying the conversion screen at that point, the game progress information received while the conversion screen was displayed is Output the video and utterance content based on the information. As a result, if there is an excessive delay, a conversion screen can be displayed during live distribution to ensure synchronization.

In addition, the specific time is the time determined according to the time required from the part transition operation to the part start operation and the time required from the scene transition operation to the scene start operation, and is the average time required from the part transition operation to the part start operation. The time required is shorter than the average time required from the scene transition operation to the scene start operation. As a result, it is possible to synchronize in advance during live distribution a delay that is likely not to be absorbed during the display of a transition screen based on game progress information including part end information and scene end information.

Next, the processing contents when viewing a live broadcast from the middle will be explained. Depending on the content of the part being distributed, the user may be at a disadvantage compared to other users if the user starts viewing from the middle. For example, if the user starts watching the live game part just before the live game starts, the user may not be able to participate in the game because the user does not understand the live game or is not ready for the game. Even so, the user may be at a disadvantage compared to users who have been viewing the program for some time. Therefore, in this embodiment, when viewing a live broadcast is started in the middle of a live game part (specific part), the timing is a period during which it may be disadvantageous even if you participate in the game in the live game part. When the game is in progress, standby information is displayed that prompts the user to wait until a predetermined timing (for example, the timing at which the next battle scene starts). A specific example will be described below using FIG. 30. FIG. 30(A) is a diagram for explaining an example of a predetermined schedule regarding the progress of the game from the start of one scene (for example, the first round) of the rock-paper-scissors game part to the notification of the result. (B) to FIG. 30(E) are diagrams for explaining display examples on the touch screen 15 of the user terminal 100 when viewing a live broadcast from the middle.

As shown in Figure 30 (A), when the first round of the rock-paper-scissors game begins, first, a predetermined conversation (greetings, reports on recent events, results and impressions from the previous game, etc.) and This is a ``conversation/explanation period'' during which part previews and explanations (for example, information about the start of a live game, an overview of the live game, etc.) are given. After that, there will be an "operation reception period" in which operations are accepted from the user, and after the operation reception period ends, there will be a "performer availability period", and finally a "results period" in which results will be reported. predetermined. In the "operation acceptance period", after the performer gives a hint of the move to be made and information related to the move to be made, there is a period of 10 seconds during which operations are accepted as described above. The performer plays the rock-paper-scissors game according to such a schedule. Specifically, by operating the controller 540, the performer causes the transition from the "conversation/explanation period" to the "operation reception period", and from the "operation reception period" to the "performer handout period". and transition from the "performer handout period" to the "result period". In other words, similarly to part transitions and scene transitions, it is possible to transition to each period at the timing of the performer.

In addition, during the above period, for example, during the "conversation/explanation period," hints of the moves given by the performer are not yet discussed, so no matter when you start watching, there will be a difference between you and the user who is already watching. is unlikely to cause a difference in the degree of advantage. On the other hand, the "operation acceptance period" occurs after the performer has given hints about the moves, etc., which may be at a disadvantage compared to users who were watching, and if the remaining time of the operation acceptance period is short. It will also be disadvantageous. Furthermore, during the "performer availability period" and "results period," the current round has already ended, so mid-participation is not possible. As described above, the degree of advantage of a user who starts viewing from the middle of the live game part differs depending on which period the user starts viewing.

Therefore, the user terminal 100 in this embodiment determines whether or not it is a period that may be disadvantageous to the user when starting viewing a live broadcast from the middle, and if it is a period that may be disadvantageous to the user, information corresponding to the period is determined. Performs processing to notify. In the live game part in this embodiment, as shown in FIG. 30(A), the "period for conversation/explanation, etc." is set as the period during which participation is possible, and the "operation acceptance period" to "result period" are set as the period during which participation is not possible. It shall be determined in the possible period. A specific example will be explained using FIGS. 30(B) to 30(E).

FIG. 30(B) is a display example when an icon B1 for starting viewing of a live stream being distributed is displayed on the touch screen 15. At the top of the touch screen 15, a message such as "live game distribution in progress" is scrolled and displayed, and an icon B1 written with "live viewing start" is displayed in the center of the screen. Thereby, the user can be informed that the live game part is currently being distributed and that the user can view (participate in) the live game part from the middle by touching the icon B1. When the icon B1 is touched, a conversion screen is uniformly displayed as shown in FIG. 30(C), triggered by the touch operation. Thereby, it is possible to notify the user in an easy-to-understand manner that the viewing start operation has been effectively accepted and processing for live viewing is being performed. The conversion screen in FIG. 30(C) is an example of the same screen as the conversion screen displayed when changing to part-time, etc., but is not limited to this, and is a screen different from the conversion screen displayed when changing to part-time, etc. It may be.

FIGS. 30(D) and 30(E) are diagrams illustrating a state in which video and speech content to be live distributed are output in response to a touch operation on the icon B1. Of these, FIG. 30(D) shows a display example when viewing is started during the "conversation/explanation period" of the live game part (for example, FIG. 28(D), etc.). More specifically, the video is displayed based on the received game progress information. FIG. 30(D) shows a situation in which a story related to the rock-paper-scissors game and an explanation of the rules are being given.

On the other hand, FIG. 30(E) shows a display example when viewing of the live game part is started during the "operation acceptance period" (for example, FIG. 28(E), etc.). The user terminal 100 determines whether the current period is a period after the "operation acceptance period" during which participation is not possible (a period in which participation may be at a disadvantage), based on the game progress information received when viewing the live broadcast. to identify. In addition, when the user terminal 100 is in a period in which participation is not possible after the "operation reception period," the user terminal 100 displays a standby display that is superimposed on the video based on the received game progress information (priority is given to the video) and prompts the user to wait until the next round. Image 188 is displayed. In FIG. 30E, a standby image 188 is shown that scrolls and displays a message such as "Game in progress... Please wait until the next game (second round starts)!". This prevents the user from being disadvantaged if they start watching in the middle, and also clearly informs the user that they can participate from a predetermined timing (for example, when the second round starts). can. On the other hand, when viewing is started during the "conversation/explanation period," the standby image 188 is not displayed as shown in FIG. You can prevent this from happening to you.

Furthermore, in FIG. 30(E), the selection UI 180 is hidden because the user cannot participate in the rock-paper-scissors game. On the other hand, the rule explanation display 181, item input icon 185, and comment input icon 186 are displayed in the same manner as in FIG. 30(D). This allows you to check the rules in advance even if you are unable to participate. In addition, inputting items, inputting comments, etc. can be performed according to the intention of the user who is viewing the program midway through, since this does not cause any disadvantage to the user. Note that when the first round ends and the second round starts, the user can participate in the rock-paper-scissors game, so the selection UI 180 is also displayed (see FIG. 28(H)). .

In the live distribution game of this embodiment, an avatar object 610 is placed in the virtual space 600B. The game play terminal 300 (server 200) simultaneously broadcasts game progress information including voice data and motion data according to the voice and motion input by the performer to a plurality of user terminals 100 live. Each of the plurality of user terminals 100 controls the display state of the avatar object 610 by reproducing motion data corresponding to the reproduced audio data in accordance with the reproduction of the audio data among the received game progress information.

In the live distribution game, the same virtual space 600B as the virtual space 600A defined by the game play terminal 300 is defined by the user terminal 100. The game play terminal 300 distributes game progress information (see FIG. 31(A)) including a header, action instruction data, and virtual space control data to the plurality of user terminals 100 via the server 200. The motion instruction data includes audio data corresponding to the voice uttered by the performer and motion data corresponding to the motion of the performer. Further, the virtual space control data is data for controlling progress within the virtual space 600B, and specifically, is data for operating various objects placed in the virtual space 600B. The virtual space control data also includes game information 132, information for specifying the type of option object (goo, scissor, pass) determined as the move of the avatar object 610, and the like. User terminal 100 reproduces motion data and virtual space control data based on the audio data, and controls the movement of avatar object 610 and various objects in virtual space 600B.

The game progress information is distributed every predetermined time (for example, 1/60 seconds), and the game progress information distributed within the predetermined time corresponds to one frame. That is, each of the audio data, motion data, and virtual space control data is composed of data in units of frames. The header includes a frame number that identifies the frame.

When an operation to start a live streaming game is performed, the user terminal 100 identifies audio data from the game progress information distributed from the game play terminal 300 via the server 200, and performs input/output based on the audio data. The speaker included in the IF 14 outputs the voice uttered by the performer.

The memory 11 of the user terminal 100 stores in advance multiple types of object data for displaying objects including the avatar object 610. Furthermore, the memory 11 stores an object ID for identifying the type of object.

The user terminal 100 identifies motion data and virtual space control data from the game progress information distributed from the game play terminal 300, and analyzes (rendering) the motion data and virtual space control data to render the avatar object 610. Specify the type, position, orientation, etc. of the included object. Further, the user terminal 100 arranges objects including the avatar object 610 in the virtual space 600B based on the analysis result and the object data and object ID stored in the memory 11. The user terminal 100 displays on the touch screen 15 a view image 660 of the view area 640B according to the position, orientation, and tilt of the virtual camera 620B arranged in the virtual space 600B.

The audio data, motion data, and virtual space control data included in each frame of game progress information distributed from the game play terminal 300 are input to a buffer memory provided in the memory 11 or the like. The user terminal 100 identifies a second timing to start playing the audio data based on the first timing at which the audio data is received, and reproduces the audio data, motion data, and virtual space of the same frame stored in the buffer memory. Reproduction of each piece of control data is started from the second timing (see FIG. 31(B)). The second timing may be the timing at which the reproduction of the previously received audio data ends, or may be the timing at which a predetermined time has elapsed. After the playback is started, the playback is continued based on the system time managed by the processor 10, for example. This makes it possible to synchronize audio and motion during playback.

However, due to deterioration of the communication environment around the user terminal 100 (for example, a decrease in reception strength or generation of noise), some of the audio data, motion data, and virtual space control data may be lost or lost. The data may be damaged and, as a result, the data may not be playable.

Therefore, when the user terminal 100 cannot reproduce the audio data of a certain frame, it skips the reproduction of the motion data and virtual space control data included in the frame (see frame m in FIG. 32(A)). That is, when audio data cannot be received or cannot be reproduced, the motion data and virtual space control data corresponding to the audio data are not reproduced even if the motion data and virtual space control data corresponding to the audio data can be reproduced. Note that in FIG. 32, the frame surrounding the data is a dotted line frame to indicate that the data cannot be received or cannot be reproduced.

On the other hand, if the user terminal 100 can reproduce the audio data of a certain frame, the user terminal 100 can reproduce the motion data or virtual space control data of the frame even if the motion data or virtual space control data of the frame cannot be reproduced. In the same way as when possible, playback of the audio data of the frame is started (see frame n+1 in FIG. 32(B)). That is, when the audio data can be received and played back, the playback of the audio data is started even if the motion data and virtual space control data corresponding to the audio data cannot be played back. Note that when starting the reproduction of the audio data, if there is data that can be reproduced among the motion data and the virtual space control data, the reproduction of the audio data is started, and the reproduction of that data is also started.

In FIG. 32(A), the motion data and virtual space control data included in the game progress information of frame m can be received and played back, while the audio data cannot be received or played back. ing. In this case, the user terminal 100 skips reproduction of the motion data and virtual space control data of frame m. In FIG. 32A, the word "skip" is attached to motion data and virtual space control data whose reproduction is to be skipped. Note that if the game progress information for the next frame m+1 is received normally, a second timing to start playing the audio data included in the game progress information is specified, and the audio data is played at the second timing. If it is possible, the audio data is played back, and based on the playback of the audio data, the playback of the motion data and virtual space control data of the same frame stored in the buffer memory is started.

FIG. 32(B) shows a situation where motion data and virtual space control data included in the game progress information of frame n+1 cannot be received or cannot be played back, while audio data can be received and played back. ing. In this case, the user terminal 100 starts playing back the audio data of the frame n+1. In addition, in a situation where either the motion data or the virtual space control data included in the game progress information of frame n+1 can be received and reproduced, the data is transmitted from the second timing when the playback of the audio data of frame n+1 is started. Start playing.

According to this embodiment, the server 200 distributes audio data and motion data corresponding to the audio and motion input by the performer playing the avatar object 610 in the virtual space 600B. In accordance with the reproduction of the audio data, the user terminal 100 reproduces motion data corresponding to the reproduced audio data to control the movement, that is, the display state, of the avatar object 610 in the virtual space 600B. This makes it possible to synchronize the display state of the character with the sound by playing back the motion data in response to the playback of the sound data using the sound data as a reference, and the display state of the character separate from the sound. This makes it possible to avoid as much as possible inconveniences that may cause a sense of discomfort during viewing, such as when the content is controlled.

Further, according to the present embodiment, the user terminal 100 plays motion data based on the audio data, and controls the movement of other objects within the virtual space 600B, that is, the progress within the virtual space 600B. As a result, it is possible to further suppress the feeling of discomfort during viewing, such as when the progress in the virtual space 600B is controlled apart from the audio.

Furthermore, according to the present embodiment, the user terminal 100 identifies the second timing to start playing the audio data based on the first timing at which the audio data is received, and generates motion data corresponding to the played audio data. playback is started from the second timing. This makes it possible to reliably synchronize the motion playback with reference to the voice input by the performer.

Further, according to this embodiment, the audio data is composed of data in units of frames, and the motion data is also composed of data in units of frames. When the user terminal 100 cannot reproduce the audio data of the first frame configuring the audio data, it skips the reproduction of the motion data of the first frame configuring the motion data (for example, see FIG. 32(A)). This avoids an inconvenience in which the display state of the character is controlled without sound being output, which causes a sense of discomfort during viewing.

Furthermore, according to the present embodiment, when the user terminal 100 can reproduce the audio data of the second frame constituting the audio data but cannot reproduce the motion data of the second frame, the user terminal 100 reproduces the motion data of the second frame. As in the case where it is possible, the reproduction of the audio data of the second frame is started (for example, see FIG. 32(B)). As a result, the performer's voice can be provided to the user as much as possible, and the user can recognize and understand the content etc. being distributed from the voice.

The audio data in this embodiment includes part end information and part start information related to the display of the conversion screen shown in FIG. 28(C) and the like. The audio data also includes scene end information and scene start information related to the display of the transition screen shown in FIG. 28(G) and the like. Furthermore, the audio data includes the distribution time related to the display of the conversion screen shown at timing 2 in FIG. 29(B). As a result, when the video based on the game progress information is displayed on the user terminal 100, it is a case where the audio data can be played back, so the display of the conversion screen can be reliably started at the timing when it should be displayed, and the conversion screen The display can be reliably finished at the timing when it should be finished. The audio data also includes information for specifying the current part, scene, and period related to the display of the standby image 188 shown in FIG. 30(D). As a result, when the video based on the game progress information is displayed on the user terminal 100, it is a case where the audio data can be played back, so when starting viewing during the unavailable period, the standby image is reliably displayed. can.

<Modified example>
Modifications of the embodiment described above are listed below.

(1) The avatar object 610 in the above embodiment is a signal detected by a sensor included in the HMD 500 attached to the performer's head and a motion sensor 520 attached to the performer's hands, legs, waist, etc. An example of operating based on data has been described. However, the avatar object 610 is not limited to this, as long as it moves according to the movement of the performer. For example, the avatar object 610 may be used to analyze images captured from multiple directions of the performer using multiple cameras. By specifying the performer's movements on the game play terminal 300 side (distribution side) and outputting movement instruction data (game progress information including motion data) for making the avatar object 610 move based on the identification result, The user terminal 100 may display a display in which the avatar object 610 moves according to the movements of the performer.

Furthermore, the performer in the above embodiment described an example in which objects in the virtual space are confirmed from images displayed on the HMD 500 worn on the head. However, the present invention is not limited to this, as long as the performer can check the image in the virtual space. It may also display images in virtual space in different directions. In this case, the performer can confirm the virtual space by viewing images displayed on a plurality of display units arranged around the performer.

(2) In the above embodiment, a transition screen is exemplified as a specific video, and the transition screen displayed at the time of part transition (for example, FIG. 28(C)), the transition screen at the time of scene transition (for example, FIG. 28(G)), Further, an example has been described in which the conversion screen (for example, FIG. 29(B)) that is displayed when the delay time is determined to be equal to or longer than the specific time is the same screen. However, even if the transition screen has different forms depending on whether it is a part transition, a scene transition, or when the delay time is determined to be longer than a specific time, good. Further, the conversion screen at the time of part-time transition may have a different aspect depending on the type of part before transition or part after transition, even at the time of part-time transition. Further, the transition screen at the time of scene transition may have different modes depending on the type of scene before transition or scene after transition, even at the time of scene transition. In addition, the transition screen that is displayed when the delay time is determined to be longer than a certain time is a screen that is different from the transition screen when changing parts or scenes (for example, displaying a message such as "Synchronizing...") It may be. The conversion screen is not limited to the one shown in FIG. 28 as long as the user cannot visually recognize the avatar object 610 according to the movement of the performer, and for example, it may be a screen in which the display of the touch screen 15 is darkened (displaying a black screen). It may be.

(3) In the above embodiment, game progress information including information regarding the display of the conversion screen (part end information, part start information, etc.) is distributed according to the performer's operation, and the time during which the conversion screen can be displayed depends on the performer. An example was explained. However, the time during which the conversion screen can be displayed may be predetermined (for example, 10 seconds, etc.). In this case, the game play terminal 300 (distribution side) distributes information (part end information, etc.) for starting the display of the conversion screen according to the operation of the performer (distribution side), and displays the conversion screen. When a predetermined period of time (for example, 10 seconds, etc.) has elapsed after distributing the information to start, information for ending the display of the conversion screen (part-time start information, etc.) is automatically distributed. There may be. Furthermore, the time during which the transition screen can be displayed may be set differently depending on when the part changes and when the scene changes. For example, the time during the scene change is shorter than the time when the part changes ( For example, 5 seconds, etc.) may be set.

(4) In FIG. 29(B) in the above embodiment, at the timing (for example, timing 2) of displaying the video based on the received game progress information, a specific time has elapsed since the distribution based on the distribution time of the game progress information. An example of determining whether the time has elapsed has been described. However, the determination method is not limited to this, and for example, regardless of the timing of displaying the video based on the received game progress information, the determination method is not limited to this, and for example, regardless of the timing of displaying the video based on the received game progress information, the judgment method Based on the distribution time of the oldest game progress information, it may be determined at predetermined time intervals (for example, 1/60 seconds) whether or not a specific time period has passed since the distribution.

(5) In the above embodiment, the game using the selection UI 180 is a game in which the user competes against a performer (avatar object 610), and the selection UI 180 is used to perform operations related to the battle result (selecting the move the user makes). An example of accepting a manual selection operation for However, the competitive game is not limited to this, and may be a card game (for example, poker). In addition, games that use the selection UI 180 are not limited to competitive games, but include games where the goal is to clear a mission given by a performer (avatar object 610), and a game where the goal is to clear a mission given by a performer (avatar object 610), and a game where the goal is to complete a mission given by a performer (avatar object 610). It may also be a game in which players compete against each other.

(6) The image in the virtual space including the avatar object 610 played by the performer in the above embodiment receives movement instruction data, which is data distributed from the outside and includes motion data and audio data, and is displayed on the user terminal 100 side. An example (real-time rendering method) in which a moving image (video) is generated (rendered) and played back using the motion instruction data has been described. However, the game play terminal 300 and the distribution terminal 400 distribute video data for displaying a video in the virtual space including the avatar object 610, and the user terminal 100 uses the video data to display the video including the avatar object 610. It may also be a method (video distribution method) for reproducing moving images (videos). In other words, in addition to displaying images of the virtual space 600B on the user terminal 100 using a real-time rendering method, images of the virtual space 600B can be displayed on the user terminal using a video distribution method that distributes video data composed of multiple frames of images. 100 may be displayed.

(7) The display mode of the avatar object 610 in the above embodiment may be different for each user depending on the degree of progress of the game (for example, each time the player wins a rock-paper-scissors game). For example, even if the game progress information transmitted from the game play terminal 300 side is the same, the costume of the avatar object 610 may be changed according to the degree of progress of the game. In this case, a plurality of types of costume data for the avatar object 610 may be stored in advance on each user terminal 100 side. Note that the display mode of the avatar object 610 is not limited to the costume, and may be changed depending on the hairstyle, skin color, level of makeup, etc. This makes it possible to increase the variations of the avatar object 610 without increasing the processing load on the side of the game play terminal 300 that outputs game progress information, thereby making it possible to attract users.

Alternatively or in addition to this, the display mode of the avatar object 610 may be made different from the display mode displayed on the user terminals of other users, depending on the charge made by the user during the game. Furthermore, the display mode of the avatar object 610 may vary depending on the ranking of the user, which is updated according to the result of the game.

(8) In the above embodiment, after the reproduction of each of the audio data, motion data, and virtual space control data is started from the second timing, the reproduction is continued based on the system time (that is, after the reproduction starts is supplemented by system time). However, the continuation of the playback is not limited to the system time, and may be performed by simply playing back the stored data itself.

(9) In the above embodiment, the audio data, motion data, and virtual space control data are delivered live (real-time delivery). However, the data may be stored in the server 200 in advance and distributed to the user terminal 100 of the user in response to the user's viewing operation.

(10) In the embodiment described above, the virtual space control data is motion data for operating various objects placed in the virtual space 600B. However, in addition to the motion data, the virtual space control data includes audio data such as sound effects in the virtual space 600B, and camera control data for controlling the position and tilt of the virtual camera 620B placed in the virtual space 600B. It may include.

<Additional notes>
The matters described in each of the above embodiments are additionally described below.

(Additional note 1):
According to an aspect of an embodiment of the present disclosure, there is provided a game program executed on a terminal device (user terminal 100) including a processor, a memory, and a display unit, the game program having the processor perform a a step of receiving live-distributed data (game progress information) for displaying an image of a virtual space including characters moving in real time; and a step of receiving an image of the virtual space (600B) based on the received data. The data includes first data (part end information, scene end information) corresponding to the end of the first part constituting the live distribution, and second data following the first part. and second data (part start information, scene start information) corresponding to the start of the part of When the video in the virtual space of the part is switched to a predetermined specific video (conversion screen) and the second data is received while the specific video is being displayed, the display time during which the specific video was displayed is Regardless, the specific video is switched to the virtual space video of the second part for display (see FIGS. 28 and 29(A)).

(Additional note 2):
In (Additional Note 1), the second data is data that is distributed when a predetermined trigger (performer's operation) is established after the first data is distributed, and the displaying step includes the display time being the data that is distributed when the first data is distributed. Even when the time required from the data distribution to the second data distribution has not been reached, if the second data is received while the specific video is being displayed, the virtual space of the second part is moved from the specific video to the virtual space of the second part. (See FIG. 29(A)).

(Additional note 3):
In (Appendix 1) or (Appendix 2), the step of displaying includes whether or not the situation is such that a specific amount of time or more is required from the live distribution of the data to the display of the image of the virtual space based on the data. If the situation requires more than the specified time, the system switches to the specific video and displays it, and then switches to and displays the video in the virtual space based on the data received while displaying the specific video ( (See FIG. 29(B)).

(Additional note 4):
In (Appendix 3), the specific time is a time (shorter than an average time) that is predetermined according to the time required from the first data distribution to the second data distribution.

(Additional note 5):
In any of (Appendix 1) to (Appendix 4), the data includes voice data and motion data corresponding to the voice and motion input by the performer (see FIG. 31), and the displaying step includes the received data. In accordance with the playback of the audio data, motion data corresponding to the audio data is played based on the audio data to display the character moving in the virtual space, and the end of the first part is indicated. Information for specifying and information for specifying the start of the second part are included in the audio data of the data.

(Appendix 6):
In any of (Appendix 1) to (Appendix 5), the part constituting the live distribution includes a specific part (live game part) in which a game is played in which a user who owns the terminal device can participate; The displaying step includes, when starting viewing the live broadcast from the middle of the specific part, displaying the video of the virtual space of the specific part based on the received data, and prompting the user to wait until a predetermined timing. Display standby information (see Figure 30).

(Appendix 7):
In (Additional note 6), the displaying step includes displaying the standby information when the timing to start viewing the live broadcast from the middle of the specific part is during a specific period (participation impossible period) of the specific part. While not displayed, the standby information is displayed during a non-specific period (participation possible period) (see FIGS. 30(D) and (E)).

(Appendix 8):
In (Additional Note 7), the specific period is a period predetermined because the user may be at a disadvantage if he or she participates in the game.

(Appendix 9):
In any one of (Appendix 6) to (Appendix 8), the step of displaying, when starting viewing the live distribution from the middle of the specific part, displays the specific video and then displays the video based on the received data. The display is switched to the video of the virtual space (see FIG. 30(C)).

(Appendix 10):
In any one of (Appendix 1) to (Appendix 9), the specific video is a video that is displayed using the entire display area of the display unit (see FIG. 28(C)).

(Appendix 11):
In any one of (Appendix 1) to (Appendix 10), the specific video includes a video corresponding to the type of the first part that ends or the type of the second part that starts after displaying the specific video.

(Appendix 12):
According to an aspect of one embodiment, there is provided a game method executed on a terminal device (user terminal 100) including a processor, a memory, and a display unit, wherein the terminal device operates in real time according to the movements of a performer. a step of receiving live-distributed data (game progress information) for displaying an image of a virtual space including a character to be played; and displaying an image of the virtual space (600B) based on the received data. and the data includes first data (part end information, scene end information) corresponding to the end of the first part constituting the live distribution, and the start of the second part following the first part. and second data (part start information, scene start information) corresponding to the virtual space of the first part when the first part is finished based on the first data. When the video is switched to a predetermined specific video (conversion screen) and the second data is received while the specific video is being displayed, regardless of the display time during which the specific video was displayed, The specific video is switched to and displayed in the virtual space of the second part (see FIGS. 28 and 29(A)).

(Appendix 13):
According to an aspect of an embodiment, a terminal device (user terminal 100) includes a storage unit that stores a game program, and a control unit that controls the operation of the terminal device by executing the game program. the control unit receives live-distributed data (game progress information) for displaying an image of a virtual space including a character that moves in real time according to the movements of a performer; displaying an image of the virtual space (600B) based on the received data, the data includes first data (part end information, scene end information) corresponding to the end of the first part constituting the live distribution. information) and second data (part start information, scene start information) corresponding to the start of a second part following the first part, and the displaying step includes: When the first part is finished, the virtual space image of the first part is switched to a predetermined specific image (conversion screen) and the second data is received while the specific image is being displayed. Sometimes, the display is switched from the specific video to the virtual space video of the second part regardless of the display time during which the specific video was displayed (see FIGS. 28 and 29(A)).

[Example of implementation using software]
The control blocks (especially the control units 110, 210, 310, 410) of the user terminal 100, the server 200, the game play terminal 300 (HMD set 1000), and the distribution terminal 400 are based on logic formed in an integrated circuit (IC chip) or the like. It may be realized by a circuit (hardware) or by software.

In the latter case, the user terminal 100, the server 200, the game play terminal 300 (HMD set 1000), and the distribution terminal 400 are equipped with a computer that executes instructions of a program that is software that implements each function. This computer includes, for example, one or more processors and a computer-readable recording medium that stores the above program. In the computer, the processor reads the program from the recording medium and executes the program, thereby achieving the object of the present invention. As the processor, for example, a CPU (Central Processing Unit) can be used. As the recording medium, in addition to "non-temporary tangible media" such as ROM (Read Only Memory), tapes, disks, cards, semiconductor memories, programmable logic circuits, etc. can be used. Further, the computer may further include a RAM (Random Access Memory) for expanding the above program. Furthermore, the program may be supplied to the computer via any transmission medium (communication network, broadcast waves, etc.) that can transmit the program. Note that one aspect of the present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

The present invention is not limited to the embodiments described above, and various modifications can be made within the scope of the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. are also included within the technical scope of the present invention.

1 System, 2 Network, 3, 3A, 3B User (first user), 4 Player (performer), 10, 20, 30, 40 Processor, 11, 21, 31, 41 Memory, 12, 22, 32, 42 Storage , 13, 23, 33, 43 communication IF, 14, 24, 34, 44 input/output IF, 15, 45 touch screen, 17 camera, 18 ranging sensor, 51 monitor, 52 gaze sensor, 53 1st camera, 54 1st 2 camera, 55 microphone, 56 speaker, 100, 100A, 100B, 100C user terminal (computer, first computer, first information processing device), 110, 210, 310, 410 control unit (first control unit, second control unit) 111,311,413 Operation reception unit, 112,312,412 Display control unit, 113,313 UI control unit, 114,314 Animation generation unit, 115,315 Game progress unit, 116,316 Virtual space control unit, 117 Video playback section, 120, 220, 320, 420 Storage section (first storage section, second storage section), 131, 231, 331 Game program (program, first program), 132, 232, 332 Game information, 133 , 233, 333 user information, 151, 451 input unit, 152, 452 display unit (display), 200 server, 211 communication intermediary unit, 212 log generation unit, 213 list generation unit, 234, 421 user list, 300 game play terminal (external device, second external device), 317 reaction processing unit, 400 distribution terminal (external, first external device, computer, second information processing device), 411 communication control unit, 414 audio reception unit, 415 motion identification unit, 416 Operation instruction data generation unit, 422 Motion list, 423 Distribution program (program, second program), 540, 1020, 1021 Controller, 500 HMD, 510 HMD sensor, 520 Motion sensor, 530 Display, 600A, 600B Virtual space, 610 Avatar object (character), 620A, 620B Virtual camera, 631, 632, 633, 634 Object, 640A, 640B View area, 650, 660 View image, 671 Enemy object, 672, 673 Obstacle object, 674 Performance object, 691, 692 Utterance, 701,702,703A,70B,704A,704B,705,706,711,711A,711B,711C,711D,722,723,745,745A,745B,745C,752,762,763,930,2011 , 2022, 2031, 2032, 2033, 2034, 2037, 2038, 2051, 2063, 2072, 2073, 2075 UI image (message UI, UI), 721 Download screen, 731 User list screen (list), 732, 732A, 732B , 732C, 742, 742A, 742B, 742C Record image, 733, 733A, 733B, 733C User name, 734, 734A, 734B, 734C Tag information, 735, 735A, 735B, 735C Icon, 741 Motion list screen (choices), 743,743A,743B,743C Motion name, 744,744A,744B,744C,753 Motion image, 751 Distribution screen, 761 Distribution completion screen, 810A,810B Motion video, 820A,820B Speech audio, 910A,910B Video, 920A, 920B audio, 1000 HMD set, 1010 object, 1030 storage medium

Claims (11)

A video display program executed on a terminal device including a processor, a memory, and a display unit,
the processor;
receiving live-distributed data for displaying an image of a virtual space including characters that move in real time according to the movements of a performer;
displaying an image of the virtual space based on the received data;
The data includes first data corresponding to the end of the first part constituting the live distribution, and second data corresponding to the start of the second part following the first part,
The displaying step includes:
Switching and displaying the video of the virtual space of the first part to a predetermined specific video when ending the first part based on the first data;
When the second data is received while the specific video is being displayed, the video is switched from the specific video to a virtual space video of the second part and displayed, regardless of the display time during which the specific video was displayed. Display program . The second data is data that is distributed when a predetermined trigger is established after the first data is distributed;
In the displaying step, the second data is received while the specific video is being displayed, even when the display time has not reached the time required from the first data distribution to the second data distribution. 2. The video display program according to claim 1, wherein the specific video is sometimes switched to a virtual space video of the second part for display. In the displaying step, it is determined whether or not a specific time or more is required from when the data is live-distributed until displaying the image of the virtual space based on the data, and the display step determines whether or not a specific time or more is required from when the data is live-distributed until displaying the image of the virtual space based on the data, and 3. The video according to claim 1 or 2, wherein the video is switched to and displayed in the specific video when the specific video is displayed, and then switched to and displayed in the virtual space based on data received during the display of the specific video. Display program . 4. The video display program according to claim 3, wherein the specific time is a time predetermined according to the time required from the first data distribution to the second data distribution. The data includes audio data and motion data corresponding to the audio and motion input by the performer,
The displaying step includes playing the motion data corresponding to the audio data based on the audio data in accordance with the reproduction of the audio data among the received data, and displaying the character moving in the virtual space. ,
Claims 1 to 4, wherein information for specifying the end of the first part and information for specifying the start of the second part are included in the audio data of the data. A video display program described in any of the above. The part constituting the live distribution includes a specific part in which a game is played in which a user who owns the terminal device can participate,
The displaying step includes, when starting viewing the live broadcast from the middle of the specific part, displaying the video of the virtual space of the specific part based on the received data, and prompting the user to wait until a predetermined timing. The video display program according to any one of claims 1 to 5, which displays standby information. The step of displaying does not display the standby information when the timing to start viewing the live broadcast from the middle of the specific part is during a specific period of the specific part, while not displaying the standby information when the timing is during a non-specific period. The video display program according to claim 6, which displays standby information. 8. The video display program according to claim 7, wherein the non-specific period is a period predetermined because participating in the game may put the user at a disadvantage. 7. The step of displaying, when starting viewing the live broadcast from the middle of the specific part, displays the specific video and then switches to display the video in the virtual space based on the received data. - A video display program according to any one of claims 8 to 9. 10. The video display program according to claim 1, wherein the specific video is a video displayed using the entire display area of the display unit. The video according to any one of claims 1 to 10, wherein the specific video includes a video corresponding to the type of the first part that ends or the type of the second part that starts after displaying the specific video. Display program .

Images