JP6726345B1 - Program, method, and terminal device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a program, a method, and a terminal device capable of making a user feel the presence of a virtual character. SOLUTION: First display data for enabling display of an image of a virtual space including a first character that moves in synchronization with a motion of a performer is received. An image in which the first character moves is displayed in the virtual space based on the first display data. A predetermined first object is associated with the first character, and the display mode of the first object is changed according to a predetermined rule. [Selection diagram] Fig. 28

Description

The present invention relates to a program, a method, and a terminal device.

Conventionally, there are systems for displaying virtual characters controlled by a performer. In such a system, there is a system in which a decorative object, which is requested to be displayed by a user, is attached to a virtual character and displayed.

Patent No. 6420930

However, in the conventional system, the part corresponding to the request from the user is decorated according to the request. Therefore, for example, it is not possible to give the user a sense of reality that the display mode of the specific part of the virtual character changes spontaneously, and there is room for improvement.

The present invention has been conceived in view of such circumstances, and an object thereof is to provide a program, a method, and a terminal device that can make a user feel the presence of a virtual character.

According to an aspect of an embodiment shown in the present disclosure, a program executed in a terminal device including a processor, wherein the processor displays an image of a virtual space including a first character that moves in synchronization with a performer's motion. Performing a step of receiving the first display data for enabling display, and a step of displaying an image of the first character moving in the virtual space based on the first display data, The first character is associated with a predetermined first object, and the display mode of the first object is changed according to a predetermined rule.

According to the present invention, it is possible to make a user feel the presence of a virtual character.

It is a figure showing an example of an outline of a system according to a certain embodiment. It is a block diagram which shows an example of the hardware constitutions of the user terminal according to a certain embodiment. It is a block diagram which shows an example of the hardware constitutions of the server according to a certain embodiment. It is a block diagram which shows an example of the hardware constitutions of the game play terminal according to a certain embodiment. It is a block diagram which shows an example of the hardware constitutions of the delivery terminal according to a certain embodiment. It is a block diagram showing an example of functional composition of a user terminal, a server, and a HMD set according to a certain embodiment. FIG. 3 is a block diagram showing an example of a functional configuration of a distribution terminal according to an embodiment. 7 is a flowchart showing a part of a process executed in a user terminal and a game play terminal according to an embodiment. FIG. 6 is a diagram showing an example of a virtual space provided to a player and a visual field image visually recognized by the player according to an embodiment. FIG. 6 is a diagram showing an example of a virtual space provided to a user of a user terminal and a view image visually recognized by the user according to an embodiment. It is a figure which shows the other example of a visual field image visually recognized by the user of a user terminal. It is a figure which shows another example of the view image visually recognized by the user of a user terminal. It is a flow chart showing a part of processing performed in a game play terminal according to some embodiments. 7 is a flowchart showing a part of processing executed in a user terminal according to an embodiment. 6 is a flowchart showing a part of processing executed in a server according to an embodiment. It is a figure showing an example of a list of users who participated in a game according to a certain embodiment. 7 is a flowchart showing a part of a process executed in a distribution terminal according to an embodiment. It is a figure which shows a specific example of the screen displayed on the distribution terminal according to an embodiment. It is a figure which shows the other specific example of the screen displayed on the distribution terminal according to a certain embodiment. FIG. 6 is a diagram showing a specific example of a voice input by a player according to an embodiment. FIG. 14 is a diagram showing still another specific example of the screen displayed on the distribution terminal according to the embodiment, and an outline of distribution of operation command data. It is a figure which shows the other specific example of the audio|voice input by a player according to one embodiment. FIG. 14 is a diagram showing still another specific example of the screen displayed on the distribution terminal according to the embodiment, and an outline of distribution of operation command data. It is a figure showing an outline of transmission of game progress information from a game play terminal to a user terminal according to a certain embodiment. 7 is a flowchart showing a part of processing executed in a user terminal according to an embodiment. It is a figure which shows one specific example of moving image reproduction. It is a figure which shows the other specific example of a moving image reproduction. (A) is a flowchart showing an example of a display mode control process on the game play terminal side, and (B) is a flowchart showing an example of a display mode setting process on the user terminal side. (A) is a figure which shows an example of a live delivery video of the Nth time, (B) is a figure which shows an example of a live delivery video of the (N+1)th time, (C) shows an example of a live delivery video of the (N+2)th time. It is a figure. (A) is a figure which shows an example of the live delivery video in normal viewing mode, (B) is a figure which shows an example of the live delivery video in AR viewing mode.

The system according to the present disclosure is a system for providing a game to a plurality of users. Hereinafter, the system will be described with reference to the drawings. It should be noted that the present invention is not limited to these exemplifications, and is shown by the scope of the claims, and it is intended that all modifications within the scope and meaning equivalent to the scope of the claims are included in the present invention. It In the following description, the same elements will be denoted by the same reference symbols in the description of the drawings, without redundant description.

<Outline of operation of system 1>
FIG. 1 is a diagram showing an outline of a system 1 according to this embodiment. The system 1 includes a plurality of user terminals 100 (computers), a server 200, a game play terminal 300 (external device, second external device), and a distribution terminal 400 (external, first external device). 1 illustrates the user terminals 100A to 100C, in other words, three user terminals 100 as an example of the plurality of user terminals 100, but the number of user terminals 100 is not limited to the illustrated example. .. In addition, in the present embodiment, when it is not necessary to distinguish the user terminals 100A to 100C, the user terminals 100A to 100C are described as "user terminal 100". The user terminal 100, the game play terminal 300, and the distribution terminal 400 are connected to the server 200 via the network 2. The network 2 includes various mobile communication systems constructed by the Internet and a wireless base station (not shown). Examples of this mobile communication system include so-called 3G and 4G mobile communication systems, LTE (Long Term Evolution), and a wireless network (for example, Wi-Fi (registered trademark)) that can be connected to the Internet by a predetermined access point. To be

(Outline of the game)
In the present embodiment, as an example of a game provided by the system 1 (hereinafter, this game), a game mainly played by a user of the game play terminal 300 will be described. Hereinafter, the user of the game play terminal 300 will be referred to as a “player”. As an example, the player (actor) operates the character appearing in the game to advance the game. In addition, in this game, the user of the user terminal 100 plays a role of supporting the progress of the game by the player. Details of this game will be described later. The game provided by the system 1 is not limited to this example as long as it is a game in which a plurality of users participate.

(Game play terminal 300)
The game play terminal 300 advances the game according to the input operation by the player. Further, the game play terminal 300 sequentially delivers the information generated by the player's game play (hereinafter, game progress information) to the server 200 in real time.

(Server 200)
The server 200 transmits the game progress information (second data) received from the game play terminal 300 in real time to the user terminal 100. Further, the server 200 mediates transmission and reception of various information among the user terminal 100, the game play terminal 300, and the distribution terminal 400.

(Distribution terminal 400)
The distribution terminal 400 generates motion instruction data (first data) according to an input operation by the user of the distribution terminal 400, and distributes the motion instruction data to the user terminal 100 via the server 200. The action instruction data is data for reproducing a moving image on the user terminal 100, and specifically, data for moving a character appearing in the moving image.

In the present embodiment, as an example, the user of the distribution terminal 400 is the player of this game. Further, as an example, the moving image reproduced on the user terminal 100 based on the operation instruction data is a moving image in which the character operated by the player in the game moves. The "movement" is to move at least a part of the character's body, and includes utterance. Therefore, the motion instruction data according to the present embodiment includes, for example, voice data for causing the character to speak and motion data for moving the character's body.

The operation instruction data is, for example, transmitted to the user terminal 100 after the end of the present game. Details of the operation instruction data and the moving image reproduced based on the operation instruction data will be described later.

(User terminal 100)
The user terminal 100 receives the game progress information in real time, generates a game screen using the information, and displays the game screen. In other words, the user terminal 100 reproduces the game screen of the game played by the player by real-time rendering. This allows the user of the user terminal 100 to view the same game screen as the game screen viewed by the player while playing the game at substantially the same timing as the player.

In addition, the user terminal 100 generates information for supporting the progress of the game by the player in response to the input operation by the user, and transmits the information to the game play terminal 300 via the server 200. Details of the information will be described later.

Further, the user terminal 100 receives the operation instruction data from the distribution terminal 400, generates a moving image (video) using the operation instruction data, and reproduces it. In other words, the user terminal 100 renders and reproduces the motion instruction data.

<Hardware configuration of system 1>
FIG. 2 is a diagram showing a hardware configuration of the user terminal 100. FIG. 3 is a diagram showing a hardware configuration of the server 200. FIG. 4 is a diagram showing a hardware configuration of the game play terminal 300. FIG. 5 is a diagram showing a hardware configuration of the distribution terminal 400.

(User terminal 100)
In the present embodiment, an example in which the user terminal 100 is realized as a smartphone will be described as an example, but the user terminal 100 is not limited to the smartphone. For example, the user terminal 100 may be realized as a feature phone, a tablet computer, a laptop computer (so-called laptop computer), a desktop computer, or the like. Further, the user terminal 100 may be a game device suitable for game play.

As shown in FIG. 2, the user terminal 100 includes a processor 10, a memory 11, a storage 12, a communication interface (IF) 13, an input/output IF 14, a touch screen 15 (display unit), a camera 17, a measurement unit. The distance sensor 18 is provided. These components included in the user terminal 100 are electrically connected to each other by a communication bus. The user terminal 100 may include an input/output IF 14 to which a display (display unit) configured separately from the main body of the user terminal 100 can be connected instead of or in addition to the touch screen 15.

Further, as shown in FIG. 2, the user terminal 100 may be configured to be able to communicate with one or more controllers 1020. The controller 1020 establishes communication with the user terminal 100 according to a communication standard such as Bluetooth (registered trademark). The controller 1020 may have one or more buttons or the like, and transmits an output value based on a user's input operation to the buttons or the like to the user terminal 100. Further, the controller 1020 may have various sensors such as an acceleration sensor and an angular velocity sensor, and outputs output values of the various sensors to the user terminal 100.

The user terminal 100 may have the camera 17 and the distance measuring sensor 18 instead of or in addition to the camera 17 and the distance measuring sensor 18.

It is desirable that the user terminal 100 causes a user who uses the controller 1020 to input user identification information such as a name or a login ID of the user via the controller 1020, for example, at the start of the game. This allows the user terminal 100 to associate the controller 1020 with the user, and specifies which user the output value belongs to based on the transmission source (controller 1020) of the received output value. be able to.

When the user terminal 100 communicates with the plurality of controllers 1020, each controller 1020 is held by each user, so that the one user terminal 100 does not communicate with other devices such as the server 200 via the network 2. Multiplayer can be realized with. In addition, each user terminal 100 communicates with each other according to a wireless standard such as a wireless LAN (Local Area Network) standard (communicatively connects without passing through the server 200), thereby realizing multi-play locally by a plurality of user terminals 100. You can also do it. When the above-mentioned multiplay is locally realized by one user terminal 100, the user terminal 100 may further include at least a part of various functions of the server 200, which will be described later. When the above-described multiplay is locally realized by a plurality of user terminals 100, the plurality of user terminals 100 may be provided with various functions of the server 200, which will be described later, in a distributed manner.

Note that the user terminal 100 may communicate with the server 200 even when the above-described multiplay is realized locally. For example, information indicating a play result such as a score or a win or loss in a game and user identification information may be associated with each other and transmitted to the server 200.

Further, the controller 1020 may be configured to be attachable to and detachable from the user terminal 100. In this case, a coupling part with the controller 1020 may be provided on at least one surface of the housing of the user terminal 100. When the user terminal 100 and the controller 1020 are connected by wire via the connecting unit, the user terminal 100 and the controller 1020 send and receive signals via wire.

As shown in FIG. 2, the user terminal 100 may accept mounting of a storage medium 1030 such as an external memory card via the input/output IF 14. As a result, the user terminal 100 can read the program and data recorded in the storage medium 1030. The program recorded in the storage medium 1030 is, for example, a game program.

The user terminal 100 may store the game program acquired by communicating with an external device such as the server 200 in the memory 11 of the user terminal 100, or the game program acquired by reading from the storage medium 1030 in the memory 11. May be stored in.

As described above, the user terminal 100 includes the communication IF 13, the input/output IF 14, the touch screen 15, the camera 17, and the distance measuring sensor 18 as an example of a mechanism for inputting information to the user terminal 100. Each of the above-described units as a mechanism for inputting can be regarded as an operation unit configured to receive an input operation of a user.

For example, when the operation unit is configured by at least one of the camera 17 and the distance measuring sensor 18, the operation unit detects an object 1010 near the user terminal 100 and performs an input operation from the detection result of the object. Identify. As an example, the user's hand as the object 1010, a marker having a predetermined shape, or the like is detected, and the input operation is specified based on the color, shape, movement, or type of the object 1010 obtained as a detection result. It More specifically, when the user's hand is detected from the image captured by the camera 17, the user terminal 100 operates the gesture (a series of movements of the user's hand) detected based on the captured image by the user's input operation. As specified and accepted. The captured image may be a still image or a moving image.

Alternatively, when the operation unit includes the touch screen 15, the user terminal 100 identifies and accepts the user's operation performed on the input unit 151 of the touch screen 15 as the user's input operation. Alternatively, when the operation unit is configured by the communication IF 13, the user terminal 100 identifies and receives a signal (for example, an output value) transmitted from the controller 1020 as a user input operation. Alternatively, when the operation unit is configured by the input/output IF 14, a signal output from an input device (not shown) different from the controller 1020 connected to the input/output IF 14 is specified and accepted as the user's input operation.

(Server 200)
The server 200 may be, for example, a general purpose computer such as a workstation or a personal computer. The server 200 includes a processor 20, a memory 21, a storage 22, a communication IF 23, and an input/output IF 24. These components included in the server 200 are electrically connected to each other by a communication bus.

(Game play terminal 300)
The game play terminal 300 may be, for example, a general-purpose computer such as a personal computer. The game play terminal 300 includes a processor 30, a memory 31, a storage 32, a communication IF 33, and an input/output IF 34. These components included in the game play terminal 300 are electrically connected to each other by a communication bus.

As shown in FIG. 4, the game play terminal 300 according to the present embodiment is included in an HMD (Head Mounted Display) set 1000, as an example. That is, it can be said that the HMD set 1000 is included in the system 1, and also that the player plays the game using the HMD set 1000. The device for the player to play the game is not limited to the HMD set 1000. As an example, the device may be any device that allows a player to virtually experience the game. Further, the device may be realized as a smartphone, a feature phone, a tablet computer, a laptop computer (so-called notebook computer), a desktop computer, or the like. Further, the device may be a game device suitable for game play.

The HMD set 1000 includes a game play terminal 300, an HMD 500, an HMD sensor 510, a motion sensor 520, a display 530, and a controller 540. The HMD 500 includes a monitor 51, a gaze sensor 52, a first camera 53, a second camera 54, a microphone 55, and a speaker 56. Controller 540 may include motion sensor 520.

The HMD 500 can be mounted on the player's head and provide the player with a virtual space during operation. More specifically, the HMD 500 displays an image for the right eye and an image for the left eye on the monitor 51, respectively. When each eye of the player visually recognizes each image, the player can recognize the image as a three-dimensional image based on the parallax of both eyes. The HMD 500 may include both a so-called head mounted display including a monitor and a head mounted device to which a terminal having a monitor such as a smartphone can be attached.

The monitor 51 is realized, for example, as a non-transmissive display device. In one aspect, the monitor 51 is arranged on the main body of the HMD 500 so as to be located in front of both eyes of the player. Therefore, the player can immerse himself in the virtual space when viewing the three-dimensional image displayed on the monitor 51. In one aspect, the virtual space includes, for example, a background, an object operable by the player, and an image of a menu selectable by the player. In one aspect, the monitor 51 can be realized as a liquid crystal monitor or an organic EL (Electro Luminescence) monitor included in a so-called smartphone or other information display terminal.

In another aspect, the monitor 51 can be realized as a transmissive display device. In this case, the HMD 500 may be an open type such as a glasses type instead of the closed type that covers the eyes of the player as shown in FIG. The transmissive monitor 51 may be temporarily configured as a non-transmissive display device by adjusting its transmittance. The monitor 51 may include a configuration for displaying a part of the image forming the virtual space and the real space at the same time. For example, the monitor 51 may display an image of the real space taken by a camera mounted on the HMD 500, or may make the real space visible by setting a part of the transmittances high.

In one aspect, monitor 51 may include a sub-monitor for displaying an image for the right eye and a sub-monitor for displaying an image for the left eye. In another aspect, the monitor 51 may be configured to integrally display the image for the right eye and the image for the left eye. In this case, the monitor 51 includes a high speed shutter. The high-speed shutter operates so that the image for the right eye and the image for the left eye can be displayed alternately so that the image is recognized by only one of the eyes.

In one aspect, the HMD 500 includes a plurality of light sources (not shown). Each light source is realized by, for example, an LED (Light Emitting Diode) that emits infrared rays. The HMD sensor 510 has a position tracking function for detecting the movement of the HMD 500. More specifically, the HMD sensor 510 reads a plurality of infrared rays emitted by the HMD 500 and detects the position and inclination of the HMD 500 in the physical space.

In another aspect, the HMD sensor 510 may be implemented by a camera. In this case, the HMD sensor 510 can detect the position and inclination of the HMD 500 by executing the image analysis process using the image information of the HMD 500 output from the camera.

In another aspect, the HMD 500 may include a sensor (not shown) as a position detector instead of the HMD sensor 510 or in addition to the HMD sensor 510. The HMD 500 can detect the position and inclination of the HMD 500 itself using the sensor. For example, when the sensor is an angular velocity sensor, a geomagnetic sensor, or an acceleration sensor, the HMD 500 may use any one of these sensors instead of the HMD sensor 510 to detect its own position and tilt. As an example, when the sensor included in the HMD 500 is an angular velocity sensor, the angular velocity sensor detects the angular velocity around the three axes of the HMD 500 in the physical space over time. The HMD 500 calculates the temporal change of the angle around the three axes of the HMD 500 based on each angular velocity, and further calculates the inclination of the HMD 500 based on the temporal change of the angle.

The gaze sensor 52 detects the direction in which the right and left eyes of the player are directed. That is, the gaze sensor 52 detects the line of sight of the player. The detection of the direction of the line of sight is realized by, for example, a known eye tracking function. The gaze sensor 52 is realized by a sensor having the eye tracking function. In one aspect, the gaze sensor 52 preferably includes a sensor for the right eye and a sensor for the left eye. The gaze sensor 52 may be, for example, a sensor that emits infrared light to the right and left eyes of the player, and receives the reflected light from the cornea and iris with respect to the emitted light to detect the rotation angle of each eyeball. The gaze sensor 52 can detect the line of sight of the player based on each detected rotation angle.

The first camera 53 photographs the lower part of the player's face. More specifically, the first camera 53 photographs the player's nose and mouth. The second camera 54 photographs the player's eyes and eyebrows. The housing of the HMD 500 on the player side is defined as the inside of the HMD 500, and the housing of the HMD 500 opposite to the player is defined as the outside of the HMD 500. In one aspect, the first camera 53 may be arranged outside the HMD 500 and the second camera 54 may be arranged inside the HMD 500. The images generated by the first camera 53 and the second camera 54 are input to the game play terminal 300. In another aspect, the first camera 53 and the second camera 54 may be realized as one camera, and the one camera may be used to capture the face of the player.

The microphone 55 converts the utterance of the player into an audio signal (electrical signal) and outputs it to the game play terminal 300. The speaker 56 converts the audio signal into audio and outputs it to the player. In another aspect, HMD 500 may include earphones instead of speaker 56.

The controller 540 is connected to the game play terminal 300 by wire or wirelessly. The controller 540 receives an input of a command from the player to the game play terminal 300. In one aspect, the controller 540 is configured to be held by a player. In another aspect, the controller 540 is configured to be attachable to a player's body or a part of clothing. In yet another aspect, controller 540 may be configured to output at least one of vibration, sound, and light based on a signal transmitted from game play terminal 300. In still another aspect, the controller 540 receives an operation from the player for controlling the position and movement of the object placed in the virtual space.

In an aspect, the controller 540 includes multiple light sources. Each light source is realized by, for example, an LED that emits infrared rays. The HMD sensor 510 has a position tracking function. In this case, the HMD sensor 510 reads a plurality of infrared rays emitted by the controller 540 and detects the position and inclination of the controller 540 in the physical space. In another aspect, the HMD sensor 510 may be implemented by a camera. In this case, the HMD sensor 510 can detect the position and inclination of the controller 540 by executing the image analysis process using the image information of the controller 540 output from the camera.

The motion sensor 520 is attached to the player's hand in a certain situation, and detects the movement of the player's hand. For example, the motion sensor 520 detects the rotation speed, the rotation speed, etc. of the hand. The detected signal is sent to the game play terminal 300. The motion sensor 520 is provided in the controller 540, for example. In one aspect, the motion sensor 520 is provided, for example, in the controller 540 that is configured to be held by the player. In another aspect, for safety in the physical space, the controller 540 is attached to something that does not easily fly by being attached to the player's hand, such as a glove type. In yet another aspect, a sensor not attached to the player may detect movement of the player's hand. For example, a signal from a camera that captures an image of the player may be input to the game play terminal 300 as a signal representing the action of the player. The motion sensor 520 and the game play terminal 300 are wirelessly connected to each other, for example. In the case of wireless communication, the communication form is not particularly limited, and for example, known communication methods such as Bluetooth are used.

The display 530 displays an image similar to the image displayed on the monitor 51. This allows a user other than the player wearing the HMD 500 to view the same image as the player. The image displayed on the display 530 does not have to be a three-dimensional image, and may be an image for the right eye or an image for the left eye. Examples of the display 530 include a liquid crystal display and an organic EL monitor.

The game play terminal 300 causes the character to be operated by the player to operate based on various information acquired from each unit of the HMD 500, the controller 540, and the motion sensor 520, and advances the game. The “motion” here means moving each part of the body, changing the posture, changing the facial expression, moving, speaking, touching or moving an object placed in the virtual space, and the character. Includes the use of grasping weapons, tools, etc. That is, in this game, when the player moves each part of the body, the character also moves each part of the body similarly to the player. In this game, the character utters the contents uttered by the player. In other words, in this game, the character is an avatar object that behaves as a player's alter ego. As an example, at least a part of the motion of the character may be performed by an input to the controller 540 by the player.

In the present embodiment, the motion sensor 520 is attached to, for example, both hands of the player, both feet of the player, the waist of the player, and the head of the player. The motion sensors 520 attached to both hands of the player may be provided in the controller 540 as described above. Further, the motion sensor 520 attached to the head of the player may be provided in the HMD 500. Motion sensor 520 may also be attached to both elbows and knees of the user. By increasing the number of motion sensors 520 attached to the player, the movement of the player can be more accurately reflected on the character.
Further, the player may wear a suit to which one or more motion sensors 520 are attached, instead of attaching the motion sensor 520 to each part of the body. That is, the method of motion capture is not limited to the example using the motion sensor 520.

(Distribution terminal 400)
The distribution terminal 400 may be a mobile terminal such as a smartphone, a PDA (Personal Digital Assistant), or a tablet computer. The distribution terminal 400 may be a so-called stationary terminal such as a desktop personal computer.

As shown in FIG. 5, the distribution terminal 400 includes a processor 40, a memory 41, a storage 42, a communication IF 43, an input/output IF 44, and a touch screen 45. The distribution terminal 400 may include an input/output IF 44 to which a display (display unit) configured separately from the main body of the distribution terminal 400 can be connected instead of or in addition to the touch screen 45.

The controller 1021 may have a physical input mechanism such as one or more buttons, levers, sticks, wheels and the like. The controller 1021 transmits to the distribution terminal 400 an output value based on an input operation input to the input mechanism by an operator (a player in the present embodiment) of the distribution terminal 400. Further, the controller 1021 may include various sensors such as an acceleration sensor and an angular velocity sensor, and the output values of the various sensors may be transmitted to the distribution terminal 400. The output value described above is accepted by the distribution terminal 400 via the communication IF 43.

The distribution terminal 400 may include a camera and a distance measuring sensor (both not shown). Instead of or in addition to the distribution terminal 400, the controller 1021 may include a camera and a distance measuring sensor.

As described above, the distribution terminal 400 includes the communication IF 43, the input/output IF 44, and the touch screen 45 as an example of a mechanism for inputting information to the distribution terminal 400. Each of the above-described units as a mechanism for inputting can be regarded as an operation unit configured to receive an input operation of a user.

When the operation unit is configured by the touch screen 45, the distribution terminal 400 identifies and accepts the user operation performed on the input unit 451 of the touch screen 45 as the user input operation. Alternatively, when the operation unit is configured by the communication IF 43, the distribution terminal 400 identifies and receives a signal (for example, an output value) transmitted from the controller 1021 as a user input operation. Alternatively, when the operation unit is composed of the input/output IF 44, the distribution terminal 400 identifies and receives a signal output from an input device (not shown) connected to the input/output IF 44 as the user's input operation.

<Hardware components of each device>
The processors 10, 20, 30, and 40 control the overall operations of the user terminal 100, the server 200, the game play terminal 300, and the distribution terminal 400, respectively. The processors 10, 20, 30, 40 include a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a GPU (Graphics Processing Unit). The processors 10, 20, 30, and 40 read programs from the storages 12, 22, 32, and 42 described below, respectively. Then, the processors 10, 20, 30, and 40 develop the read programs in the memories 11, 21, 31, and 41 described later, respectively. The processors 10, 20, 30 execute the expanded program.

The memories 11, 21, 31, 41 are main storage devices. The memories 11, 21, 31, and 41 are configured by storage devices such as ROM (Read Only Memory) and RAM (Random Access Memory). The memory 11 provides a work area to the processor 10 by temporarily storing a program and various data read by the processor 10 from a storage 12 described later. The memory 11 also temporarily stores various data generated while the processor 10 operates according to a program. The memory 21 provides a work area to the processor 20 by temporarily storing various programs and data read by the processor 20 from a storage 22 described later. The memory 21 also temporarily stores various data generated while the processor 20 operates according to the program. The memory 31 provides a work area to the processor 30 by temporarily storing various programs and data read by the processor 30 from a storage 32 described later. The memory 31 also temporarily stores various data generated while the processor 30 operates according to the program. The memory 41 provides a work area to the processor 40 by temporarily storing a program and various data read by the processor 40 from a storage 42 described later. The memory 41 also temporarily stores various data generated while the processor 40 operates according to the program.

In the present embodiment, the program executed by the processors 10 and 30 may be a game program of this game. In the present embodiment, the program executed by the processor 40 may be a distribution program for realizing distribution of operation instruction data. Further, the processor 10 may further execute a viewing program for realizing playback of a moving image.

In the present embodiment, the program executed by the processor 20 may be at least one of the above-mentioned game program, distribution program, and viewing program. The processor 20 executes at least one of a game program, a distribution program, and a viewing program in response to a request from at least one of the user terminal 100, the game play terminal 300, and the distribution terminal 400. The distribution program and the viewing program may be executed in parallel.

That is, the game program may be a program that realizes a game by cooperation of the user terminal 100, the server 200, and the game play terminal 300. The distribution program may be a program that realizes the distribution of the operation instruction data by the cooperation of the server 200 and the distribution terminal 400. The viewing program may be a program that realizes the reproduction of a moving image by the cooperation of the user terminal 100 and the server 200.

The storages 12, 22, 32, 42 are auxiliary storage devices. The storages 12, 22, 32, and 42 are storage devices such as flash memories or HDDs (Hard Disk Drives). The storages 12 and 32 store, for example, various data regarding games. The storage 42 stores various data regarding distribution of operation instruction data. In addition, the storage 12 stores various data relating to reproduction of moving images. The storage 22 may store at least a part of various data relating to each of a game, distribution of motion instruction data, and reproduction of a moving image.

The communication IFs 13, 23, 33, 43 control transmission/reception of various data in the user terminal 100, the server 200, the game play terminal 300, and the distribution terminal 400, respectively. The communication IFs 13, 23, 33, and 43 are, for example, communication using a wireless LAN (Local Area Network), wired LAN, wireless LAN, or Internet communication via a mobile phone network, and communication using short-distance wireless communication. Control.

The input/output IFs 14, 24, 34, 44 are interfaces for the user terminal 100, the server 200, the game play terminal 300, and the distribution terminal 400 to receive data input and output data, respectively. The input/output IFs 14, 24, 34, 44 may input/output data via a USB (Universal Serial Bus) or the like. The input/output IFs 14, 24, 34, 44 may include physical buttons, cameras, microphones, speakers, mice, keyboards, displays, sticks, levers and the like. Further, the input/output IFs 14, 24, 34, 44 may include a connection unit for transmitting/receiving data to/from peripheral devices.

The touch screen 15 is an electronic component that combines an input unit 151 and a display unit 152 (display). The touch screen 45 is an electronic component that combines an input unit 451 and a display unit 452. The input units 151 and 451 are, for example, touch-sensitive devices, and are configured by a touch pad, for example. The display units 152 and 452 are configured by, for example, a liquid crystal display, an organic EL (Electro-Luminescence) display, or the like.

The input units 151 and 451 detect a position at which a user operation (mainly a physical contact operation such as a touch operation, a slide operation, a swipe operation, and a tap operation) on the input surface is detected, and information indicating the position is detected. Is provided as an input signal. The input units 151 and 451 may include a touch sensing unit (not shown). The touch sensing unit may be of any type such as a capacitance type or a resistance film type.

Although not shown, each of the user terminal 100 and the distribution terminal 400 may include one or more sensors for specifying the holding postures of the user terminal 100 and the distribution terminal 400, respectively. This sensor may be, for example, an acceleration sensor or an angular velocity sensor.

When the user terminal 100 and the distribution terminal 400 include a sensor, the processors 10 and 40 specify the holding attitudes of the user terminal 100 and the distribution terminal 400 from the outputs of the sensors, respectively, and perform processing according to the holding attitudes. It also becomes possible. For example, the processors 10 and 40 may perform vertical screen display for displaying vertically long images on the display units 152 and 452, respectively, when the user terminal 100 and the distribution terminal 400 are held vertically. On the other hand, when the user terminal 100 and the distribution terminal 400 are held sideways, a landscape screen may be displayed in which a landscape image is displayed on the display unit. In this way, the processors 10 and 40 may be capable of switching between the vertical screen display and the horizontal screen display according to the holding postures of the user terminal 100 and the distribution terminal 400, respectively.

<Functional configuration of system 1>
FIG. 6 is a block diagram showing the functional configurations of the user terminal 100, the server 200, and the HMD set 1000 included in the system 1. FIG. 7 is a block diagram showing a functional configuration of the distribution terminal 400 shown in FIG.

The user terminal 100 has a function as an input device that receives a user's input operation and a function as an output device that outputs a game image or sound. The user terminal 100 functions as the control unit 110 and the storage unit 120 in cooperation with the processor 10, the memory 11, the storage 12, the communication IF 13, the input/output IF 14, the touch screen 15, and the like.

The server 200 has a function of mediating transmission/reception of various information among the user terminal 100, the HMD set 1000, and the distribution terminal 400. The server 200 functions as the control unit 210 and the storage unit 220 in cooperation with the processor 20, the memory 21, the storage 22, the communication IF 23, the input/output IF 24, and the like.

The HMD set 1000 (game play terminal 300) uses a function as an input device that receives an input operation of a player, a function as an output device that outputs a game image and a sound, and game progress information via a server 200 to a user. It has a function of transmitting to the terminal 100 in real time. The HMD set 1000 includes the processor 30, the memory 31, the storage 32, the communication IF 33, the input/output IF 34, the HMD 500, the HMD sensor 510, the motion sensor 520, the controller 540, and the like of the game play terminal 300, and the control unit 310. And functions as the storage unit 320.

The distribution terminal 400 has a function of generating operation instruction data and transmitting the operation instruction data to the user terminal 100 via the server 200. The distribution terminal 400 functions as the control unit 410 and the storage unit 420 in cooperation with the processor 40, the memory 41, the storage 42, the communication IF 43, the input/output IF 44, the touch screen 45, and the like.

(Data stored in the storage unit of each device)
The storage unit 120 stores a game program 131 (program), game information 132, and user information 133. The storage unit 220 stores a game program 231, game information 232, user information 233, and a user list 234. The storage unit 320 stores a game program 331, game information 332, and user information 333. The storage unit 420 stores a user list 421, a motion list 422, and a distribution program 423 (program, second program).

The game programs 131, 231, 331 are game programs executed by the user terminal 100, the server 200, and the HMD set 1000, respectively. This game is realized by the devices operating in cooperation with each other based on the game programs 131, 231, 331. The game programs 131 and 331 may be stored in the storage unit 220 and downloaded to the user terminal 100 and the HMD set 1000, respectively. In the present embodiment, it is assumed that the user terminal 100 renders the data received from the distribution terminal 400 based on the game program 131 and reproduces the moving image. In other words, the game program 131 is also a program for playing a moving image using the moving image instruction data distributed from the distribution terminal 400. The program for playing the moving image may be different from the game program 131. In this case, the storage unit 120 stores a program for playing the moving image, separately from the game program 131.

The game information 132, 232, 332 is data that the user terminal 100, the server 200, and the HMD set 1000 respectively refer to when executing a game program. The user information 133, 233, 333 is data regarding the account of the user of the user terminal 100. The game information 232 is the game information 132 of each user terminal 100 and the game information 332 of the HMD set 1000. The user information 233 is user information 133 of each user terminal 100 and user information of the player included in the user information 333. The user information 333 is the user information 133 of each user terminal 100 and the user information of the player.

The user list 234 and the user list 421 are lists of users who participated in the game. The user list 234 and the user list 421 may include a list of users who participated in the latest game play by the player, and a list of users who participated in each game play before the game play. The motion list 422 is a list of a plurality of motion data created in advance. The motion list 422 is, for example, a list in which motion data is associated with each piece of information (for example, motion name) that identifies each motion. The distribution program 423 is a program for realizing distribution of operation instruction data for reproducing a moving image on the user terminal 100 to the user terminal 100.

(Functional configuration of the server 200)
The control unit 210 centrally controls the server 200 by executing the game program 231 stored in the storage unit 220. For example, the control unit 210 mediates transmission/reception of various information among the user terminal 100, the HMD set 1000, and the distribution terminal 400.

The control unit 210 functions as the communication mediation unit 211, the log generation unit 212, and the list generation unit 213 according to the description of the game program 231. The control unit 210 can also function as other functional blocks (not shown) for mediating transmission/reception of various information relating to game play and distribution of motion instruction data, and for supporting progress of the game.

The communication mediation unit 211 mediates transmission and reception of various information among the user terminal 100, the HMD set 1000, and the distribution terminal 400. For example, the communication mediating unit 211 transmits the game progress information received from the HMD set 1000 to the user terminal 100. The game progress information includes data indicating the movement of the character operated by the player, the parameter of the character, information on items and weapons possessed by the character, enemy characters, and the like. The server 200 transmits the game progress information to the user terminals 100 of all users participating in the game. In other words, the server 200 transmits the common game progress information to the user terminals 100 of all users participating in the game. As a result, the game progresses in the user terminals 100 of all the users participating in the game, similarly to the HMD set 1000.

Further, for example, the communication mediating unit 211 transmits the information received from any of the user terminals 100 for supporting the progress of the game by the player, to the other user terminals 100 and the HMD set 1000. The information may be, for example, item information for the player to advantageously advance the game, and may be item information indicating an item provided to the player (character). The item information includes information indicating the user who provided the item (user name, user ID, etc.). The communication mediation unit 211 may mediate the distribution of the operation instruction data from the distribution terminal 400 to the user terminal 100.

The log generation unit 212 generates a game progress log based on the game progress information received from the HMD set 1000. The list generation unit 213 generates the user list 234 after the game play ends. As will be described in detail later, each user in the user list 234 is associated with a tag indicating the content of the support provided to the player by the user. The list generation unit 213 generates a tag based on the game progress log generated by the log generation unit 212 and associates the tag with the corresponding user. The list generation unit 213 may associate, as a tag, the content of support provided to each player by each user, which is input by the game operator or the like using a terminal device such as a personal computer, as a tag. .. As a result, the content of the support provided by each user becomes more detailed. When the user participates in the game, the user terminal 100 transmits information indicating the user to the server 200 based on the user's operation. For example, the user terminal 100 transmits the user ID input by the user to the server 200. That is, the server 200 holds information indicating each user for all users participating in the game. The list generation unit 213 may use the information to generate the user list 234.

(Functional configuration of HMD set 1000)
The control unit 310 centrally controls the HMD set 1000 by executing the game program 331 stored in the storage unit 320. For example, the control unit 310 advances the game according to the game program 331 and the operation of the player. Further, the control unit 310 communicates with the server 200 as needed to transmit and receive information while the game is in progress. The control unit 310 may directly transmit/receive information to/from the user terminal 100 without passing through the server 200.

The control unit 310, in accordance with the description of the game program 331, the operation reception unit 311, the display control unit 312, the UI control unit 313, the animation generation unit 314, the game progression unit 315, the virtual space control unit 316, and the reaction processing unit 317. Function as. The control unit 310 can also function as other functional blocks (not shown) for controlling the characters appearing in the game depending on the nature of the game to be executed.

The operation reception unit 311 detects and receives an input operation by the player. The operation receiving unit 311 receives signals input from the HMD 500, the motion sensor 520, the controller 540, and the like, determines what input operation has been performed, and outputs the result to each element of the control unit 310.

The UI control unit 313 controls a user interface (hereinafter, UI) image displayed on the monitor 51, the display 530, and the like. The UI image is a tool for the player to input to the HMD set 1000 necessary for the progress of the game, or a tool for obtaining information output from the HMD set 1000 during the progress of the game. The UI image is, for example, but not limited to, an icon, a button, a list, a menu screen, or the like.

The animation generation unit 314 generates animations showing the motions of various objects based on the control modes of various objects. For example, the animation generation unit 314 may generate an animation or the like that represents an object (for example, a player's avatar object) moving as if it were there, moving its mouth, or changing its facial expression. ..

The game progression unit 315 progresses the game based on the game program 331, an input operation by the player, an action of the avatar object in response to the input operation, and the like. For example, the game progression unit 315 performs a predetermined game process when the avatar object performs a predetermined action. Further, for example, the game progression unit 315 may receive information indicating a user's operation on the user terminal 100, and may perform a game process based on the user's operation. In addition, the game progression unit 315 generates game progress information according to the progress of the game and transmits it to the server 200. The game progress information is transmitted to the user terminal 100 via the server 200. As a result, the progress of the game in the HMD set 1000 is shared by the user terminal 100. In other words, the progress of the game on the HMD set 1000 and the progress of the game on the user terminal 100 are synchronized.

The virtual space control unit 316 performs various controls regarding the virtual space provided to the player according to the progress of the game. As an example, the virtual space control unit 316 creates various objects and arranges them in the virtual space. The virtual space control unit 316 also arranges the virtual camera in the virtual space. In addition, the virtual space control unit 316 operates various objects arranged in the virtual space according to the progress of the game. Further, the virtual space control unit 316 controls the position and inclination of the virtual camera arranged in the virtual space according to the progress of the game.

The display control unit 312 outputs, to the monitor 51 and the display 530, a game screen in which the processing result executed by each of the above-described elements is reflected. The display control unit 312 may display an image based on the field of view from the virtual camera arranged in the virtual space on the monitor 51 and the display 530 as a game screen. Further, the display control unit 312 may include the animation generated by the animation generation unit 314 in the game screen. Further, the display control unit 312 may superimpose and draw the above-mentioned UI image controlled by the UI control unit 313 on the game screen.

The reaction processing unit 317 receives feedback regarding the reaction of the user of the user terminal 100 with respect to the player's game play, and outputs the feedback to the player. In the present embodiment, for example, the user terminal 100 can create a comment (message) addressed to the avatar object based on the user's input operation. The reaction processing unit 317 receives the comment data of the comment and outputs it. The reaction processing unit 317 may display text data corresponding to the user's comment on the monitor 51 and the display 530, or may output voice data corresponding to the user's comment from a speaker (not shown). In the former case, the reaction processing unit 317 may superimpose an image corresponding to the text data (that is, an image including the content of the comment) on the game screen and draw it.

(Functional configuration of the user terminal 100)
The control unit 110 centrally controls the user terminal 100 by executing the game program 131 stored in the storage unit 120. For example, the control unit 110 advances the game according to the game program 131 and the user's operation. Further, the control unit 110 communicates with the server 200 as needed to transmit and receive information while the game is in progress. The control unit 110 may directly send and receive information to and from the HMD set 1000 without going through the server 200.

The control unit 110, according to the description of the game program 131, the operation reception unit 111, the display control unit 112, the UI control unit 113, the animation generation unit 114, the game progression unit 115, the virtual space control unit 116, and the moving image reproduction unit 117. Function as. The control unit 110 can also function as other functional blocks (not shown) for the progress of the game depending on the nature of the game to be executed.

The operation receiving unit 111 detects and receives a user's input operation on the input unit 151. The operation reception unit 111 determines what input operation has been performed based on the user's action on the console via the touch screen 15 and the other input/output IF 14, and outputs the result to each element of the control unit 110. To do.

For example, the operation receiving unit 111 receives an input operation on the input unit 151, detects the coordinates of the input position of the input operation, and specifies the type of the input operation. The operation receiving unit 111 identifies, as the type of input operation, for example, a touch operation, a slide operation, a swipe operation, a tap operation, or the like. Further, the operation receiving unit 111 detects that the touch input is released from the touch screen 15 when the continuously detected input is interrupted.

The UI control unit 113 controls a UI image displayed on the display unit 152 in order to construct a UI in accordance with at least one of a user input operation and received game progress information. The UI image is a tool for the user to input necessary for the progress of the game to the user terminal 100, or a tool for obtaining information output during the progress of the game from the user terminal 100. The UI image is, for example, but not limited to, an icon, a button, a list, a menu screen, or the like.

The animation generation unit 114 generates animations showing the motions of various objects based on the control modes of various objects.

The game progression unit 115 progresses the game based on the game program 131, the received game progress information, the input operation by the user, and the like. When a predetermined game process is performed by the user's input operation, the game progression unit 115 transmits information about the game process to the HMD set 1000 via the server 200. As a result, the predetermined game process is shared by the HMD set 1000. In other words, the progress of the game on the HMD set 1000 and the progress of the game on the user terminal 100 are synchronized. The predetermined game process is, for example, a process of providing an item to the avatar object, and in the case of this example, the information about the game process is the item information described above.

The virtual space control unit 116 performs various controls regarding the virtual space provided to the user according to the progress of the game. As an example, the virtual space control unit 116 creates various objects and arranges them in the virtual space. In addition, the virtual space control unit 116 arranges the virtual camera in the virtual space. In addition, the virtual space control unit 116 operates various objects arranged in the virtual space according to the progress of the game, specifically, the received game progress information. In addition, the virtual space control unit 316 controls the position and tilt of the virtual camera arranged in the virtual space according to the progress of the game, specifically, the received game progress information.

The display control unit 112 outputs, to the display unit 152, a game screen in which the processing result executed by each of the above-described elements is reflected. The display control unit 112 may display an image based on the field of view from the virtual camera arranged in the virtual space provided to the user on the display unit 152 as a game screen. Further, the display control unit 112 may include the animation generated by the animation generation unit 114 in the game screen. Further, the display control unit 112 may superimpose and draw the above-described UI image controlled by the UI control unit 113 on the game screen. In any case, the game screen displayed on the display unit 152 is the same as the game screen displayed on the other user terminals 100 and the HMD set 1000.

The moving image reproducing unit 117 analyzes (renders) the operation instruction data received from the distribution terminal 400 and reproduces the moving image.

(Functional configuration of distribution terminal 400)
The control unit 410 centrally controls the distribution terminal 400 by executing a program (not shown) stored in the storage unit 420. For example, the control unit 410 generates the action instruction data according to the program and the operation of the user (player in the present embodiment) of the distribution terminal 400, and distributes it to the user terminal 100. Further, the control unit 410 communicates with the server 200 to send and receive information as necessary. The control unit 410 may directly transmit/receive information to/from the user terminal 100 without passing through the server 200.

The control unit 410 functions as a communication control unit 411, a display control unit 412, an operation reception unit 413, a voice reception unit 414, a motion identification unit 415, and a motion instruction data generation unit 416 according to the description of the program. The control unit 410 can also function as other functional blocks (not shown) in order to generate and distribute operation instruction data.

The communication control unit 411 controls transmission/reception of information to/from the server 200 or the user terminal 100 via the server 200. As an example, the communication control unit 411 receives the user list 421 from the server 200. In addition, the communication control unit 411, as an example, transmits the operation instruction data to the user terminal 100.

The display control unit 412 outputs, to the display unit 452, various screens on which the processing result executed by each element is reflected. The display control unit 412 displays a screen including the received user list 234, as an example. Further, as an example, the display control unit 412 displays a screen including a motion list 422 for allowing the player to select the motion data for operating the avatar object included in the operation instruction data to be distributed.

The operation receiving unit 413 detects and receives an input operation by the player on the input unit 151. The operation reception unit 111 determines what input operation is performed based on the action of the player on the console via the touch screen 45 and the other input/output IF 44, and outputs the result to each element of the control unit 410. To do.

For example, the operation receiving unit 413 receives an input operation on the input unit 451, detects the coordinates of the input position of the input operation, and specifies the type of the input operation. The operation reception unit 413 identifies, as the type of input operation, for example, a touch operation, a slide operation, a swipe operation, a tap operation, or the like. Further, the operation receiving unit 413 detects that the contact input is released from the touch screen 45 when the continuously detected input is interrupted.

The voice receiving unit 414 receives voice generated around the distribution terminal 400 and generates voice data of the voice. The voice receiving unit 414 receives, for example, a voice uttered by the player and generates voice data of the voice.

The motion identification unit 415 identifies the motion data selected by the player from the motion list 422 according to the input operation by the player.

The motion instruction data generation unit 416 generates motion instruction data. As an example, the motion instruction data generation unit 416 generates motion instruction data including the generated voice data and the identified motion data.

Note that the functions of the HMD set 1000, the server 200, and the user terminal 100 shown in FIG. 6 and the functions of the distribution terminal 400 shown in FIG. 7 are merely examples. Each device of the HMD set 1000, the server 200, the user terminal 100, and the distribution terminal 400 may have at least a part of the functions of another device. Furthermore, another device other than the HMD set 1000, the server 200, the user terminal 100, and the distribution terminal 400 may be a component of the system 1, and the other device may execute a part of the processing in the system 1. That is, the computer that executes the game program in the present embodiment may be any of the HMD set 1000, the server 200, the user terminal 100, the distribution terminal 400, and another device other than the HMD set 1000, and a plurality of these devices. It may be realized by a combination of the above devices.

<Virtual space control processing>
FIG. 8 is a flowchart showing an example of the flow of control processing of the virtual space provided to the player and the virtual space provided to the user of the user terminal 100. FIG. 9 is a diagram showing a virtual space 600A provided to a player and a view image visually recognized by the player according to an embodiment. FIG. 10 is a diagram showing a virtual space 600B provided to the user of the user terminal 100 and a view image visually recognized by the user according to an embodiment. Note that, hereinafter, when it is not necessary to distinguish between the virtual spaces 600A and 600B, it will be referred to as "virtual space 600".

In step S1, the processor 30 defines the virtual space 600A shown in FIG. 9 as the virtual space control unit 316. The processor 30 defines the virtual space 600A using virtual space data (not shown). The virtual space data may be stored in the game play terminal 300, may be generated by the processor 30 based on the game program 331, or may be acquired by the processor 30 from an external device such as the server 200. Good.

As an example, the virtual space 600 has a spherical structure that covers the entire point defined as the center in the 360-degree direction. 9 and 10, in order not to complicate the description, the celestial sphere in the upper half of the virtual space 600 is illustrated.

In step S2, the processor 30 arranges the avatar object 610 (character) in the virtual space 600A as the virtual space control unit 316. The avatar object 610 is an avatar object associated with the player, and operates according to the input operation of the player.

In step S3, the processor 30 as the virtual space control unit 316 arranges other objects in the virtual space 600A. In the example of FIG. 9, the processor 30 arranges the objects 631 to 634. The other objects imitate, for example, a character object (so-called non-player character, NPC) that operates according to the game program 331, an operation object such as a virtual hand, an animal, a plant, an artificial object, or a natural object arranged according to the progress of the game. It may include objects and the like.

In step S4, the processor 30 arranges the virtual camera 620A in the virtual space 600A as the virtual space control unit 316. As an example, the processor 30 arranges the virtual camera 620A at the position of the head of the avatar object 610.

In step S5, the processor 30 displays the visual field image 650 on the monitor 51 and the display 530. The processor 30 defines a view area 640A that is a view from the virtual camera 620A in the virtual space 600A according to the initial position and the tilt of the virtual camera 620A. Then, the processor 30 defines the view image 650 corresponding to the view region 640A. The processor 30 displays the visual field image 650 on the HMD 500 and the display 530 by outputting the visual field image 650 to the monitor 51 and the display 530.

In the example of FIG. 9, as shown in FIG. 9A, part of the object 634 is included in the visual field region 640A, so the visual field image 650 is one of the objects 634 as shown in FIG. 9B. Including parts.

In step S6, the processor 30 transmits the initial arrangement information to the user terminal 100 via the server 200. The initial placement information is information indicating initial placement positions of various objects in the virtual space 600A. In the example of FIG. 9, the initial placement information includes information on initial placement positions of the avatar object 610 and the objects 631 to 634. The initial placement information can also be expressed as one of the game progress information.

In step S7, the processor 30 as the virtual space control unit 316 controls the virtual camera 620A according to the movement of the HMD 500. Specifically, the processor 30 controls the orientation and inclination of the virtual camera 620A according to the movement of the HMD 500, that is, the posture of the player's head. As described below, when the player moves the head (changes the posture of the head), the processor 30 moves the head of the avatar object 610 in accordance with this movement. For example, the processor 30 controls the orientation and inclination of the virtual camera 620A so that the direction of the line of sight of the avatar object 610 and the direction of the line of sight of the virtual camera 620A match. In step S8, the processor 30 updates the view field image 650 according to the change of the orientation and inclination of the virtual camera 620A.

In step S9, the processor 30, as the virtual space control unit 316, moves the avatar object 610 according to the movement of the player. As an example, the processor 30 moves the avatar object 610 in the virtual space 600A in response to the player moving in the physical space. Further, the processor 30 moves the head of the avatar object 610 in the virtual space 600A in response to the player moving the head in the physical space.

In step S10, the processor 30 causes the virtual space control unit 316 to move the virtual camera 620A so as to follow the avatar object 610. That is, the virtual camera 620A is always at the position of the head of the avatar object 610 even if the avatar object 610 moves.

The processor 30 updates the view field image 650 according to the movement of the virtual camera 620A. That is, the processor 30 updates the view field area 640A according to the posture of the player's head and the position of the virtual camera 620A in the virtual space 600A. As a result, the view image 650 is updated.

In step S11, the processor 30 transmits the motion instruction data of the avatar object 610 to the user terminal 100 via the server 200. The motion instruction data here includes motion data that captures the motion of the player during the virtual experience (for example, during game play), voice data of the voice uttered by the player, and operation data indicating the content of the input operation to the controller 540. At least one is included. When the player is playing the game, the action instruction data is transmitted to the user terminal 100 as game progress information, for example.

The processes of steps S7 to S11 are continuously and repeatedly executed while the player is playing the game.

In step S21, the processor 10 of the user terminal 100 of the user 3 defines the virtual space 600B shown in FIG. 10 as the virtual space control unit 116. The processor 10 defines a virtual space 600B using virtual space data (not shown). The virtual space data may be stored in the user terminal 100, may be generated by the processor 10 based on the game program 131, or may be acquired by the processor 10 from an external device such as the server 200. Good.

In step S22, the processor 10 receives the initial placement information. In step S23, the processor 10, as the virtual space control unit 116, arranges various objects in the virtual space 600B according to the initial arrangement information. In the case of the example in FIG. 10, the various objects are an avatar object 610 and objects 631 to 634.

In step S24, the processor 10 as the virtual space control unit 116 arranges the virtual camera 620B in the virtual space 600B. As an example, the processor 10 arranges the virtual camera 620B at the position shown in FIG.

In step S25, the processor 10 displays the visual field image 660 on the display unit 152. The processor 10 defines a visual field 640B that is a visual field from the virtual camera 620B in the virtual space 600B according to the initial position and the inclination of the virtual camera 620B. Then, the processor 10 defines the visual field image 660 corresponding to the visual field region 640B. The processor 10 displays the visual field image 660 on the display unit 152 by outputting the visual field image 660 to the display unit 152.

In the example of FIG. 10, as shown in FIG. 10A, the avatar object 610 and the object 631 are included in the visual field region 640B, and therefore the visual field image 660 shows the avatar object 610 as shown in FIG. 10B. And an object 631.

In step S26, the processor 10 receives the operation instruction data. In step S27, the processor 10, as the virtual space control unit 116, moves the avatar object 610 in the virtual space 600B according to the operation instruction data. In other words, the processor 10 reproduces the image of the moving avatar object 610 by real-time rendering.

In step S28, the processor 10 controls the virtual camera 620B as the virtual space control unit 116 according to the user's operation accepted by the operation accepting unit 111. In step S29, the processor 10 updates the view image 660 in response to the position of the virtual camera 620B in the virtual space 600B, the orientation and the inclination of the virtual camera 620B being changed. Note that in step S28, the processor 10 may automatically control the virtual camera 620B according to the movement of the avatar object 610, for example, the movement of the avatar object 610 or the change of the orientation. For example, the processor 10 may automatically move the virtual camera 620B so as to always photograph the avatar object 610 from the front, or change the orientation and the inclination. In addition, as an example, the processor 10 automatically moves the virtual camera 620B so as to always photograph the avatar object 610 from the rear according to the movement of the avatar object 610, or changes the orientation and tilt. Good.

Thus, in the virtual space 600A, the avatar object 610 moves according to the movement of the player. The operation instruction data indicating this operation is transmitted to the user terminal 100. In the virtual space 600B, the avatar object 610 operates according to the received operation instruction data. As a result, the avatar object 610 performs the same operation in the virtual space 600A and the virtual space 600B. In other words, the user 3 can use the user terminal 100 to visually recognize the action of the avatar object 610 according to the action of the player.

<Game outline>
FIG. 11 is a diagram showing another example of the visual field image displayed on the user terminal 100. Specifically, it is a diagram showing an example of a game screen of a game (main game) executed by the system 1, which is being played by the player.

In this game, as an example, an avatar object 610 that operates a weapon such as a gun or a knife, and a plurality of enemy objects 671 that are NPCs are made to appear in the virtual space 600, and the avatar object 610 battles against the enemy object 671. It is a game that lets you. Various game parameters such as the physical strength of the avatar object 610, the number of magazines that can be used, the number of remaining bullets of the gun, and the remaining number of enemy objects 671 are updated as the game progresses.

A plurality of stages are prepared in this game, and the player can clear the stage by satisfying a predetermined achievement condition associated with each stage. The predetermined achievement condition is established by, for example, defeating all the appearing enemy objects 671, defeating the boss object among the appearing enemy objects 671, acquiring a predetermined item, or reaching a predetermined position. It may include a condition to do. The achievement conditions are defined in the game program 131. In this game, the player clears the stage when the achievement condition is satisfied in accordance with the content of the game, in other words, the avatar object 610 wins over the enemy object 671 (the avatar object 610 and the enemy object 671 Between the winning and losing) is determined. On the other hand, for example, when the game executed in the system 1 is a racing game or the like, the rank of the avatar object 610 is determined when the condition of reaching the goal is satisfied.

In this game, in order to share the virtual space between the HMD set 1000 and the plurality of user terminals 100, the game progress information is live-distributed to the plurality of user terminals 100 at predetermined time intervals. As a result, the visual field image of the visual field defined by the virtual camera 620B corresponding to the user terminal 100 is displayed on the touch screen 15 of the user terminal 100 watching the game. Parameter images representing the physical strength of the avatar object 610, the number of usable magazines, the number of remaining bullets of the gun, the number of remaining enemy objects 671 and the like are superimposed and displayed in the upper right and upper left of the view image. .. This view image can also be expressed as a game screen.

As described above, the game progress information includes the motion data that captures the player's motion, the voice data of the voice uttered by the player, and the operation data indicating the content of the input operation to the controller 540. These data are, for example, information for specifying the position, posture, direction, etc. of the avatar object 610, information for specifying the position, posture, direction, etc. of the enemy object 671, and other objects (for example, obstacle objects 672, 673). This is information for identifying the position of the. The processor 10 identifies the position, orientation, orientation, etc. of each object by analyzing (rendering) the game progress information.

The game information 132 includes data of various objects such as an avatar object 610, an enemy object 671, and obstacle objects 672 and 673. The processor 10 updates the position, orientation, orientation, etc. of each object using the data and the analysis result of the game progress information. As a result, the game progresses, and each object in the virtual space 600B moves in the same manner as each object in the virtual space 600A. Specifically, in the virtual space 600B, each object including the avatar object 610 operates based on the game progress information regardless of whether or not the user operates the user terminal 100.

On the touch screen 15 of the user terminal 100, as an example, UI images 701 and 702 are displayed so as to be superimposed on the view image. The UI image 701 is a UI image that receives an operation for displaying on the touch screen 15 a UI image 711 that receives an item input operation for supporting the avatar object 610 from the user 3. The UI image 702 receives an operation for inputting a comment on the avatar object 610 (in other words, the player 4) and receiving an operation for transmitting from the user 3 (described later) for displaying on the touch screen 15. It is a UI image. The operation received by the UI images 701 and 702 may be, for example, an operation of tapping the UI images 701 and 702.

When the UI image 701 is tapped, the UI image 711 is displayed by being superimposed on the view field image. The UI image 711 includes, for example, a UI image 711A with a magazine icon, a UI image 711B with a first aid box icon, a UI image 711C with a triangular cone icon, and a UI with a barricade icon. Image 711D is included. The item input operation corresponds to, for example, an operation of tapping one of the UI images.

As an example, when the UI image 711A is tapped, the number of remaining bullets of the gun used by the avatar object 610 increases. When the UI image 711B is tapped, the physical strength of the avatar object 610 is restored. When the UI images 711C and 711D are tapped, obstacle objects 672 and 673 that obstruct the movement of the enemy object 671 are arranged in the virtual space. The obstacle objects 672 and 673 may be ones that hinder the movement of the enemy object 671 more than the other.

The processor 10 transmits to the server 200 the item input information indicating that the item input operation has been performed. The item input information includes at least information for identifying the type of item designated by the item input operation. The item input information may include other information about the item, such as information indicating the position where the item is arranged. The item input information is transmitted to the other user terminals 100 and the HMD set 1000 via the server 200.

FIG. 12 is a diagram showing another example of the visual field image displayed on the user terminal 100. Specifically, it is a diagram showing an example of a game screen of the present game, and is a diagram for explaining communication between the player who is playing the game and the user terminal 100.

In the example of FIG. 12A, the user terminal 100 causes the avatar object 610 to execute the utterance 691. Specifically, the user terminal 100 causes the avatar object 610 to perform the utterance 691 in accordance with the voice data included in the game progress information. The content of the utterance 691 is “No bullet!” spoken by the player 4. That is, the content of the utterance 691 is to inform each user that the means for attacking the enemy object 671 is likely to be lost because the magazine has become 0 and the bullet loaded in the gun has become 1.

Note that in FIG. 12A, a speech bubble is used to visually indicate the utterance of the avatar object 610, but in reality, a voice is output from the speaker of the user terminal 100. The speech balloon shown in FIG. 12A (that is, the speech balloon including the text of the audio content) may be displayed in the visual field image together with the audio output. This also applies to the utterance 692 described later.

Upon accepting the tap operation on the UI image 702, the user terminal 100 displays the UI images 705 and 706 (message UI) superimposed on the view image, as shown in FIG. 12B. The UI image 705 is a UI image that displays a comment for the avatar object 610 (in other words, the player). The UI image 706 is a UI image that receives a comment transmission operation from the user 3 for transmitting the input comment.

As an example, when the user terminal 100 receives a tap operation on the UI image 705, the user terminal 100 causes the touch screen 15 to display a UI image imitating a keyboard (not shown, hereinafter simply referred to as “keyboard”). The user terminal 100 causes the UI image 705 to display text according to the user's input operation on the keyboard. In the example of FIG. 12B, the text “Magazine send” is displayed on the UI image 705.

For example, when the user terminal 100 receives a tap operation on the UI image 706 after the text is input, the user terminal 100 transmits comment information including information indicating the input content (text content) and information indicating the user to the server 200. To do. The comment information is transmitted to other user terminals 100 and the HMD set 1000 via the server 200.

The UI image 703A is a UI image showing the user name of the user who sent the comment, and the UI image 704A is a UI image showing the content of the comment sent by the user. In the example of FIG. 12B, the user whose user name is “BBBBBB” has transmitted the comment information of “dangerous!” using his or her own user terminal 100, so that the UI image 703A and the UI image 704A are transmitted. Is displayed. The UI image 703A and the UI image 704A are displayed on the touch screens 15 of all the user terminals 100 participating in the game and the monitor 51 of the HMD 500. The UI images 703A and 704A may be one UI image. That is, one UI image may include the user name and the content of the comment.

In the example of FIG. 12C, the user of the user terminal 100 shown in FIG. 12, who has the user name “AAAAA”, inputs and transmits the comment as described above, so that the UI image is displayed on the touch screen 15. 703B and 704B are displayed. The UI image 703B includes the user name “AAAAA”, and the UI image 704B includes the comment “Send magazine!” entered in the example of FIG. 12B.

In the example of FIG. 12C, the user “AAAAA” further inputs the tap operation on the UI image 701, displays the UI image 711 on the touch screen 15, and inputs the tap operation on the UI image 711A. It is a later view image 611. In other words, as a result of the item insertion information indicating the magazine being transmitted from the user terminal 100 of the user “AAAAA” to the other user terminals 100 and the HMD set 1000, the user terminal 100 and the HMD set 1000 have the effect object 674 (described later). Are arranged in the virtual space 600. As an example, the user terminal 100 and the HMD set 1000 execute the effect regarding the effect object 674 and execute the process of activating the effect of the item object after the elapsed time indicated by the item input information has elapsed.

In the example of FIG. 12D, the number of magazines is increased from 0 to 1 by executing the process of activating the effect of the item object. As a result, the player utters "Thank you!" to the user "AAAAA", and the voice data of the utterance is transmitted to each user terminal 100. As a result, each user terminal 100 outputs the voice "Thank you!" as the utterance 692 of the avatar object 610.

As described above, in the present game, the communication between the user and the avatar object 610 is realized by outputting the utterance voice of the avatar object 610 based on the utterance of the player and inputting the comment by each user.

(Game progression process in the game play terminal 300)
FIG. 13 is a flowchart showing an example of the flow of the game progress processing executed by the game play terminal 300.

In step S31, the processor 30, as the game progression unit 315, advances the game based on the game program 331 and the movement of the player. In step S32, the processor 30 generates game progress information and delivers it to the user terminal 100. Specifically, the processor 30 transmits the generated game progress information to each user terminal 100 via the server 200.

When the processor 30 receives the item input information in step S33 (YES in S33), the processor 30 arranges the item object in the virtual space 600A based on the item input information in step S34. As an example, the processor 30 arranges the effect object 674 in the virtual space 600A before the arrangement of the item object (see FIG. 11C). The effect object 674 may be, for example, an object imitating a present box. As an example, the processor 30 may execute the effect regarding the effect object 674 after the elapsed time indicated by the item input information has elapsed. The effect may be, for example, an animation in which the lid of the present box opens. After executing the animation, the processor 30 executes processing for activating the effect of the item object. For example, in the example of FIG. 11D, the obstacle object 673 is arranged.

The processor 30 may arrange the item object corresponding to the UI image subjected to the tap operation in the virtual space 600A after the execution of the animation. For example, when a tap operation is performed on the UI image 711A, the processor 30 arranges a magazine object indicating a magazine in the virtual space 600A after executing the animation. Further, when the tap operation is performed on the UI image 711B, the processor 30 arranges the first aid kit object indicating the first aid kit in the virtual space 600A after the execution of the animation. The processor 30 may execute a process of activating the effect of the magazine object or the first aid box object when the avatar object 610 moves to the position of the magazine object or the first aid box object, for example.

The processor 30 continues and repeats the processes of steps S31 to S34 until the game ends. When the game ends, for example, when the player inputs a predetermined input operation for ending the game (YES in step S35), the process illustrated in FIG. 13 ends.

(Game progression process in user terminal 100)
FIG. 14 is a flowchart showing an example of the flow of the game progress processing executed by the user terminal 100.

In step S41, the processor 10 receives the game progress information. In step S42, the processor 10 causes the game progress section 115 to progress the game based on the game progress information.

In step S43, when processor 10 accepts the item input operation by user 3 (YES in step S43), processor 10 consumes virtual currency and places effect object 674 in virtual space 600B in step S44. Here, the virtual currency is purchased (charged for this game) by the user 3 performing a predetermined operation on the processor 10 before or during participation in the game. It may be provided, or may be given to the user 3 when a predetermined condition is satisfied. The predetermined condition may be one that requires participation in this game, such as clearing a quest in this game, or one that does not require participation in this game, such as answering a questionnaire. The amount of virtual currency (amount of virtual currency possessed) is stored in the user terminal 100 as the game information 132, for example.

In step S45, the processor 10 transmits the item input information to the server 200. The item input information is transmitted to the game play terminal 300 via the server 200.

The processor 10 arranges the item object in the virtual space 600A when a predetermined time passes after the arrangement of the effect object 674. In the example of FIG. 11, the obstacle object 673 is arranged. That is, when the user 3 inputs the tap operation on the UI image 711C, a predetermined amount of virtual currency is consumed and the obstacle object 673 is arranged.

The processor 10 continues and repeats the processing of steps S41 to S45 until the game ends. When the game is over, for example, when the player performs a predetermined input operation for ending the game, or when the user 3 performs a predetermined input operation for leaving the game halfway (YES in step S46). ), the process shown in FIG. 14 ends.

(Game progression process in server 200)
FIG. 15 is a flowchart showing an example of the flow of the game progress processing executed by the server 200.

In step S51, the processor 20 receives the game progress information from the game play terminal 300. In step S52, the processor 20, as the log generation unit 212, updates the game progress log (hereinafter, play log). The play log is generated by the processor 20 when the initial placement information is received from the game play terminal 300, for example.

In step S53, the processor 20 transmits the received game progress information to each user terminal 100.

When the item input information is received from any of the user terminals 100 in step S54 (YES in step S54), the processor 20 updates the play log as the log generation unit 212 in step S55. In step S56, the processor 20 transmits the received item insertion information to the game play terminal 300.

The processor 20 continues and repeats the processing of steps S51 to S56 until the game ends. When the game is over, for example, when the information indicating that the game is over is received from the game play terminal 300 (YES in step S57), in step S58, the processor 20 sets the list generation unit 213 from the play log. A list of users who participated in the game (user list 234) is generated. The processor 20 stores the generated user list 234 in the server 200.

FIG. 16 is a diagram showing a specific example of the user list 234. Information (for example, a user name) indicating each user who participated in the game is stored in the “user” column. In the “tag” column, information (tag) generated based on the support provided to each player by each user is stored. In the example of FIG. 16, among the tags stored in the “tag” column, those without the brackets are the information automatically generated by the processor 20, and those with the brackets are those that the operator of the game plays. The information is manually entered.

In the example of FIG. 16, the user “AAAAA” is associated with information such as magazine 10F, boss, and “win boss by present of magazine”. This indicates that, for example, in the boss battle on the stage of 10F, the user “AAAAA” has inserted a magazine, and the avatar object 610 has won the boss with a bullet of the inserted magazine.

In addition, the user "BBBBBB" is associated with information such as a first aid kit, 3F, Zako, and "Recovery on the verge of game over". For example, in the battle with the Zako enemy on the stage of 3F, the user " BBBBB” has inserted the first aid box, and as a result, the avatar object 610 has recovered its physical strength on the verge of reaching 0 (game over).

In addition, the user “CCCCC” is associated with information such as barricade, 5F, Zako, and “barricade stops two zombies.” This is because, for example, in the battle with the Zaco enemy on the stage of 5F, the user “CCCCC” throws in a barricade (obstacle object 672 in FIG. 11), and as a result, the two Zaco enemies are successfully stopped. Showing.

In the example of FIG. 16, one support provided is associated with the user name of each user 3, but the user name of the user 3 who has provided support multiple times includes tags for each support multiple times. Is associated. It is preferable that the respective tags are distinguished in the user list 234. As a result, after the game ends, the player who refers to the user list 421 using the distribution terminal 400 can accurately understand the content of each support.

<Distribution of operation instruction data>
(Distribution process in distribution terminal 400)
FIG. 17 is a flowchart showing an example of the flow of distribution processing executed by the distribution terminal 400. FIG. 18 is a diagram showing a specific example of a screen displayed on distribution terminal 400. FIG. 19 is a diagram showing another specific example of the screen displayed on the distribution terminal.

In step S61, the processor 40, as the operation receiving unit 413, receives the first operation for displaying the list of users who have participated in the game (user list 234). A download screen 721 shown in FIG. 18A is a screen for downloading the user list 234 from the server 200 and displaying it on the display unit 452. The download screen 721 is, for example, a screen displayed immediately after the activation operation of the application that executes the distribution process illustrated in FIG. 17 is input to the distribution terminal 400.

Download screen 721 includes UI images 722 and 723, for example. The UI image 722 receives an operation for downloading the user list 234, that is, the first operation. The first operation may be, for example, an operation of tapping the UI image 722. The UI image 723 receives an operation for ending the application. The operation may be, for example, an operation of tapping the UI image 723.

Upon accepting the tap operation on the UI image 722, in step S62, the processor 40, as the communication control unit 411, acquires (receives) the user list 234 from the server 200. In step S63, the processor 40 causes the display unit 452 to display the user list 234 as the display control unit 412. Specifically, the processor 40 causes the display unit 452 to display the user list screen generated based on the user list 234. The user list screen may be, for example, the user list screen 731 shown in FIG. The user list screen 731 includes record images corresponding to the records in the user list 234. In the example of FIG. 18B, record images 732A to 732C are described as record images, but the number of record images is not limited to three. In the example of FIG. 18B, when the number of records in the user list 234 is larger than 3 (that is, the number of users who participated in the game is larger than 3), the player performs, for example, an operation of scrolling the screen (for example, By inputting a drag operation or a flick operation) on the touch screen 45, another record image can be displayed on the display unit 452.

As an example, the record images 732A to 732C include user names 733A to 733C, tag information 734A to 734C, and icons 735A to 735C, respectively. After that, when it is not necessary to distinguish the record images 732A to 732C, the user names 733A to 733C, the tag information 734A to 734C, and the icons 735A to 735C, the "record image 732", the "user name 733", and the "user name 733", respectively. It is described as "tag information 734" and "icon 735".

The user name 733 is information stored in the “user” column of the user list 234 and indicating each user who participated in the game. The tag information 734 is information indicating a tag associated with each piece of information indicating each user who participated in the game in the user list 234. For example, the record image 732A includes "AAAAAA" as the user name 733A. Therefore, the record image 732A includes, as the tag information 734A, “magazine 10F, boss, “win boss by magazine present”, which is associated with “AAAAA” in the user list 234. The icon 735 is, for example, an image preset by the user.

Note that the processor 40 may store the received user list in the distribution terminal 400 (user list 421 in FIG. 7). The download screen 721 may include a UI image (not shown) for displaying the user list 421 on the display unit 452. In this example, when the UI image is tapped, the processor 40 does not download the user list 234, reads the user list 421, generates a user list screen from the user list 421, and displays it on the display unit 452. ..

In step S64, the processor 40, as the operation receiving unit 413, receives the second operation for selecting one of the users included in the user list screen 731. The second operation may be, for example, an operation of tapping any one of the record images 732 on the user list screen 731. In the example of FIG. 18B, the player has input a tap operation on the record image 732A. That is, the player has selected the user “AAAAAA” as the user who distributes the motion instruction data.

Upon accepting the tap operation on the record image 732, in step S65, the processor 40 causes the display unit 452 to display the motion list 422 as the display control unit 412. Specifically, the processor 40 causes the display unit 452 to display the motion list screen generated based on the motion list 422. The motion list screen may be the motion list screen 741 shown in FIG. 19 as an example. The motion list screen 741 includes record images corresponding to each record in the motion list 422. In the example of FIG. 19, record images 742A to 742C are described as record images, but the number of record images is not limited to three. In the example of FIG. 19, when the number of records in the motion list 422 is more than 4, the player inputs another record by inputting an operation of scrolling the screen (for example, a drag operation or a flick operation) on the touch screen 45. The image can be displayed on the display unit 452.

As an example, the record images 742A to 742C include motion names 743A to 743C, motion images 744A to 744C, and UI images 745A to 745C, respectively. After that, if it is not necessary to distinguish between the record images 742A to 742C, the motion names 743A to 743C, the motion images 744A to 744C, and the UI images 745A to 745C, respectively, the “record image 7432”, the “motion name 743”, It is described as "motion image 744" and "UI image 745".

The motion name 743 is information for identifying a motion stored in the motion list 422. The motion image 744 is an image generated from the motion data associated with each motion name in the motion list 422. As an example, the processor 40 includes the image of the avatar object 610 that takes the initial posture in each motion data in the record image 742 as the motion image 744. The motion image 744 may be a UI image that receives a predetermined operation by the player (for example, a tap operation on the motion image 744). When the processor 40 receives the predetermined operation, the processor 40 may reproduce the motion video in which the avatar object 610 operates based on the motion data. The processor 40 may automatically redisplay the motion list screen 741 when the motion video ends.

Note that the record image 742 may include, for example, a UI image including the text “motion playback” instead of the motion image 744.

In step S66, the processor 40, as the operation receiving unit 413, receives the third operation for selecting a motion. The third operation may be, for example, a tap operation on the UI image 745. That is, the UI image 745 receives an operation of selecting motion data corresponding to each record image 742. Upon receiving the third operation, the processor 40 causes the motion identifying unit 415 to identify the motion data selected by the player.

In step S67, the processor 40, as the display control unit 412 and the voice reception unit 414, receives the voice input of the player while the avatar object 610 reproduces the motion video that operates based on the selected motion data.

FIG. 20 is a diagram showing a specific example of voice input by the player 4. As shown in FIG. 20, the player 4 is inputting the uttered voice 820A while reproducing the motion video 810A. The utterance voice 820A is an utterance voice addressed to the user 3 (hereinafter, user 3A) whose user name is “AAAAAA”. That is, in the example of FIG. 20, the player 4 selects the user 3A (first user) in step S64, and creates motion instruction data addressed to the user 3A. The user terminal 100 used by the user 3A is assumed to be the user terminal 100A.

Since the utterance voice 820A is the utterance voice addressed to the user 3A, the utterance voice 820A is the utterance voice based on the content of the support provided by the user 3A to the avatar object 610 (in other words, the player 4). Specifically, the user 3A inserts a magazine in the boss battle on the stage of 10F, and the avatar object 610 wins the boss with a bullet of the inserted magazine. Therefore, the utterance voice 820A is "Thank you for presenting the magazine in the boss battle! The timing was perfect! Thanks to Mr. AAAAA, I was able to clear it!" As described above, it is preferable that the uttered voice includes the content of the support provided by the user 3 in the game and the appreciation to the user 3.

In a certain aspect, the player 4 creates the utterance content addressed to the user 3 before starting voice input, that is, before inputting the third operation to the distribution terminal 400. In another aspect, the utterance content addressed to the user 3 may be automatically generated by the processor 40. Moreover, the processor 40 may superimpose the tag associated with the user 3 selected by the second operation on the motion video 810A and display the tag.

The processor 40 converts the received voice into voice data. In step S68, the processor 40, as the motion instruction data generation unit 416, generates motion instruction data including the voice data and the motion data of the selected motion.

In step S69, the processor 40, as the communication control unit 411, distributes the generated operation instruction data to the user terminal 100 (first computer) of the selected user 3 (user 3A in the example of FIG. 20). FIG. 21 is a diagram showing still another specific example of the screen displayed on the distribution terminal 400. After the execution of step S68, the processor 40 causes the display unit 452 to display the distribution screen as the display control unit 412. The distribution screen may be, for example, the distribution screen 751 shown in FIG. The distribution screen 751 includes a UI image 752 and a motion image 753A. Further, the distribution screen 751 may include information indicating the user to whom the operation instruction data is distributed, as shown in FIG.

The UI image 752 receives an operation for delivering the action instruction data to the selected user 3. The operation may be, for example, a tap operation on the UI image 752. The motion image 753A is a UI image that receives an operation for reproducing a moving image based on the generated operation instruction data, that is, a moving image based on the operation instruction data generated for the user 3A. The operation may be, for example, a tap operation on the motion image 753A. Note that the UI image that accepts the operation for playing the generated moving image is not limited to the motion image 753A. For example, it may be a UI image including a text of “play video”. The processor 40 may automatically redisplay the distribution screen 751 when the moving image ends.

The distribution screen 751 preferably further includes a UI image for accepting an operation for returning to acceptance of voice input. The operation may be, for example, a tap operation on the UI image. When the distribution screen 751 includes the UI image, the player 4 can perform voice input again when voice input fails, for example, when the content of utterance is wrong. The UI image may be a UI image that receives an operation for returning to the selection of motion data.

Upon accepting the tap operation on the UI image 752, the processor 40 transmits the operation instruction data to the server 200 together with the information indicating the user 3A. The server 200 identifies the destination user terminal 100 of the action instruction data based on the information indicating the user 3A, and transmits the action instruction data to the identified user terminal 100 (that is, the user terminal 100A).

When the transmission of the operation instruction data is completed, the processor 40 may display the distribution completion screen 761 shown in FIG. 21B on the display unit 452 as an example. The distribution completion screen 761 includes UI images 762 and 763 as an example. Further, the distribution completion screen 761 may include a text indicating that the transmission of the motion instruction data has been completed, as shown in FIG.

The UI image 762 receives an operation for starting the creation of motion instruction data addressed to another user 3. The operation may be, for example, an operation of tapping the UI image 762. Upon receiving the tap operation, the processor 40 causes the display unit 452 to display the user list screen again. That is, when the tap operation is accepted, the distribution process returns to step S63. At this time, the processor 40 may generate a user list screen based on the user list 421 stored in the distribution terminal 400, and display the user list screen on the display unit 452. The UI image 763 receives an operation for ending the application. The operation may be, for example, an operation of tapping the UI image 763. When the operation is accepted, the distribution process ends.

In the example described with reference to FIGS. 20 and 21, as illustrated in FIG. 21C, the distribution terminal 400 transmits the motion instruction data of the moving image addressed to the user 3A (user 3 whose user name is “AAAAAA”). , To only the user terminal 100A.

FIG. 22 is a diagram showing another specific example of the voice input by the player 4. As shown in FIG. 22, the player 4 is inputting the uttered voice 820B while reproducing the motion video 810B. The utterance voice 820B is an utterance voice addressed to the user 3 (hereinafter, user 3B) whose user name is “BBBBBB”. That is, in the example of FIG. 22, the player 4 inputs the tap operation to the record image 732B corresponding to the user 3B in step S64, and creates the action instruction data addressed to the user 3B. The user terminal 100 used by the user 3B is assumed to be the user terminal 100B.

Since the uttered voice 820B is the uttered voice addressed to the user 3B, the uttered voice 820B is the uttered voice based on the content of the support provided by the user 3B to the avatar object 610 (in other words, the player 4). Specifically, the user 3B is on the verge of having the physical strength of the avatar object 610 become 0 (the game is over) as a result of the user "BBBBB" throwing the first aid kit in the battle with the Zako enemy on the stage 3F. I have recovered my physical strength. For this reason, the utterance voice 820B has the content, "Thanks to the first aid kit that Mr. BBBBB gave me, the game will not be over in 3F. Thank you very much!"

FIG. 23 is a diagram showing yet another specific example of the screen displayed on the distribution terminal 400. The distribution screen 751 illustrated in FIG. 23A includes a UI image 752 and a motion image 753B. Upon receiving the tap operation, the motion image 753B reproduces a moving image based on the action instruction data generated for the user 3B.

Upon accepting the tap operation on the UI image 752, the processor 40 transmits the operation instruction data to the server 200 together with the information indicating the user 3B. The server 200 specifies the destination user terminal 100 of the operation instruction data based on the information indicating the user 3B, and transmits the operation instruction data to the specified user terminal 100 (that is, the user terminal 100B).

In the example described with reference to FIGS. 22 and 23, as illustrated in FIG. 23C, the distribution terminal 400 transmits the motion instruction data of the moving image addressed to the user 3B (user 3 having the user name “BBBBBB”). , To only the user terminal 100B.

As described above, the content of the sound based on the sound data included in the operation instruction data is based on the content of the support provided to the player 4 by the user 3 in participating in the latest game. Since the contents of the support are different for each user 3, the contents of the voice are different for each user 3. That is, after the end of the game, the operation instruction data including different sounds are transmitted to at least some of the user terminals 100 of the users 3 who participated in the game.

The motion of the avatar object 610 in the example of FIG. 22 is different from the motion in the example of FIG. That is, the player 4 selects motion data that is different from that at the time of generating the motion instruction data addressed to the user 3A when generating the motion instruction data addressed to the user 3B. Specifically, in step S66, the player 4 inputs a tap operation on the UI image 745B for selecting motion data corresponding to the record image 742B. In this way, the player 4 can make the motion data included in the motion instruction data different for each user 3.

Then, the motion instruction data for each user 3, including the voice data having different contents for each user 3 and the motion data selected for each user 3, is transmitted only to the user terminal 100 of each user 3. In other words, unique motion instruction data for each user terminal 100 is transmitted to each of the user terminals 100 of the selected user 3.

FIG. 24 is a diagram showing an outline of transmission of game progress information from the game play terminal 300 to the user terminal 100. While the operation instruction data for moving image reproduction in the user terminal 100 is unique for each user terminal 100, as shown in FIG. 24, the user terminal 100 of all the users 3 participating in the game during the game execution. The game progress information transmitted to each of the user terminals 100 is common. That is, the action instruction data included in the game progress information is also common to each user terminal 100. Thus, it can be said that the motion instruction data for reproducing the moving image and the motion instruction data for advancing the game are different data in terms of the difference between the user terminals 100 and the destination.

(Movie reproduction process in the user terminal 100)
FIG. 25 is a flowchart showing an example of the flow of a moving image reproduction process executed by the user terminal 100.

In step S71, the processor 10 as the moving image reproducing unit 117 receives the operation instruction data. In step S72, the processor 10, as the moving image reproducing unit 117, notifies the user 3 of the reception of the operation instruction data. As an example, the processor 10 displays a notification image on the display unit 152, reproduces a notification sound from a speaker (not shown), lights a lighting unit (not shown) including an LED (light-emitting diode), or the like. The user 3 is notified of the reception of the operation instruction data by at least one of blinking.

In step S73, the processor 10 as the operation reception unit 111 receives the first reproduction operation for reproducing the moving image. The first reproduction operation may be, for example, an operation of tapping the notification image. In step S74, the processor 10, as the moving image reproduction unit 117, renders the motion instruction data and reproduces the moving image. As an example, the processor 10 may start an application for playing this game and play a video, or may start an application for playing a video different from the application and play a video. Good. Hereinafter, the moving image will be referred to as a "thank you video".

FIG. 26 is a diagram showing a specific example of reproduction of a thank you video. Specifically, it is a diagram showing an example of reproduction of a thank you video on the user terminal 100 of the user 3A. In the thank you video 910A reproduced on the user terminal 100, the avatar object 610 speaks a voice 920A while executing a certain motion. In other words, the processor 10 outputs the sound 920A from the speaker (not shown) while reproducing the thank you video 910A including the avatar object 610 that executes a certain motion.

The motion in the thank you video 910A is based on the motion data selected by the player 4 in the generation of the motion instruction data addressed to the user 3A, and the voice 920A is the utterance voice input by the player 4 in the generation of the motion instruction data. It is based on the audio data generated from 820A. That is, the sound 920A is a sound including the content of the support provided by the user 3A in the game and the appreciation for the support. In this manner, the user 3A can view the thank-you video in which the avatar object 610 utters the content of the support that the user 3A performed in the game and the gratitude for the support, by inputting the first reproduction operation.

As an example, the user terminal 100 may display at least one UI image on the touch screen 15 after the reproduction of the thank you video 910A is completed. The UI image may be, for example, a UI image that receives an operation for reproducing the thank you video 910A again, a UI image that receives an operation for transitioning to another screen, or an application. It may be a UI image that receives an operation for ending.

Moreover, the user terminal 100 may display at least one UI image on the touch screen 15 during reproduction of the thank you video 910A, for example. The UI image may be, for example, a plurality of UI images that respectively accept an operation of temporarily stopping or terminating the thank-you video 910A being reproduced, or changing a scene to be reproduced.

Note that these UI images displayed during the reproduction of the thank you video 910A and after the hunting of the thank you video 910A does not include a UI image for making a response to the avatar object 610. That is, the thank you video 910A according to the present embodiment does not include means for making a response to the avatar object 610.

FIG. 27 is a diagram showing another specific example of reproduction of a thank you video. Specifically, it is a diagram showing an example of reproduction of a thank-you video on the user terminal 100 of the user 3B. In the thank you video 910B reproduced on the user terminal 100, the avatar object 610 speaks a voice 920B while executing a certain motion. In other words, the processor 10 outputs the sound 920B from the speaker (not shown) while playing the thank you video 910B including the avatar object 610 that executes a certain motion.

The motion in the thank you video 910B is based on the motion data selected by the player 4 in the generation of the motion instruction data addressed to the user 3B, and the voice 920B is the utterance voice input by the player 4 in the generation of the motion instruction data. It is based on the audio data generated from 820B. Therefore, in the example of FIG. 27, the motion performed by the avatar object 610 is different from the motion of the example of FIG. Further, the voice 920B is a voice including the content of the support provided by the user 3B in the game and thanks for the support. Therefore, in the example of FIG. 27, the content of the sound 920B is different from the content of the sound 920A in the example of FIG.

In this way, the thank you video received by at least some of the user terminals 100 of the users 3 who participated in the game after the game is a video in which the utterance content of the avatar object 610 differs for each user 3.

Note that the processor 10 may display the UI image 930 including the content prompting participation in the next game by superimposing it on the moving image 910. The UI image 930 may be distributed together with the operation instruction data, or may be stored in the user terminal 100 as the game information 132.

In this game, the game play terminal 300 distributes game progress information, which is distribution content, to a plurality of user terminals 100 via the server 200. Each of the plurality of user terminals 100 displays the distribution content on the touch screen 15 based on the game progress information. Specifically, the memory 11 of the user terminal 100 stores in advance a plurality of types of object data for displaying objects including the avatar object 610. When the user terminal 100 receives the game progress information from the game play terminal 300, the user terminal 100 specifies the action instruction data from the game progress information, and analyzes (renders) the action instruction data to determine the type/position/orientation of the object. Etc. are specified. Further, the user terminal 100 arranges objects including the avatar object 610 in the virtual space 600B using the object data stored in the memory 11 based on the analysis result. On the touch screen 15, a visual field image 660 of the visual field 640B corresponding to the current position and orientation of the virtual camera 620B is displayed. It should be noted that objects including the avatar object 610 are also arranged in the virtual space 600A defined by the game play terminal 300, and the position/posture of the object is the same in the virtual spaces 600A and 600B.

The distribution content includes a plurality of types of content with different characters displayed as the avatar objects 610. The plurality of types of content include, for example, first distribution content that is distributed once or irregularly, and second distribution content that is regularly distributed every day or at a predetermined time on a predetermined day of the week.

The first distribution content and the second distribution content are, for example, a plurality of types of characters having different characters, such as a character who appears as a protagonist or a facilitator, characters that appear in the content, progress, and others, and the type of game played in the content. Is provided. Further, in the second distribution content, although the same character according to the type (for example, the character a according to the content a) appears and the character background (for example, a wreath object etc.) is displayed every time, the content of the same type ( For example, the content a) is different every time.

Decorative objects (texture images) such as clothes and accessories are associated with the character displayed as the avatar object 610 for each type of character. Hereinafter, a specific method for specifying the display mode of the decoration object will be exemplified.

The distribution content includes the first character content in which the first character appears as the avatar object 610 in the virtual space 600B, the second character content in which the second character appears as the avatar object 610 in the virtual space 600B, and the like. When delivering the first character content, the action instruction data is data (first display data) for enabling display of the image of the virtual space 600B in which the object including the first character is arranged. On the other hand, when delivering the second character content, the action instruction data is data (second display data) for enabling display of the image of the virtual space 600B in which the object including the second character is placed. ..

In each of the first character and the second character, the upper body, the lower body, the hands, the feet, the neck, the ear lobes, the eyes, and the like correspond to clothes and accessories (necklaces, earrings, glasses, etc.). The decoration object is associated in advance. The type and the part of the decorative object to be associated are defined so as to differ depending on the type of the character.

In the present embodiment, the first character includes a first object corresponding to a shirt and a third object that covers a part of the first object by wearing it on the first object (shirt). Decorative objects including (blouse) and are associated. On the other hand, the second character is associated with a decoration object including a second object of the same type as the first object (a shirt having a different design or the like).

Here, the display mode of the first object associated with the first character is determined according to a predetermined rule (for example, random number lottery) on the side of the game play terminal 300, for each distribution such as when distribution is started. .. The game play terminal 300 distributes game progress information including display mode information capable of specifying the display mode to the user terminal 100, and the user terminal 100 displays the first object display mode based on the display mode information. Specify and set. On the other hand, the display mode of the decoration objects other than the first object associated with the first character is fixed in advance, including the display mode of the third object. The user terminal 100 sets the display mode of the decoration object other than the first object to the fixed display mode. On the other hand, the display mode of the decoration object associated with the second character is fixed in advance, including the display mode of the second object of the same type as the first object. The user terminal 100 sets the display mode of the decoration object to a fixed display mode.

Therefore, the display mode of the first object among the decoration objects associated with the first character is changed for each distribution. Here, changing the display mode includes changing the texture image, and the texture image of the first object is changed for each distribution. On the other hand, regarding the decoration objects other than the first object associated with the first character and the decoration objects associated with the second character, the display mode is regardless of the number of times of distribution (distribution period). Fixed without.

The display mode is a mode in which the decorative object is displayed on the display unit 152. In the present embodiment, an example in which the display mode determined for the first object (shirt) is the display color of the first object (that is, the shirt color) is shown. Therefore, of the decorative objects such as shirts and blouses associated with the first character, the shirt color is specified from a plurality of colors (white, black, red, green, blue, purple, etc.).

In the present embodiment, the second distribution content that is periodically distributed includes, for example, the first character content in which the first character in which the display mode of the first object is changed appears. Further, in the present embodiment, as the first character, for example, a character having the appearance of an office lady is exemplified. However, the first character is not limited to this, and for example, a character (for example, a high school student character) whose age set to the character is less than a predetermined age (18 years old), a character that looks like a cheering party, or a female college student. It may be a character having the appearance of.

The content is distributed from the game play terminal 300 during a predetermined distribution period. When the distribution of the content is started, the processor 30 of the game play terminal 300 executes the display mode control process shown in FIG. The viewing of the content on the user terminal 100 side is started by a user's touch operation on a predetermined content icon. When the viewing of the content is started by the touch operation, the processor 10 on the user terminal 100 side executes the display mode setting process shown in FIG.

28A, in step S81, the game play terminal 300 stores in the memory 31 whether or not the current timing coincides with the content distribution start timing that is predetermined for each content. It is determined based on the distribution schedule. That is, in the memory 31, information such as a time slot for distributing the content and the type of the content (characters that appear, that is, whether the content is the first character content, the second character content, or the like) is stored as a distribution schedule. Remembered In step S81, it is determined based on the delivery schedule whether or not they match.

If it is determined that the timing matches the start timing, the process proceeds to step S82. In step S82, it is determined whether the content to be distributed this time is the first character content based on the distribution schedule. When it is determined that the content is the first character content, the process proceeds to step S83.

In step S83, the display mode (that is, the color of the shirt) of the first object among the decoration objects associated with the first character in the current distribution period is displayed in a plurality of types (white, black, red, green, Blue, purple, etc.) (randomly). More specifically, the first character content is the second distribution content that is periodically distributed as described above, and in step S83, the display mode at the time of the previous distribution, which was determined at the time of the previous distribution of the first character content, is displayed. A display mode different from that is determined by random number lottery. As a result, the display mode of the first object (that is, the color of the shirt) is changed to a display mode different from the previous display mode.

In step S84, display mode information (first data) that can specify the display mode determined in step S83 is set as the information included in the game progress information. When the first character content is distributed, the game progress information, which is information according to the motion of the performer and includes display mode information according to the determination in step S83, is performed every predetermined time (for example, 1/60 seconds). be delivered.

When the process of step S84 is completed, when it is not determined that the current timing coincides with the start timing of content distribution in step S81, or it is not determined that the content to be distributed this time is the first character content in step S82. When it returns. When the content other than the first character content is distributed, the game progress information according to the motion of the performer is distributed every predetermined time (for example, 1/60 seconds).

With reference to FIG. 28B, the user terminal 100 determines in step S91 whether or not a content viewing start operation has been performed based on an input operation on the touch screen 15. If it is not determined that the viewing start operation has been performed, the process returns, and if it is determined that the viewing start operation has been performed, the process proceeds to step S92. In step S92, it is determined whether the game progress information received from the game play terminal 300 includes the display mode information by analyzing the game progress information.

If it is determined that the display mode information is included, the process proceeds to step S93, and the display mode of the first object in the current distribution period is set based on the display mode information. That is, among the display modes of the decoration object in the current distribution period, the display mode of the first object (that is, the color of the shirt) is the display mode specified by the display mode information (white, black, red, green, Either blue or purple). When the process of step S93 is completed, or when it is not determined in step S92 that the game progress information including the display mode information is received, the process proceeds to step S94.

In step S94, a fixed display mode is set as the display mode of objects other than the first object among the decoration objects associated with the first character. Further, when viewing content in which characters other than the first character appear, in step S94, a decoration object (including a shirt) associated with a character different from the first character (for example, the second character). A fixed display mode is set as the display mode of. That is, when the first character content is distributed, the decoration objects (blouse, pants, shoes, etc.) other than the first object (shirt) associated with the first character are fixed. Set the display mode (fixed color). On the other hand, when the content different from the first character content (for example, the second character content) is distributed, all the costume objects associated with the character different from the first character (for example, the second character). Is set to a fixed display mode.

Through the processes of steps S93 and S94, model data of the avatar object 610 to be operated in the virtual space 600B is generated. While viewing the content, the avatar object 610 based on the model data operates based on the operation instruction data included in the game progress information distributed from the game play terminal 300. When the process of step S94 is completed, the process returns.

FIG. 29 shows a display example of an object including the first character while viewing the first character content. In FIG. 29, a wreath object reminiscent of a wreath is arranged behind the first character. The first character operates based on the motion instruction data.

Decorative objects associated with the first character include a shirt worn on the upper body, a blouse draped over the shirt, pants worn on the lower body, shoes worn on the legs, and the like. Further, for the decorative objects other than the shirt, which is the first object, the colors as the display mode are uniformly set. For example, white is defined for blouses and shoes, and gray is defined for pants.

On the other hand, with respect to the shirt, which is the first object, as shown in step S83 of FIG. 28, a color different from that of the previous distribution is set for each content distribution. As the display mode of the shirt, white is set as shown in FIG. 29A in the N-th live distribution targeting the first character content. Also, in the N+1th live distribution targeting the first character content, a gray color is set as shown in FIG. 29B, and in the N+2nd live distribution targeting the first character content, As shown in FIG. 29(C), blue is set (the striped pattern in the figure indicates blue). As described above, with respect to the first character content, the same first character appears by wearing a shirt of a different color for each content distribution.

<Effect of this embodiment>
According to the present embodiment, the image of the virtual space 600B including the first character that moves in association with the motion of the performer is based on the motion instruction data (first display data) distributed from the game play terminal 300. Is displayed. Here, the first character is associated with a predetermined first object (shirt), and the display mode (shirt color) of the first object is changed according to a predetermined rule. As a result, the display mode of the first object can be changed spontaneously, and as a result, it is possible to make the user feel the presence of the first character.

Further, according to the present embodiment, the video of the virtual space 600B including the second character that moves in association with the motion of the performer becomes the motion instruction data (second display data) distributed from the game play terminal 300. It is displayed based on. Here, the second character is associated with a second object (shirt) of the same type as the first object, and the display mode (shirt color) of the second object is fixed. Thereby, the presence of the first character can be emphasized.

Further, according to the present embodiment, the display mode of the first object is changed for each distribution period to a display mode different from the display mode in the previous distribution period. As a result, it is possible to provide the user with an impression that the display mode of the first object is intentionally changed for each distribution, and also to provide the user with the fun of what kind of display mode will appear during the current distribution period. The audience rating of can be improved.

Further, according to the present embodiment, the first display data is data distributed from the game play terminal 300 and includes motion data corresponding to the motion input by the performer. The video of the first character moving in the virtual space 600B is displayed by moving the first character in real time in the virtual space 600B based on the motion data. This makes it possible to suppress the amount of data to be distributed.

Further, according to the present embodiment, the display mode of the first object changed according to the predetermined rule is specified by the display mode information (first data) distributed from the game play terminal 300. The display mode of the first object is set based on the first data. Thereby, the display mode of the first object on the plurality of user terminals 100 can be uniformly controlled on the game play terminal 300 side.

Further, according to the present embodiment, a predetermined third object (blouse) is associated with the first character. The third object is an object that covers at least a part of the first object. With this, it becomes possible to control the display state of the first object by the third object.

<Modification>
Modifications and the like of the embodiment described above are listed below.

(1) In the above-described embodiment, the avatar object 610 and the like are operated in the same virtual space 600B as the virtual space 600A, and the virtual camera 620B arranged in the virtual space 600B and swiping the touch screen 15 of the user terminal 100. The normal viewing mode in which the view image 660 of the view area 640B corresponding to the position, orientation, and tilt of the virtual camera 620B that changes according to the operation is displayed on the touch screen 15 of the user terminal 100 has been described.

However, the space in which the avatar object 610 or the like is operated is not limited to the virtual space and may be the real space. The physical space may be, for example, a space specified by the acquired image captured by the camera (imaging unit) 17. Specifically, for example, when switched to the AR viewing mode, in the user terminal 100, a flat surface portion such as a floor surface is specified by analyzing the acquired image captured by the camera 17, and for example, the avatar object 610. The virtual space for AR in which at least is arranged is arranged at a predetermined position of the plane portion (for example, the center position of the plane portion). Further, the virtual camera 620B in this case is arranged at a position where the AR virtual space is captured from the same direction as the shooting direction of the camera 17 when the captured image is shot. As a result, an image in the augmented reality space in which the view image from the virtual camera 620B is superimposed on the acquired image from the camera 17 is displayed on the display unit 152. Further, the user terminal 100 detects the actual position, orientation, inclination, etc. of the camera 17, and changes the position, orientation, inclination, etc. of the virtual camera 620B according to the position, orientation, inclination, etc. Thereby, the acquired image can be changed according to the change of the position, orientation, inclination, etc. of the camera 17.

In the user terminal 100 in which the AR viewing mode is selected, the avatar object 610 and the like operate in the augmented reality space in which objects including the avatar object 610 are arranged in the acquired image acquired from the camera 17. As a result, the user can have an impression (realism) as if the avatar object 610 or the like is actually operating in the real space in front of the user.

FIG. 30 shows an example in which a character imitating a woman wearing a hat (hereinafter, referred to as a 1'character) appears in the virtual space 600B. The first' character is associated with a decorative object such as a dress worn over the whole body of the first' character, a hat worn on the head, and earrings attached to the ears. In FIG. 30, the piercing of the decoration objects associated with the first' character corresponds to the first object. For this reason, the display mode of the piercing is set to a different display mode for each distribution in step S83 of FIG. It should be noted that the second distribution content that is periodically distributed includes, for example, character content in which the first' character in which the display mode of the first object is changed appears.

First, the normal viewing mode will be described with reference to FIG. FIG. 30A-1 is a display example when the first' character faces the front and holds the index finger of the right hand in front. FIG. 30(A-2) is a display example when the first' character is facing diagonally right to the front and is pointing forward with the index finger of the right hand. In the normal viewing mode, the position, orientation, and tilt of virtual camera 620B are changed according to the swipe operation. However, the pierce is covered with a hat corresponding to the third object, and no matter how the swipe operation is performed, the position where the pierce can be seen (for example, FIG. 30(B-3) or FIG. The virtual camera 320B cannot be moved to a position where -4) or the like can be displayed). In the normal viewing mode, as shown in FIG. 30A, an AR mode transition icon 182 for switching to the AR viewing mode is normally displayed. It is possible to switch to the AR viewing mode by detecting a touch operation or the like on the AR mode transition icon 182 in the normal viewing mode.

The AR viewing mode will be described with reference to FIG. 30(B-1) and 30(B-2) show display examples in the AR viewing mode at the same timings as in FIGS. 30(A-1) and 30(A-2), respectively. In the AR viewing mode, the image from the virtual camera 620B is displayed on the display unit 152 among the images in the augmented reality space in which the first' character is arranged with respect to the acquired image captured by the camera 17. Is displayed. The 1'th character operates based on the operation instruction data distributed from the game play terminal 300. Further, the position, orientation, tilt, etc. of the virtual camera 620B are changed according to the actual position, orientation, tilt, etc. of the camera 17. In the AR viewing mode, the AR mode transition icon 182 is displayed in a highlighted manner (black-and-white inverted display in FIG. 30B) to notify that the AR viewing mode is in progress.

Therefore, if the position, orientation, etc. of the camera 17 are adjusted so that the left ear is looked up from the diagonally lower left side of the first' character, the virtual camera 320B moves to a position where the piercing can be seen, and the image of the piercing is displayed. It is displayed on the display unit 152 (see (B-3) in FIG. 30B). That is, the display mode of the piercing can be confirmed by selecting the AR viewing mode and adjusting the position/direction of the camera 17 and the like. As a result, the user can be motivated to view in the AR viewing mode, and the taste can be improved. If the position, orientation, etc. of the camera 17 are adjusted so as to look up at the left half of the first' character from the feet of the first' character, as shown in (B-4) of FIG. 30(B). , An image of the left half of the body including the legs is displayed on the display unit 152.

As described above, in the AR viewing mode, by adjusting the position/direction of the camera 17 and the like, a position where the piercing, which is the first object associated with the first′ character, can be displayed (for example, the first′ character). The virtual camera 620B can be moved to a position where the virtual camera 620B looks up from the neck of the character, a position where the character looks up from the foot of the 1'character, or the like. As a result, the piercing that is the first object can be displayed, and the user can confirm the display mode of the piercing. On the other hand, in the normal viewing mode, as described above, no matter how the operation is performed, the position where the piercing, which is the first object associated with the first' character, can be displayed (for example, the first' The virtual camera 620B cannot be moved to a position where the virtual camera 620B looks up from the character's neck, a position where it looks up from the foot of the 1'character, or the like. As a result, the piercing that is the first object cannot be displayed, and the user cannot confirm the display mode of the piercing.

In the AR viewing mode, it is possible to switch to the normal viewing mode by detecting a touch operation or the like on the AR mode transition icon 182 displayed in the highlighted mode. In the AR viewing mode, the AR mode transition icon 182 is displayed in a highlighted manner, the AR mode reset icon is normally displayed, and the AR mode is reset in response to the touch on the AR mode reset icon ( You may make it restart). When the AR mode is reset, the flat surface portion such as the floor surface is specified again, and the view field image in the AR virtual space 600C is arranged at a predetermined position of the flat surface portion.

In addition, when the tap operation is performed on the AR mode transition icon 182 displayed in the emphasized state in the state where the first object is displayed in the AR viewing mode, the viewpoint of the virtual camera 620B is predetermined. To the existing viewpoint (for example, the viewpoint that captures the virtual space 600B from the front), or is changed to a viewpoint that minimizes the movement amount of the virtual camera 620B within the movable range of the virtual camera 620B in the normal mode. It As a result, the deviation of the viewpoint before and after the mode switching can be reduced as much as possible. Further, in the state where the first object is displayed in the AR viewing mode (FIG. 30(B-3), etc.), the AR mode transition icon 182 may be temporarily hidden.

Also, in the AR viewing mode, the virtual camera 620B is controlled in synchronization with changes in the position, orientation, tilt, etc. of the camera 17. However, the virtual camera 620B may control the position, orientation, tilt, and the like according to an input operation (for example, a swipe operation, a touch operation, etc.) from the user. Thereby, the visual field of the virtual camera 620B can be changed according to the input operation from the user, and as a result, the visual field image arranged on the acquired image can be changed. Specifically, a swipe operation can rotate or move an avatar object or the like (for example, only the avatar object 610, the avatar object 610 and other objects, the entire virtual space) arranged on the acquired image. Thereby, for example, even when the avatar object 610 or the like is operated by generating the augmented reality space in a narrow space where the position, orientation, etc. of the user terminal 100 cannot be sufficiently adjusted, input to the touch screen 15 is performed. It is possible to rotate/move the avatar object 610 or the like by an operation.

Further, as the viewing mode, a TV mode is provided in which the position, orientation, tilt, etc. of the virtual camera 620B are changed based on the operation of the switcher on the operator side of the system 1, and the TV mode is switched to the TV mode according to the user's operation. You may make it possible. However, in the TV mode, the virtual camera 320B cannot be moved to a position where the first object can be seen, that is, the first object cannot be displayed on the display unit 152.

(2) In the above embodiment, the video in which the avatar object 610 is operating is reproduced by the real-time rendering method. However, the video may be reproduced by a moving image distribution system. In this case, for the first character content, the first object whose display mode is changed according to a predetermined rule is associated with the first character in the game play terminal 300, and the first character moves in the virtual space. Video data including an image to be displayed is distributed from the game play terminal 300 as the first display data. On the touch screen 15 of the user terminal 100, an image in which the first character moves in the virtual space is displayed based on the moving image data.

(3) The display mode of the costume or the like of the avatar object 610 in the above-described embodiment is uniformly set according to the type of the avatar object 610. However, the display mode of the costume or the like of the avatar object 610 may be different for each user depending on, for example, the degree of progress of the game (for example, every time the game is won or each time an item is inserted). For example, even if the game progress information transmitted from the game play terminal 300 side is the same, the costume of the avatar object 610 may be the costume according to the degree of progress of the game. In this case, a plurality of types of costume data of the avatar object 610 may be stored in advance on each user terminal 100 side. The target for changing the display mode of the avatar object 610 is not limited to the costume, but may be the hairstyle, the skin color, the degree of makeup, or the like. Accordingly, the variation of the avatar object 610 can be increased without increasing the processing load on the game play terminal 300 side that outputs the game progress information, and thus the user can be attracted.

Instead of or in addition to this, the display mode of the avatar object 610 may be made different from the display mode displayed on the user terminal 100 of another user according to the charge by the user during the game. .. Further, the display mode of the avatar object 610 may be different according to the ranking of the user, which is updated according to the result of the game. Further, the user terminals 100 that can receive the first character content may be limited to some of the user terminals 100, such as the user terminals 100 of users who are charged a lot during the game.

(4) In the above embodiment, the color of the display mode of the first object (shirt) is changed by the process of step S83. However, in step S83, the shape of the first object may be changed instead of or together with the color. For example, the shirt may be changed from a V-neck to a round neck, or the piercing may be changed from a star-shaped one to a ring-shaped one. Further, in step S83, instead of or in addition to the one for determining the color, shape, type, etc., the display (for example, wear)/non-display (not wear) of the first object itself is determined. Good.

(5) In the above embodiment, one of the decorative objects associated with the first character (eg, shirt, piercing) is set as the first object, and the display mode of the first object is set. I am trying to change it. However, the first object is not limited to a shirt or piercing as long as it is an object associated with the first character, and may be an object such as socks, pants, hair clip, gloves, hat, and underwear. Further, the object is not limited to the decorative object, and may be an object (for example, a hairstyle, nail, etc.) related to the first character itself. Furthermore, the display mode of the first object associated with the first character may be changed, and the display mode of the background object (eg, a wreath object) of the first character may be changed.

(6) In the above-described embodiment, the display mode of the decoration objects other than the first object among the decoration objects associated with the first character is fixed in advance. However, the display mode of the decoration object other than the first object may be changed, for example, every predetermined number of times (five times), every season, every time period, or the like, instead of every delivery. Similarly, in the above embodiment, the display mode of the decoration object associated with the second character is fixed in advance, but this display mode is also distributed every predetermined number of times (5 times) or the season. It may be changed every time or every time period.

(7) In the above embodiment, the display mode of the first object associated with the character (the first character, the first' character, or the like) that appears in the second distribution content that is regularly distributed is distributed. The example changed every time was explained. However, the display mode of the first object associated with the character appearing in the first distribution content that is distributed irregularly may be changed to a display mode different from that at the time of the previous distribution.

<Appendix>
The matters described in each of the above embodiments will be additionally described below.

(Appendix 1):
According to an aspect of one embodiment shown in the present disclosure, a program executed in a terminal device (user terminal 100) including a processor, wherein the processor causes the first character ( A step (S26) of receiving first display data (motion instruction data) for enabling display of an image of a virtual space including an avatar object 610), and the virtual display based on the first display data. A step (S29) of displaying an image of the action of the first character in the space is executed, and the first character is associated with a predetermined first object, and the first character is associated with the first character. The display mode (color) of the object is changed according to a predetermined rule.

(Appendix 2):
In (Supplementary Note 1), the processor receives second display data for enabling display of a video in a virtual space including a second character (another avatar object 610) that moves in synchronization with a performer's motion. The step of performing (S26) and the step of displaying an image of the second character moving in the virtual space based on the second display data (S29). Is associated with a shirt of a second object of the same type as the first object, and the display mode (color) of the second object is fixed.

(Appendix 3):
In (Supplementary Note 1) or (Supplementary Note 2), the display mode of the first object is changed for each distribution period to a display mode different from the display mode in the previous distribution period.

(Appendix 4):
In any one of (Supplementary Note 1) to (Supplementary Note 3), the first display data is data distributed from the outside, and includes the motion data corresponding to the motion input by the performer, and the displaying step is performed. Causes the first character to move in real time in the virtual space based on the motion data, and displays an image of the movement of the first character in the virtual space.

(Appendix 5):
(Supplementary Note 4), in the processor, a step of receiving first data (display mode information) for specifying a display mode of the first object changed by the predetermined rule (S92); The step (S93) of setting the display mode of the first object based on the first data is executed.

(Appendix 6):
In (Supplementary Note 4) or (Supplementary Note 5), the processor is set to one of a plurality of types of viewing modes (normal viewing mode, AR viewing mode) in which at least part of the aspect for specifying the viewpoint is different. And executing the step of displaying the image of the virtual space viewed from a specified viewpoint, when the first viewing mode (AR viewing mode) of the plurality of viewing modes is displayed. Is capable of moving the viewpoint to the position where the first object is displayed. In a mode other than the first viewing mode, the viewpoint cannot be moved to the position where the first object is displayed. is there.

(Appendix 7):
In (Supplementary Note 6), the terminal device includes a detection unit that detects the position and orientation of the terminal device, and a touch screen, and the first viewing mode is the position and orientation detected by the detection unit. It is a viewing mode in which a viewpoint can be specified based on the plurality of types of viewing modes, and the plurality of types of viewing modes include a second viewing mode (normal viewing mode) that identifies a viewpoint based on a touch operation on the touch screen. In the case of the second viewing mode of the viewing modes, the viewpoint cannot be moved to the position where the first object is displayed.

(Appendix 8):
In any one of (Supplementary Note 1) to (Supplementary Note 3), the first display data is data distributed from the outside and is associated with a first object whose display mode is changed by the predetermined rule. The displayed first character is moving image data including a moving image in the virtual space, and the displaying step displays a moving image of the first character in the virtual space based on the moving image data. indicate.

(Appendix 9):
In any one of (Supplementary Note 1) to (Supplementary Note 8), a predetermined third object is associated with the first character, and the third object is at least the first object. An object that covers a part.

(Appendix 10):
According to an aspect of an embodiment shown in the present disclosure, there is provided a method executed in a terminal device including a processor, which is capable of displaying a video in a virtual space including a first character that moves in association with a motion of a performer. Receiving a first display data for performing a display, and displaying a video image of the first character moving in the virtual space based on the first display data, A predetermined first object is associated with the first character, and the display mode of the first object is changed according to a predetermined rule.

(Appendix 11):
According to an aspect of an embodiment shown in the present disclosure, a terminal device including a processor for displaying a virtual space image including a first character that moves in synchronization with a motion of a performer is provided. Of the first display data is received, and an image of the movement of the first character in the virtual space is displayed based on the first display data. The first character has a predetermined first Objects are associated with each other, and the display mode of the first object is changed according to a predetermined rule.

[Example of software implementation]
The control blocks (particularly the control units 110, 210, 310, 410) of the user terminal 100, the server 200, the game play terminal 300 (HMD set 1000), and the distribution terminal 400 are logics formed in an integrated circuit (IC chip) or the like. It may be realized by a circuit (hardware) or software.

In the latter case, the user terminal 100, the server 200, the game play terminal 300 (HMD set 1000), and the distribution terminal 400 are equipped with a computer that executes the instructions of a program that is software that realizes each function. The computer includes, for example, one or more processors and a computer-readable recording medium that stores the program. Then, in the computer, the processor reads the program from the recording medium and executes the program to achieve the object of the present invention. As the processor, for example, a CPU (Central Processing Unit) can be used. As the recording medium, a "non-transitory tangible medium" such as a ROM (Read Only Memory), a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. Further, a RAM (Random Access Memory) for expanding the program may be further provided. The program may be supplied to the computer via any transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. Note that one aspect of the present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

The present invention is not limited to the above-described embodiments, but various modifications can be made within the scope of the claims, and embodiments obtained by appropriately combining the technical means disclosed in the different embodiments Is also included in the technical scope of the present invention.

1 system, 2 network, 3, 3A, 3B user (first user), 4 player (actor), 10, 20, 30, 40 processor, 11, 21, 31, 41 memory, 12, 22, 32, 42 storage , 13, 23, 33, 43 Communication IF, 14, 24, 34, 44 Input/output IF, 15, 45 Touch screen, 17 camera, 18 Distance measuring sensor, 51 Monitor, 52 Gaze sensor, 53 First camera, 54th 2 cameras, 55 microphones, 56 speakers, 100, 100A, 100B, 100C user terminals (computer, first computer, first information processing device), 110, 210, 310, 410 control unit (first control unit, second control) Section), 111, 311, 413 operation acceptance section, 112, 312, 412 display control section, 113, 313 UI control section, 114, 314 animation generation section, 115, 315 game progression section, 116, 316 virtual space control section, 117 video reproducing section, 120, 220, 320, 420 storage section (first storage section, second storage section), 131, 231, 331 game program (program, first program), 132, 232, 332 game information, 133 , 233, 333 user information, 151, 451 input unit, 152, 452 display unit (display), 200 server, 211 communication mediation unit, 212 log generation unit, 213 list generation unit, 234, 421 user list, 300 game play terminal (External device, second external device), 317 reaction processing unit, 400 distribution terminal (external, first external device, computer, second information processing device), 411 communication control unit, 414 voice reception unit, 415 motion identification unit, 416 Operation instruction data generation unit, 422 Motion list, 423 Distribution program (program, second program), 540, 1020, 1021 controller, 500 HMD, 510 HMD sensor, 520 motion sensor, 530 display, 600A, 600B virtual space, 610 Avatar object (character), 620A, 620B virtual camera, 631, 632, 633, 634 object, 640A, 640B view area, 650, 660 view image, 671 enemy object, 672, 673 obstacle object, 674 staging object, 691, 692 utterances, 701, 702, 703A, 70B, 704 A, 704B, 705, 706, 711, 711A, 711B, 711C, 711D, 722, 723, 745, 745A, 745B, 745C, 752, 762, 763, 930, 2011, 2022, 2031, 2032, 2033, 2034, 2037, 2038, 2051, 2063, 2072, 2073, 2075 UI image (message UI, UI), 721 download screen, 731 user list screen (list), 732, 732A, 732B, 732C, 742, 742A, 742B, 742C record Image, 733, 733A, 733B, 733C User name, 734, 734A, 734B, 734C Tag information, 735, 735A, 735B, 735C icon, 741 motion list screen (option), 743, 743A, 743B, 743C Motion name, 744 , 744A, 744B, 744C, 753 motion image, 751 delivery screen, 761 delivery completion screen, 810A, 810B motion video, 820A, 820B uttered voice, 910A, 910B video, 920A, 920B voice, 1000 HMD set, 1010 object, 1030 Storage medium, 2010 home screen, 2020 ranking screen, 2021 title image, 2026, 2026A, 2026B ranking, 2027, 2027A, 2027B billing amount, 2028, 2028A, 2028B transmission count, 2029 sent notification, 2030 last transmission date, 2035 Details Display area, 2036 scroll bar, 2040 detailed screen, 2050, 2060 preparation screen, 2052 text, 2053, 2061 selection image, 2054A, 2054B, 2054C, 2062A, 2062B, 2062C, 2062D, 2062E, 2062F option, 2070 voice input screen, 2074 tag image

Claims (11)

A program executed in a terminal device including a processor,
In the processor,
Receiving first display data for enabling display of an image of a virtual space including a first character that moves in synchronization with the motion of the performer;
Displaying a video of the first character moving in the virtual space based on the first display data,
A predetermined first object is associated with the first character,
The display mode of the first object is changed regardless of whether there is a request from the user who owns the terminal device, and the display mode is changed to a display mode regardless of the progress of the user.
A program in which the user can specify the first object regardless of which display mode is changed. In the processor,
Receiving second display data for enabling display of an image of a virtual space including a second character that moves in synchronization with the motion of the performer;
Displaying a video of the second character moving in the virtual space based on the second display data,
A second object of the same type as the first object is associated with the second character,
The program according to claim 1, wherein a display mode of the second object is fixed. The program according to claim 1 or 2, wherein the display mode of the first object is changed for each distribution period to a display mode different from the display mode in the previous distribution period. The first display data is data distributed from the outside, and includes motion data corresponding to a motion input by the performer,
The step of displaying, moving the first character in real time in the virtual space based on the motion data, and displaying an image of the first character moving in the virtual space. The program according to claim 3. In the processor,
Receiving a first data for specifying a display mode of the first objects change,
The program according to claim 4, which causes a step of setting a display mode of the first object to be executed based on the first data. In the processor,
Performing a step of setting one of a plurality of types of viewing modes in which at least part of the aspect for specifying the viewpoint is different,
The step of displaying displays an image of the virtual space viewed from a specified viewpoint,
In the case of the first viewing mode of the plurality of types of viewing modes, the viewpoint can be moved to the position where the first object is displayed, and the viewing mode is other than the first viewing mode. The program according to claim 4 or 5, wherein the viewpoint cannot be moved to a position where the first object is displayed. The terminal device includes a detection unit that detects the position and orientation of the terminal device, and a touch screen,
The first viewing mode is a viewing mode in which a viewpoint can be specified based on the position and orientation detected by the detection unit,
The plurality of types of viewing modes include a second viewing mode in which a viewpoint is specified based on a touch operation on the touch screen,
The program according to claim 6, wherein the viewpoint cannot be moved to a position where the first object is displayed in the second viewing mode of the plurality of viewing modes. It said first display data is data to be distributed from the outside, including a picture first character first object table示態recommendation has been changed is associated to operate in the virtual space Video data,
The program according to any one of claims 1 to 3, wherein the displaying step displays an image of the first character moving in the virtual space based on the moving image data. A predetermined third object is associated with the first character,
The program according to any one of claims 1 to 8, wherein the third object is an object that covers at least a part of the first object. A method executed in a terminal device comprising a processor, comprising:
Receiving first display data for enabling display of an image of a virtual space including a first character that moves in synchronization with the motion of the performer;
Displaying an image of the first character moving in the virtual space based on the first display data.
A predetermined first object is associated with the first character,
The display mode of the first object is changed regardless of whether there is a request from the user who owns the terminal device, and the display mode is changed to a display mode regardless of the progress of the user.
The method in which the first object can be identified by the user regardless of which display mode is changed. A terminal device including a processor,
Receiving first display data for enabling display of an image of a virtual space including a first character that moves in synchronization with the motion of the performer,
An image of the first character moving in the virtual space based on the first display data,
A predetermined first object is associated with the first character,
The display mode of the first object is changed regardless of whether there is a request from the user who owns the terminal device, and the display mode is changed to a display mode regardless of the progress of the user.
The terminal device is such that the user can specify the first object regardless of which display mode is changed.