JP7686847B1 - VIRTUAL SPACE CONTENT DELIVERY SYSTEM, VIRTUAL SPACE CONTENT DELIVERY PROGRAM, AND VIRTUAL SPACE CONTENT DELIVERY METHOD - Google Patents

Abstract

[Problem] To provide a virtual space content distribution system, a virtual space content distribution program, and a virtual space content distribution method that can prevent a decline in interest in virtual space content by reducing the processing load for displaying and outputting viewer avatars in order to suppress the occurrence of display output problems.
[Solution] The appearance of a second viewer avatar included in content when the content of a performance by a first performer avatar is output on a viewer user terminal is different from the appearance of a second viewer avatar included in content when the content of a performance by a second performer avatar is output on a viewer user terminal.
[Selection] Figure 56

Description

The present invention relates to a virtual space content distribution system, a virtual space content distribution program, and a virtual space content distribution method capable of distributing virtual space content including at least the content of a performance performed by a performer avatar in a virtual space.

Conventionally, there is known a system that connects multiple viewer terminals owned by viewers (viewer users) to a server computer capable of executing processes related to virtual space content and processes related to the distribution of virtual space content via a communications network, and allows virtual characters (viewer avatars) representing viewers to participate in a virtual space in which content is distributed by a distributor's virtual character (performer avatar) (see, for example, Patent Document 1).

JP 2019-186797 A

In Patent Document 1, a viewer avatar corresponding to each viewer user can participate in the virtual space, and so the viewer avatars participating in these virtual spaces must also be displayed and output as avatars participating in the virtual space of the virtual space content. However, if the number of viewer avatars participating in the virtual space increases depending on the progress of these virtual space contents, the processing load related to the display and output of the virtual space content increases, and problems may occur in the display and output of the virtual space content. Therefore, in order to prevent such problems from occurring, it is conceivable to change the mode of the viewer avatar (mode of the first viewer avatar) to a mode (mode of the second viewer avatar) that requires a smaller processing load for display and output.

In addition, in Patent Document 1, it is conceivable that multiple performer avatars (first performer avatar, second performer avatar) can participate in the virtual space, and virtual space content can be distributed to each performer avatar.

However, while it is possible to change the appearance of the viewer avatar to that of the second viewer avatar in order to reduce the processing load associated with the display output of virtual space content, if virtual space content can be delivered for each performer avatar, and the appearance of the second viewer avatar is uniformly the same when the virtual space content of the first performer avatar is delivered and when the virtual space content of the second performer avatar is delivered, there is a risk that the viewer will not be able to feel immersed in the virtual space content of each performer avatar, and interest in the virtual space content will decrease.

The present invention was made in response to these problems, and aims to provide a virtual space content distribution system, a virtual space content distribution program, and a virtual space content distribution method that can prevent a decline in interest in virtual space content by reducing the processing load for displaying and outputting viewer avatars in order to reduce the occurrence of display output problems.

The virtual space content delivery system according to claim 1 comprises:
A virtual space content distribution system capable of distributing virtual space content including at least content of a performance (e.g., a virtual live performance) performed by a performer avatar in a virtual space,
A viewer user terminal (e.g., viewer terminal 300) capable of outputting an image of the virtual space content and allowing a viewer user to view the image;
A server computer (e.g., a delivery server computer 100) that is connected to the viewer user terminal via a communication network and is capable of executing at least processes related to the virtual space and processes related to the delivery of the virtual space content;
At least
The viewer users are multiple, and the viewer user terminal is assigned to each viewer user;
The virtual space content may include a viewer avatar by allowing the viewer user to participate in the virtual space that is the subject of the virtual space content using the viewer user terminal (for example, as shown in FIG. 13, a portion of the viewer terminal 300 in which the viewer avatar of each viewer is displayed as if virtually participating in a virtual live venue),
the viewer avatars include a first viewer avatar (e.g., a full avatar) and a second viewer avatar (e.g., a simplified avatar) that requires a smaller processing load for displaying than the first viewer avatar;
The viewer user terminal is capable of outputting, as the virtual space content, content of a performance by a first performer avatar (e.g., a first performance by a performer avatar of performer A) and content of a performance by a second performer avatar different from the first performer avatar (e.g., a second performance by a performer avatar of performer B) (e.g., a portion displaying an image generated by the viewer terminal 300 executing a viewer viewpoint image generation process),
The appearance of the second viewer avatar corresponding to the first performer, which is the second viewer avatar included in the content when the content of the performance by the first performer avatar is being output on the viewer user terminal, is different from the appearance of the second viewer avatar corresponding to the second performer, which is the second viewer avatar included in the content when the content of the performance by the second performer avatar is being output on the viewer user terminal (for example, as shown in FIG. 56 , a simplified avatar of appearance α is displayed as the viewer avatar during the first performance, and a simplified avatar of appearance β is displayed as the viewer avatar during the second performance).
It is characterized by the following.
According to this feature, when the viewer avatar is set to the second viewer avatar which has a small display processing load, the appearance of the second viewer avatar differs depending on the performer avatar performing the performance, thereby preventing a decrease in interest in the virtual space content due to the appearance of the second viewer avatar being common regardless of the performer.

The virtual space content delivery system of claim 2 is the virtual space content delivery system according to claim 1,
The appearance of the first viewer avatar is common when the virtual space content output on the viewer user terminal is content of a performance by the first performer avatar and when it is content of a performance by the second performer avatar (for example, as shown in variant example 4-7, in the case where viewer avatars are displayed as full avatars during part of the periods of "First Performance" and "Second Performance", the appearance of the full avatar during the periods of "First Performance" and "Second Performance" is common for a specific viewer avatar).
It is characterized by the following.
According to this feature, the appearance of the first viewer avatar, which requires a large processing load to display, is common to each performer, thereby preventing an increase in the processing load and an increase in the amount of display data.

The virtual space content delivery system of claim 3 is the virtual space content delivery system according to claim 1 or 2,
The viewer user terminal is capable of outputting, as the virtual space content, collaboration content in which the first performer avatar and the second performer avatar simultaneously perform a performance in the same virtual space,
When the collaboration content is output, the viewer user can select on the viewer user terminal whether the appearance of the second viewer avatar should be that of the first performer avatar or the second performer avatar (for example, as shown in FIG. 66, when a joint performance is performed between the performer avatar of performer A and the performer avatar of performer B, the viewer user can select the appearance of a simplified avatar on the viewer terminal 300).
It is characterized by the following.
This feature can increase the interest of the collaborative content.

A virtual space content delivery system according to claim 4 is the virtual space content delivery system according to claim 1 or 2,
The viewer user terminal is capable of outputting, as the virtual space content, collaboration content in which the first performer avatar and the second performer avatar simultaneously perform a performance in the same virtual space,
The state of the second viewer avatar when the virtual space content output on the viewer user terminal is the collaboration content is a special state different from the state when the performance of the first performer avatar is output and the state when the performance of the second performer avatar is output (for example, as shown in FIG. 56, when the performer avatar of performer A and the performer avatar of performer B are performing a collaborative performance, a simplified avatar of state γ is displayed as the viewer avatar).
It is characterized by the following.
This feature can increase the interest of the collaborative content.

A virtual space content delivery system according to claim 5 is the virtual space content delivery system according to claim 1,
the virtual space content includes a first content portion of a performance by the first performer avatar and a second content portion of a performance by the second performer avatar;
The appearance of the second viewer avatar when the first content portion is output on the viewer user terminal corresponds to the first performer avatar, and the appearance of the second viewer avatar when the second content portion is output on the viewer user terminal corresponds to the second performer avatar (for example, as shown in FIG. 67, during a performance by performer A's performer avatar, a heart-shaped simplified avatar corresponding to the performer avatar of performer A is displayed, and during a performance by performer B's performer avatar, a star-shaped simplified avatar corresponding to the performer avatar of performer B is displayed).
It is characterized by the following.
This feature makes it possible to prevent a decline in interest in virtual space content that partially includes performances by multiple performer avatars.

A virtual space content delivery system according to claim 6 is the virtual space content delivery system according to claim 1,
The viewer avatar is changeable from the first viewer avatar to the second viewer avatar,
The timing at which the multiple viewer avatars participating in the virtual space change to the second viewer avatar is the same (for example, as shown in FIG. 55, all viewer avatars participating in a virtual live show change to simple avatars from the start of the “first MC”).
It is characterized by the following.
According to this feature, it is possible to prevent a sense of inequality that would be caused to viewer users due to differences in the timing of changing into the second viewer avatar.

The virtual space content delivery system of claim 7 is the virtual space content delivery system according to claim 6,
When the viewer avatar changes to the second viewer avatar, the performer avatar does not change to a simplified performer avatar that requires a small processing load for display (for example, as shown in FIG. 56, a simplified performer avatar is not provided as the performer avatar, so when the viewer avatar changes to the simplified avatar, the performer avatar does not change to the simplified avatar).
It is characterized by the following.
This feature makes it possible to prevent the virtual space content from losing its appeal.

The virtual space content delivery system of claim 8 is the virtual space content delivery system according to claim 1,
the state of the second viewer avatar when the content of the program by the first performer avatar is outputted in the viewer user terminal includes an element corresponding to an attribute of the first performer avatar;
The appearance of the second viewer avatar when the content of the performance by the second performer avatar is output on the viewer user terminal includes elements corresponding to the attributes of the second performer avatar (for example, as shown in FIG. 67, when performer A's performer avatar is performing, a heart-shaped simplified avatar having the same shape as the hair accessory of the performer avatar of the performer A is displayed as the viewer avatar, and when performer B's performer avatar is performing, a star-shaped simplified avatar having the same shape as the hair accessory of the performer B's performer avatar is displayed as the viewer avatar).
It is characterized by the following.
This feature makes it possible to prevent the virtual space content from losing its appeal.

A virtual space content delivery system according to claim 9 is the virtual space content delivery system according to claim 1,
In a case where the first performer avatar performs a performance in a first virtual space and the second performer avatar performs a performance in a second virtual space different from the first virtual space,
The appearance of the second viewer avatar participating in the first virtual space is different from the appearance of the second viewer avatar participating in the second virtual space (for example, as shown in FIG. 68, when a performance of a performer avatar of performer A is performed in a first virtual live venue and a performance of a performer avatar of performer B is performed in a second virtual live venue, the appearance of a simplified avatar as a viewer avatar participating in the first virtual live venue is different from the appearance of a simplified avatar as a viewer avatar participating in the second virtual live venue).
It is characterized by the following.
This feature makes it possible to prevent the virtual space content from losing its appeal.

The virtual space content delivery program according to claim 10,
A virtual space content distribution program for distributing virtual space content including at least content of a performance (e.g., a virtual live performance) performed by a performer avatar in a virtual space,
The virtual space content includes:
A content including a video that can be viewed by a viewer user at a viewer user terminal (e.g., viewer terminal 300) assigned to each of a plurality of viewer users,
The viewer may include a viewer avatar by being able to participate in the virtual space that is the target of the virtual space content by using the viewer user terminal connected via a communication network to a server computer (e.g., a distribution server computer 100) that is capable of executing at least processes related to the virtual space and processes related to the distribution of the virtual space content (e.g., as shown in FIG. 13, a portion of the viewer terminal 300 in which the viewer avatar of each viewer is displayed as if virtually participating in a virtual live venue),
the viewer avatars include a first viewer avatar (e.g., a full avatar) and a second viewer avatar (e.g., a simplified avatar) that requires a smaller processing load for displaying than the first viewer avatar;
a virtual space content output process for outputting, as the virtual space content, content of a performance by a first performer avatar (e.g., a first performance by a performer avatar of performer A) and content of a performance by a second performer avatar different from the first performer avatar (e.g., a second performance by a performer avatar of performer B) at the viewer user terminal (e.g., a portion for displaying an image generated by the viewer terminal 300 executing a viewer viewpoint image generation process);
a viewer avatar behavior process (e.g., a portion in which the viewer terminal 300 executes the viewer viewpoint video generation process shown in FIG. 59 ) that sets different behaviors between a first viewer avatar corresponding to a first performer, which is the second viewer avatar included in the content when the content of a performance by the first performer avatar is being output on the viewer user terminal, and a second viewer avatar corresponding to a second performer, which is the second viewer avatar included in the content when the content of a performance by the second performer avatar is being output on the viewer user terminal;
The present invention is characterized in that it includes:
According to this feature, when the viewer avatar is set to the second viewer avatar which has a small display processing load, the appearance of the second viewer avatar differs depending on the performer avatar performing the performance, thereby preventing a decrease in interest in the virtual space content due to the appearance of the second viewer avatar being common regardless of the performer.

The virtual space content delivery method according to claim 11,
A virtual space content distribution method for distributing virtual space content including at least content of a performance (e.g., a virtual live performance) performed by a performer avatar in a virtual space, comprising:
The virtual space content includes:
A content including a video that can be viewed by a viewer user at a viewer user terminal (e.g., viewer terminal 300) assigned to each of a plurality of viewer users,
The viewer may include a viewer avatar by being able to participate in the virtual space that is the target of the virtual space content by using the viewer user terminal connected via a communication network to a server computer (e.g., a distribution server computer 100) that is capable of executing at least processes related to the virtual space and processes related to the distribution of the virtual space content (e.g., as shown in FIG. 13, a portion of the viewer terminal 300 in which the viewer avatar of each viewer is displayed as if virtually participating in a virtual live venue),
the viewer avatars include a first viewer avatar (e.g., a full avatar) and a second viewer avatar (e.g., a simplified avatar) that requires a smaller processing load for displaying than the first viewer avatar;
a virtual space content output procedure (e.g., a portion in which the viewer terminal 300 displays an image generated by executing a viewer viewpoint image generation process) for outputting, as the virtual space content, content of a performance by a first performer avatar (e.g., a first performance by a performer avatar of performer A) and content of a performance by a second performer avatar different from the first performer avatar (e.g., a second performance by a performer avatar of performer B) at the viewer user terminal;
a viewer avatar appearance procedure (e.g., a portion in which the viewer terminal 300 executes the viewer viewpoint video generation process shown in FIG. 59 ) in which an appearance of a second viewer avatar corresponding to a first performer, which is the second viewer avatar included in the content when the content of a performance by the first performer avatar is outputted on the viewer user terminal, is different from an appearance of a second viewer avatar corresponding to a second performer, which is the second viewer avatar included in the content when the content of a performance by the second performer avatar is outputted on the viewer user terminal;
The present invention is characterized in that it includes:
According to this feature, when the viewer avatar is set to the second viewer avatar which has a small display processing load, the appearance of the second viewer avatar differs depending on the performer avatar performing the performance, thereby preventing a decrease in interest in the virtual space content due to the appearance of the second viewer avatar being common regardless of the performer.

The present invention may have only the invention-specific features described in the claims of the present invention, or may have the invention-specific features described in the claims of the present invention as well as features other than the invention-specific features.

1 is a block diagram showing an example of a system configuration of a virtual space content delivery system according to a first embodiment of the present invention. 1 is a diagram illustrating an example of the configuration of a delivery server computer in a virtual space content delivery system according to a first embodiment of the present invention. 3 is a diagram showing various data stored in a storage device of a delivery server computer according to the first embodiment of the present invention; FIG. 1 is a diagram illustrating an example of the configuration of a performer terminal in a virtual space content distribution system according to a first embodiment of the present invention. 2 is a diagram illustrating an example of the configuration of a viewer terminal in the virtual space content delivery system according to the first embodiment of the present invention. FIG. 2 is a diagram illustrating an example of the configuration of an administrator terminal in the virtual space content delivery system according to the first embodiment of the present invention. 3 is a diagram showing an example of the configuration of event data in the virtual space content delivery system according to the first embodiment of the present invention; FIG. A figure showing an example of the configuration of performer user data in the virtual space content distribution system of Example 1 of the present invention. 2 is a diagram showing an example of the configuration of viewer user data in the virtual space content delivery system according to the first embodiment of the present invention; FIG. 2 is an explanatory diagram showing data exchanged between each terminal and a server computer constituting the virtual space content distribution system according to the first embodiment of the present invention; FIG. 1 is a schematic diagram showing the facilities of a studio used by a performer user in a virtual space content delivery system according to a first embodiment of the present invention. FIG. FIG. 1 is a diagram showing a virtual live venue, which is a virtual space used in a virtual space content distribution system according to a first embodiment of the present invention. 1 is a diagram showing an example of a display on a viewer terminal constituting the virtual space content delivery system according to the first embodiment of the present invention; 1 is a diagram showing an example of a display on a viewer terminal constituting the virtual space content delivery system according to the first embodiment of the present invention; 1 is a diagram showing an example of a display on a performer terminal constituting the virtual space content distribution system of Example 1 of the present invention. FIG. 1A to 1C are diagrams illustrating examples of display by zooming operations on a viewer terminal constituting the virtual space content delivery system according to the first embodiment of the present invention. 1A to 1C are diagrams illustrating examples of display by zooming operations on a viewer terminal constituting the virtual space content delivery system according to the first embodiment of the present invention. 1 is a diagram showing an example of a display on a viewer terminal constituting the virtual space content delivery system according to the first embodiment of the present invention; A figure showing an example of the display of a flying performance period on a viewer terminal constituting the virtual space content distribution system of Example 1 of the present invention. 10 is a flow diagram showing an example of a viewer viewpoint video control process executed in a viewer terminal constituting the virtual space content delivery system according to the first embodiment of the present invention. FIG. FIG. 11 is an explanatory diagram showing the contents of a virtual live performance delivered by a virtual space content delivery system according to a second embodiment of the present invention. 11 is an explanatory diagram showing data exchanged between a delivery server computer and a viewer terminal constituting a virtual space content delivery system according to a second embodiment of the present invention; FIG. 13 is a diagram showing a participant user table used in a delivery server computer constituting a virtual space content delivery system according to a second embodiment of the present invention. FIG. 11 is an explanatory diagram showing data exchanged between a delivery server computer and a viewer terminal that constitute a virtual space content delivery system according to a second embodiment of the present invention. FIG. 11 is an explanatory diagram showing data exchanged between a delivery server computer and a viewer terminal constituting a virtual space content delivery system according to a second embodiment of the present invention; FIG. 11 is an explanatory diagram showing data exchanged between a delivery server computer and a viewer terminal constituting a virtual space content delivery system according to a second embodiment of the present invention; FIG. 11 is an explanatory diagram showing the relationship between the period type of a virtual live show and the entity that generates the virtual camera viewpoint video in a virtual space content distribution system according to a second embodiment of the present invention. FIG. 10 is an explanatory diagram showing the generation status of a virtual camera viewpoint image and a viewer viewpoint image in a distribution server computer and a viewer terminal that constitute a virtual space content distribution system of Example 2 of the present invention. FIG. FIG. 11 is a flow diagram showing a viewer viewpoint video generation process in a modified example of the second embodiment of the present invention. FIG. 13 is a diagram showing the contents of virtual space update data transmitted from a delivery server computer in a modified example of the second embodiment of the present invention. FIG. 11 is an explanatory diagram showing the contents of a virtual live performance delivered by a virtual space content delivery system according to a third embodiment of the present invention. A figure showing a virtual live venue, which is a virtual space used in a virtual space content distribution system of Example 3 of the present invention. A figure showing a virtual live venue, which is a virtual space used in a virtual space content distribution system of Example 3 of the present invention. A figure showing a virtual live venue, which is a virtual space used in a virtual space content distribution system of Example 3 of the present invention. A figure showing a virtual live venue, which is a virtual space used in a virtual space content distribution system of Example 3 of the present invention. FIG. 11 is a diagram showing an example of the configuration of time schedule data (TS data) stored in a delivery server computer constituting a virtual space content delivery system according to a third embodiment of the present invention. 13 is an explanatory diagram showing data exchanged between a delivery server computer and a viewer terminal that constitute a virtual space content delivery system according to a third embodiment of the present invention. FIG. 13 is an explanatory diagram showing data exchanged between a delivery server computer and a viewer terminal that constitute a virtual space content delivery system according to a third embodiment of the present invention. FIG. 13 is an explanatory diagram showing data exchanged between a delivery server computer and a viewer terminal that constitute a virtual space content delivery system according to a third embodiment of the present invention. FIG. FIG. 11 is an explanatory diagram showing a friend user data table stored in a viewer terminal constituting a virtual space content delivery system according to a third embodiment of the present invention. FIG. 11 is an explanatory diagram showing an example of an avatar simplification determination process in the virtual space content delivery system according to the third embodiment of the present invention. FIG. 11 is an explanatory diagram showing an example of a viewer viewpoint video generation process in a virtual space content delivery system according to a third embodiment of the present invention. 13 is an explanatory diagram showing an example of a virtual camera viewpoint video generation process (viewer terminal) in the virtual space content delivery system according to the third embodiment of the present invention. FIG. 1A is an explanatory diagram showing the appearance of each avatar generated in the viewer viewpoint video generation process in a virtual space content distribution system according to a third embodiment of the present invention, and FIG. 1B is an explanatory diagram showing the appearance of each avatar generated in the virtual camera line of sight video generation process (viewer terminal) in a virtual space content distribution system according to a third embodiment of the present invention. 1A is an explanatory diagram showing the appearance of a performer avatar in a virtual space content distribution system of Example 3 of the present invention, and FIG. 1B is an explanatory diagram showing the appearance of a viewer avatar in a virtual space content distribution system of Example 3 of the present invention. 13 is an explanatory diagram showing changes in the appearance of an avatar in a virtual space content delivery system according to a third embodiment of the present invention. FIG. 13 is an explanatory diagram showing changes in the appearance of an avatar in a virtual space content delivery system according to a third embodiment of the present invention. FIG. 13 is an explanatory diagram showing changes in the appearance of an avatar in a virtual space content delivery system according to a third embodiment of the present invention. FIG. 13 is an explanatory diagram showing changes in the appearance of a viewer avatar in a virtual space content delivery system according to a third embodiment of the present invention. FIG. 13 is an explanatory diagram showing changes in the appearance of a performer avatar in a modified example of Example 3 of the present invention. FIG. 13 is an explanatory diagram of a simplified avatar as a viewer avatar in a modified example of the third embodiment of the present invention. FIG. 13 is an explanatory diagram showing data exchanged between a delivery server computer and a viewer terminal that constitute a virtual space content delivery system in a modified example of the third embodiment of the present invention. FIG. 13 is an explanatory diagram illustrating an example of a pre-data transmission process in a modified example of the third embodiment of the present invention. FIG. 13 is an explanatory diagram of a simplified avatar as a viewer avatar in a modified example of the third embodiment of the present invention. An explanatory diagram showing the transition of the appearance of a viewer avatar in a virtual space content delivery system in Example 4 of the present invention. 13 is an explanatory diagram of performer avatars and viewer avatars in a virtual space content delivery system according to a fourth embodiment of the present invention. FIG. 13 is an explanatory diagram of a display screen of a viewer terminal when an item selection operation is performed in the virtual space content delivery system according to the fourth embodiment of the present invention. FIG. 13 is an explanatory diagram showing the interaction between a distribution server computer and a viewer terminal during a live virtual space participation period in a virtual space content distribution system according to a fourth embodiment of the present invention. 13 is a flowchart of a viewer viewpoint video generation process in a virtual space content delivery system according to a fourth embodiment of the present invention. FIG. 13 is an explanatory diagram comparing a full avatar and a simple avatar in a virtual space content delivery system according to a fourth embodiment of the present invention. An explanatory diagram of the colors of items corresponding to performer avatars in a virtual space content distribution system in variant example 4-1 of the present invention. FIG. 4 is an explanatory diagram of the operation of a chemilight in a virtual space content distribution system in a modified example 4-2 of the present invention. FIG. 4 is an explanatory diagram showing the interaction between a distribution server computer and a viewer terminal in a virtual space content distribution system in variant example 4-3 of the present invention. An explanatory diagram of a friend user data table in a virtual space content distribution system in variant example 4-3 of the present invention. An explanatory diagram of a full avatar and a simple avatar in a virtual space content distribution system in variant example 4-3 of the present invention. An explanatory diagram for determining the appearance of a simple avatar at the start of a collaborative program in a virtual space content distribution system in variant example 4-4 of the present invention. An explanatory diagram of a performer avatar and a simple avatar in a virtual space content distribution system in variant example 4-5 of the present invention. An explanatory diagram of a performer avatar and a simple avatar in a virtual space content distribution system in variant example 4-6 of the present invention. An explanatory diagram showing the transition of the appearance of a viewer avatar in a virtual space content delivery system in Example 5 of the present invention. FIG. 13 is an explanatory diagram of a viewer avatar in the virtual space content delivery system according to the fifth embodiment of the present invention. An explanatory diagram showing the interaction between a distribution server computer and a viewer terminal during a live virtual space participation period in a virtual space content distribution system in Example 5 of the present invention. FIG. 13 is an explanatory diagram of a participant user table in the virtual space content delivery system according to the fifth embodiment of the present invention. FIG. 13 is an explanatory diagram showing the interaction between a delivery server computer and a viewer terminal relating to a friend registration function in a virtual space content delivery system according to a fifth embodiment of the present invention. FIG. 13 is an explanatory diagram of a friend user data table in the virtual space content delivery system according to the fifth embodiment of the present invention. FIG. 13 is an explanatory diagram of a period during which a simplified target area can be designated in a virtual space content distribution system according to a fifth embodiment of the present invention. 13 is a flowchart showing a simplification target area designation operation process in the virtual space content delivery system according to the fifth embodiment of the present invention. FIG. 13 is an explanatory diagram showing the display mode of a viewer terminal when a simplified target area designation operation process is executed in a virtual space content distribution system according to a fifth embodiment of the present invention. 13 is a flowchart showing a viewer viewpoint video generation process in a virtual space content delivery system according to a fifth embodiment of the present invention. An explanatory diagram of the appearance of viewer avatars in areas subject to simplification and areas not subject to simplification in a virtual space content delivery system according to Example 5 of the present invention. An explanatory diagram showing the display mode of a viewer terminal in a virtual space content distribution system in variant example 5-1 of the present invention. An explanatory diagram showing the display mode of a viewer terminal in a virtual space content distribution system in variant example 5-2 of the present invention. FIG. 5 is an explanatory diagram of a case where the setting of the simplification target area is not completed during the period during which the simplification target area can be designated in a virtual space content distribution system in a variant example 5-3 of the present invention. An explanatory diagram showing the interaction between a distribution server computer and a viewer terminal during a live virtual space participation period in a virtual space content distribution system in variant example 5-4 of the present invention. An explanatory diagram of the number of simplification target areas for each viewer terminal specification in a virtual space content distribution system in variant example 5-4 of the present invention. An explanatory diagram of the appearance of viewer avatars in areas subject to simplification and areas not subject to simplification in a virtual space content distribution system in variant example 5-5 of the present invention. An explanatory diagram showing the display mode of a viewer terminal in a virtual space content distribution system in variant example 5-6 of the present invention. An explanatory diagram of a virtual live venue in a virtual space content distribution system in Example 6 of the present invention. 13 is a flowchart of a virtual space update process in a virtual space content delivery system according to a sixth embodiment of the present invention. FIG. 23 is an explanatory diagram of whether or not collision judgment is performed for each avatar in the virtual space content delivery system according to the sixth embodiment of the present invention. An explanatory diagram showing the interactions between the distribution server computer, viewer terminal, performer terminal, and administrator terminal during a live virtual space participation period in a virtual space content distribution system in variant example 6-1 of the present invention. 6 is a flowchart of a virtual space update process in a virtual space content delivery system in a modified example 6-1 of the present invention. An explanatory diagram of whether or not collision detection is performed between a viewer avatar and a transparent object in a virtual space content distribution system in variant example 6-1 of the present invention. An explanatory diagram showing the display mode of a viewer terminal in a virtual space content distribution system in variant example 6-2 of the present invention. An explanatory diagram of a viewer avatar in a virtual space content delivery system in variant example 6-3 of the present invention. An explanatory diagram of whether or not collision detection is performed between each avatar and a transparent object in a virtual space content distribution system in variant example 6-4 of the present invention. FIG. 23 is an explanatory diagram of a virtual space content delivery system in a virtual space content delivery system according to a seventh embodiment of the present invention. FIG. 23 is an explanatory diagram of the storage of a delivery server computer in the virtual space content delivery system according to the seventh embodiment of the present invention. FIG. 23 is an explanatory diagram showing the interaction between a delivery server computer and a viewer terminal in a virtual space content delivery system according to a seventh embodiment of the present invention. FIG. 23 is an explanatory diagram of a participant user table in the virtual space content distribution system according to the seventh embodiment of the present invention. 13 is a flowchart showing a process for determining a participating virtual live venue in a virtual space content distribution system according to a seventh embodiment of the present invention. An explanatory diagram showing the interactions between the viewer terminal of a viewer avatar participating in a sub virtual live venue, the distribution server computer, and the performer terminal in a virtual space content distribution system in Example 7 of the present invention. FIG. 13 is an explanatory diagram showing the interactions between viewer terminals, performer terminals, and a chat information server computer in the virtual space content distribution system according to the seventh embodiment of the present invention. 13 is an explanatory diagram showing the display mode of a performer terminal and a viewer terminal in a virtual space content distribution system according to Example 7 of the present invention. FIG. 23 is an explanatory diagram comparing information outputted at each terminal in the virtual space content delivery system according to the seventh embodiment of the present invention. An explanatory diagram comparing information outputted at each terminal in a virtual space content distribution system in variant example 7-1 of the present invention. FIG. 7 is an explanatory diagram of an image displayed on a virtual display in a virtual space content delivery system according to a modified example 7-2 of the present invention. An explanatory diagram showing the interaction between a distribution server computer and a viewer terminal during a live virtual space participation period in a virtual space content distribution system in variant example 7-3 of the present invention. An explanatory diagram showing the display mode of a viewer terminal in a virtual space content distribution system in variant example 7-3 of the present invention. An explanatory diagram regarding whether a viewer avatar can move to another virtual live venue in a virtual space content distribution system in variant example 7-3 of the present invention.

The following describes the form for implementing the virtual space content distribution system of the present invention based on the embodiment and the drawings. Note that the same reference numerals are used to designate identical or similar components in multiple drawings, and duplicated explanations will be omitted. The virtual space content distribution program of the present invention may be a program used by the entire system consisting of the server computer and each terminal described below that constitute the virtual space content distribution system, or it may be a program that operates only on a portion of the devices that constitute the virtual space content distribution system, such as the server computer or each terminal, and each process described below is executed by a program stored in the device on which the process is performed.

Figure 1 is a block diagram showing an example of the system configuration of a virtual space content distribution system in Example 1, which is an embodiment of the present invention. In the system of Example 1 shown in Figure 1, a virtual live performance in which a performer avatar 1 corresponding to a performer user performs a live event in a virtual live venue shown in Figure 12 is distributed as virtual space content. Note that in the following, distribution means transmission to multiple people, and is synonymous with transmission.

As shown in FIG. 1, the virtual space content distribution system of this embodiment 1 is mainly composed of a distribution server computer 100 capable of executing various processes related to the virtual live venue shown in FIG. 11, which is a virtual space, and various processes related to the distribution of the virtual live event, an administrator terminal 150 used by an event manager M of an event management organization that manages and operates the virtual live event to manage the distribution of the virtual live event, a performer terminal 200 used by a performer user, and a viewer terminal 300 that can be used by a viewer user who virtually participates in and watches the virtual live event, and the distribution server computer 100, the administrator terminal 150, the performer terminal 200, and the viewer terminal 300 are connected to each other so that data communication can be performed via the Internet network N, which is an open computer network.

In this embodiment 1, the administrator terminal 150 and the performer terminal 200 are also connected to the distribution server computer 100 via the Internet network N, but the present invention is not limited to this. For example, if the studio shown in FIG. 11 is a studio operated by an event management organization that provides studios to performer users, and the distribution server computer 100 is installed in these studios, the administrator terminal 150 and the performer terminal 200 may be locally connected to the distribution server computer 100 for data communication, or may be connected to the distribution server computer 100 for communication via a local area network LAN within the studio facility.

As shown in FIG. 1, there are multiple viewer users, such as viewer user A, viewer user B, viewer user C, etc., and viewer terminals 300 include viewer terminals 300a, 300b, 300c, etc. that can be used by each of viewer users A, B, C, etc. Although there are usually multiple viewer users, there may be one or four or more, and the number may be determined appropriately depending on the virtual space, such as a virtual live venue. Hereinafter, viewer user A, viewer user B, viewer user C, etc. may be collectively referred to as viewer users, and viewer terminals 300a, 300b, 300c, etc. may be collectively referred to as viewer terminals 300.

In addition, in this embodiment 1, an example is given in which there is one performer user, but the present invention is not limited to this, and the virtual live performance may be a joint event in which multiple performer users who are working individually virtually participate, or a group event in which a group of multiple people who are working together virtually participate. In this way, when multiple performers are appearing, performer terminal 200a, performer terminal 200b, performer terminal 200c, etc. may be provided for each performer user, or when a group of multiple people are performing, one performer terminal 200 may be used by multiple people.

In addition, in FIG. 1, the distribution server computer 100 is illustrated as a single unit, but the present invention is not limited to this. The distribution server computer 100 may be configured with multiple server computers, such as a first server computer that mainly performs virtual space processing and a second server computer that mainly performs distribution processing, or may be configured as a cloud server, etc., and the number of server computers may be changed appropriately depending on the scale of the event, the number of participating listener users, etc.

Similarly, while the administrator terminal 150 is illustrated as a single unit in FIG. 1, the present invention is not limited to this, and it goes without saying that there may be multiple administrator terminals 150 depending on the number of performer users and viewer users virtually participating in the live performance.

<Distribution server computer>
Fig. 2 is a diagram showing the configuration of the distribution server computer 100 used in this embodiment 1. As described above, the distribution server computer 100 used in this embodiment 1 is a normal server computer with relatively excellent processing power installed in a studio operated by an event management organization, and as shown in Fig. 2, has a processor (CPU) 101 connected to a data bus 108, a memory (RAM) 102, a storage 103 such as a hard disk, a communication interface (I/F) 104 capable of two-way communication by TCP/IP protocol via the Internet, which is an open computer network, and an input/output interface (I/F) 105 to which an input device such as a keyboard can be connected.

In the first embodiment, an input/output interface (I/F) 105 is provided to enable local operation, but if remote operation is possible via a communication interface (I/F) 104, the input/output interface (I/F) 105 may not be provided.

In addition to an operation system (server OS) for providing server functions (not shown), various data and programs are stored in the storage 103 of the distribution server computer 100 as shown in FIG. 3. Specifically, mainly stored are an event management program 110 for providing an event management function for managing live events, a virtual space generation program 120 for providing a virtual space for a virtual live venue, a performer user management program 130 for providing a performer user management function for managing performer users, and a viewer user management program 132 for providing a viewer user management function for managing viewer users.

The event management program 110 is a program for managing a virtual live event, which is an event held at a virtual live venue (see FIG. 12), in cooperation with the administrator terminal 150, and is capable of managing the progress of the virtual live event using event data 111 (see FIG. 7), time schedule data (TS data) 112, and music data 113 stored in the storage 103 together with the event management program 110.

An example of the event data 111 used in this embodiment 1 is shown in FIG. 7. The event data 111 stores information such as the event name, the file name of the time schedule (TS) data, a performer ID uniquely assigned to the performer user who is the performer, the file name of the music list, the file name of the participating viewer list in which the information of special viewer users who are permitted virtual participation is registered in the third area of the virtual audience area described below, and the file name of the participating viewer list in which the information of viewer users who have reserved virtual participation is registered in the fourth area of the virtual audience area described below, in association with the event ID uniquely assigned to the event.

The time schedule data (TS data) 112 is data that describes the order of progression of the virtual live event to be held and the programs such as songs in chronological order, and the virtual live event is managed so that it progresses based on the time schedule described in the time schedule data (TS data) 112. Furthermore, the time schedule data (TS data) 112 and event data 111 can be changed, updated, or added by the event manager M operating the manager terminal 150.

The song data 113 is data on the performance parts (so-called karaoke) of each song sung by the performer user, and is played by the audio control program 124 (described later) based on the time schedule data (TS data) 112 by the event management program 110, and is transmitted to the performer terminal 200 and the viewer terminal 300 (see FIG. 11).

The virtual space generation program 120 is a program for providing a virtual live venue, which is the virtual space shown in FIG. 12, and has the function of providing a virtual live venue in which performer avatars and viewer avatars are virtually participating, using virtual space data 121 describing the structure of the virtual live venue, which is stored in storage 103 together with the virtual space generation program 120, object data 122 for placing various objects (including virtual placement object 43, which is a star decoration on the stage, and virtual aerial objects 50-52, which are star objects in the sky) to be placed in the virtual space, and performer avatar data included in performer user data 131 described below and viewer avatar data included in viewer user data 133.

As shown in FIG. 12, the viewpoint image control program 123 is a program that provides a function to generate viewpoint images from virtual cameras C1-C4 virtually set in the virtual live venue and an airborne moving virtual camera (not shown), and also provides a function to display images from the viewpoints of viewer avatars 11-14 virtually participating in the third area of the virtual live venue and viewer avatars 21-32 virtually participating in the fourth area on the viewer terminal 300. Note that virtual camera C2 is a virtual camera set to provide an image from the viewpoint of performer avatar 1 on the stage of the virtual live venue, and the viewpoint image from virtual camera C2 is projected onto a screen S installed in the studio by a projector 211 (described later) connected to the performer terminal 200 in cooperation with the performer terminal 200.

The audio control program 124 is a program for controlling the audio (including music) distributed during the virtual live event, and specifically provides a function for playing music data 113 and a function for synthesizing the audio produced by the performer user with the played music and distributing it to the administrator terminal 150 and the viewer terminal 300.

The comment control program 125 cooperates with a viewer comment control program stored in the viewer terminal 300 described below to provide a function for displaying various comments entered by each viewer user in the viewer terminal 300 in chronological order on the performer terminal 200 and the viewer terminal 300 during the virtual live event.

The gift control program 126 is a program for controlling gifts given from viewer users to performer users in cooperation with a viewer program stored in the viewer terminal 300 described below, and specifically provides functions such as a function for the viewer user to purchase gifts based on information on the unit purchase price set for various gifts stored in the gift data 134, a processing function for the viewer user to give gifts purchased using the gift object data included in the gift data 134 to the performer user (including processing for presentation when giving the gift), and a processing function for changing the owner of a gift given from the viewer user to the performer user.

The performer user management program 130 provides management functions and authentication functions for information regarding performer users based on the performer user data 131 shown in FIG. 8, which is stored in the storage 103 together with the performer user management program 130.

The performer user data 131 used in this Example 1 is illustrated in FIG. 8. The performer user data 131 stores various information, such as the performer user's account (email address), name, authentication information, the file name of the avatar data of the avatar used in the virtual space, and the file name of the item list in which the items owned are registered, in association with a performer user ID uniquely assigned to the performer user. Although not illustrated in FIG. 8, information on virtual value such as points owned by each performer user may also be stored as the performer user data 131.

Performer users can be authenticated by matching their performer user ID, account, and authentication information, and items (gifts) that each performer user has received from viewers and that they come to own can be identified from an item list.

The viewer user management program 132 provides functions such as management of information about viewer users and authentication functions based on the viewer user data 133 shown in FIG. 9, which is stored in the storage 103 together with the program.

The viewer user data 133 used in the present embodiment 1 is illustrated in FIG. 9. The viewer user data 133 stores various information associated with a viewer user ID uniquely assigned to the viewer user, such as the viewer user's account (email address), authentication information, the file name of the avatar data of the avatar to be used in the virtual space, owned points which are the number of points that can be used in the virtual space, the file name of an item list in which items (gifts) purchased using points are registered, and personal information such as name, date of birth, and telephone number. Although not shown in FIG. 9, a nickname (comment name) displayed together with a comment is also registered in the viewer user data 133, and the nickname (comment name) is displayed together with the comment. Points can be increased, for example, by purchasing points from a specified operating company.

<Performer terminal>
Fig. 4 is a diagram showing an example of the configuration of a performer terminal 200 in the virtual space content delivery system of this embodiment 1. In this embodiment 1, as shown in Fig. 11, the performer terminal 200 is installed in a control room adjacent to a studio where a performer user performs acting movements, and uses a normal computer with relatively excellent processing power, and as shown in Fig. 4, has a processor (CPU) 201 connected to a data bus 208, a memory (RAM) 202, a storage 203 such as a hard disk, a communication interface (I/F) 204 capable of two-way communication by TCP/IP protocol via the Internet, which is an open computer network, an image processing unit 206 including a graphics processing unit (GPU) to which display devices A210 to C212 are connected, and an input/output interface (I/F) 205 to which various input/output devices are connected.

The input/output interface (I/F) 205 is connected to the following input devices: a motion sensor 220 including multiple wearable sensors 220C1-220C5 (see FIG. 11) worn by the performer user on the left and right limbs and waist; an expression input device 221 consisting of an imaging camera for inputting the performer's facial expressions; a voice input device 222 consisting of a sound-collecting microphone worn on the performer user's head; and an operation input device 224 such as a keyboard or touch panel that can be operated by an assistant operator O who assists the performer user. Note that by providing these input devices with a controller that the performer user can hold to perform various operations, the performer himself may be able to perform various operations without the assistance of the assistant operator O.

In this embodiment 1, the performer user performs performance actions in accordance with the music (including musical actions such as singing and dancing), and an assistant operator O assists with operations during the performance. However, there may be multiple assistant operators O, or the operation contents may be set in advance in the performer terminal 200, for example, in a sequence program, so that an assistant operator O is not provided.

The input/output interface (I/F) 205 is connected to an audio output device 223 including high-performance earphones (ear monitors) worn by the performer user as output devices and speakers placed in the control room, so that the sound of the music being played sent from the distribution server computer 100 is output to the performer user via the high-performance earphones (ear monitors), and the music audio is output from the speakers along with the performer's voice, allowing an assistant operator O, etc., to check the status of the music, including the voice produced by the performer user, in the control room.

In this embodiment 1, high-performance earphones (in-ear monitors) are used to avoid the inconvenience of the reproduced music audio being input from the sound collection microphone worn by the performer user, but the present invention is not limited to this, and the music audio may be output from speakers in the studio, etc., as long as the inconvenience of the music audio being input can be avoided.

Note that, in this embodiment 1, an example is shown in which the sound collection microphones are worn by the performers, but these sound collection microphones may also be stationary microphones that are installed on the floor, walls, or ceiling of the studio.

Here, we will explain the motion sensor 220 used in the performer terminal 200 of this Example 1. Any motion sensor can be used as long as it can properly detect (measure) the body movements (movements) of the performer user. In this Example 1, in order to enable more accurate and shorter-period detection of the performer's movements, multiple wearable sensors 220C1 to 220C5 are used that are worn by the performer user on the body. Note that the motion sensor 220 may be of a type that does not require the performer to wear any equipment, such as a LiDER that uses laser light or the like.

In this embodiment 1, in order to reduce the burden on the performer user's head, particularly the burden on the performer user's head, caused by wearing sensors when performing a performance, as described below, the performer user's head movements are detected by image recognition using images captured by the imaging camera constituting the facial expression input device 221, and five wearable sensors 220C1 to 220C5 are used to detect movements other than the head. However, for example, wearable sensors may also be provided on the performer user's head, or more wearable sensors (seven or more) may be worn to detect more detailed movements.

As shown in FIG. 11, the mounting sensors 220C1 to 220C5 detect their own position and orientation in cooperation with base station 220a and base station 220b installed in a studio room separated from the adjacent adjustment room by a glass window.

Base station 220a and base station 220b can use, as an example, a multi-axis laser emitter. After emitting a flashing light for synchronization, base station 220a scans the laser light, for example, around a vertical axis. Base station 220b scans the laser light, for example, around a horizontal axis.

Each of the mounting sensors 220C1 to 220C5 may include multiple optical sensors that detect the incidence of blinking light and laser light from the base station 220a and base station 220b.

Each of the mounting sensors 220C1 to 220C5 can detect its own position and orientation based on the time difference between the incidence of the blinking light and the incidence of the laser light, the light reception time at each optical sensor, the incidence angle of the laser light detected by each optical sensor, and other information as necessary. For example, ViveTracker or a base station provided by HTC CORPORATION can be suitably used.

Base station 220a and base station 220b emit flashing light and perform laser light scanning at regular intervals, so the detection information of each worn sensor 220C1-220C5 is updated at each interval. Detection information indicating the position and orientation of each motion sensor calculated in each of these worn sensors 220C1-220C5 is input to the performer terminal 200 via short-range wireless communication and transmitted to the distribution server computer 100 as performer avatar information together with facial motion information (see FIG. 10).

Note that in this embodiment 1, an example is shown in which two base stations, base station 220a and base station 220b, are used, but the present invention is not limited to this, and three or more base stations may be used.

Motion sensor 220 may also be configured such that each of a number of motion sensors worn by the performer user is equipped with a number of infrared LEDs or visible light LEDs, and the position and orientation of each of the motion sensors is detected by detecting the light from these LEDs with an infrared camera installed on the floor or wall of the studio.

Moreover, in order to reduce the burden on the performer user, the motion sensor 220 may be replaced with a motion sensor using a lighter reflective marker instead of the wearable sensors 220C1-220C5. In other words, the motion sensor may detect the performer user's movements by photographing reflective markers attached to various parts of the performer user's body with adhesive tape or the like to generate photographic data, and then detecting the position and orientation of the reflective markers through image processing of the photographic data. In this case, a cameraman who photographs the performer user may be positioned in the studio, and the cameraman may photograph the performer user from a direction suitable for detection, thereby enabling the performer user's movements to be detected well.

Motion sensor 220 may also be a suit with a built-in inertial sensor, such as a suit with a built-in inertial sensor for the MVN motion capture system marketed by Xsens.com, and the movements of the performer user may be detected by analyzing the sensor signal output from the inertial sensor.

Any facial expression input device 221 can be used as long as it can properly detect the movement of the head, including the face, of the performer user. In this embodiment 1, specifically, it is configured with an imaging camera (digital camera) arranged so as to be able to continuously capture head images, including the face, of the performer user, and the head movement and facial movement (facial expression) of the performer user captured by the imaging camera are detected by image recognition, and the facial motion information, which is the detected head movement and facial movement (facial expression), is transmitted to the distribution server computer 100 as performer avatar information together with body motion information, as shown in FIG. 10.

The facial expression input device 221 is not limited to the imaging camera (digital camera) used in the first embodiment. For example, the facial expression input device 221 may be a 3D camera capable of detecting the depth of a person's face, or a mobile terminal such as a smartphone equipped with a LiDER device. In this case, the performer user may wear such a mobile terminal.

As described above, the performer terminal 200 of this embodiment 1 has three display devices A210 to C212. Display device A210 is an LCD monitor or the like installed in the control room as shown in FIG. 11. Display device B211 is a projector that projects an image onto a screen S installed in the studio. Display device C212 is a large vertical display installed adjacent to the screen S in the studio.

Display device A210, which is placed in the adjustment room of the studio, displays images from each of the viewpoints of virtual cameras C1 to C4, viewpoint images from a virtual camera moving in the air during the flying performance period described below, the contents of comments, and a message input window into which assistant operator O inputs any message he or she wishes to convey to the performer user.

On the other hand, a screen S projected from a projector, which is a display device B211 installed in the studio, displays an image from the viewpoint of a virtual camera C2 (see FIG. 12), which is the viewpoint of the performer, for example, as shown in FIG. 15, including avatars of viewers virtually participating in the live performance in the third and fourth areas, allowing the performer user to check the status of the viewer users virtually participating in the virtual live performance through the image.

In addition, comments from viewer users, as well as comments and messages entered by the assistant operator O, are displayed on the display device C212 installed in the studio, allowing the performer user to check the comments from viewer users and messages from the assistant operator O even during the performance.

In addition to an operating system (OS) for operating the computer that is the performer terminal 200, the storage 203 stores a performer program that is executed by the processor (CPU) 201 etc. to provide various functions in cooperation with the distribution server computer 100, including the function of allowing the performer user to control the movement of the performer avatar 1 in a virtual live performance, which is virtual space content.

As shown in FIG. 4, the performer programs include a performer avatar control program, a performer viewpoint image control program, a performer viewpoint image control program, a performer voice control program, a performer comment control program, performer authentication data, etc.

The performer avatar control program is a program that provides the function of primarily scanning the body motion of the performer user using the motion sensor 220 and scanning the facial motion (facial expression) of the performer user using the facial expression input device 221 to generate performer avatar information for operating the performer avatar and transmit it to the distribution server computer 100.

The performer's viewpoint image control program is a program that provides the function of generating and outputting the viewpoint image of virtual camera C2, which is the performer's viewpoint, based on the virtual space update data distributed from the distribution server computer 100.

The performer's voice control program is a program that provides the function of outputting the playback sound of a song based on the playback music data distributed from the distribution server computer 100 via the audio output device 223, converting the audio input from the audio input device 222 into data and sending it to the distribution server computer 100 as performer voice data, and generating and outputting a monitor voice consisting of the playback sound of the song and the performer's voice.

The performer comment control program is a program that provides the function of displaying each comment by the viewer user that is distributed from the distribution server computer 100.

The performer authentication data is used to authenticate the identity of the performer user in a communication connection with the distribution server computer 100.

The performer avatar control program includes a motion parameter generation program capable of generating parameter information for the positions and rotation angles of the bones that make up the rig data (sometimes called skeleton data) that indicates the skeleton of the performer avatar included in the performer avatar data, as well as an image recognition processing program for image recognition of the performer user's head position and facial expression from the image captured by the imaging camera.

In addition, in this embodiment 1, the performer terminal 200 is illustrated as a form in which a computer serving as the performer terminal 200 installed in a studio operated by an event management organization is used jointly by multiple performer users to hold a live event, and each performer user can input performer authentication data to use the computer as the performer terminal 200 of that performer user. However, the present invention is not limited to this, and the performer terminal 200 may be used only by the performer user who owns it, such as when a performer user appears in a virtual live performance from his or her home, and the performer terminal 200 is a computer installed in the performer user's home and dedicated to the performer user's own use, and in such cases, storage of performer authentication data is not necessarily required.

In this way, by using the performer terminal 200 constituting the virtual space content distribution system of this embodiment 1, the performer user can move the performer avatar 1 placed on the virtual stage G in the virtual live venue in conjunction with the performer user's own movements, and can have the performer user's own facial expressions reflected in the facial expressions of the performer avatar 1. In addition, the performer user can distribute his or her own voice as the voice of the performer avatar 1 to viewers via the distribution server computer 100.

<Viewer terminal>
Fig. 5 is a diagram showing an example of the configuration of a viewer terminal 300 in the virtual space content delivery system of the present embodiment 1. In the present embodiment 1, the viewer terminal 300 is a smartphone P carried by a viewer user or a regular desktop computer (PC) installed at a viewer user's home or the like, and as shown in Fig. 5, has a processor (CPU) 301 connected to a data bus 308, a memory (RAM) 302, a storage 303 such as a hard disk or non-volatile memory, a communication interface (I/F) 304 capable of two-way communication by TCP/IP protocol via the Internet, which is an open computer network, an image processing unit 306 including a graphics processing unit (GPU) to which a display device 310 is connected, and an input/output interface (I/F) 305 to which various input/output devices are connected.

The display device 310 may be any device capable of displaying various viewpoint images generated based on the virtual space data of the virtual live venue, which is a virtual space updated by the virtual space update data (see FIG. 10) distributed from the distribution server computer 100, by the viewer viewpoint image control program included in the viewer program stored in the storage 303, as described below, and the flying viewpoint images distributed from the distribution server computer 100 during the flying performance period described below. If the viewer terminal 300 is a smartphone P, the display device 310 corresponds to the display of the smartphone P, and if the viewer terminal 300 is a normal computer (PC), the display device 310 corresponds to a stationary display constituting the computer (PC). Note that these displays are not limited to those that display planar images (2D images), but may be, for example, those that can display stereoscopic images (3D images) by displaying right-eye images and left-eye images, such as a head-mounted display (HMD) that can be worn by the viewer.

The input/output interface (I/F) 305 is connected to an operation input device 321 consisting of one or more devices that allow the viewer to perform various operations as input devices, and an audio output device 322 such as earphones or speakers that can output live audio, etc., distributed from the distribution server computer 100.

It goes without saying that the audio output device 322 need not be a separate device, but may be integrated with the display device 310, such as the head-mounted display (HMD) described above.

The operation input device 321 may be any device that allows the viewer to perform various operations such as motion operations including movement of the viewer avatar, zoom operations, operations related to comments, operations related to gifts, etc. For example, it may be composed of at least one of a transparent touch panel formed on the display surface of a smartphone, a keyboard constituting a computer, a game controller, etc.

In this embodiment 1, as described below, the virtual audience area in the virtual live venue is divided into a third area near the virtual stage G in which only viewer users who have been specially permitted by the performer user can virtually participate, and a fourth area around the third area in which general viewer users can virtually participate, as shown in FIG. 12. Viewer users who virtually participate in the fourth area may have an operation input device 321 such as a touch panel, keyboard, or game controller as described above, but viewer users who virtually participate in the third area possess an operation input device 321 such as an MVN motion capture suit with the above-mentioned inertial sensor built in, and thus can precisely move the viewer avatar with their own movements, just like the performer users.

However, even viewer users who are permitted to virtually participate in the third area cannot change the facial expression of the viewer avatar, but the present invention is not limited to this, and the facial expressions of these viewer avatars may be changed, for example, as described above, by the viewer user using the camera function of a smartphone or a camera connected to a computer to recognize the facial expression of the viewer user, even if it is at a level different from the level of change made by the performer user.

The various input/output devices connected to the input/output interface (I/F) 305 may be built-in or external, and in the case of external devices, they may be connected in either a wired or wireless manner.

In addition to an operating system (OS) for operating the viewer terminal 300, which is a smartphone or computer, the storage 303 also stores a viewer program as a viewing application that is executed by the processor (CPU) 301 or the like to provide various functions, including a live viewing function for viewer users in a virtual live, which is virtual space content, in cooperation with the distribution server computer 100.

As shown in FIG. 5, the viewer program includes a viewer viewpoint image control program that provides the function of generating each viewpoint image of the virtual live to be displayed on the display device 310, a viewer audio control program that provides the function of outputting live audio from the audio output device 322 based on the live audio (data) of the virtual live distributed from the distribution server computer 100, a viewer comment control program that displays comments based on comment data distributed from the distribution server computer 100 and provides functions related to comments such as comments entered by the viewer and transmitted to the distribution server computer 100, and viewer authentication data for authenticating the identity of the viewer user in a communication connection with the distribution server computer 100.

Note that although not shown in FIG. 5, in addition to the above-mentioned programs, there is included a gift control program that controls gifts, and an area designation program for reserving an area (position) in which a viewer avatar will virtually participate in the virtual live venue in advance before the start of the live event, as shown in variant example 3-1 (see FIGS. 42 and 43) described below, but other programs may also be included.

In this embodiment 1, the comments entered by the viewer are short messages in text format, but are not limited to these short messages in text format and may be, for example, messages in still image format, messages in video format, or any other electronic message format. The comments are displayed on the display device 310, superimposed on each viewpoint video.

In this way, by using the viewer terminal 300 constituting the virtual space content distribution system of this embodiment 1, a viewer user can move and operate the viewer avatar virtually participating in the virtual live venue, and by changing the viewpoint of the viewer avatar, the viewpoint image displayed on the display device 310 can be changed, and the zoom state (viewpoint state) can also be changed, allowing the viewer avatar to operate and cheer on the performer avatar 1.

Note that a viewer avatar that virtually participates in the third area is only allowed to move within the third area, and cannot move to the fourth area. On the other hand, a viewer avatar that virtually participates in the fourth area is only allowed to move within the fourth area, and cannot move to the third area. However, the present invention is not limited to this, and a viewer avatar that virtually participates in three areas may be allowed to move to the fourth area. Note that a viewer avatar naturally cannot move onto the virtual stage G, etc., so visibility of the performer avatar 1 is not obstructed by the viewer avatar moving onto the virtual stage G.

In addition, by displaying viewpoint images on these display devices 310 and outputting live audio, users can enjoy the virtual live performance with the feeling that they are actually present at the virtual live venue.

Although we will not go into detail here, the comment function provided by the viewer comment control program allows viewers to enjoy the virtual live show while checking comments entered by themselves and other viewer users, and the gift function provided by the gift control program allows viewers to give items they own to the performer avatar 1 to liven up the virtual live show.

<Administrator terminal>
Fig. 6 is a diagram showing an example of the configuration of the administrator terminal 150 in the virtual space content delivery system of this embodiment 1. In this embodiment 1, the administrator terminal 150 uses a normal computer installed in a control room adjacent to the studio together with the performer terminal 200, and as shown in Fig. 6, has a processor (CPU) 151 connected to a data bus 158, a memory (RAM) 152, a storage 153 such as a hard disk, a communication interface (I/F) 154 capable of two-way communication by the TCP/IP protocol via the Internet, which is an open computer network, an image processing unit 156 including a graphics processing unit (GPU) to which a display device 160 is connected, and an input/output interface (I/F) 155 to which various input/output devices are connected.

As described below, the display device 160 is configured as one or more stationary displays capable of individually displaying each viewpoint image of each of the virtual cameras C1 to C4 described below and each viewpoint image of the virtual camera moving in the air during the flying performance period, which are generated based on the virtual space data of the virtual live venue, which is a virtual space updated by the virtual space update data (see FIG. 10) distributed from the distribution server computer 100, by the administrator viewpoint image control program included in the administrator program stored in the storage 153, as well as being capable of displaying comments based on the comment data distributed from the distribution server computer 100.

The input/output interface (I/F) 155 is connected to an operation input device 161 consisting of one or more devices that allow the event manager M to perform various operations as input devices, and an audio output device 162 such as earphones, headphones, or speakers that can output live audio, etc., distributed from the distribution server computer 100.

The operation input device 161 can be any device that allows the event manager M to perform various operations such as input and settings related to the virtual live event, and may be composed of at least one of, for example, a keyboard constituting a computer, a transparent touch panel formed on the surface of a display, a viewpoint switching controller, etc.

The various input/output devices connected to the input/output interface (I/F) 155 may be connected in either a wired or wireless manner.

In addition to an operating system (OS) for operating the computer that is the administrator terminal 150, the storage 153 also stores an administrator program that is executed by the processor (CPU) 151 etc. to provide various functions in cooperation with the distribution server computer 100, including a distribution management function for the distribution of virtual live shows, which are virtual space content.

As shown in FIG. 6, the administrator programs include an administrator viewpoint video control program, an administrator audio control program, an administrator comment control program, a viewer management program, a performer management program, etc.

The administrator viewpoint image control program is a program that provides the function of generating viewpoint images from each virtual camera to be displayed on the display device 160, as well as the function of changing and switching between each viewpoint.

The administrator audio control program is a program that provides the function of outputting live audio from the audio output device 162 based on the audio data of the virtual live broadcast distributed from the distribution server computer 100.

The administrator comment control program is a program that displays comments based on comment data distributed from the distribution server computer 100, and provides comment-related functions such as prohibiting comments from being distributed and selecting viewers to prevent them from being distributed.

The viewer management program is a program for managing viewer users who virtually participate in a virtual live performance, and the performer management program is a program for managing performer users who virtually participate in a virtual live performance.

Note that although not shown in FIG. 6, the system includes an authentication program for authenticating whether the administrator who is the operator is the actual person, and a schedule program for editing the time schedule and song order of the virtual live performance, but may also include other programs.

In this way, the event manager M can operate the manager terminal 150 constituting the virtual space content distribution system of this embodiment 1 to implement various settings for executing the event, such as the program, order of songs, settings related to the performance, the time schedule, and the trajectory of the virtual camera moving in the air during the flying performance period, and the virtual live performance is managed based on the settings made in this way.

<Virtual live venue>
A virtual live performance venue, which is the virtual space used in Example 1, is shown in Fig. 12. As shown in Fig. 12, the virtual live performance venue, like a live performance venue in real space, is a virtual space having a virtual stage G where performer users appear as performer avatars 1 and a virtual audience area where audience users virtually participate as audience avatars.

As shown in FIG. 12, virtual stage G has a relatively large trapezoidal floor surface, and performer avatar 1 can move around on virtual stage G in the same way as in a live venue in real space. On the side of virtual stage G opposite the virtual audience seats, as shown in FIG. 12, a virtual stage wall is formed that is divided into three parts: a center, a right side, and a left side, and virtual displays 40, 41, and 42 are provided in front of these virtual stage walls, and images and videos for the performance are virtually displayed on the virtual displays 40, 41, and 42.

A star virtual placement object 43 related to the performer user is placed between the virtual displays 40, 41, and 42, and large star virtual aerial objects 50 to 52 are placed in a second area, which is an area on the virtual stage G above the first area surrounded by the virtual stage walls, as shown in FIG. 12. The virtual aerial objects 50 to 52 are capable of moving within the second area.

A virtual audience area is provided in front of the virtual stage G, where viewer users can virtually participate in the virtual live venue as viewer avatars. As shown in FIG. 12, this virtual audience area is made up of a third area, which is a special area close to the center of the virtual stage G where the performer avatar 1 appears, and a fourth area, which is formed to surround the third area and is located at a greater distance from the center of the virtual stage G than the third area.

In these third areas, special viewer users who are permitted to participate virtually can participate virtually up to a certain upper limit. In this embodiment 1, as shown in FIG. 12, for example, viewer avatars 11 to 14 of four viewer users who have met a certain virtual participation record and have been permitted to participate virtually by the performer user are placed.

The fourth area also contains viewer avatars 21-32 of general viewer users who have booked virtual participation before the start of the virtual live event. Note that in FIG. 12, viewer avatars 21-32 are shown in a simplified manner for convenience, but are the same avatars as viewer avatars 11-14.

In addition, each viewer avatar 11 to 14 arranged in the third area can be moved within the third area by the viewer user operating the viewer terminal, and each viewer avatar 21 to 32 arranged in the fourth area can be moved within the fourth area by the viewer user operating the viewer terminal.

As shown in FIG. 12, four virtual cameras C1 to C4 are virtually positioned (set) in the virtual live venue. Virtual camera C1 is virtually positioned in front of performer avatar 1 so as to face the performer avatar 1, and the viewpoint image captured by virtual camera C1 is a viewpoint image viewed from a position close to the front of the performer avatar 1, as shown in FIG. 13(a).

The virtual camera C2 is a camera virtually placed (set) above the head of the performer avatar 1, and the viewpoint image captured by the virtual camera C2 is a performer viewpoint image viewed from the performer avatar 1 toward the virtual audience area, as shown in FIG. 15.

Virtual camera C3 is a virtual camera virtually placed on the virtual display 41 diagonally behind the performer avatar 1, and the viewpoint image from virtual camera C3 is a viewpoint image looking at the virtual audience area from diagonally behind the performer avatar 1, as shown in Figure 13 (c).

Virtual camera C4 is a virtual camera virtually placed on the virtual display 42 diagonally behind the performer avatar 1, and the viewpoint image from virtual camera C4 is a viewpoint image looking at the virtual audience area from diagonally behind the performer avatar 1, as shown in FIG. 13(b).

In addition, in this embodiment 1, in addition to the fixedly placed (set) virtual cameras C1 to C4, an air-moving virtual camera (not shown) is virtually placed so that it can move through the air in the virtual live venue space, which is a virtual space. The aerial trajectories along which these air-moving virtual cameras move and the viewpoint directions (angles) at each position on the aerial trajectory are preset by the administrator terminal 150, and during the flying performance period described below, the performer avatar 1 virtually flies so as to follow the aerial trajectory of the air-moving virtual camera, thereby generating a viewpoint image in which the flying performer avatar 1 is captured by the air-moving virtual camera.

In this embodiment 1, as shown in FIG. 12, four virtual cameras C1 to C4 are virtually arranged as fixedly arranged (set) virtual cameras, but the present invention is not limited to this. The number of these fixedly arranged (set) virtual cameras may be five or more. Conversely, without providing fixedly arranged (set) virtual cameras, only the viewpoint images seen from the viewpoints corresponding to each avatar virtually participating in the virtual live venue may be displayed on the performer terminal 200 and the viewer terminal 300.

In addition, in this embodiment 1, an example is given of a configuration in which there is one virtual camera moving in the air, but the present invention is not limited to this, and a configuration in which there are multiple virtual cameras moving in the air, or conversely, a configuration in which no virtual camera moving in the air is virtually placed, may also be used.

<Studio>
11 is a diagram showing the studio used in Example 1, and as described above, the studio is adjacent to the control room separated by a glass window, and the performer user performs each action corresponding to singing in the studio, while the assistant operator O assists with the operation of the performer terminal 200 installed in the control room. Around the performer user wearing the mounting sensors 220C1 to 220C5, base stations 220a and 220b are mounted on a stand, and an imaging camera constituting the facial expression input device 221 is mounted on a stand and positioned almost in front of the performer user.

A screen S is provided on the wall facing the performer user in the studio, and a projector (display device B211) placed on the ceiling of the studio projects the viewpoint image of virtual camera C2 onto screen S, so that the viewpoint image seen from performer avatar 1 as looking at the virtual audience area is displayed on screen S, as shown in FIG. 15. This allows the performer user to proceed with the live performance while keeping track of the behavior of the viewer avatars virtually participating in the virtual audience area (the audience's reaction to the live performance).

A large vertical display serving as display device C212 is placed to the side of screen S, and is configured to display comments from viewers and messages that the assistant operator O wishes to convey to the performer user. Note that display device C212 may also be configured to display information (performer support information) such as the lyrics of songs sung by performers during live performances and lines.

<Operation of Virtual Space Content Distribution System>
FIG. 10 is an explanatory diagram for explaining the operation of the virtual space content delivery system of the first embodiment, specifically, the flow of delivery of video and audio of a virtual live performance.

First, the video system will be described. As mentioned above, the distribution server computer 100 is capable of providing a virtual space of a virtual live venue based on the virtual space generation program 120 and avatar data of performer users and viewer users, and the movements of each avatar of these performer users and viewer users are reflected in the movements of the performer avatar 1, and the movements (operations) of the viewer users are reflected in the movements of the viewer avatar by updating the virtual space data describing the state of the virtual space with performer avatar information transmitted from the performer terminal 200 and viewer avatar information transmitted from the viewer terminal 300.

Although not shown in FIG. 10, when the performer terminal 200 and the viewer terminal 300 communicate with the distribution server computer 100 to virtually participate in the virtual live performance, virtual space data of the virtual live performance venue is delivered in advance from the distribution server computer 100 and stored in the performer terminal 200 and the viewer terminal 300. As described below, the virtual space data thus prestored is updated sequentially by virtual space update data successively delivered from the distribution server computer 100, so that the latest state of the virtual live performance venue can be identified on the performer terminal 200 and the viewer terminal 300.

Specifically, the movements and facial expressions of the performer user in the studio are scanned at predetermined time intervals by the body motion scan and face motion scan functions of the performer avatar control program described above, and performer avatar information is generated and transmitted to the distribution server computer 100.

On the other hand, viewer users who are permitted to virtually participate in the third area use motion sensors, just like the performers, and are scanned at predetermined time intervals using the body motion scan and face motion scan functions to generate viewer avatar information that is then sent to the distribution server computer 100.

In addition, a viewer user who is permitted to virtually participate in the fourth area can use the touch panel, keyboard, or controller of the viewer terminal 300 to perform various operations such as movement operations, operations to change the viewpoint direction, operations to change the viewpoint state (zoom), operations to raise hands, operations to clap hands, operations to jump, etc., to cause the viewer avatar to perform the corresponding action. Then, of these operations, viewer avatar information is generated based on the operations related to the actions of the viewer avatar and is transmitted to the distribution server computer 100.

In addition, viewer users virtually participating in the third area can also use a controller or the like to perform various operations such as movement, changing the viewpoint (zoom), raising their hands, clapping their hands, jumping, etc. In this case, menu items corresponding to each operation are displayed, and the menu items corresponding to each operation can be selected using a controller or the like.

In this way, the viewer terminal 300 executes an avatar action operation process that accepts operations corresponding to each action to be performed by the viewer avatar. The avatar action operation process also includes processes such as displaying and erasing menu items corresponding to each of the above-mentioned actions.

In this way, based on the performer avatar information transmitted from the performer terminal 200 and the viewer avatar information transmitted from the viewer terminal 300, the distribution server computer 100 executes a virtual space update process that updates the virtual space data, thereby reflecting the actions (operations) of the performer user and the viewer user in each avatar virtually participating in the virtual live venue.

Then, a virtual space update data generation process is executed to generate virtual space update data based on the virtual space data before the update and the virtual space data after the update, and the generated virtual space update data is distributed to the performer terminal 200 and the viewer terminal 300.

As shown in FIG. 10, in the performer terminal 200 and the viewer terminal 300 to which the virtual space update data has been delivered, the pre-stored virtual space data is updated based on the delivered virtual space update data, and a performer perspective video generation process and a viewer perspective video generation process are executed using the updated virtual space data, thereby generating performer perspective video and viewer perspective video based on the updated virtual space data, and the generated performer perspective video and viewer perspective video are displayed on the performer terminal 200 and the viewer terminal 300 (viewer perspective video output process).

In this way, in this embodiment 1, the viewpoint video of each viewer user who is virtually participating in the virtual live venue through a viewer avatar is generated and displayed on the viewer terminal 300 of each viewer user, and by generating the viewer avatar viewpoint video of each viewer user on the distribution server computer 100, it is possible to avoid a situation in which the processing load on the distribution server computer 100 becomes significantly large, which would result in many viewer users being unable to participate in the virtual live, and it is also possible to avoid a situation in which distribution becomes difficult due to an increased processing load.

In this embodiment 1, a viewpoint image in which the almost front of the performer avatar 1 is displayed large, which is the viewpoint image seen from the viewer avatar 13 close to the performer avatar 1, as shown in FIG. 13(d), is generated and displayed on the display device 310 of the viewer terminal 300 of the viewer user of the viewer avatar 13 who is virtually participating in the virtual stage G in the third region close to the performer avatar 1, while a viewpoint image in which the almost front of the performer avatar 1 is displayed large, which is the viewpoint image seen from the viewer avatar 13 close to the performer avatar 1, as shown in FIG. 13(d), is displayed on the display device 310 of the viewer terminal 300 of the viewer user of the viewer avatar 28 who is virtually participating in the virtual stage G in the fourth region far from the performer avatar 1, for example, at a position behind the viewer avatar 13, as shown in FIG. 13(e).

In addition, the display device 310 of the viewer terminal 300 of the viewer user of the viewer avatar 32 who is virtually participating at the edge position of the virtual stage G in the fourth area, which is set to be U-shaped when viewed from above, displays a viewpoint image as seen from the viewer avatar 32 positioned diagonally forward of the performer avatar 1, as shown in Figure 13 (f), in which the performer avatar 1 appears relatively small through the viewer avatar 14 who is virtually participating in the third area.

The display examples shown in FIG. 13 all illustrate the case where the viewer terminal 300 is a stationary computer (PC) and the display device 310 is a stationary display. However, for example, if the viewer terminal 300 is a smartphone P or a tablet, the viewpoint image of the virtual camera C1 is displayed as shown in FIG. 14(a), and if the viewer terminal 300 is the viewer terminal 300 of the viewer avatar 13, it is displayed as shown in FIG. 14(b).

Next, regarding the audio system, in the distribution server computer 100, music data 113 is played by the audio control program 124 based on the time schedule data (TS data) 112, and is transmitted to the performer terminal 200 as a played music piece (data) (music playback process).

On the performer terminal 200, the music audio from the played music (data) sent from the distribution server computer 100 is output to the performer user through high-performance earphones (ear monitors) worn by the performer user (played music output process), and when the performer user sings along with the output played music, the performer voice input from the sound collection microphone (audio input device 222) is digitized and sent to the distribution server computer 100 as the performer voice (data).

The distribution server computer 100 generates distribution audio (data) from the performer audio (data) received from the performer terminal 200 and the music played in the music playback process described above (distribution audio generation process), and distributes the generated distribution audio (data) to the viewer terminal 300 (generated audio distribution process).

In the viewer terminal 300, the distribution audio (data) distributed from the distribution server computer 100 is output from an audio output device 322 such as an earphone or speaker, allowing the viewer to hear the live audio as if the performer avatar 1 were singing in the virtual live venue.

In addition, as described above, in the performer terminal 200, the performer voice (data) is sent to the distribution server computer 100, and as shown in FIG. 10, the monitor voice generation process and the monitor voice output process are executed, so that the music voice of the played music and the performer voice are output from the speakers located in the control room, allowing the assistant operator O, etc. to check the singing status of the performer user in the control room.

<Viewer's viewpoint video control processing>
Next, a viewer's viewpoint video control process executed in the viewer terminal 300 based on a viewer's viewpoint video control program included in the viewer's program will be described with reference to FIG.

In the viewer viewpoint video control process, first, it is determined whether or not the performer avatar 1 is in a flying performance period in which it is virtually flying, as shown in FIG. 19 (step S1). Whether or not the performance period is in a flying performance period can be determined from performance status data indicating that the performance period is in a flying performance period, sent from the distribution server computer 100. The distribution server computer 100 transmits performance status data indicating that the performance period is in a flying performance period to the viewer terminal 300 when the flying performance period is reached based on the time schedule data.

If it is during the flying performance period (Y in step S1), proceed to step S21. If it is not during the flying performance period (N in step S1), it is determined whether viewpoint-related operations including viewpoint movement operations (including avatar movement operations and viewpoint changes due to motion operations) and zoom operations are disabled (step S2).

If the viewpoint-related operation is disabled (Y in step S2), proceed to step S7. If the viewpoint-related operation is not disabled (N in step S2), determine whether or not there is an operation that changes the viewpoint, such as an operation to move the avatar or an operation to move the motion (step S3), and if there is an operation (Y in step S3), change the viewer viewpoint according to the operation, and if there is no operation (N in step S3), proceed to step S5 and determine whether or not there is a zoom operation (step S5).

If a zoom operation is performed (Y in step S5), the image area displayed on the display device 310 is changed to an image area corresponding to the operation, and if a zoom operation is not performed (N in step S5), it is determined whether or not a motion tracking setting is set (step S7).

This motion tracking setting can be set at the viewer's terminal 300 at will, for example, before the start of a live performance or during a live performance, depending on the type of device the viewer uses for operation and his/her level of proficiency in operation. Specifically, the motion tracking setting is set when, for example, the viewer is not familiar with operating the operation input device 321 such as a touch panel, keyboard, or game controller, or when the viewer is not familiar with changing the viewpoint by moving or moving the avatar, or when, as described below, the virtual participation position of the viewer's avatar is far from the performer avatar 1 and the viewer is zoomed in and displayed in an enlarged manner, so that even the slightest movement of the performer avatar 1 prevents the performer avatar 1 from being displayed properly, and the viewer is unable to change the viewpoint in response to the movement or movement of the performer avatar 1.

If there is no motion tracking setting (N in step S7), proceed to step S11, whereas if there is a motion tracking setting (Y in step S7), it is further determined whether the motion tracking condition is met (step S8).

In this embodiment 1, the motion tracking condition is a condition under which it is almost certain that the performer avatar 1 will no longer be displayed clearly on the display device 310. Specifically, if the head (or face) of the performer avatar 1 is within a specified range around the display area of the display device 310, it is highly likely that the movement or motion of the performer avatar 1 will cause the head (face) to move outside the display area, causing it to no longer be displayed clearly, and therefore the motion tracking condition is determined to be met.

In this way, in this embodiment 1, by determining that the motion tracking condition is met before the head (face) of the performer avatar 1 disappears from the display area, it is possible to significantly reduce the occurrence of the head (face) of the performer avatar 1 disappearing from the display area, but the present invention is not limited to this, and these motion tracking conditions may be the condition that the head (face) moves outside the display area, or other conditions may be used as the motion tracking conditions. In other words, any condition can be used as the motion tracking condition as long as it can prevent the performer avatar 1 from disappearing from the display area properly.

If the action-following condition is met (Y in step S8), after disabling the viewpoint-related operation (step S12), the viewpoint image displayed in the display area of the display device 310 is automatically changed to a motion-following viewpoint in which the head (face) of the performer avatar 1 is within a non-metrology range surrounded by a predetermined range around the display area, and the viewpoint image seen from the motion-following viewpoint is displayed in the display area of the display device 310 (step S13).

On the other hand, if the motion tracking condition is not met (N in step S8), it is further determined whether or not the viewpoint-related operation is disabled (step S9). If the viewpoint-related operation is disabled (Y in step S9), the disabled viewpoint-related operation is enabled (step S10) and the process proceeds to step S11. If the viewpoint-related operation is not disabled (N in step S9), the process proceeds to step S11 without going through step S10.

In step 11, since the motion tracking setting is not set or the motion tracking condition is not met, the viewpoint image of the viewer avatar continues to be displayed in the display area of the display device 310 (step S11).

Also, as mentioned above, if the flying performance period in which the performer avatar 1 virtually flies during a live performance has begun, step S1 is judged as Y and the process proceeds to step S21, where it is determined whether or not it is time to start the flying performance period (step S21).

If it is time to start the flying performance period (Y in step S21), the flying viewpoint video delivered from the delivery server computer 100 is displayed (step S22) in place of the viewer avatar's viewpoint video or the action-tracking viewpoint video that was displayed in the display area of the display device 310 before the start of the flying performance period, and then viewpoint-related operations are disabled (step S23) and the process returns to step S1.

In this embodiment 1, since the same flying viewpoint video is displayed on the viewer terminals 300 of all viewer users during the flying performance period, the flying viewpoint video is not generated on each viewer terminal 300 side like the viewer avatar viewpoint video that is different for each viewer user, but is generated on the distribution server computer 100 side and distributed to the viewer terminal 300, so that the flying viewpoint video, which involves airborne movement of the viewpoint and therefore has a large rendering load for the viewpoint video, can be displayed well even on viewer terminals 300 with low processing power, but the present invention is not limited to this, and these flying viewpoint videos may be generated on the individual viewer terminals 300 side, similar to the viewer avatar viewpoint video.

On the other hand, if it is not the start timing of the flying performance period (N in step S21), it is further determined whether it is the end timing of the flying performance period (step S24).

Whether or not it is the end timing of the flying performance period may be determined based on data indicating the end contained in the flying viewpoint video data transmitted from the distribution server computer 100, or may be determined based on the aforementioned performance status data that is different from the flying viewpoint video.

If it is not the end of the flying performance period (N in step S24), i.e., if it is during the flying performance period, proceed to step S26, update the flying viewpoint video, and then return to step 1; on the other hand, if it is the end of the flying performance period (Y in step S24), enable viewpoint-related operations (step S25), proceed to step S11, display the viewer avatar's viewpoint video in the display area of display device 310 instead of the flying viewpoint video that was displayed in the display area of display device 310 during the flying performance period, and then return to step S1.

In this manner, the viewer viewpoint video control process shown in FIG. 20 is executed in the viewer terminal 300 of this embodiment 1, and the viewer viewpoint videos displayed on the viewer terminal 300 are explained using FIG. 16 to FIG. 19.

Figure 16 is a diagram showing the display contents of the display device 310 when a zoom operation is performed on the viewer terminal 300 of this Example 1. In the default state where no zoom operation is performed, a viewpoint image of the image area centered on the upper body of the performer avatar 1 is displayed as shown in Figure 16 (a). In this default state, when the viewer user performs a zoom-up operation, for example, by touching the "+" display (not shown) if the viewer terminal 300 is a smartphone P, or by operating the "+" key on the keyboard if the viewer terminal 300 is a computer (PC), a viewpoint image of the image area centered on the head (face) of the performer avatar 1 is displayed as shown in Figure 16 (b), for example.

On the other hand, if the viewer user performs a zoom-down operation, for example, by touching the "- " display (not shown) if the viewer terminal 300 is a smartphone P, or by operating the "- " key on the keyboard if the viewer terminal 300 is a computer (PC), then a viewpoint image of the image area including the entire body of the performer avatar 1 will be displayed, as shown in FIG. 16(c), for example.

Note that in Figure 16, in order to make the zoom function easier to understand, the explanation was given using the viewpoint image of virtual camera C1 rather than the viewpoint of the viewer avatar, but as shown in Figure 17, zooming up and down can also be performed on the viewpoint of the viewer avatar in the same way.

Although not described in detail in the viewer viewpoint image control process of FIG. 20, for example, when the performer avatar 1 appears on the virtual stage G for the first time when the live performance starts, the viewpoint of the viewer avatar is not directed toward the performer avatar 1, and therefore there is a possibility that the performer avatar 1 is not displayed on the viewer terminal 300 for a long period of time. Therefore, in this way, the period when the performer avatar 1 first appears may be set as an appearance performance period, and during this performance period, the viewpoint image displayed on the viewer terminal 300 may be automatically switched to the viewpoint image of virtual camera C1, etc., instead of the viewpoint image of the viewer avatar, and each viewer user may be allowed to perform zooming in, etc. on the viewpoint images of virtual camera C1.

Figure 17 shows the viewpoint image of viewer avatar 28, who is virtually participating in the fourth area, as an example of the viewpoint image of a viewer avatar. As shown in Figure 17, the viewpoint image of viewer avatar 28 includes the heads of viewer avatar 13 and viewer avatar 14, who are virtually participating in the third area, and the image shows performer avatar 1 seen through the heads of viewer avatar 13 and viewer avatar 14.

Therefore, the viewer user of viewer avatar 28 can, for example, perform a zoom-in operation to change the image area so that the upper body of performer avatar 1 is displayed at the center, as shown in FIG. 17(b), thereby preventing viewer avatar 13 and viewer avatar 14 from being displayed and allowing the viewer user to enjoy the live performance.

However, in the zoomed-in state shown in FIG. 17(b), for example, as the live performance progresses and the performer avatar 1 begins to move and make large movements, if the viewer user is unable to properly change the viewpoint of the viewer avatar 28 in accordance with the movements of the performer avatar 1, the performer avatar 1 may not be displayed properly, as shown in FIGS. 18(d)-(f). In that case, if the viewer user has to pay attention to changing the viewpoint, it may become difficult to concentrate on the live performance and the viewer may not be able to fully enjoy the live performance.

However, in this Example 1, when the viewer user sets the motion-following setting as described above, as shown in Figures 18(a) to (c), the head of the performer avatar 1 enters within a specified peripheral area of the display area, the motion-following condition is met, the viewpoint is automatically changed to the motion-following viewpoint, and the viewpoint image of the motion-following viewpoint is displayed. This means that the viewer user does not need to pay attention to the viewpoint operation, and can concentrate on the live performance and enjoy it to the fullest.

As the live performance progresses and the flying performance period begins, as shown in Figure 19, the viewpoint images of the individual viewer avatars are automatically changed to flying viewpoint images seen from a virtual camera moving in the air, without any operation by the viewer user.

Specifically, before the performer avatar 1 begins the virtual flight, an image is displayed in which a wing object grows on the back of the performer avatar 1. Then, as the performer avatar 1 looks up at a virtual aerial star object 50 that is virtually placed in the second area of the virtual live venue, the viewpoint of the viewer avatar also moves in a similar manner. At this time, light effects 61 appear around the performer avatar 1.

After that, when the performer avatar 1 starts virtual flight, the viewpoint image displayed on the display device 310 automatically switches from the viewpoint image of the viewer avatar up to FIG. 19(b) to a flying viewpoint image seen from a virtual camera moving in the air, as shown in FIG. 19(c).

During the flying performance period, the airborne virtual camera moves through the second area based on a preset aerial trajectory as described above, and the flying viewpoint image seen from this airborne moving virtual camera is displayed on the display device 310 as shown in FIG. 19(d).

<Modification of the First Embodiment>
In the above-mentioned first embodiment, the general action (first action) is an action in which the performer avatar 1 stands without moving to the virtual stage G, the special action (second action) is an action in which the performer avatar 1 performs a virtual flight, and the specific action (third action) is an action in which the performer avatar 1 moves rapidly in the lateral direction. However, the present invention is not limited to this, and the specific action may be an action according to the content to be distributed. For example, the general action (first action) may be an action (normal action) that the performer avatar performs for the longest period in the distributed content, the special action (second action) may be an action that is performed in the shortest special period in the distributed content or the least frequently performed action, and the specific action (third action) may be an action that is performed for a shorter period than the general action (first action) but for a longer period than the special action, or an operation that is performed less frequently than the general action (first action) but more frequently than the special action. Note that the special action (second action) may include the specific action (third action).

In addition, in the above-mentioned Example 1, an embodiment is exemplified in which there is both a function of switching the viewpoint corresponding to a special action (second action) and a function of following a specific action (third action), but the present invention is not limited to this, and the virtual space content delivery system may have only one of these functions.

In the above-mentioned Example 1, an example was given of a form in which the viewpoint of the viewer user is changed and followed in accordance with the movements of the performer avatar, but the present invention is not limited to this. For example, the change and tracking of the viewpoint may be performed in accordance with specific effects such as smoke or fireworks (pillars of fire) devices, special lighting, and deformation of the virtual stage G, which are effects on the virtual stage G.

Next, the virtual space content distribution system of the second embodiment will be described below with reference to Figs. 21 to 28. Note that the same components as those of the first embodiment will be denoted by the same reference numerals and will not be described.

The configuration of the virtual space content delivery system of Example 2 is the same as that of Example 1 shown in FIG. 1, but the progress of the virtual live performance is different, and the period during which the viewpoint images from the virtual cameras C1 to C4 and the airborne moving virtual camera (hereinafter collectively referred to as virtual camera viewpoint images) are delivered and the entity that generates these virtual camera viewpoint images are different from those of Example 1, which is a characteristic feature of Example 2.

Fig. 21 is a diagram showing the progress of a virtual live performance distributed in the virtual space content distribution system of Example 2. In the virtual live performance of Example 2, a "live virtual space participation period" and a "pre-performance speech" period are set as periods before the start of the virtual live performance, and during the live virtual space participation period, viewer users can virtually participate in the virtual live performance venue, which is the virtual space that is the target of the virtual live performance.

The period up to the pre-data download completion check timing, which is set just before the end of the "pre-show speech" period, is the pre-data download (DL) period, during which virtual space data (basic data) of the virtual live venue, avatar data of performer avatar 1, and avatar data of other viewer users who have participated virtually are downloaded (DL) as pre-data from the distribution server computer 100 to the viewer terminal 300 of the viewer user who has participated virtually during the period during which participation in the live virtual space is possible.

In addition, at the pre-data DL completion check timing, which is the end timing of the pre-data download (DL) period, it is checked whether the pre-data download (DL) has been completed at each viewer terminal 300.

If the download (DL) of the advance data is not complete by the time of the advance data DL completion check timing due to differences in the model, processing power, or communication environment of the viewer terminal 300, the download (DL) of the advance data will continue. However, in this case, data that is highly important for the distribution of the virtual live performance, such as the virtual space data (basic data) of the virtual live performance venue required for the opening performance and the first MC, and the avatar data of performer avatar 1, will be downloaded (DL) with priority.

In addition, in the second embodiment, as described above, the data required to generate viewpoint images at the viewer terminal 300 during the distribution of a virtual live performance is downloaded (DL) as advance data, but the present invention is not limited to this, and this data, particularly special data required individually for each performance and each MC described below, may be downloaded (DL) sequentially each time at a timing corresponding to the timing at which each performance and each MC is distributed.

During the "pre-show introduction" period, explanations about the virtual live show are given through video footage, etc. Therefore, by making the "pre-show introduction" period a period during which advance data can be downloaded (DL), each viewer terminal 300 can also download (DL) from the distribution server computer 100 the avatar data of viewer avatars who virtually participate in the live virtual space just before the end of the participation period.

As shown in FIG. 21, the virtual live performance of Example 2 progresses as follows: "Opening performance" → "First MC" → "First performance" → "Second MC" → "Second performance" ... "Opening performance" is a performance in which performer avatar 1 appears on virtual stage G. In the "First MC", performer avatar 1 appears on virtual stage G, greets the audience through talk, and introduces the first performance to be performed first. The "First performance" is a performance that includes singing and dancing by performer avatar 1 (performer), and also includes a flying performance in the middle of the performance, in which the performer avatar 1 virtually flies through the virtual live venue and is virtually captured by a virtual camera moving in the air. In the "Second MC", performer avatar 1 (performer) mainly introduces the second performance through talk. The "second performance" includes singing and dancing by the performer avatar 1 (performer), as well as a costume change performance using footage virtually captured by the virtual camera C1 of the performer avatar 1 changing costumes, and an avatar splitting into multiple people using footage virtually captured by the virtual camera C1.

The processing load on the distribution server computer 100 during each period in which these effects and performances are performed is the lowest at "low" during the period in which participation in the live virtual space is possible, while it is the highest at "high" during the period in which the special effects "flying effect", "costume change effect", and "double-ego effect" are performed. The processing load on the distribution server computer 100 during the period of the "first performance" from the "introduction", "opening effect", "first MC", and "flying effect" is "medium", and the processing load is also "medium" during the period of the "first performance" after the "flying effect", the period of the "second performance" until the "second MC" and "costume change effect", the period of the "second performance" from the "costume change effect" to the "double-ego effect" is also "medium", and the period of the "second performance" after the "double-ego effect" is also "medium".

Next, regarding each period in the virtual live in Example 2, the viewpoint images that can be viewed on the viewer terminal 300 will be explained. During the period during which participation in the live virtual space is possible, from virtual participation until the start of the "pre-show speech," only the viewer avatar viewpoint images can be displayed.

When the "pre-show speech" begins, only the virtual camera viewpoint image, specifically the viewpoint image of virtual camera C1, can be displayed. In this way, the viewpoint image during the "pre-show speech" is limited to the virtual camera viewpoint image even though it is before the start of the virtual live performance. This is because the pre-show speeches also include explanations of points to note about the virtual live performance, and the purpose is to prevent viewer users from not viewing the pre-show speech taking place on virtual stage G due to the viewer avatar viewpoint image being displayed on the viewer terminal 300.

As described below, these virtual camera viewpoint images of the "introduction" are generated by the distribution server computer 100 and distributed to each viewer terminal 300, so they can be displayed even on viewer terminals 300 that have not yet completed downloading the advance data.

For viewer terminals 300 for which the download of advance data has been completed at the above-mentioned advance data DL completion check timing, during the period of the "first performance" up to the "first MC" and the "flying performance", as well as the period after the "flying performance", viewer avatar viewpoint video can be displayed in addition to the virtual camera viewpoint video.

On the other hand, for viewer terminals 300 that have not completed downloading the advance data at the above-mentioned advance data DL completion check timing, the viewer avatar viewpoint video cannot be generated accurately, so the virtual camera viewpoint video generated by the distribution server computer 100 is distributed, making it possible to display only the virtual camera viewpoint video during the entire virtual live performance, and it is not possible to display the viewer avatar viewpoint video.

In addition, among the images of the "Flying Performance", "Costume Change Performance", and "Double Ego Performance", which have a "high" processing load on the distribution server computer 100, the virtual camera viewpoint images of virtual camera C1 for the "Costume Change Performance" and "Double Ego Performance", like the virtual camera viewpoint images during the second performance period other than these "Costume Change Performance" and "Double Ego Performance", are generated on each viewer terminal 300 rather than on the distribution server computer 100, thereby reducing the processing load on the distribution server computer 100 for generating the virtual camera viewpoint images of virtual camera C1 for these "Costume Change Performance" and "Double Ego Performance".

The "flying performance" period is also a period in which the processing load on the distribution server computer 100 is high, so it is considered to be generated on the viewer terminal 300, similar to the "costume change performance" and "doppelganger performance". In this case, detailed trajectory data of the movement trajectory of the virtual camera moving in the air within the virtual live venue must be distributed in advance to each viewer terminal 300 to generate the video. Therefore, in Example 2, the video for the "flying performance" is generated and distributed on the distribution server computer 100 side, but the present invention is not limited to this, and the video for the "flying performance" may also be generated on each viewer terminal 300 in order to reduce the processing load on the distribution server computer 100.

Here, we briefly explain the participant user table used in the distribution server computer 100 of Example 2 using Figure 23.

The participant user table is a data table used to identify the participation status of viewer users in a virtual live event and the download status of advance data for each virtual live event registered in the event data shown in FIG. 7, and as shown in FIG. 23, it is a table that can store data such as participation identification data (participation identification flag) that can identify the participation status of the participant (viewer user) in association with a participant ID (viewer user ID), the participation area of the participant (viewer user), and advance data DL identification data (advance data DL identification flag) that can identify the download status of advance data.

In addition, the participant ID field will contain all the viewer user IDs of the people who are expected to participate and are described in the file for each participant group stored in the event data in association with the virtual live event.

When a participant ID (viewer user ID) is registered, the participation identification data (participation identification flag) is registered as "0", which corresponds to a non-participating state, and is updated to "1", indicating participation, when the participation of each participant (viewer user) is permitted by authentication. Therefore, the participation identification data (participation identification flag) makes it possible to identify, among the viewer users who are scheduled to participate, those who are participating and those who were unable to participate for some reason.

The participation area data is data that can identify whether the participation area of each participant (viewer user) is the third area or the fourth area, and is registered when the participation of each participant (viewer user) is permitted by authentication.

The pre-data DL identification data (pre-data DL identification flag) is updated based on a check as to whether the download (DL) of the pre-data is complete at the pre-data DL completion check timing described above, and if it is complete, it is updated from "0", indicating incomplete, to "1", indicating complete. Therefore, by using this pre-data DL identification data (pre-data DL identification flag), it is possible to easily identify all participants (viewer users) who have not completed the download (DL) of their pre-data among the participants (viewer users) who are virtually participating.

Next, the flow of downloading advance data by a viewer user virtually participating in a virtual live performance venue in Example 2 will be described with reference to FIG. 22.

First, the viewer user performs an operation to join the virtual live show that he or she plans to virtually participate in, for example, by selecting a participation menu or operating a specific key on the keyboard to which the participation operation is assigned, in a viewer program (viewing application) running on the viewer terminal 300.

By executing this participation operation, a participation request including the viewer user ID and authentication data for authenticating the viewer user is sent from the viewer terminal 300 to the distribution server computer 100.

The distribution server computer 100 that receives the participation request executes a participant authentication process on the condition that it is a period during which participation in the live virtual space is possible. In the participant authentication process, it is determined whether the viewer user is a prospective participant of the corresponding virtual live depending on whether the viewer user ID included in the received participation request is registered as a participant ID in the participant user database corresponding to the virtual live to be participated in, and the viewer user is authenticated by comparing the authentication data described in the authentication information file identified from the authentication information file data stored in association with the viewer user ID in the viewer user data shown in FIG. 9 with the authentication data included in the participation request.

If the viewer user is a prospective participant and has completed authentication using the authentication data, the participant identification data (participation identification flag) in the participant user table is updated to "1", which corresponds to "participation".

Then, the distribution server computer 100 transmits (responds to) advance data DL instruction information to the viewer terminal 300 that sent the participation request, in order to have the viewer terminal 300 download advance data corresponding to the virtual live event to be participated in. By receiving the advance data DL instruction information, the viewer terminal 300 can confirm that participation has been permitted.

In response to receiving the advance data DL instruction information, the viewer terminal 300 identifies the advance data to be downloaded and transmits an advance data DL request to the distribution server computer 100, requesting the download of the identified advance data.

In response to receiving this advance data DL request, the distribution server computer 100 executes the advance data transmission process.

In this advance data transmission process, the requested advance data is transmitted to the viewer terminal 300 that sent the advance data DL request. As mentioned above, the advance data includes virtual space data (basic data) of the virtual live venue, avatar data of the performer avatar 1, and avatar data of other viewer users who are already participating virtually.

In addition, in the advance data transmission process, the avatar data of the viewer user who sent the advance data DL request is additionally distributed as advance data to the viewer terminals 300 other than the viewer terminal 300 that sent the advance data DL request, but of viewer users whose participation has already been confirmed by the participant data table. As a result, the avatar data of all viewer users who are virtually participating in the virtual live show is distributed in advance to and stored in the viewer terminals 300 of each viewer user who is virtually participating in the virtual live show, along with the virtual space data (basic data) and avatar data of the performer avatars.

Then, at the pre-data DL completion check timing described above (see FIG. 21), it is checked whether or not all of the pre-data has been downloaded, and if the pre-data download has been completed, the pre-data DL identification data (pre-data DL identification flag) in the participant user table is updated from "0", indicating incomplete, to "1", indicating complete.

As described above, for participants (viewer users) whose advance data download has not been completed and whose advance data DL identification data (advance data DL identification flag) remains at "0," indicating incomplete download, only the virtual camera viewpoint image can be displayed.

Next, we will use Figures 24 to 27 to explain how the subject generating the various viewpoint videos displayed on the viewer terminal 300 changes during each period of the virtual live broadcast.

First, during the live virtual space participation period when only the viewer avatar viewpoint video can be displayed on the viewer terminal 300, as shown in FIG. 24, the virtual space data already stored in the viewer terminal 300 is updated sequentially by receiving the advance data based on the virtual space update data transmitted from the distribution server computer 100, including the performer avatar action data for generating the actions of the performer avatar 1 that perform actions linked to the actions of the performer user, and the viewer viewpoint video generation process is executed in the viewer terminal 300 based on the updated virtual space data to generate the viewer avatar viewpoint video, which is output (displayed) by the video output process. Note that although the performer terminal 200 is omitted in FIG. 24, the virtual space update data is also transmitted to the performer terminal 200 as shown in FIG. 10 in Example 1.

Furthermore, during each of the periods of "introduction", "opening performance", and "flying performance" in which only virtual camera viewpoint images can be displayed on the viewer terminals 300, as shown in FIG. 25, the distribution server computer 100 transmits virtual space update data only to the performer terminals 200 in the virtual space update data distribution process. Then, following the virtual space update data distribution process, a virtual camera viewpoint image generation process is executed in which a virtual camera viewpoint image is generated, and a virtual camera viewpoint image data distribution process is executed in which the image data of the virtual camera viewpoint image generated in the virtual camera viewpoint image generation process is distributed to all viewer terminals 300 participating in the virtual live.

The virtual camera viewpoint video data distributed from the distribution server computer 100 in these virtual camera viewpoint video data distribution processes is displayed by executing a video output process on the viewer terminal 300 that receives the virtual camera viewpoint video data.

In other words, as shown in FIG. 27, during the periods of "pre-show," "opening performance," and "flying performance," in which only virtual camera viewpoint images can be displayed, the virtual camera viewpoint images are generated by the distribution server computer 100.

In addition, during the performance period, excluding the MC period and the flying performance period, during which the viewer terminal 300 can select and display either the viewer avatar viewpoint image or the virtual camera viewpoint image, as shown in FIG. 26, the distribution server computer 100 executes a virtual space update data distribution process, and virtual space update data is distributed to the viewer terminal 300 of each viewer user participating in the virtual live performance, thereby executing the virtual space update process on the viewer terminal 300.

Then, in the viewer terminal 300, it is determined whether or not the viewer avatar viewpoint video has been selected as the video to be displayed. If the viewer avatar viewpoint video has been selected, the viewer avatar viewpoint video is generated by executing the viewer viewpoint video generation process, similar to the flow shown in FIG. 24, and the generated viewer avatar viewpoint video is displayed by the video output process.

On the other hand, when the viewer avatar viewpoint video is not selected as the video to be displayed, i.e., when the virtual camera viewpoint video is selected as the video to be displayed, the virtual camera viewpoint video generation process is executed in the viewer terminal 300 to generate the virtual camera viewpoint video, and the generated virtual camera viewpoint video is displayed by the video output process.

The viewer user may be allowed to select which of the virtual cameras C1, C3, or C4 the virtual camera viewpoint image generated in this virtual camera viewpoint image should be, or the distribution server computer 100 may instruct all viewer terminals 300.

In other words, as shown in FIG. 27, during the performance period excluding the MC period and the flying performance period when either the virtual camera viewpoint video or the viewer avatar viewpoint video can be displayed on the viewer terminal 300, the virtual camera viewpoint video is generated on the viewer terminal 300. In this way, during the period when either the virtual camera viewpoint video or the viewer avatar viewpoint video can be selected and displayed on the viewer terminal 300, switching between the virtual camera viewpoint video and the viewer avatar viewpoint video is performed at the viewer user's choice, so that switching between these videos can be performed without problems such as video interruption and while preventing an increase in the processing load on the distribution server computer 100 that accompanies switching between these videos, compared to when the virtual camera viewpoint video data is displayed by distribution from the distribution server computer 100.

Meanwhile, during each of the periods of "pre-show", "opening performance", and "flying performance", when only the virtual camera viewpoint video can be displayed on the viewer terminal 300, the virtual camera viewpoint video is generated only on the distribution server computer 100 and distributed to each viewer terminal 300. In this way, during the period when only the virtual camera viewpoint video can be displayed on the viewer terminal 300, the virtual camera viewpoint video is generated only on the distribution server computer 100, so that it is possible to prevent unnecessary increases in the processing load of the entire system due to the process of generating the virtual camera viewpoint video being executed redundantly on the viewer terminal 300.

Here, the situation when the viewpoint video is switched on the viewer terminal 300 will be explained with reference to FIG. 28, taking as an example the period of "flying effect," during which only the virtual camera viewpoint video can be displayed.

For example, as shown in FIG. 28, on a viewer terminal 300 on which viewer avatar viewpoint video is displayed during the "first MC" period and the first performance period up to the "flying performance", if the distribution server computer 100 starts distributing the virtual camera viewpoint video (viewpoint video from a virtual camera moving in the air) from the start of the "flying performance" period in which only the virtual camera viewpoint video (viewpoint video from a virtual camera moving in the air) can be displayed, the virtual camera viewpoint video (viewpoint video from a virtual camera moving in the air) will not be displayed on the viewer terminal 300 during the period from the start of these distributions until the virtual camera viewpoint video is displayed, which could result in a problem in which the viewer is unable to view the first video of the "flying performance".

For this reason, as shown in FIG. 28, at the start of a "flying performance" in which the viewpoint image is forcibly changed from the viewer avatar viewpoint image to the virtual camera viewpoint image (viewpoint image from a virtual camera moving in the air), the distribution server computer 100 of Example 2 transmits a BF advance notification to the viewer terminal 300 at a specific timing prior to the start of the "flying performance", thereby notifying the viewer terminal 300 of the distribution start (buffering start) timing for starting advance distribution of the virtual camera viewpoint image (viewpoint image from a virtual camera moving in the air) before the start of the flying performance period.

In response to receiving the BF advance notification, the viewer terminal 300 identifies the timing at which distribution of the virtual camera viewpoint video (viewpoint video from a virtual camera moving in the air) will begin from the distribution server computer 100, and begins buffering the received virtual camera viewpoint video (viewpoint video from a virtual camera moving in the air) from the time that this timing occurs.

The timing at which distribution of these virtual camera viewpoint images (viewpoint images from virtual cameras moving in the air) begins, i.e., the timing at which buffering begins, can be determined appropriately taking into account differences in the image storage capabilities of the viewer terminals 300, but can be set to a period equivalent to several tens of frames of image, for example.

In this way, the distribution server computer 100 of Example 2 starts generating and distributing virtual camera viewpoint video (viewpoint video from a virtual camera moving in the air) of the flying performance before the flying performance period starts, and the viewer terminal 300 buffers the distributed virtual camera viewpoint video, so that when the flying performance period starts, the video can be instantly changed from the viewer avatar viewpoint video that was displayed before the flying performance period started to the virtual camera viewpoint video, thereby preventing the occurrence of a problem in which the viewer is unable to view the initial video of the "flying performance".

Furthermore, when the period of the "flying performance" ends, because the viewer avatar viewpoint video was selected before the start of the "flying performance", the viewpoint video switches from the flying performance's virtual camera viewpoint video (viewpoint video from the virtual camera moving in the air) to the viewer avatar viewpoint video in response to the end of the period of the "flying performance". However, even in this case, if the viewer avatar viewpoint video is generated and displayed from the end of the period of the "flying performance", there is a risk that a malfunction will occur in which the viewer avatar viewpoint video is not displayed on the viewer terminal 300 during the period until the viewer avatar viewpoint video is generated and displayed, just as at the start of the "flying performance".

For this reason, even when the period of these "flying effects" ends, the distribution server computer 100 of Example 2 notifies the viewer terminal 300 in advance of the end timing of the "flying effects" period when distribution of the virtual camera viewpoint video (viewpoint video from a virtual camera moving in the air) will end, by sending an end TM (timing) advance notification to the viewer terminal 300 as shown in FIG. 28.

In response to receiving the end TM (timing) advance notification, the viewer terminal 300 identifies in advance the timing at which the distribution of the virtual camera viewpoint video (viewpoint video from a virtual camera moving in the air) from the distribution server computer 100 will be stopped, and starts the viewer viewpoint video generation process at a specified timing prior to that timing, thereby pre-generating the viewer avatar viewpoint video.

The distribution server computer 100 starts distributing the virtual space update data from the time of sending the end TM (timing) advance notification, and the viewer terminal 300 executes virtual space processing to update the virtual space data using the received virtual space update data, so that at the specified timing mentioned above, a viewer avatar viewpoint image that reflects the situation of the virtual live venue at that time can be generated.

In this way, the distribution server computer 100 of Example 2 transmits an end TM (timing) advance notification to the viewer terminal 300 before the end of the flying performance period, enabling the viewer terminal 300 to generate the viewer avatar viewpoint video in advance before the end of the flying performance period, thereby preventing the viewer terminal 300 from failing to display the viewer avatar viewpoint video.

Note that in Figure 28, the "flying performance" has been explained as an example, but similar control is also performed during the "pre-show" and "opening performance" periods, when the viewpoint image is forcibly changed from the viewer avatar viewpoint image to the virtual camera viewpoint image.

As described above, in the virtual space content distribution system of Example 2, there are multiple viewer users, and each viewer user is assigned a viewer terminal 300. When the viewer terminal 300 displays the viewer avatar viewpoint video, which is a viewpoint video of the virtual live venue (virtual space) corresponding to the viewer avatar, the viewer avatar viewpoint video is generated based on the virtual space update data (virtual space data) transmitted from the distribution server computer 100. Since the generation of these viewer avatar viewpoint videos, which have a large processing load, is performed by the viewer terminal 300, more viewer users can participate in the virtual live.

The virtual space content of Example 2 is content that allows a performer user to participate in a virtual live venue as performer avatar 1, and is content in which performer avatar 1 performs musical actions (performances) such as singing and dancing in the virtual live venue, thereby enhancing the interest of the virtual space content.

In addition, since the virtual space update data (virtual space data) of Example 2 includes performer avatar movement data for generating the movements of performer avatar 1, the movements of performer avatar 1 can be accurately reflected in the viewer avatar viewpoint image generated in the viewer terminal 300.

In addition, the common viewpoint video displayed in common on each viewer terminal 300 in Example 2, such as the "pre-show speech," "opening performance," and "pre-show performance" videos, is generated in the distribution server computer 100 and distributed to each viewer terminal 300, so that it is possible to prevent unnecessary increases in the processing load of the entire system due to duplicate processing being executed on each viewer terminal 300.

In addition, in the viewer terminal 300 of Example 2, during the performance period (specific period) other than the "MC period" and "flying performance" periods, during which either the viewer avatar viewer's ...

In addition, during special periods such as "costume change performance" and "doppelganger performance" in which the processing load on the distribution server computer 100 in Example 2 is high, each viewer terminal 300 also generates and displays virtual camera viewpoint images (common viewpoint images), thereby preventing the processing load on the distribution server computer 100 from becoming temporarily extremely high.

In addition, the distribution server computer 100 of Example 2 transmits advance data including basic data of the virtual live venue (virtual space) and basic data of the performer avatar 1 and viewer avatars who will virtually participate in the virtual live venue to each viewer terminal 300 before the virtual live starts, thereby preventing the occurrence of a problem in which the viewer avatar's viewpoint image is not displayed properly at the start of the virtual live.

In addition, the distribution server computer 100 of Example 2 transmits virtual camera viewpoint video (common viewpoint video) to viewer terminals 300 for which it cannot confirm that they have completed receiving the advance data. In this way, it is possible to prevent the processing load on the distribution server computer 100 from increasing excessively due to the transmission of basic data, etc. after the start of the virtual live broadcast, while also preventing viewer users from being unable to watch the virtual live broadcast.

<Modification of the second embodiment>
FIG. 29 is a flowchart showing the process of generating a viewer's viewpoint video in the modification 2-1 of the second embodiment.

In the above-mentioned Example 2, a form was shown in which the viewer avatar viewpoint images including the movements of the performer avatar 1 were always generated uniformly, but the present invention is not limited to this, and these viewer avatar viewpoint images may be generated in multiple forms with different processing loads in the image generation, depending on the selection of the viewer user at the viewer terminal 300.

In other words, as shown in Example 1, the viewer terminal 300 used by the viewer user may be a stationary personal computer (PC) with high processing power, or a smartphone terminal or tablet terminal with relatively lower processing power than a personal computer (PC). It is assumed that terminals with different processing powers will be used. If the viewer avatar viewpoint video is always generated uniformly on terminals with different processing powers, the viewer avatar viewpoint video cannot be generated properly on a terminal with low processing power, and there is a risk that a mismatch will occur between the viewer avatar viewpoint video and the audio, or the viewer avatar viewpoint video will become a discontinuous and unnatural video. For example, when generating a viewer avatar viewpoint video including performer movements, the viewer may be allowed to select, as appropriate, according to the processing power of the terminal he or she uses, whether to generate a viewer avatar viewpoint video including high-definition performer movements, which requires a high processing load for generating the video, or to generate a viewer avatar viewpoint video including low-definition performer movements, which requires a low processing load for generating the video.

Specifically, the viewer viewpoint video generation process executed in the viewer terminal 300 may be as shown in the flow diagram of FIG. 29. In the viewer viewpoint video generation process, first, the performer movement generation mode selected by the viewer user in the viewer terminal 300 is identified (step S101). Then, it is determined whether the identified performer movement generation mode is a high processing load mode (step S102). If the high processing load mode is selected (Y in step S102), a viewer avatar viewpoint video including high-definition performer movements is generated (step S103). On the other hand, if the high processing load mode is not selected (N in step S102), a viewer avatar viewpoint video including low-definition performer movements is generated (step S104).

Note that, although FIG. 29 illustrates an example of two modes, a high processing load mode and a low processing load mode, the present invention is not limited to this, and three or more processing load modes may be used.

In addition, in the above-mentioned Example 2 (similar to Example 1), an example is given in which all actions, including the movement of the viewer avatar of the viewer user, are reflected in each viewer avatar by distributing virtual space update data, but the present invention is not limited to this. For example, in a case where the actions of these viewer avatars are actions that are normally impossible to perform but are possible when a special fee is paid, for example, an operation to perform a special action that allows the performer avatar 1 to appeal to the viewer avatar, is made possible on the viewer terminal 300, the special action is directed to the performer user and not to other viewer users, so that it is possible to prevent the virtual space update data including these special actions from being distributed to the viewer terminals 300 of other viewer users, thereby making it possible to prevent an excessive increase in the processing load of the distribution server computer 100 by making these special actions executable.

In this way, if virtual space update data including special actions is not distributed to the viewer terminals 300 of other viewer users, as shown in variant example 2-2 of FIG. 30, when viewer user A performs an operation corresponding to a special action, causing viewer user A's viewer avatar to perform a special action, virtual space update data A corresponding to viewer user A's viewer avatar performing a special action is transmitted from the distribution server computer 100 to viewer user A's viewer terminal 300, but virtual space update data B in which viewer user A's viewer avatar is not performing a special action is distributed to the viewer terminal 300 of viewer user B, who is different from viewer user A.

As a result, different virtual space update data is delivered from the delivery server computer 100 to the viewer terminal 300 of viewer user A and the viewer terminal 300 of viewer user B, which increases the degree of freedom of the virtual space update data and makes it possible to generate different viewer avatar viewpoint images on the viewer terminal 300 of viewer user A and the viewer terminal 300 of viewer user B.

In addition, viewer terminal 300 of viewer user A can generate and display an image of viewer avatar of viewer user A performing a special action different from the actions that viewer avatar of viewer user B is capable of performing, based on virtual space update data A that is different from virtual space update data B distributed to viewer terminal 300 of viewer user B, thereby increasing viewer user A's interest in the virtual live performance, which is virtual space content.

In addition, in the above-mentioned Example 2 (which is also the same as Example 1), the viewer avatars of the viewer users virtually participating in the fourth area are illustrated as normal avatars in which the avatar appearance differs for each viewer user, but the present invention is not limited to this. For example, in order to reduce the processing load when generating various viewpoint images in a virtual live performance, the viewer avatars of the viewer users virtually participating in the fourth area may be simplified avatars in which the avatar appearance is simplified.

Note that FIG. 30 shows an example of a special action that becomes possible when a special fee is paid, but for example, in the case of using a simplified avatar as described above, normal actions other than the movement of the simplified avatar, such as waving, clapping, and changing the direction of the viewer avatar, may be prevented from being included in the virtual space update data distributed to other viewer users, just like special actions, thereby further reducing the processing load on the distribution server computer 100.

In the above-mentioned first and second embodiments, an example was given of the video displayed on the viewer terminal 300 being automatically switched between viewer's viewpoint video and virtual camera's viewpoint video in accordance with time schedule data preset by the administrator, such as when performing the flying performance shown in FIG. 19, but the present invention is not limited to this, and the video displayed on the viewer terminal 300 may be suitably switched between virtual camera's viewpoint video and viewer's viewpoint video in response to the operation of the viewer user. In this way, the viewer user can watch the virtual live show from various viewpoints by their own operation, thereby increasing the interest of the virtual live show.

Furthermore, even if the image displayed on the viewer terminal 300 can be switched between virtual camera viewpoint image and viewer viewpoint image by the viewer user's operation as described above, the operation of the viewer terminal 300 by the viewer user (switching the image to be displayed) may be disabled during the execution period of a performance that is intended to leave a strong impression on the viewer user, such as a flying performance (see FIG. 19), to prevent the viewer user from missing the performance.

Next, the virtual space content distribution system of the third embodiment will be described below with reference to Figs. 31 to 56. Note that the same components as those of the first and second embodiments will be given the same reference numerals and the description will be omitted.

As shown in FIG. 31, the virtual space content distribution system of Example 3 has the same live progress content as Example 2. In particular, in Example 3, a performance is provided by the avatars of two performers, Performer A and Performer B, during the live performance.

Immediately after the start of the first MC period, performer avatar 1A of performer A, the first performer, appears, and then a solo performance by performer avatar 1A of performer A is performed as the first performance. During the second MC period, performer avatar 1B of performer B, the second performer, appears (the number of performer avatars increases from one to two), and then a joint performance by performer avatar 1A of performer A and performer avatar 1B of performer B is performed as the second performance.

In addition, in this embodiment 3, it is possible to execute a lighting effect in which the virtual aerial objects 50 to 52 are illuminated as part of the virtual live performance. In particular, lighting effects are not executed during the virtual live performance venue participation period and the pre-performance period (lighting effect light intensity: none), but the virtual aerial objects 50 to 52 are illuminated with a relatively weak lighting intensity during the performance opening period, the first MC period, and the second MC period (lighting effect light intensity: weak), and the virtual aerial objects 50 to 52 are illuminated with a stronger lighting intensity during the first performance period and the second performance period than during the performance opening period, the first MC period, and the second MC period (lighting effect light intensity: strong).

As shown in FIG. 31, in the period immediately up until the start of the first MC period, performer A's performer avatar 1A does not appear, and therefore there is no movement of performer A's performer avatar 1A (performer A's performer avatar movement: none); in the first MC period and the second MC period after performer A's performer avatar 1A appears, the movement of performer A's performer avatar 1A is relatively small (performer A's performer avatar movement: small); and in the first performance period and the second performance period, performer A's performer avatar 1A dances and sings, and therefore the movement of performer A's performer avatar is larger (more intense) than in other periods (performer A's performer avatar movement: large).

Similarly, the period before performer B's performer avatar 1B appears during the second MC period is a period in which performer B's performer avatar 1B makes no movements (performer B's performer avatar movements: none), the second MC period after performer B's performer avatar 1B appears is a period in which performer B's performer avatar 1B makes relatively small movements (performer B's performer avatar movements: small), and the second performance period is a period in which performer B's performer avatar 1B makes larger (more intense) movements than in other periods because performer B's performer avatar dances and sings together with performer A's performer avatar 1B (performer B's performer avatar movements: large).

The period during which participation in the virtual live venue is possible is a period during which multiple viewer terminals 300 connect to the distribution server computer 100 in sequence, and therefore is a period during which viewer avatars who virtually participate in the virtual live venue through these connections are gradually added and placed in the virtual space over time.

For this reason, as shown in Figures 31 and 32 (A), immediately after the start of the virtual live venue participation period, no light-emitting effects are being performed, no performer avatars 1A, 1B of each performer are being displayed, and the number of viewer avatars virtually participating in the virtual live venue is small, so there are few viewer avatars that require updates of location information, etc. in the distribution server computer 100, and the update processing load for updating the virtual space is relatively low. Also, for each viewer terminal 300, as described above, the number of viewer avatars virtually participating in the virtual live venue is small, so the drawing processing load for drawing the viewer avatars is relatively low.

If the number of viewer avatars virtually participating in the virtual live venue increases from this state, as shown in FIG. 32(B), the amount of viewer avatar information that is the subject of updating the virtual space data in the distribution server computer 100 increases, so the update processing load for updating the virtual space by the distribution server computer 100 increases. Also, as the number of viewer avatars virtually participating in the virtual live venue increases, the number of viewer avatars to be drawn in each viewer terminal 300 also increases, so the drawing processing load for drawing the viewer avatars also increases.

In addition, as the number of viewer avatars virtually participating in the virtual live venue increases, the number of viewers performing movement operations from the viewer terminal 300 also increases, as shown in Figures 31 and 33, and the number of viewer avatars moving around the virtual live venue may also increase.

A moving viewer avatar changes information such as its current location in the virtual live venue more frequently than a viewer avatar that is stationary in a specific location, so as the number of viewer avatars moving within the virtual live venue increases as shown in Figures 31 and 33, the update processing load for updating the virtual space by the distribution server computer 100 increases. At this time, if the number of moving viewer avatars increases in each viewer terminal 300, the amount of drawing due to the movement of each viewer avatar also increases, so the drawing processing load on the viewer terminal 300 also increases.

When performing light-emitting effects during each performance, as shown in Figures 31 and 34 (A), the delivery server computer 100 must calculate how the light hits each viewer avatar and performer avatar and the shadows they generate based on the current position of each avatar and the distance and angle from the light source (virtual aerial objects 50-52 in this embodiment 3), increasing the update load of the delivery server computer 100 for updating the virtual space. At this time, the drawing processing load also increases in each viewer terminal 300, as it must draw the virtual aerial objects 50-52 to emit light in accordance with the light-emitting effects, draw the way the light hits each viewer avatar differently, and draw shadows for each viewer avatar.

In addition, in this embodiment 3, the light emission intensity of the virtual aerial objects 50 to 52 may be increased as a light emission effect (see FIG. 31). When the light emission intensity is increased in this way, the processing load associated with the calculation processing of the way the light hits each viewer avatar and performer avatar and the shadows that are generated in the distribution server computer 100 described above further increases, which results in a further increase in the update load of the virtual space update by the distribution server computer 100, and similarly, the drawing processing load in each viewer terminal 300 also further increases.

When performer avatar 1A appears, as shown in FIG. 31 and FIG. 34(B), performer avatar information is added to the information to be updated in the virtual space data in the distribution server computer 100, so the update processing load of the virtual space update by the distribution server computer 100 increases. When performer avatar 1B appears, as shown in FIG. 31 and FIG. 34(B), performer avatar information of performer avatar 1B is added to the information to be updated in the virtual space data in the distribution server computer 100 in addition to the performer avatar information of performer avatar 1A, so the update processing load of the virtual space update by the distribution server computer 100 increases further. At the viewer terminal 300, the process of drawing performer avatar 1A of performer A starts from the timing of the appearance of performer avatar 1A, so the drawing processing load increases. When performer avatar 1B appears, the process of drawing performer avatar 1B of performer B starts in addition to the process of drawing performer avatar 1A, so the drawing processing load increases further. In this embodiment, after performer B's performer avatar 1B appears (after the number of performer avatars becomes two), the number of performer avatars does not decrease, so the process of drawing performer avatar 1A and the process of drawing performer avatar 1B continue until the end of the virtual live performance.

During each performance, as shown in Figures 31 and 35, in addition to increasing the light emission intensity of the virtual aerial objects 50-52 as a light emission effect, each performer avatar 1A, 1B performs a performance such as singing or dancing. Therefore, as described above, the processing load associated with the calculation processing of the light hitting and the shadows generated for each viewer avatar and performer avatar in the distribution server computer 100 further increases, and the update processing load of updating the virtual space by the distribution server computer 100 further increases. In addition, the drawing processing load also increases in each viewer terminal 300 for drawing the light hitting and shadows for each viewer avatar caused by illuminating the virtual aerial objects 50-52, and for drawing the performer avatars 1A, 1B.

In this embodiment 3, when the processing load on the distribution server computer 100 or the viewer terminal 300 increases due to an increase in the number of viewer avatars and performer avatars placed in the virtual live venue as described above, an increase in the light emission intensity of the light emission effects, an increase in the number of moving viewer avatars and performer avatars, an increase in the amount of movement of the performer avatars, etc., it is possible to suppress an excessive increase in the processing load on the entire equipment constituting the virtual space content distribution system, such as the distribution server computer 100 and the viewer terminal 300, by changing from a full avatar, which requires a relatively large processing load for displaying the display mode of the viewer avatar, to a simplified avatar, which requires a relatively small processing load for displaying.

<Time schedule data>
The virtual live event of Example 3, which proceeds as described above, is managed by the distribution server computer 100 based on time schedule data (TS data), as shown in Fig. 36. The time schedule data is stored in the storage 103 of the distribution server computer 100 in a form included in the event management program 110.

The time schedule data is data in a table format that stores the scheduled time of each timing, the time elapsed since the start of the live, the content of that timing, and the file name of the associated data related to that timing, in association with a timing ID that is assigned to each individual timing in the virtual live.

As shown in FIG. 36, these timings include the start and end timings of the opening performance and the performer entrance performance, the timings when performer avatars 1A and 1B appear during the performer entrance performance, as well as the start and end timings of each transition period and the timings when each viewer avatar is simplified (first viewer avatar simplified timing T1, second viewer avatar simplified timing T2, third viewer avatar simplified timing T3). Hereinafter, in this embodiment, these first viewer avatar simplified timing T1, second viewer avatar simplified timing T2, and third viewer avatar simplified timing T3 may be referred to simply as timings T1, T2, and T3.

Timing T1 is the timing immediately before the start of the first performance period when the light intensity of the light-emitting effects increases and the movements of performer avatar 1A become more intense, increasing the drawing processing load on the viewer terminal 300. Timing T2 is the timing immediately before the start of the second MC period when the drawing processing load on the viewer terminal 300 increases as performer avatar 1B is added. Timing T3 is the timing at the start of the second performance period when the light intensity of the light-emitting effects increases and the movements of performer avatars 1A and 1B become more intense, increasing the drawing processing load on the viewer terminal 300. These timings T1 to T3 are set in advance to change the viewer avatar from a full avatar to a simple avatar before the processing load on the viewer terminal 300 increases, as described above.

<Video generation during the period when live virtual space participation is possible>
Furthermore, during the live virtual space participation period in which only the viewer avatar viewpoint video can be displayed on the viewer terminal 300 in this embodiment 3, as shown in Figure 37, in the same manner as in embodiment 2, after avatar action operation processing is executed on the viewer terminal 300, and virtual space update processing and virtual space update data generation processing are executed on the distribution server computer 100, an avatar simplification determination process is further executed on the distribution server computer 100 to determine whether the viewer avatar should be changed from a full avatar to a simplified avatar.

After executing the avatar simplification determination process, the distribution server computer 100 executes the virtual space update data distribution process in the same manner as in Example 2, thereby transmitting the simplification determination value data, which is the result of the above-mentioned avatar simplification determination process, together with the virtual space update data to the viewer terminal 300.

Based on the received virtual space update data and simplification judgment value data, the viewer terminal 300 executes a virtual space update process, a viewer viewpoint video generation process, and a video output process. In particular, in this embodiment 3, the received simplification judgment value data is used to perform a process of drawing (generating) a viewer avatar as a full avatar or a simplified avatar in the viewer viewpoint video generation process, as will be described in detail later.

<Video production during the show, excluding MC periods and false start periods>
Furthermore, during the performance period excluding the MC period and the flying performance period when the viewer avatar viewpoint video and the virtual camera viewpoint video can be displayed on the viewer terminal 300 in this embodiment 3, as shown in Figure 38, after the viewer terminal 300 executes the avatar movement operation process, and the distribution server computer 100 executes the virtual space update process and the virtual space update data generation process, as in embodiment 2, the distribution server computer 100 further executes the avatar simplification determination process which determines whether or not to change the viewer avatar display from a full avatar to a simplified avatar. Note that this avatar simplification determination process is the same process as the avatar simplification determination process shown in Figure 38.

After executing the avatar simplification determination process, the distribution server computer 100 executes the virtual space update data distribution process in the same manner as in Example 2 to transmit the simplification determination value data, which is the result of the above-mentioned avatar simplification determination process, together with the virtual space update data to the viewer terminal 300.

The viewer terminal 300 executes a virtual space update process based on receiving the virtual space update data and the simplified judgment value data, a viewer viewpoint video generation process if a viewer avatar viewpoint video is selected as the video to be displayed, and a virtual camera designated video generation process if a virtual camera viewpoint video is selected as the video to be displayed. In particular, in this embodiment 3, the received simplified judgment value data is used to perform a process of drawing (generating) the viewer avatar as a full avatar or a simplified avatar in the viewer viewpoint video generation process and the virtual camera designated video generation process, as will be described in detail later.

<About friend registration>
Furthermore, in the virtual space content delivery system of the third embodiment, a viewer can register other viewers as his/her friends by operating the viewer terminal 300. The storage 303 of the viewer terminal 300 of the third embodiment stores a friend user data table (see FIG. 40) that stores the user IDs and nicknames of viewer users who have been registered as friends in advance.

As shown in FIG. 39, the viewer terminal 300 executes a viewer avatar selection process to select a specific viewer avatar from among multiple viewer avatars by accepting operation of the viewer's operation input device 321 during the period during which participation in the live virtual space is possible, and then executes a viewer avatar inquiry process to identify the file name of the selected viewer avatar (viewer avatar file name) and transmit viewer avatar inquiry information including the identified viewer avatar file name to the distribution server computer 100.

When the distribution server computer 100 receives viewer avatar inquiry information from the viewer terminal 300, it executes a viewer avatar identification process to identify the viewer user ID stored in association with the viewer avatar file name included in the viewer avatar inquiry information from the viewer user data shown in FIG. 9. The distribution server computer 100 then transmits viewer user ID information including the viewer user ID to the viewer terminal 300 that has transmitted the viewer avatar inquiry information.

When the viewer terminal 300 receives the viewer user ID information, it executes a friend addition operation process on the display device 310 to prompt the viewer to register the viewer (viewer user ID) of the selected viewer avatar as a friend by associating it with the viewer avatar file name.

At this time, if the viewer performs an operation to register a friend, a friend addition process is executed to store the viewer user ID and the viewer avatar file name in the friend user data table in association with each other, and if the operation to register a friend is not performed, the viewer terminal 300 executes a friend addition cancellation process to display on the display device 310 an indication that the friend registration will not be performed.

<Avatar simplification determination process>
Fig. 41 is a flow chart showing an example of the avatar simplification determination process shown in Fig. 38. In the avatar simplification determination process, the delivery server computer 100 first determines whether or not it is during the virtual live venue participation period (see Fig. 31) by referring to the time schedule data or the like (step S401). If it is not during the virtual live venue participation period (step S401; No), the process proceeds to step S407, and if it is during the virtual live venue participation period (step S401; Yes), the process further determines whether or not the participant number threshold reached flag is set (step S402). The participant number threshold reached flag is a flag indicating that the number of viewer avatars placed in the virtual live venue (the number of viewers watching the virtual live) has already reached a predetermined threshold.

If the participant number threshold reached flag is not set (step S402; Yes), the distribution server computer 100 proceeds to step S407. If the participant number threshold reached flag is not set (step S402; No), the distribution server computer 100 further refers to the participant user table shown in FIG. 23 to identify the number of participants in the live performance (the number of viewer avatars virtually participating in the virtual live performance venue) (step S403), and determines whether the identified number of participants in the live performance has reached the threshold (step S404).

The threshold value for the number of participants is a value determined from the number of full viewer avatars that can be drawn at one time on each viewer terminal 300 and the number of full viewer avatars that can be placed in the virtual live venue. For example, if the limit for the number of full viewer avatars that can be drawn at one time on each viewer terminal 300 and the limit for the number of full viewer avatars that can provisionally participate in the virtual live venue is 200, the administrator of the distribution server computer 100 may set the threshold value for the number of participants to 180, which is about 10% lower than the limit. The threshold value for the number of participants is not simply the number of viewer avatars to be drawn, but is a value determined solely by the processing load corresponding to the number of viewer avatars that can be drawn on the viewer terminal 300.

The threshold number of participants may be set to a value different from the above by the administrator of the distribution server computer 100 on the administrator terminal 150, taking into consideration the performance (drawing processing capability) of each viewer terminal 300, etc.

If the number of participants has not reached the threshold (step S404; No), proceed to step S407. If the number of participants in the live event has reached the threshold (step S404; Yes), the distribution server computer 100 adds 1 to the simplification determination value based on the fact that the rendering processing load of the viewer avatar in full avatar form on the viewer terminal 300 is approaching its limit (step S405), sets a participant number threshold reached flag (step S406), and proceeds to step S407. The simplification determination value, which will be described in detail later, is a value for specifying the range to be displayed as a simplified avatar with a smaller number of polygons that make up each viewer avatar displayed on each viewer terminal 300, and is set to "0" as the initial value.

In step S407, the distribution server computer 100 determines whether it is during the virtual live venue participation period or the MC period by referring to the time schedule data, etc. (step S407). If it is neither during the virtual live venue participation period nor the MC period (step S407; No), the process proceeds to step S413, and if it is during the virtual live venue participation period or the MC period (step S407; Yes), the process determines whether the moving number threshold reached flag has been set (step S408). The moving number threshold reached flag is a flag indicating that the number of viewer avatars moving within the virtual live venue as full avatars, among the viewer avatars placed in the virtual live venue, has already reached a predetermined threshold.

If the moving number threshold reached flag is set (step S408; Yes), proceed to step S413; if the moving number threshold reached flag is not set (step S408; No), the distribution server computer 100 determines the number of viewer avatars currently moving (the number of viewer terminals 300 being moved by each viewer) from the viewer avatar information received from each viewer terminal 300 (step S409).

In step S409, the distribution server computer 100 stores the position information of each viewer avatar within the virtual live venue for each unit time (e.g., 10 ms), compares the stored position information of each viewer avatar within the virtual live venue with the previously stored position information of each viewer avatar within the virtual live venue, and identifies the number of viewer avatars currently moving by tallying up the number of viewer avatars whose stored position information differs from that previously stored.

Then, the delivery server computer 100 determines whether the identified number of moving viewer avatars (the number of viewer avatars currently moving) has reached a threshold value (step S410). The threshold value for the number of moving viewer avatars in step S410 is a value determined from the number of moving viewer avatars in full avatars that can be rendered at one time on each viewer terminal 300. For example, if the limit for the number of moving viewer avatars in full avatars that can be rendered at one time on each viewer terminal 300 is 200, the threshold value for the number of moving viewer avatars can be set by the administrator of the delivery server computer 100 to 150, which is 25% lower than the limit. The threshold value for the number of moving viewer avatars is not simply the number of viewer avatars to be rendered, but is a value determined solely by the processing load corresponding to the number of viewer avatars that can be rendered on the viewer terminal 300.

The threshold value for the number of moving viewer avatars may be set by the administrator of the distribution server computer 100, etc., as appropriate, taking into account the performance (drawing processing capability) of each viewer terminal 300, etc.

If the number of moving viewer avatars has not reached the threshold value (step S410; No), proceed to S413. If the number of moving viewer avatars has reached the threshold value (step S410; Yes), the distribution server computer 100 adds 1 to the simplification judgment value based on the fact that the rendering processing load of the full avatar viewer avatars on the viewer terminal 300 is approaching its limit (step S411), and sets the moving number threshold reached flag and proceeds to step S413 (step S412).

In step S413, the distribution server computer 100 determines whether or not the current time is timing T1 immediately before the start of the first performance (step S413). If it is determined that the current time is timing T1 immediately before the start of the first performance (step S413; Yes), the process proceeds to step S416.

If it is determined that the current time is not timing T1 immediately before the start of the first performance (step S413; No), the distribution server computer 100 determines whether it is timing T2 when performer B's performer avatar 1B appears (step S414). If it is determined that the current time is timing T2 immediately before performer B's performer avatar 1B appears (step S414; Yes), the process proceeds to step S416.

If it is determined that the time is not timing T2 immediately before performer B's performer avatar 1B appears (step S414; No), the distribution server computer 100 determines whether the time is timing T3 immediately before the start of the second performance (step S415). If it is determined that the time is timing T3 immediately before the start of the second performance (step S415; Yes), it proceeds to step S416, and if it is determined that the time is not timing T3 immediately before the start of the second performance (step S415; No), it proceeds to step S417.

In step S416, the distribution server computer 100 increments the simplification determination value by 1 based on the fact that the rendering processing load on the viewer terminal 300 increases immediately after the above-mentioned timings T1, T2, and T3 are reached, and proceeds to step S417 (step S416).

In step S417, the distribution server computer 100 executes a composite simplification judgment process. For example, even if the threshold number of participants has not been reached in step S404, the threshold number of moving viewer avatars has not been reached in step S410, or it is not any of the timings T1 to T3 in steps S413 to S415, the composite simplification judgment process judges whether the combined processing load of the viewer avatar as a full avatar in the viewer terminal 300, the calculation of the shadow generation of each viewer avatar, the drawing processing load of the performer avatar, etc. has reached the limit of the processing load of the viewer terminal 300, and if this combined processing load has reached the limit of the processing load of the viewer terminal 300, the simplification judgment value may be increased by 1 in the same manner as in steps S405, S411, and S416.

In this embodiment 3, an example is given of a form in which the simplification judgment value is increased by 1 at timing T2 based on the fact that the number of performer avatars will immediately become 2 (performer avatar 1B appears in addition to performer avatar 1A). In other words, the threshold value for the number of performer avatars is set to "2", but this threshold value for the number of performer avatars is not simply the number of performer avatars to be drawn, but rather is a value determined by the processing load corresponding to the number of performer avatars that can be drawn on the viewer terminal 300.

The threshold number of performer avatars may be set by the administrator of the distribution server computer 100 to a value different from the above, taking into consideration the performance (drawing processing capability) of each viewer terminal 300. In particular, in this embodiment 3, if the threshold number of performer avatars is set to "1", the simplification judgment value will be incremented by 1 immediately before performer avatar 1A of performer A appears during the first MC period.

In the composite simplification judgment process of this embodiment 3, an example is given of a form in which it is judged whether the combined processing load of the drawing processing load of the viewer avatar as a full avatar in the viewer terminal 300, the calculation of the shadow generation of each viewer avatar, the drawing processing load of the performer avatar, etc. is approaching the limit of the processing load of the viewer terminal 300, but the present invention is not limited to this, and the composite simplification judgment process may add 1 to the simplification judgment value on the condition that the processing load of the viewer terminal 300 has reached a processing load different from the processing load corresponding to the threshold value of the number of participants or the threshold value of the number of moving viewer avatars.

After executing the process of step S417, the distribution server computer 100 only sets the transmission of the simplification determination value and ends the avatar simplification determination process (step S418).

<Viewer's viewpoint video generation process>
Fig. 42 is a flow chart showing an example of the viewer viewpoint video generation process shown in Fig. 38. In the viewer viewpoint video generation process, the viewer terminal 300 first generates a video of the performer avatar in the full avatar shown in Fig. 45 (A) (step S501). As shown in Fig. 45 (A), the performer avatars 1A and 1B include full avatars created based on the preferences of each performer (performer A and performer B in this embodiment 3) and simplified avatars that are a simplification of the full avatars. The simplified avatars are avatar models created with fewer polygons than the full avatars (number of polygons: performer's full avatar >performer's simplified avatar), and the rendering process load on each viewer terminal 300 is lighter than that of the full avatars.

Next, it is determined whether the simplification determination value is 0 (step S502). If the simplification determination value is 0 (step S502; Yes), images of both the viewer avatars of the viewers who are registered as friends and the viewer avatars of the viewers who are not registered as friends are generated as full avatars, and the process proceeds to step S509 (step S503). As shown in FIG. 45 (B), full avatars are provided as viewer avatars, created based on the preferences of each viewer. These full avatars as viewer avatars are avatar models created with the same number of polygons as the performer's simplified avatar (number of polygons: performer's simplified avatar ≒ viewer's full avatar), and the shape, color scheme, etc. differ for each viewer.

In addition, in this embodiment 3, an example is shown in which the performer's full avatar and the viewer's full avatar are created with approximately the same number of polygons, but the present invention is not limited to this, and the number of polygons in the performer's full avatar and the viewer's full avatar may be less than the number of polygons in the performer's full avatar, the number of polygons in the viewer's full avatar may be more than the number of polygons in the performer's simplified avatar, and the number of polygons in the performer's simplified avatar may be more than the number of polygons in the viewer's full avatar.

As shown in FIG. 45(B), simplified avatars are also provided as viewer avatars, which are created with fewer polygons than each viewer's full avatar. The simplified avatars as viewer avatars have a common form regardless of the form of each viewer's full avatar, and are created with fewer polygons than each viewer's full avatar (number of polygons: viewer's full avatar > viewer's simplified avatar), which means that the rendering processing load on each viewer terminal 300 is lighter than that of a full avatar.

Returning to FIG. 42, if the simplification judgment value is not 0 (step S502; No), the viewer terminal 300 further judges whether the simplification judgment value is 1 or not (step S504). If the simplification judgment value is 1 (step S504; Yes), it judges whether the friend viewer avatar identified flag is set or not (step S505a). The friend viewer avatar identified flag is a flag indicating that the viewer avatar file name existing in the friend user data table has been identified. If the friend viewer avatar identified flag is set (step S505a; Yes), the process proceeds to step S506. If the friend viewer avatar identified flag is not set (step S505a; No), the friend user data table shown in FIG. 40 is referenced, the viewer avatar data file name existing in the friend user data table is identified, and the identified viewer avatar data file name is stored (step S505b). The viewer terminal 300 also sets the friend viewer avatar identified flag and proceeds to step S506 (step S505c).

Then, in step S506, the viewer terminal 300 generates a full avatar image of the viewer avatar (the friend's viewer avatar) corresponding to the viewer avatar data file name identified in step S505 (step S506), while generating simplified avatar images of the viewer avatars (other people's viewer avatars) other than the viewer avatar corresponding to the viewer avatar data file name identified in step S505b (step S507), and proceeds to step S509.

Also, if the simplification determination value is not 1 (step S504; No), the viewer terminal 300 generates images of all viewer avatars as simplified avatars, regardless of whether the viewer is registered as a friend, since the simplification determination value is 2 or more (step S508), and proceeds to step S509.

In step S509, the viewer terminal 300 generates viewer perspective video other than the performer avatars 1A, 1B and the viewer avatar (step S509), and ends the viewer perspective video generation process.

As described above, in this embodiment 3, as shown in FIG. 44(A), the performer avatars 1A and 1B generated in the viewer viewpoint video generation process are always generated as full avatars regardless of the value of the simplification determination process. In other words, performer avatars 1A and 1B are always displayed as full avatars from the viewer avatar viewpoint of the viewer operating the viewer terminal 300.

In addition, among the viewer avatars, for viewer avatars of viewers who are registered as friends, if the simplification judgment value is 1 or less, a full avatar image is generated, whereas if the simplification judgment value is 2 or more, a simplified avatar image is generated. In other words, from the viewpoint of the viewer avatar of the viewer operating the viewer terminal 300, a viewer avatar who is registered as a friend is displayed as a full avatar if the simplification judgment value is 1 or less, whereas if the simplification judgment value is 2 or more, a simplified avatar image is generated.

Furthermore, for viewer avatars of viewers who are not registered as friends, if the simplification determination value is 0, a full avatar image is generated, whereas if the simplification determination value is 1 or greater, a simplified avatar image is generated. In other words, from the viewpoint of the viewer avatar of the viewer operating the viewer terminal 300, viewer avatars who are not registered as friends are displayed as full avatars only if the simplification determination value is 0, whereas if the simplification determination value is 1 or greater, a simplified avatar image is generated.

<Virtual camera viewpoint video generation process (viewer terminal)>
Fig. 43 is an example of a flowchart showing the virtual camera viewpoint video generation process (the virtual camera viewpoint video generation process executed by the viewer terminal 300) shown in Fig. 38. In the virtual camera viewpoint video generation process, the viewer terminal 300 first generates the video of the performer avatars 1A and 1B as full avatars (step S511), and determines whether the simplification determination value is 0 (step S512). If the simplification determination value is 0 (step S512; Yes), the viewer terminal 300 generates the video of the viewer's own viewer avatar operating the viewer terminal 300 and all other viewer avatars regardless of whether they are registered as friends (step S513), and proceeds to step S523.

If the simplification judgment value is not 0 (step S512; No), it is further judged whether the simplification judgment value is 1 or not (step S514). If the simplification judgment value is 1 (step S514; Yes), it is judged whether the friend viewer avatar specified flag is set or not (step S515a). If the friend viewer avatar specified flag is set (step S515a; Yes), it proceeds to step S516, and if the friend viewer avatar specified flag is not set (step S515a; No), it refers to the friend user data shown in FIG. 40 and specifies the viewer avatar data file name present in the friend user data (step S515b). In addition, the viewer terminal 300 sets the friend viewer avatar specified flag and proceeds to step S516 (step S515c).

Then, the viewer terminal 300 generates full avatar images of the viewer's own viewer avatar operating the viewer terminal 300 and the viewer avatar corresponding to the viewer avatar data file name identified in step S515b (the friend's viewer avatar) (step S516), while generating simplified avatar images of the viewer's own viewer avatar operating the viewer terminal 300 and the viewer avatar corresponding to the viewer avatar data file name identified in step S515 (other people's viewer avatars) (step S517), and proceeds to step S523.

If the simplification judgment value is not 1 (step S514; No), it is further judged whether the simplification judgment value is 2 or not (step S518). If the simplification judgment value is 2 (step S518; No), the viewer terminal 300 generates an image of the viewer avatar of the viewer operating the viewer terminal 300 as a full avatar (step S520), while generating an image of the viewer avatar of a viewer other than the viewer's own viewer user as a simplified avatar (step S521), and proceeds to step S523.

If the simplification judgment value is not 2 (step S518; No), the viewer terminal 300 determines that the simplification judgment value is 3 or greater, generates images of the viewer's own viewer avatar operating the viewer terminal 300 and all other viewer avatars, regardless of whether they are registered as friends, using simplified avatars (step S522), and proceeds to step S523. In step S523, the viewer terminal 300 generates other virtual camera viewpoint images and ends the virtual camera viewpoint image generation process (step S523).

As described above, in this embodiment 3, as shown in FIG. 44(B), the performer avatars 1A and 1B generated by the viewer viewpoint video generation process of the viewer terminal 300 are always generated as full avatars regardless of the value of the simplification determination process. In other words, from the viewer avatar viewpoint of the viewer operating the viewer terminal 300, the performer avatars are always displayed as full avatars.

In addition, among the viewer avatars, when the simplification judgment value is 2 or less, a full avatar image is generated for the viewer's own viewer avatar operating the viewer terminal 300, whereas when the simplification judgment value is 3 or more, a simplified avatar image is generated. In other words, from the virtual camera viewpoint, the viewer's own viewer avatar operating the viewer terminal 300 is displayed as a full avatar when the simplification judgment value is 2 or less, whereas when the simplification judgment value is 3 or more, a simplified avatar image is generated.

In addition, for viewer avatars of viewers who are registered as friends, if the simplification judgment value is 1 or less, a full avatar image is generated, whereas if the simplification judgment value is 2 or more, a simplified avatar image is generated. In other words, from the virtual camera viewpoint, the viewer avatar of a viewer who is registered as a friend is displayed as a full avatar if the simplification judgment value is 1 or less, whereas if the simplification judgment value is 2 or more, a simplified avatar image is generated.

Furthermore, for viewer avatars of viewers who are not registered as friends, if the simplification determination value is 0, a full avatar image is generated, whereas if the simplification determination value is 1 or greater, a simplified avatar image is generated. In other words, from the virtual camera viewpoint, viewer avatars of viewers who are not registered as friends are displayed as full avatars only if the simplification determination value is 0, whereas if the simplification determination value is 1 or greater, a simplified avatar image is generated.

In this embodiment 3, as shown in Figures 38, 42, and 43, an example is given in which each viewer avatar in the viewer viewpoint video and virtual camera viewpoint video generated in the viewer terminal 300 is changed from a full avatar to a simplified avatar in response to an increase in the drawing processing load in the viewer terminal 300. However, the present invention is not limited to this, and as shown in Figure 25, each viewer avatar in the virtual camera viewpoint video generated in the distribution server computer 100 may also be changed from a full avatar to a simplified avatar. In this case, each viewer avatar may be changed from a full avatar to a simplified avatar in response to an increase in the drawing processing load in the distribution server computer 100.

Furthermore, since it is only necessary to prevent an excessive increase in the processing load of the entire virtual space content distribution system of the present invention, the viewer avatar in the viewer viewpoint image and the virtual camera viewpoint image generated by the viewer terminal 300 may be changed from a full avatar to a simplified avatar in response to an increase in the drawing processing load in the distribution server computer 100, or the viewer avatar in the virtual camera viewpoint image generated by the distribution server computer 100 may be changed from a full avatar to a simplified avatar in response to an increase in the drawing processing load in the viewer terminal 300.

In this embodiment 3, as described above, an example is given of a form in which the viewer avatar is changed from a full avatar to a simplified avatar in the viewer terminal 300 in response to the processing load corresponding to the number of avatars that can be drawn in the viewer terminal 300. However, the present invention is not limited to this, and the viewer avatar may be changed from a full avatar to a simplified avatar simply based on exceeding a threshold value for the number of participants, a threshold value for the number of moving viewer avatars, or a threshold value for the number of performer avatars.

Next, the changes in the display of performer avatars 1A, 1B and viewer avatars within the virtual live venue will be explained using as an example the display from the viewpoint of virtual camera C2 on the viewer terminal 300 of the viewer operating viewer avatar 11. In addition, below, viewer avatars 12, 13, and 14 will be explained as avatars of viewers who have been registered as friends by the viewer operating viewer avatar 11, and the other viewer avatars will be explained as avatars of viewers who have not been registered as friends by the viewer operating viewer avatar 11.

In this embodiment 3, we will explain the changes in the display of performer avatars 1A, 1B and viewer avatars from the viewpoint of virtual camera C2. However, we will not explain the display of performer avatars 1A, 1B and viewer avatars from the viewer's viewpoint (the viewpoint from viewer avatar 11) because it is the same as from the viewpoint of virtual camera C2, except that viewer avatar 11 is not displayed.

In addition, in this embodiment 3, since the composite simplification judgment process (step S417) is executed in the avatar simplification judgment process shown in FIG. 41, the viewer avatar does not necessarily change from a full avatar to a simplified avatar when the number of participants reaches a threshold value, when the number of moving viewer avatars reaches a threshold value, or at any of the times T1 to T3. However, to avoid complicating the explanation, the following explanation uses an example in which the viewer avatar changes from a full avatar to a simplified avatar when the number of participants reaches a threshold value, when the number of moving viewer avatars reaches a threshold value, or at times T1 to T3.

First, as shown in FIG. 46, in the case where neither the threshold value for the number of participants nor the threshold value for the number of moving viewer avatars is reached within the virtual live venue, performer avatar 1A of performer A is displayed as a full avatar from the time when he appears, which is immediately after the start of the first MC period, and performer avatar 2B of performer B is displayed as a full avatar from the time when he appears during the second MC period. Note that both performer avatar 1A of performer A and performer avatar 1B of performer B are maintained as full avatars without changing to simple avatars.

Meanwhile, from the start of display during the virtual live venue participation period until timing T1, immediately before the start of the performance (first performance) by performer A's performer avatar 1A, all viewer avatars are displayed as full avatars (see FIG. 49(A)). From timing T1, the illuminating intensity of the lighting effects increases from the start of the first performance immediately following, and the movements of performer avatar 1A become more intense, increasing the load of the rendering process on the viewer terminal 300. In response to this, the simplification determination value is updated from 0 to 1, and the viewer avatars of viewers who are not registered as friends by the viewer operating viewer avatar 11 begin to be displayed as simplified avatars (see FIG. 49(B)).

Furthermore, immediately after timing T2, performer avatar 1B appears; in other words, as the number of performer avatars increases from 1 to 2, the rendering processing load on the viewer terminal 300 increases, and the simplification determination value is updated from 1 to 2, so that simplified avatars begin to be displayed as viewer avatars for viewers who have been registered as friends by the viewer operating viewer avatar 11 (see FIG. 49 (C)).

Then, immediately after timing T3, the light intensity of the light-emitting effects increases and the second performance begins, in which the movements of performer avatars 1A, 2B of performers A and B become more intense. In response, the simplification judgment value is updated from 2 to 3, and the viewer avatar 11 itself begins to be displayed as a simplified avatar (see Figure 49 (D)).

As shown in FIG. 47, when a determination is made as to whether the participant number threshold has been reached within a virtual live venue, performer A's performer avatar 1A is displayed as a full avatar from the time he appears, which is immediately after the start of the first MC period, and performer B's performer avatar 1B is displayed as a full avatar from the time he appears during the second MC period. Note that both performer A's performer avatar 1A and performer B's performer avatar 1B remain displayed as full avatars without changing to simplified avatars.

On the other hand, from the time when display begins during the virtual live venue participation period until it is determined that the number of participants threshold has been reached during the virtual live venue participation period, all viewer avatars are displayed as full avatars (see FIG. 49(A)). However, if it is determined that the number of participants threshold has been reached during the virtual live venue participation period, the simplified judgment value is updated from 0 to 1 at the timing of this reaching judgment, and simplified avatars begin to be displayed as viewer avatars for viewers who are not registered as friends by the viewer operating the viewer avatar 11 (see FIG. 49(B)).

From timing T1, the first performance begins, during which the light intensity of the light-emitting effects increases and the movements of the performer avatar 1A become more intense. As a result, the simplification determination value is updated from 1 to 2 in response to an increase in the drawing processing load on the viewer terminal 300, and simplified avatars begin to be displayed as viewer avatars for viewers who have been registered as friends by the viewer operating the viewer avatar 11 (see Figure 49 (C)).

Then, immediately after timing T2, performer avatar 1B appears; in other words, as the number of performer avatars increases from 1 to 2, the rendering processing load on the viewer terminal 300 increases, and the simplification determination value is updated from 2 to 3, and the viewer avatar 11 itself begins to be displayed as a simplified avatar (see Figure 49 (D)).

As shown in FIG. 48, when a threshold-reaching determination is performed for the number of participants and the number of moving viewer avatars in a virtual live venue, performer A's performer avatar 1A is displayed as a full avatar from the time he appears, which is immediately after the start of the first MC period, and performer B's performer avatar 1B is displayed as a full avatar from the time he appears during the second MC period. Note that both performer A's performer avatar 1A and performer B's performer avatar 1B remain displayed as full avatars without changing to simple avatars.

On the other hand, from the time when display begins during the virtual live venue participation period until it is determined that the number of participants threshold has been reached during the virtual live venue participation period, all viewer avatars are displayed as full avatars (see FIG. 49(A)). However, if it is determined that the number of participants threshold has been reached during the virtual live venue participation period, the simplified judgment value is updated from 0 to 1 at the timing of this reaching judgment, and simplified avatars begin to be displayed as viewer avatars for viewers who are not registered as friends by the viewer operating the viewer avatar 11 (see FIG. 49(B)).

In addition, if it is determined that the number of moving viewer avatars during the period during which participation in the virtual live venue is possible has reached a threshold value, the simplification determination value is updated from 1 to 2 at the time of reaching the threshold value, and simplified avatars will begin to be displayed as viewer avatars for viewers who have been registered as friends by the viewer operating the viewer avatar 11 (see Figure 49 (C)).

Then, immediately after timing T1, the first performance begins, during which the light intensity of the light-emitting effects increases and the movements of the performer avatar 1A become more intense. As a result, the rendering processing load on the viewer terminal 300 increases, and the simplification determination value is updated from 2 to 3, and the viewer avatar 11 itself begins to be displayed as a simplified avatar (see Figure 49 (D)).

In this embodiment 3, even after the simplification judgment value is updated to 3 and all viewer avatars start to be displayed as simplified avatars, the simplification judgment value may be updated to 4, 5, etc., but even if the simplification judgment value is updated to 4 or more, all viewer avatars will not change further from the simplified avatars.

As described above, in this embodiment 3, as shown in Figures 41 to 44 and Figures 46 to 49, the number of displayed performer avatars increases to two, which increases the processing load for drawing the performer avatars; the movement of the performer avatars becomes larger (more intense) with the start of the first performance period or the second performance period, which increases the processing load for updating the virtual space and the processing load for drawing the performer avatars; the light emission intensity of the light emission effects increases based on the start of the first performance period or the second performance period, which increases the processing load for drawing the virtual space and each avatar; and the number of viewer avatars virtually participating in the virtual live venue (the number of participants in the live performance) exceeds a threshold number, which increases the processing load for updating the virtual space and the processing load for viewing By changing the viewer avatars in the virtual live venue from full avatars to simplified avatars based on the occurrence of either of the following (the fulfillment of a specific condition of the present invention): an increase in the rendering processing load of the viewer avatars; or an increase in the update processing load of the virtual space and the rendering processing load of the viewer avatars due to the number of moving viewer avatars (number of people moving) among the viewer avatars virtually participating in the virtual live venue exceeding a threshold value, the processing load on the viewer terminal 300 for displaying (drawing) these viewer avatars is reduced, thereby preventing an excessive increase in the processing load on the entire virtual space content distribution system due to the fulfillment of the specific condition.

In addition, in this embodiment 3, time schedule data for managing the progress of the virtual live is stored in the storage 103 of the distribution server computer 100, and the distribution server computer 100 is able to identify the timing immediately before the processing load increases from the time schedule data using the avatar simplification determination process shown in FIG. 41, so that the timing at which the processing load increases can be accurately identified as timings T1 to T3 before the processing load actually increases.

In particular, in this embodiment 3, the viewer avatar is changed from a full avatar to a simple avatar from times T1 to T3 before the timing when the rendering processing load of the performer avatar increases as the number of performer avatars displayed increases to 2, the timing when the movement of the performer avatar becomes larger (more intense) as the first performance period or the second performance period begins, causing the virtual space update processing load and the rendering processing load of the performer avatar to increase, and the timing when the rendering processing load of the virtual space and each avatar increases as the light emission intensity of the light emission effect increases based on the start of the first performance period or the second performance period. This prevents the processing load required for these processes from overlapping and causing an excessive increase in the processing load of the entire virtual space content distribution system.

In this embodiment 3, the time schedule data is stored in the storage 103 of the distribution server computer 100, and the distribution server computer 100 changes the viewer avatar in the viewer terminal 300 from a full avatar to a simplified avatar based on the timings T1 to T3 identified. However, the present invention is not limited to this. The time schedule data may be stored in the storage 303 of the viewer terminal 300, and the viewer terminal 300 itself may identify the timings T1 to T3 from the time schedule data and change the viewer avatar from a full avatar to a simplified avatar based on the identification.

In addition, in this embodiment 3, an example is given of a form in which the viewer avatar is changed from a full avatar to a simplified avatar from timing T1 to T3 just before the drawing processing load increases, but the present invention is not limited to this, and if the viewer terminal 300 has sufficient drawing processing capacity, the viewer avatar may be changed from a full avatar to a simplified avatar at the timing when the drawing processing load increases.

In addition, as shown in FIG. 49, the viewer avatar of the viewer operating the viewer terminal 300 (viewer avatar 11 in the example of FIG. 49) is eventually changed from a full avatar to a simplified avatar and displayed as a virtual camera-specified image, thereby further preventing an excessive increase in processing load.

In addition, in this embodiment 3, the viewer avatar of a viewer operating another viewer terminal 300 is given priority over the viewer's own viewer avatar (viewer avatar 11 in the example of Figure 49) operating the viewer terminal 300, and changes from a full avatar to a simplified avatar, thereby effectively preventing an excessive increase in processing load.

In this embodiment 3, the viewer avatars are divided into the viewer avatar of the viewer who operates the viewer terminal 300 (viewer avatar 11 in the example of FIG. 49), the viewer avatars of the viewers who are registered as friends by the viewer (viewer avatars 12, 13, and 14 in the example of FIG. 49), and the viewer avatars of the viewers who are not registered as friends by the viewer. The priority of changing from a full avatar to a simplified avatar is set to the highest for the viewer avatar of the viewer who is not registered as a friend, and the lowest for the viewer avatar of the viewer who operates the viewer terminal 300 (viewer avatar 11 in the example of FIG. 49) (priority of changing from a full avatar to a simplified avatar: viewer avatar of the viewer who is not registered as a friend > viewer avatar of the viewer who is registered as a friend > viewer avatar of the viewer who operates the viewer terminal 300). However, the present invention is not limited to this, and the priority of changing from a full avatar to a simplified avatar may be other than the above, and the priority of changing from a full avatar to a simplified avatar may not be set between the viewer avatars.

In addition, in this embodiment 3, as shown in Figures 45 to 48, full avatars and simple avatars are provided for both the performer avatars 1A, 1B and the viewer avatars, but while the full avatars change to simple avatars as the drawing processing load of the viewer terminal 300 increases, only the full avatars are displayed as the performer avatars 1A, 1B regardless of the increase in the drawing processing load of the viewer terminal 300. In other words, during the virtual live performance, simple avatars, which require a small drawing processing load to display as the performer avatars 1A, 1B, are never displayed, so it is possible to prevent a decrease in interest in the live performance due to the display of simple avatars as the performer avatars 1A, 1B.

In addition, in this embodiment 3, an example is shown in which a full avatar and a simple avatar are provided as the performer avatars 1A and 1B, but the present invention is not limited to this, and since there is no opportunity to display the simple avatars as the performer avatars 1A and 1B in the first place, it is also possible to provide only full avatars as the performer avatars 1A and 1B and not provide simple avatars.

In addition, in this embodiment 3, a performer terminal 200 that can be operated by the performer is provided as a device constituting the virtual space content distribution system, and the movements of performer avatars 1A and 1B that immediately reflect the performer's operations can be distributed as virtual space content. However, the present invention is not limited to this, and the movements of performer avatars 1A and 1B that have been filmed or created in advance may also be distributed as virtual space content.

In addition, in this embodiment 3, as shown in FIG. 41, based on the number of viewer avatars installed in the virtual live venue or the number of viewer avatars moving around reaching a predetermined threshold, the distribution server computer 100 univocally determines that the drawing processing load on the viewer terminal 300 is approaching its limit, and changes the viewer avatar on the viewer terminal 300 from a full avatar to a simplified avatar. However, the present invention is not limited to this, and in reality, it is highly unlikely that each viewer uses the same viewer terminal 300. Therefore, even if the distribution server computer 100 determines that the drawing processing load on the viewer terminal 300 is approaching its limit, if the viewer terminal 300 still has sufficient drawing processing capacity remaining to draw the viewer avatar, performer avatar, etc., the viewer terminal 300 may continue to display the viewer avatar as a full avatar.

In addition, in this embodiment 3, an example is given of a form in which the distribution server computer 100 determines that the drawing load of the viewer terminal 300 will increase and changes the viewer avatar for the viewer terminal 300 from a full avatar to a simplified avatar when the number of viewer avatars virtually participating in the virtual live venue exceeds a threshold number, when the number of moving viewer avatars among the viewer avatars virtually participating in the virtual live venue exceeds a threshold number, when the light intensity of the light-emitting effect increases, when the movements of the performer avatar become larger (more intense), or when the number of performer avatars increases; however, the present invention is not limited to this, and the viewer terminal 300 itself may determine that the drawing processing load of the viewer terminal 300 will increase and change the viewer avatar from a full avatar to a simplified avatar when the number of viewer avatars virtually participating in the virtual live venue exceeds a threshold number, when the number of moving viewer avatars among the viewer avatars virtually participating in the virtual live venue exceeds a threshold number, when the light intensity of the light-emitting effect increases, when the movements of the performer avatar become larger (more intense), or when the number of performer avatars increases. In this case, the viewer watching the virtual live using the viewer terminal 300 may operate the viewer terminal 300 to set a threshold number for the number of viewer avatars virtually participating in the virtual live venue, or a threshold number for the number of viewer avatars that are moving.

In addition, in this embodiment 3, a form has been exemplified in which the viewer avatars of viewers who are not friends of the viewer user using the viewer terminal 300 are preferentially displayed as simplified avatars over the viewer avatars of viewers who are friends and have a strong relationship with the viewer user using the viewer terminal 300, but the present invention is not limited to this. Blood relatives such as parents and siblings of the viewer user using the viewer terminal 300, and related people who belong to the same organization (e.g., school, company, club, etc.) as the viewer user using the viewer terminal 300, etc. may be registered in advance using a procedure similar to that shown in FIG. 39, and the viewer avatars of viewers who are not blood relatives or related people may be preferentially displayed as simplified avatars. In other words, the concept of "other viewer users with whom each viewer user has a strong relationship" in this embodiment 3 may include not only friends, but also blood relatives of the viewer user himself or herself and related people who belong to the same organization.

In addition, in this embodiment 3, an example was given of changing the viewer avatar from a full avatar to a simplified avatar based on an increase in the processing load of the viewer terminal 300, but the present invention is not limited to this. A plurality of simplified avatars with different drawing loads may be provided as the viewer avatar, and when the processing load of the viewer terminal 300 increases, a simplified avatar according to the drawing capability of each viewer terminal 300 may be displayed as the viewer avatar.

<Modification of the third embodiment>
The present invention has been described above based on each embodiment and modified examples, but the specific configuration is not limited to the above-mentioned embodiments and modified examples, and the present invention also includes modifications and additions that do not deviate from the gist of the present invention.

For example, in the above-mentioned Example 3, although both full avatars and simplified avatars are prepared as performer avatars 1A and 1B, a simplified avatar is not displayed as performer avatars 1A and 1B. However, the present invention is not limited to this, and as a variant example 3-1, a case may be provided in which performer avatars 1A and 1B are displayed as simplified avatars, or a period may be provided in which each performer can select whether to display performer avatars 1A and 1B as a full avatar or a simplified avatar by operating the performer terminal 200.

For example, as shown in FIG. 50, during periods other than the performance period, it is possible to select whether performer avatar 1A of performer A and performer avatar 1B of performer B are displayed as full avatars or simplified avatars by operation from each performer terminal 200. Also, during the performance period, if the performance period is a performance period in which only performer avatar 1A of performer A sings or dances, performer avatar 1A of performer A may be displayed as a full avatar while performer avatar 1B of performer B may be displayed as a simplified avatar, thereby reducing the processing load for displaying performer avatar 1B of performer B on the viewer terminal 300, and if the performance period is a performance period in which only performer avatar 1B of performer B sings or dances, performer avatar 1B of performer B may be displayed as a full avatar while performer avatar 1A of performer A may be displayed as a simplified avatar, thereby reducing the processing load for displaying performer avatar 1A of performer A on the viewer terminal 300.

In addition, if the performance period is a performance period in which both performer A's performer avatar 1A and performer B's performer avatar 1B are singing or dancing, both performer A's performer avatar 1A and performer B's performer avatar 1B may be displayed.

In addition, in the above-mentioned Example 3, an example was given in which only one type of simplified avatar is provided as the simplified avatar of the viewer avatar, regardless of whether the performer is singing or dancing, but the present invention is not limited to this, and as shown in FIG. 51 as Variation 3-2, the type of the viewer's simplified avatar may differ depending on the performer avatar performing the performance. In particular, during non-performance periods such as the virtual live venue participation period, opening and closing performance period, and MC period, the viewer avatar may be displayed in the type shown in the above-mentioned Example 3, while during each performance period, the viewer avatar may be displayed as a simplified avatar in a type that is set in advance for each performer depending on the performer avatar performing singing or dancing. In this way, the type of the simplified avatar as the viewer avatar changes depending on the performer avatar, thereby increasing the interest of the virtual live.

When the appearance of the viewer avatar is to be varied according to the performer, as shown in Figures 52 and 53, in the advance data transmission process, the distribution server computer 100 identifies the performers for whom a simple avatar has already been set for the viewer (performers for whom data of a simple avatar for the viewer is already stored in the storage 103 in the distribution server computer 100) from the advance data download request received from the viewer terminal 300 (step S601), and then sets up the data for the viewer's simple avatar for each identified performer (step S602) and sets up the data for other advance data for transmission from the received advance data download request (step S603), and transmits the advance data for which transmission settings have been made to the viewer terminal 300 (step S604).

By doing this, the viewer terminal 300 can receive data of simplified viewer avatars corresponding to each performer (performer avatars 1A, 1B) as part of the advance data from the distribution server computer 100, and can display simplified avatars of a form corresponding to each performer avatar 1A, 1B as viewer avatars during the performance period.

In addition, in this modified example 3-2, an example is given of a form in which a simplified avatar of a form corresponding to each performer avatar 1A, 1B is displayed as the viewer avatar, but the present invention is not limited to this, and during the performance period of a performer who does not have a simplified avatar set for the viewer as pre-data (a performer for whom data of a simplified avatar for the viewer is not stored in advance in storage 103 in the distribution server computer 100), a simplified avatar of the same form as during non-performance periods may be displayed as the viewer avatar, as shown in FIG. 51.

In addition, in this modified example 3-2, an example was given of a form in which a simplified avatar in a form corresponding to each performer avatar 1A, 1B is displayed as the viewer avatar, but the present invention is not limited to this, and the simplified avatar displayed as the viewer avatar may be changed over time. For example, as shown in FIG. 54 as modified example 3-3, a heart-shaped character may be displayed as the viewer's simplified avatar from the start of the performance until a predetermined timing S, while a star-shaped character may be displayed as the viewer's simplified avatar after timing S.

In addition, in the above-mentioned Example 3, an example was given in which the viewer avatar and the performer avatars 1A and 1B each have two types of avatars: a full avatar and a simplified avatar with a smaller drawing processing load than the full avatar. However, the present invention is not limited to this, and as a modified example 3-4, multiple simplified avatars with different drawing processing loads (for example, a first simplified avatar with a smaller drawing processing load than the full avatar and a second simplified avatar with a smaller drawing processing load than the first simplified avatar) may be provided as the viewer avatar and the performer avatars 1A and 1B.

In this way, when multiple simple avatars with different rendering processing loads are provided, the state of the viewer avatar may be changed in stages from full avatar, to first simple avatar, to second simple avatar, etc. so that the rendering processing load of the viewer terminal 300 decreases, under conditions such as the number of viewer avatars virtually participating in the virtual live venue exceeding a threshold number, the number of moving viewer avatars among the viewer avatars virtually participating in the virtual live venue exceeding a threshold number, or one of timings T1 to T3 is reached.

In addition, in the above-mentioned Example 3, an example was given of a form in which a viewer avatar is changed from a full avatar to a simplified avatar on the condition that the number of moving viewer avatars among those virtually participating in the virtual live venue exceeds a threshold number, but the present invention is not limited to this, and as a modified example 3-5, a viewer avatar may be changed from a full avatar to a simplified avatar on the condition that the number of viewer avatars that are not moving but are performing actions such as dancing or jumping among those virtually participating in the virtual live venue exceeds a threshold number. In other words, the "performer avatars operating in a virtual space" in the above-mentioned Example 3 may include performer avatars that are performing actions other than moving within the virtual live venue, in addition to performer avatars that are moving within the virtual live venue.

In addition, in the above-mentioned Example 3, an example was given of a form in which each viewer avatar is first displayed as a full avatar, and then changed to a simplified avatar when a specific condition is met, but the present invention is not limited to this. As a modified example 3-6, if it is expected that the number of viewer avatars virtually participating in the virtual live venue will exceed a predetermined threshold number, each viewer avatar may be displayed as a simplified avatar from the beginning.

In addition, in the third embodiment, the number of displayed performer avatars increases to two, which increases the processing load for drawing the performer avatars; the movement of the performer avatars becomes larger (more intense) with the start of the first performance period or the second performance period, which increases the processing load for updating the virtual space and the processing load for drawing the performer avatars; the light emission intensity of the light emission effects increases based on the start of the first performance period or the second performance period, which increases the processing load for drawing the virtual space and each avatar; the number of viewer avatars virtually participating in the virtual live venue (the number of participants in the live performance) exceeds a threshold number, which increases the processing load for updating the virtual space and the processing load for drawing the viewer avatars; In the present invention, a specific condition is when the number of moving viewer avatars (number of people moving) exceeds a threshold, causing an increase in the update processing load of the virtual space and the rendering processing load of the viewer avatar, and an example is given in which the viewer avatar is changed from a full avatar to a simple avatar when any of these specific conditions are met, but the present invention is not limited to this, and as a variation 3-7, the specific conditions in the present invention may be only some of the above, or may include conditions other than those listed above as long as they increase the processing load on the distribution server computer 100 and viewer terminal 300 that constitute the virtual space content distribution system of the present invention.

In the third embodiment, as shown in Figs. 41 to 43, the viewer avatar is changed from a full avatar to a simplified avatar using the same threshold value (threshold value for the number of participants and the threshold value for the number of moving viewer avatars) when generating a viewer viewpoint video and when generating a virtual camera viewpoint video. However, the present invention is not limited to this. As a modified example 3-8, the viewer avatar may be changed from a full avatar to a simplified avatar using different threshold values when generating a viewer viewpoint video and when generating a virtual camera viewpoint video. For example, as shown in Fig. 13, when generating a virtual camera C1 video, unlike when generating a virtual camera C2 video, a virtual camera C3 video, a virtual camera C4 video, and each viewer viewpoint video, almost only the performer avatar is drawn, so the threshold value for the number of participants and the threshold value for the number of moving viewer avatars are set high, while in the virtual camera C2 video, the virtual camera C3 video, the virtual camera C4 video, and each viewer viewpoint video, each viewer avatar is actively displayed in addition to the performer avatar, so the threshold value for the number of participants and the threshold value for the number of moving viewer avatars may be set high.

In addition, even in the case of viewer-perspective images, if the viewer avatar's line of sight is not directed toward the virtual stage G, it is more difficult to render a performer avatar as viewer-perspective image than when the viewer avatar's line of sight is directed toward the virtual stage G, and the rendering process load caused by rendering the performer avatar is less likely to be high. Therefore, the threshold for the number of participants and the threshold for the number of moving viewer avatars may be set higher than when the viewer avatar's line of sight is directed toward the virtual stage G.

In addition, in the above-mentioned Example 3, an example was given of a form in which, as the drawing processing load of the viewer terminal 300 increases, the viewer avatars are changed to simplified avatars in the following order: viewer avatars of viewers who are not registered as friends, viewer avatars of viewers who are registered as friends, and the viewer's own avatar. However, the present invention is not limited to this, and as a variant example 3-9, the viewer avatars may be changed to simplified avatars in the order of viewer avatars that are farthest from the viewer's own avatar each time the drawing processing load of the viewer terminal 300 increases.

Next, the virtual space content distribution system of the fourth embodiment will be described below with reference to Figs. 55 to 60. Note that the same components as those of the first and second embodiments will be given the same reference numerals and the description will be omitted.

As shown in Figures 55 and 56, the virtual space content distribution system of Example 4 provides, as viewer avatars, full avatars that have a relatively large rendering processing load and have different appearances for each viewer user (viewer terminal 300) to make each viewer avatar identifiable, and simplified avatars that have a smaller rendering processing load than the full avatars. In addition, two types of performer avatars are provided: a full avatar corresponding to performer A and a full avatar corresponding to performer B, and no simplified avatars are provided for performer avatars for performer A and performer B. Incidentally, the full avatars as performer avatars have a larger rendering processing load than the full avatars as viewer avatars.

In the virtual live event held in the virtual space content distribution system of this embodiment 4, as in the first and second embodiments, the event will proceed in the order of "live virtual space participation period," "pre-show speech," and "opening performance." After the "opening performance," the event will proceed as follows: "First MC" → "First performance" → "Second MC" → "Second performance" → "Third MC" → "Third performance."

In the "first MC", performer A's performer avatar talks alone and introduces the first performance, in the "second MC", performer B's performer avatar talks alone and introduces the second performance, and in the "third MC", performers A and B's performers avatars talk together and introduce the third performance. In addition, the "first performance" is a performance in which performer A's performer avatar sings and dances alone, while the "second performance" is a performance in which performer B's performer avatar sings and dances alone. In addition, the "third performance" is a performance in which performers A and B sing and dance together.

In the progress of such a virtual live performance, in this embodiment 4, the number of viewer avatars participating in the virtual live performance is small, lighting effects associated with the virtual live performance are not being performed, performer avatars are not appearing on the virtual stage G, etc., so the rendering processing load on the entire virtual space content distribution system is relatively low during the "live virtual space participation period," "pre-show speech," and "opening performance" periods, and viewer avatars are displayed (drawn) as full avatars, whereas from the first MC period onwards when performer avatars appear, viewer avatars are displayed (drawn) as simplified avatars.

More specifically, as shown in Fig. 55 and Fig. 56, during the "first MC" in which performer A's performer avatar talks alone and during the "first performance" in which performer A sings and dances alone, the viewer avatar is displayed as a simplified avatar in a display mode corresponding to performer A's performer avatar (heart-shaped mode α shown in Fig. 56(B)). During the "second MC" in which performer B's performer avatar talks alone and during the "second performance" in which performer B's performer avatar sings and dances alone, the viewer avatar is displayed as a simplified avatar in a display mode corresponding to performer B's performer avatar (star-shaped mode β shown in Fig. 56(B)). During the "third MC" in which performer A and performer B's performer avatars talk together and during the "third performance" in which performer A and performer B's performers sing together, the viewer avatar is displayed as a simplified avatar in a display mode different from the above-mentioned modes α and β (mode γ in a shape that combines a cone and a sphere shown in Fig. 56(B)).

In other words, in this Example 4, when the performer avatar of performer A singing or dancing alone is displayed on each viewer terminal 300, each viewer avatar is displayed as a simplified avatar of mode α, when the performer avatar of performer B singing or dancing alone is displayed on each viewer terminal 300, each viewer avatar is displayed as a simplified avatar of mode β, and when the performer avatar of performer A and the performer avatar of performer B singing or dancing together are displayed on each viewer terminal 300, each viewer avatar is displayed as a simplified avatar of mode γ.

In addition, in the virtual space content delivery system of this embodiment 4, as shown in FIG. 57, a viewer user can operate his/her own viewer terminal 300 to equip the viewer avatar with items that the viewer user has previously purchased, such as chemical lights, hats, shirts, etc. Details will be described later, but if the viewer avatar is a full avatar, all of these items are displayed as being equipped to the viewer avatar, but if the viewer avatar is a simplified avatar, the display position or display mode may change.

When equipping the viewer avatar with each item, as shown in FIG. 58, during the period during which participation in the live virtual space is possible, a shared item data request information transmission process is executed in which the viewer terminal 300 transmits purchased item data request information to the distribution server computer 100 to request item data that has been previously purchased. Note that the purchased item data request information includes the ID (viewer user ID) of the viewer user who operates the viewer terminal 300.

When the distribution server computer 100 receives the purchase item data request information, it references the viewer user data (see FIG. 9) based on the viewer user ID included in the purchase item data request information, and executes a purchase item data identification process to identify the item being purchased (purchased item). It then executes a purchase item data transmission process to transmit information on the identified purchase item data (purchased item data information) to the viewer terminal 300.

When the viewer terminal 300 receives the purchase item data information, it executes a purchase item display process that displays the purchased items on the display device 310 based on the received purchase item data. Then, as shown in FIG. 57, it executes a purchase item selection operation acceptance process that accepts a selection operation from the viewer user while the purchased items are displayed on the display device 310. At this time, the item selected by the viewer user's operation is worn by the viewer avatar.

Figure 59 is a flowchart showing the viewer's viewpoint video generation process in this embodiment 4. In this embodiment 4, the video generated in the viewer's viewpoint video generation process described below is displayed on the viewer terminal 300, thereby outputting a first performance by the performer avatar of performer A, a second performance by the performer avatar of performer B, and a third performance by the performer avatars of both performers A and B. In the viewer's viewpoint video generation process, the viewer terminal 300 first generates a video of the performer avatar with a full avatar (step S501), and determines whether the current time is during the live virtual space participation period, the pre-show period, or the opening performance period (step S502).

If it is during the live virtual space participation period, the pre-show period, or the opening performance period (step S502; Yes), an image of each viewer avatar is generated in full avatar form (step S503). In addition, an image of the chemilight is generated in a format corresponding to each full avatar (step S504), and an image of the shirt and hat is generated in a format corresponding to each full avatar (step S505).

On the other hand, if it is not during the live virtual space participation period, the pre-show introduction period, or the opening performance period (step S502; Yes), the viewer terminal 300 determines whether it is during the first MC period or the first performance period (step S506). If it is during the first MC period or the first performance period (step S506; Yes), the image of each viewer avatar is generated with a simplified avatar (mode α) corresponding to performer A (step S507), and if it is neither during the first MC period nor the first performance period (step S506; No), it further determines whether it is during the second MC period or the second performance period (step S508).

If it is during the second MC period or the second performance period (step S508; Yes), the image of each viewer avatar is generated as a simple avatar (mode β) corresponding to performer B (step S509), and if it is neither the second MC period nor the second performance period (step S508; No), the image of each viewer avatar is generated as an intermediate avatar (mode γ) corresponding to the third MC period or the third performance period (step S510).

After executing any one of the processes in step S507, step S509, or step S510, the viewer terminal 300 generates an image of the chemilight in a manner corresponding to the full avatar at a position corresponding to each simplified avatar (step S511), and generates an image of the shirt and hat in a manner corresponding to each simplified avatar.

After executing the processing of step S505 or step S512, the viewer terminal 300 executes processing to generate other viewer viewpoint images (step S513) and terminates the viewer viewpoint image generation processing.

Here, we will explain the position and display mode of each item when the viewer avatar is a full avatar and when it is a simplified avatar. As shown in FIG. 60, when the viewer avatar is a full avatar, the viewer avatar is displayed holding a chemilight in one arm, wearing a hat, and wearing a shirt.

On the other hand, when the viewer avatar is a simplified avatar, the vertical length of the simplified avatar is shorter than that of a full avatar, and the chemilights and hat are positioned lower than when the viewer is using a full avatar. The size of the chemilights does not change from when the viewer is using a full avatar, but the size of the hat is smaller than when the viewer is using a full avatar, to match the size of the head of the simplified avatar.

In addition, as shown in FIG. 60, simplified avatar modes α and β have arms and are displayed as if they are holding a chemilight in one arm, but simplified avatar mode γ does not have arms and is displayed with the chemilight floating to the side of the head.

In addition, in all of modes α to γ, the simplified avatar is set to have a smaller body proportion than the full avatar, so the shirt is not displayed.

As described above, in this embodiment 4, as shown in FIG. 60, even if the viewer avatar is changed from a full avatar to a simplified avatar, the appearance of the chemical lights themselves remains the same, so that it is possible to prevent a decrease in interest in the virtual live performance due to the viewer avatar being a simplified avatar.

In addition, as shown in FIG. 60, in this embodiment 4, when the viewer avatar changes from a full avatar to a simplified avatar, items such as hats and shirts that can be worn by the viewer avatar change to correspond to the appearance of the simplified avatar (a hat becomes smaller in size, and a shirt becomes hidden), thereby preventing the wearing of these items from becoming inappropriate for the simplified avatar.

In addition, the position of the chemilights that can be worn by the viewer avatars differs when the viewer avatar is a full avatar and when it is a simplified avatar, so it is possible to prevent a decrease in interest in the virtual live performance due to the chemilights not being held in an appropriate position according to the simplified avatar.

In addition, in this embodiment 4, as simplified avatars for the viewer avatars, there are provided mode α when the performer avatar performing on virtual stage G is the performer avatar of performer A, and mode β when the performer avatar performing on virtual stage G is the performer avatar of performer B. By changing the viewer avatar from a full avatar to a simplified avatar, the processing load related to display is reduced, while by changing the mode of the simplified avatar depending on whether the performer avatar performing on virtual stage G is the performer avatar of performer A or the performer avatar of performer B, variety can be added to the virtual live performance, and a decrease in interest due to the virtual live performance becoming monotonous can be prevented.

In addition, in this embodiment 4, when the viewer terminal 300 displays the performer avatar of performer A performing a performance, a simplified avatar of mode α is displayed as the viewer avatar, and when the viewer terminal 300 displays the performer avatar of performer B performing a performance, a simplified avatar of mode β is displayed as the viewer avatar. This reduces the processing load associated with display by using simplified avatars as the viewer avatars, while preventing a decrease in interest in the virtual live performance due to the mode of the simplified avatar being common regardless of the performer avatar.

In addition, in this embodiment 4, during the "third performance" in which the performer avatar of performer A and the performer avatar of performer B sing and dance together on the virtual stage G, the viewer avatar is displayed as a simplified avatar of mode γ, which is different from mode α corresponding to the "first performance" in which the performer avatar of performer A sings and dances alone and mode β corresponding to the "second performance" in which the performer avatar of performer B sings and dances alone, in response to the performer avatar of performer A singing and dancing together with the performer avatar of performer B. This can add variety to the virtual live performance and increase the interest in the "third performance", which is a joint performance in which the performer avatar of performer A singing and dancing together on the virtual stage G.

In addition, in this embodiment 4, during the "first performance" in which the performer avatar of performer A sings and dances alone, the viewer avatar is displayed as a simplified avatar of mode α corresponding to the performer avatar of performer A, and during the "second performance" in which the performer avatar of performer B sings and dances alone, the viewer avatar is displayed as a simplified avatar of mode β corresponding to the performer avatar of performer B, thereby preventing a decrease in interest in a virtual live performance that partially includes performances by multiple performer avatars.

In addition, as shown in FIG. 55, in this embodiment 4, each viewer avatar changes from a full avatar to a simplified avatar at a common timing (the start timing of the "first MC" period), so it is possible to prevent the viewer user from feeling a sense of inequality due to each viewer avatar changing from a full avatar to a simplified avatar at different times.

Also, as shown in FIG. 56(A), while a full avatar is provided for the performer avatar, a simplified avatar is not provided. This means that the performer avatar does not change to a simplified avatar at the same time that each viewer avatar changes from a full avatar to a simplified avatar, preventing a decrease in interest in the virtual live performance due to the performer avatar changing to a simplified avatar.

<Modification of the fourth embodiment>
In this embodiment 4, a plurality of performer avatars (performer avatar of performer A and performer avatar of performer B) are provided, and the appearance of the viewer avatar is made different depending on the performer avatar performing the performance on the virtual stage G, while the display appearance of the viewer avatar and some items displayed (chemilights in this embodiment 4) is common regardless of the performer avatar performing the performance on the virtual stage G. However, the present invention is not limited to this, and as a modified example 4-1, the display appearance of the chemilights may be made different depending on the performer avatar performing the performance on the virtual stage G. For example, as shown in Fig. 61, when the performer avatar of performer A performs a performance on the virtual stage G, the color (emission color) of the chemilights may be red, and when the performer avatar of performer B performs a performance on the virtual stage G, the color (emission color) of the chemilights may be blue.

By doing this, it is possible to add variety to the virtual live performance by changing the state of the chemilights depending on whether the performer avatar performing on virtual stage G is the performer avatar of performer A or the performer avatar of performer B, and it is possible to prevent the virtual live performance from becoming monotonous and causing a loss of interest.

Also, as shown in FIG. 61, when the display mode of the chemilights is changed depending on the performer avatar performing the performance on virtual stage G, the mode (color in the example shown in FIG. 61) of items other than the chemilights, such as hats, may also be changed depending on the performer avatar performing the performance on virtual stage G. In this way, the virtual live performance can be changed depending on the mode of the hat when the performer avatar performing the performance on virtual stage G is the performer avatar of performer A or the performer avatar of performer B, and a decrease in interest due to the virtual live performance being the responsibility of the performer can be prevented.

In addition, in this modified example 4-1, an example is given in which the color of an item is made different depending on the performer avatar performing on the virtual stage G, but the present invention is not limited to this, and the size or shape of an item may be made different depending on the performer avatar performing on the virtual stage G.

In addition, in the above-mentioned Example 4, when the viewer avatar is a full avatar or a simplified avatar of mode α or mode β, the viewer avatar is shown holding a chemilight in its arm. However, as shown in FIG. 62(A) to FIG. 62(C) as a modified example 4-2, for the full avatar or the simplified avatar of mode α or mode β, the viewer may operate the viewer's terminal 300 to swing the arm from side to side, so that the chemilight can move in accordance with the arm. In addition, when the viewer avatar is a simplified avatar of mode γ, as shown in FIG. 62(D), since the simplified avatar does not have arms, only the chemilight may be moved by the viewer's operation of the viewer's terminal 300. In this way, it is possible to prevent the effect of reducing the processing load (e.g., the drawing processing load) related to the display by making the viewer avatar a simplified avatar of mode γ from being effectively reduced by the movement of the arms.

In addition, in the above-mentioned Example 4, an example was given of a form in which a full avatar and a simple avatar are provided as viewer avatars in the virtual space content delivery system of the present invention, but the present invention is not limited to this, and the virtual space content delivery system of the present invention may also be provided with a friend registration function that allows a specific viewer user to be registered as a friend of the user.

As an example of such a friend registration function, as shown in FIG. 63 as variant example 4-3, the viewer terminal 300 executes a viewer avatar selection process to select a specific viewer avatar from among multiple viewer avatars by accepting operation of the viewer's operation input device 321 during the period during which participation in the live virtual space is possible, identifies the file name of the selected viewer avatar (viewer avatar file name), and executes a viewer avatar inquiry process to transmit viewer avatar inquiry information including the identified viewer avatar file name to the distribution server computer 100.

When the distribution server computer 100 receives viewer avatar inquiry information from the viewer terminal 300, it executes a viewer avatar identification process to identify the viewer user ID stored in association with the viewer avatar file name included in the viewer avatar inquiry information from the viewer user data shown in FIG. 9. The distribution server computer 100 then transmits viewer user ID information including the viewer user ID to the viewer terminal 300 that has transmitted the viewer avatar inquiry information.

When the viewer terminal 300 receives the viewer user ID information, it executes a friend addition operation process on the display device 310 to prompt the viewer to register the viewer user (viewer user ID) of the selected viewer avatar as a friend by associating it with the viewer avatar file name.

At this time, if the viewer performs an operation to register a friend, a friend addition process is executed to store the viewer user ID and the viewer avatar file name in the friend user data table in association with each other, and if the operation to register a friend is not performed, the viewer terminal 300 executes a friend addition cancellation process to display on the display device 310 an indication that the friend registration will not be performed.

In addition, in this modified example 4-3, a friend user data table (see FIG. 64) that stores the user IDs and avatar data of viewer users is created in advance in the storage 303 of the viewer terminal 300, and when the friend addition process is executed, the friend user data table is updated to associate the viewer user ID with the viewer avatar file name and additionally store them.

When providing a friend registration function as described above, as shown in FIG. 65, the color of the simplified avatar of a viewer user who has been registered as a friend may be made different from the color of the simplified avatar of other viewer users who have not been registered as friends, making it easy to identify which simplified avatar is the simplified avatar of a viewer user who has been registered as a friend.

In addition, in this modified example 4-3, an example is given of a form in which it is possible to easily recognize which simple avatar is the simple avatar of a viewer user who has registered as a friend by making the color of the simple avatar different depending on whether the viewer user is a friend of the viewer user or not, but the present invention is not limited to this. It is also possible to easily recognize which simple avatar is the simple avatar of a viewer user who has registered as a friend by making the shape of the simple avatar different depending on whether the viewer user is a friend of the viewer user or not, and it is also possible to easily recognize which simple avatar is the simple avatar of a viewer user who has registered as a friend by adding an object that follows the simple avatar.

In addition, in the above-mentioned Example 4, during the "third performance" in which the performer avatar of performer A and the performer avatar of performer B sing and dance together on the virtual stage G, a form was exemplified in which the viewer avatar is displayed as a simplified avatar of mode γ in response to the performer avatar of performer A and the performer avatar of performer B singing and dancing together; however, the present invention is not limited to this, and as shown in Figure 66 as variant example 4-4, during the "third performance" in which the performer avatar of performer A and the performer avatar of performer B sing and dance together on the virtual stage G, the viewer user may be allowed to select at the viewer terminal 300 whether to display his or her own viewer avatar as a simplified avatar of mode α corresponding to the performer avatar of performer A or a simplified avatar of mode β corresponding to the performer avatar of performer B. In this way, during the "third performance" in which the performer avatar of performer A and the performer avatar of performer B sing and dance together on virtual stage G, the viewer user can select a simplified avatar according to his or her own preference, thereby increasing the interest in the "third performance," which is a joint performance in which the performer avatar of performer A and the performer avatar of performer B sing and dance together on virtual stage G.

In the above-mentioned Example 4, during the "first performance" in which the performer avatar of performer A sings and dances alone on the virtual stage G, the viewer avatar is displayed as a heart-shaped simplified avatar of mode α, and during the "second performance" in which the performer avatar of performer B sings and dances alone on the virtual stage G, the viewer avatar is displayed as a star-shaped simplified avatar of mode β. However, the present invention is not limited to this, and the simplified avatars of mode α and mode β may include elements corresponding to the attributes of the corresponding performer avatar. For example, as shown in FIG. 67 as a modified example 4-5, by having the performer avatar of performer A wear a heart-shaped hair accessory, the simplified avatar of mode α may be an avatar that includes elements of the heart-shaped hair accessory worn by the performer avatar of performer A, and by having the performer avatar of performer B wear a star-shaped hair accessory, the simplified avatar of mode β may be an avatar that includes elements of the star-shaped hair accessory worn by the performer avatar of performer B. This makes it easier for viewer users to intuitively understand which performer avatar the simplified avatar in aspect α or aspect β corresponds to, preventing a decline in interest in the virtual live performance due to the viewer being confused as to which performer avatar the aspect of the simplified avatar as the viewer avatar corresponds to.

In addition, in this modified example 4-5, the shape of the simplified avatar as the viewer avatar is made the same as the shape of the hair ornament worn by the performer avatars of performer A and performer B, thereby making it easier for the viewer user to understand which performer avatar the appearance of each simplified avatar corresponds to. However, the present invention is not limited to this, and the color, pattern, etc. of the simplified avatar as the viewer avatar may be made the same as at least a part of the performer avatars of performer A and performer B, making it easier for the viewer user to understand which performer avatar the appearance of each simplified avatar corresponds to.

In the above-mentioned Example 4, when the performer avatar of performer A sings or dances alone on the virtual stage G, a simplified avatar of mode α is displayed as the viewer avatar, and when the performer avatar of performer B sings or dances alone on the virtual stage G, a simplified avatar of mode β is displayed as the viewer avatar. However, the present invention is not limited to this, and as shown in FIG. 68 as a modified example 4-6, when the performer avatar of performer A sings or dances alone in a first virtual live venue and the performer avatar of performer B sings or dances alone in a second virtual live venue different from the first virtual live venue, the mode of the simplified avatar as the viewer avatar participating in the first virtual live venue and the mode of the simplified avatar as the viewer avatar participating in the second virtual live venue may be different. In this way, the mode of the simplified avatar can be made different depending on the virtual live venue in which the viewer avatar participates, and it is possible to prevent a decrease in the interest of the virtual live.

In addition, in the above Example 4, an example was given of a form in which the viewer avatars are displayed as simplified avatars during the periods of "1st MC", "1st Performance", "2nd MC", "2nd Performance", "3rd MC", and "3rd Performance", but the present invention is not limited to this, and as a variant example 4-7, a period may be set in which the viewer avatars are displayed as full avatars during part of the periods of "1st MC", "1st Performance", "2nd MC", "2nd Performance", "3rd MC", and "3rd Performance".

In particular, if the viewer avatar is displayed as a full avatar during part of the "First Performance", "Second Performance", and "Third Performance", the appearance of the full avatar for a particular viewer avatar during the "First Performance", "Second Performance", and "Third Performance" periods may be made common. In this way, the appearance of the full avatar, which requires a large processing load to display, will be common regardless of the performer performing on the virtual stage G, preventing an increase in processing load and an increase in display data.

Next, the virtual space content distribution system of the fifth embodiment will be described below with reference to Figs. 69 to 79. Note that the same components as those of the first and second embodiments will be given the same reference numerals and the description will be omitted.

As shown in Figures 69 and 70, the virtual space content distribution system of Example 5 uses, as viewer avatars, full avatars that have a relatively large processing load (drawing processing load) for display and have different appearances for the corresponding viewer users (viewer terminals 300) so that each viewer avatar can be identified, and simplified avatars that have a smaller drawing processing load than the full avatars.

In the virtual live event held in the virtual space content distribution system of this embodiment 5, as in the first and second embodiments, the event will proceed in the order of "live virtual space participation period," "pre-show speech," and "opening performance." After the "opening performance," the event will proceed as follows: "First MC" → "First performance" → "Second MC" → "Second performance" → "Third MC" → "Third performance."

In the "First MC", "Second MC", and "Third MC", performer avatar 1 appears on virtual stage G and greets the audience through talk and introduces the next performance. The "First Performance", "Second Performance", and "Third Performance" are performances that include singing and dancing by the performer avatar (performer).

In this embodiment 5, when a virtual live performance progresses in this way, first, during the period during which participation in the live virtual space is possible, each viewer user can operate the viewer terminal 300 to allow a viewer avatar corresponding to the viewer terminal 300 to participate in the virtual live performance as a full avatar. Then, from the start timing of the "first MC" period, each viewer terminal 300 can change the viewer avatar, which had been a full avatar up until that point, to a simplified avatar, thereby reducing the processing load for displaying the viewer avatar.

In particular, in this embodiment 5, as shown in FIG. 75, the live virtual space participation period is a simplification target area designation period in which the viewer user can operate the viewer terminal 300 to select an area (simplification target area) in which the viewer avatar will be changed from a full avatar to a simplified avatar from the third and fourth areas of the virtual audience area. Note that if the viewer user does not designate a simplification target area during the live virtual space participation period in this embodiment 5, both the third and fourth areas will be areas not subject to simplification, and the viewer avatar will not be displayed as a simplified avatar.

The following describes a series of processes for selecting an area (simplification target area) in which the viewer avatar is changed from a full avatar to a simplified avatar from the third and fourth areas of the virtual audience area, with reference to Figures 71 to 79.

As shown in FIG. 71, during the period when participation in the live virtual space is possible, the viewer terminal 300 that has caused the viewer avatar to participate in the virtual live venue first executes a simplification target area designation operation process that receives an operation from the viewer user as to which area of the viewer avatar should be changed from a full avatar to a simplified avatar.

At this time, when the viewer terminal 300 receives an operation from the viewer user, it transmits simplification target area information, including the participant ID (viewer user ID) of that viewer user, to the distribution server computer 100.

When the distribution server computer 100 receives the simplification target area information from the viewer terminal 300, it identifies the participant ID (viewer user ID) and which area is to be the simplification target area from the simplification target area information, and executes a participant user table update process to associate the data of the identified simplification target area with the participant ID and store it in the participant user table, as shown in FIG. 72.

The distribution server computer 100 also executes a participant user table update completion information transmission process that transmits participant user table update completion information to the viewer terminal 300 based on the execution of the participant user table update process. Then, based on receiving the participant user table update completion information, the viewer terminal 300 executes a participant user table update completion notification process that notifies the display device 310 that the update of the participant user table has been completed, i.e., that the viewer terminal 300 has completed registration of whether the third area or the fourth area is to be the simplification target area.

In this way, by the viewer terminal 300 and the distribution server computer 100 executing the series of processes shown in Figure 71, the viewer terminal 300 can set the area to be simplified, and the distribution server computer 100 can determine whether the viewer terminal 300 has set the third or fourth area as the area to be simplified.

The virtual space content delivery system in this embodiment 5 also has a friend registration function that allows a viewer user to register other viewer users as friends by operating the viewer terminal 300.

As shown in FIG. 73, the friend registration function of this type is that the viewer terminal 300 executes a viewer avatar selection process to select a specific viewer avatar from among multiple viewer avatars by accepting operation of the viewer's operation input device 321 during the period during which participation in the live virtual space is possible, identifies the file name of the selected viewer avatar (viewer avatar file name), and executes a viewer avatar inquiry process to transmit viewer avatar inquiry information including the identified viewer avatar file name to the distribution server computer 100.

When the distribution server computer 100 receives viewer avatar inquiry information from the viewer terminal 300, it executes a viewer avatar identification process to identify the viewer user ID stored in association with the viewer avatar file name included in the viewer avatar inquiry information from the viewer user data shown in FIG. 9. The distribution server computer 100 then transmits viewer user ID information including the viewer user ID to the viewer terminal 300 that has transmitted the viewer avatar inquiry information.

When the viewer terminal 300 receives the viewer user ID information, it executes a friend addition operation process on the display device 310 to prompt the viewer to register the viewer user (viewer user ID) of the selected viewer avatar as a friend by associating it with the viewer avatar file name.

At this time, if the viewer performs an operation to register a friend, the viewer terminal 300 executes a friend addition process that associates and stores the viewer user ID and the viewer avatar file name in the friend user data table. Also, if the operation to register a friend is not performed, the viewer terminal 300 executes a friend addition cancellation process that displays on the display device 310 a message indicating that the friend registration will not be performed.

In this embodiment 5, a friend user data table (see FIG. 74) that stores the user IDs and avatar data of viewer users is created in advance in the storage 303 of the viewer terminal 300, and when the friend addition process is executed, the friend user data table is updated to associate the viewer user ID with the viewer avatar file name and store the associated data.

Figure 76 is a flowchart showing the simplification target area designation operation process shown in Figure 71. In the simplification target area designation operation process, the viewer terminal 300 first executes the simplification target area designation operation process to receive an operation from the viewer user to select either the third area or the fourth area of the virtual audience area as the simplification target area (step S801).

As shown in Figures 77(A) and 77(B), a window menu 310WA is displayed on the display device 310 during the execution of the simplification target area designation operation process, allowing the viewer user to designate the simplification target area. The window menu 310WA includes items for "third area" and "fourth area" that can be designated as simplification target areas, and below these items are check boxes for selecting whether or not to preferentially display friends as full avatars in the simplification target area.

Returning to FIG. 76, it is determined whether or not a third area designation operation has been performed in the simplification target area designation operation process to designate the third area as the simplification target area (step S802). If a third area designation operation has been performed (step S802; Yes), the third area is stored as the simplification target area (step S803).

If there is no third area designation operation (step S802; No) or after executing the processing of step S803, the viewer terminal 300 determines whether or not there has been a fourth area designation operation that sets the fourth area as the simplification target area in the simplification target area designation operation processing (step S804). If there has been a fourth area designation operation (step S804; Yes), the fourth area is stored as the simplification target area (step S805).

If the fourth area designation operation has not been performed (step S804; No) or after executing the processing of step S805, the viewer terminal 300 transmits simplification target area information corresponding to the area stored as the simplification target area to the distribution server computer 100 (step S806).

The viewer terminal 300 also determines whether or not a full avatar priority display operation has been performed for a friend in the simplification target area designation operation process, that is, whether or not a check box located at the bottom of the window menu 310WA has been checked by the viewer user as shown in FIG. 77(B) (step S807). If a full avatar priority display operation has been performed (step S807; Yes), the viewer terminal 300 sets a friend full avatar priority display flag indicating that the viewer avatars of friends located in the simplification target area are to be displayed in full avatars, and ends the simplification target area designation operation process (step S808).

If there has been no full avatar priority display operation (step S807; No), the viewer terminal 300 judges whether or not the friendly avatar priority display flag has already been set (step S809). If the friendly avatar priority display flag has not been set (step S809; No), the simplified target area designation operation process ends, and if the friendly avatar priority display flag has been set (step S809; Yes), the viewer terminal 300 further judges whether or not a cancellation operation for the friendly avatar priority display has been performed, that is, whether or not the check box located at the bottom of the window menu 310WA has been unchecked by the viewer user's operation (step S810).

If the operation to cancel the friendly avatar priority display has not been performed (step S810; No), the simplified target area designation operation process is terminated, and if the operation to cancel the friendly avatar priority display has been performed (step S810; Yes), the friendly avatar priority display flag is cleared and the simplified target area designation operation process is terminated (step S811).

Figure 78 is a flowchart showing the viewer perspective video generation process executed by the viewer terminal 300 in this embodiment 5. In the viewer perspective video generation process, the viewer terminal 300 first generates a video of the performer avatar in full avatar (step S501), and determines whether the current time is after the start timing of the first MC period in the virtual live (step S502). If the current time is before the start timing of the first MC period in the virtual live (step S502; No), the viewer avatar video of both the viewer user who has been registered as a friend and the other viewer user who has not been registered as a friend is generated in full avatar, and the viewer perspective video generation process ends (step S503).

In addition, if the current time is after the start of the first MC period in the virtual live (step S502; Yes), the viewer terminal 300 further determines whether a friend viewer avatar identified flag has been set, indicating that the viewer avatar of the viewer user who has been registered as a friend has already been identified in the viewer perspective video generation process (step S504).

If the friend viewer avatar identified flag is not set (step S504; No), the friend user data table shown in FIG. 74 is referenced, the friend viewer avatar data file name is identified and stored (step S505), and the friend viewer avatar identified flag is set (step S506).

In addition, if the friend viewer avatar identified flag is set (step S504; Yes) or after executing the processing of step S506, the viewer terminal 300 identifies the simplification target area stored in the simplification target area designation operation processing of FIG. 76 (step S507), and generates images of all viewer avatars outside the identified simplification target area in full avatars (step S508).

Next, the viewer terminal 300 generates simplified avatars for the images of other viewer avatars (viewer avatars of viewers who are not registered as friends) in the simplified area (step S509), and generates full avatars for the images of friend viewer avatars outside the simplified area (step S510). The viewer terminal 300 also determines whether the friend full avatar priority display flag is set (step S511).

If the friend full avatar priority display flag is set (step S511; Yes), a friend viewer avatar in the simplified area is generated as a full avatar and the process of step S514 is executed (step S512); if the friend full avatar priority display flag is not set (step S511; No), an image of a friend viewer avatar in the simplified area is generated as a simplified avatar and the process of step S514 is executed (step S513).

By executing the viewer perspective video generation process as described above, in this embodiment 5, when comparing the appearance of each viewer avatar generated by the viewer perspective video generation process when the friendly avatar priority display flag is not set, as shown in Figure 79 (A), all viewer avatars located in either the simplified area or the non-simplified area prior to the start of the first MC period are displayed as full avatars, regardless of whether they are viewer avatars of viewer users who are registered as friends.

On the other hand, after the start of the second MC period, all viewer avatars placed in areas subject to simplification will be displayed as simplified avatars regardless of whether they are viewer avatars of viewer users who are registered as friends, while all viewer avatars placed in areas not subject to simplification will be displayed as full avatars regardless of whether they are viewer avatars of viewer users who are registered as friends.

Furthermore, as shown in FIG. 79 (B), when the friendly full avatar priority display flag is set and the appearance of each viewer avatar generated by the viewer viewpoint video generation process is compared, all viewer avatars located in either the simplified area or the non-simplified area prior to the start of the first MC period are displayed as full avatars, regardless of whether they are viewer avatars of viewer users who are registered as friends.

On the other hand, after the start of the second MC period, among the viewer avatars placed in the simplified area, those of viewer users who are registered as friends will be displayed as full avatars, and those of viewer users who are not registered as friends will be displayed as simplified avatars. In addition, all viewer avatars placed in areas not subject to simplification will be displayed as full avatars, regardless of whether they are viewer avatars of viewer users who are registered as friends.

As described above, in the virtual space content distribution system in this embodiment 5, the viewer user himself can perform the setting related to making the viewer avatar a simplified avatar by executing the simplified target area designation operation process shown in FIG. 76 and the viewer viewpoint video generation process shown in FIG. 78. This allows the viewer user's intention to be reflected in the viewer avatar displayed as a simplified avatar, and therefore it is possible to prevent a decrease in interest in the virtual space live performance caused by making the viewer avatar a simplified avatar, which has a smaller processing load for displaying and outputting the viewer avatar in order to prevent the occurrence of display output problems.

In addition, in the virtual space content distribution system of this embodiment 5, by executing the simplified target area designation operation process shown in FIG. 76 and the viewer viewpoint video generation process shown in FIG. 78, the viewer user can operate the viewer terminal 300 to designate the simplified target area for the viewer avatar and simplified avatar from the third and fourth areas, thereby further preventing a decrease in interest in the virtual live performance due to the viewer avatar in an area not intended by the viewer user becoming a simplified avatar.

In addition, in the virtual space content distribution system in this embodiment 5, as shown in Figures 73 and 74, other viewer users can be registered as their own friends, and as shown in Figure 79, the viewer avatar of a viewer user who has become a friend can be set to be displayed as a full avatar even when the viewer user is participating in a simplified area, so that the viewer avatar of a viewer user who is a friend is more likely to be displayed as a full avatar, and when multiple viewer avatars are participating in a virtual live, it is easier to distinguish between the viewer avatar of a viewer user who is a friend and the viewer avatar of a viewer user who is not a friend.

In addition, in this embodiment 5, the viewer user can operate the viewer terminal 300 to set only one of the third and fourth areas of the virtual audience area as the simplification target area. In addition, by setting both the third and fourth areas as the simplification target area, the processing load related to the display output can be reduced compared to when only one area is set as the simplification target area, so that the level of the processing load related to the display output can be changed depending on the situation.

<Modification of the fifth embodiment>
In the fifth embodiment, as shown in Fig. 76 and Fig. 77, the simplification target area designation operation process is executed to select whether or not the third and fourth areas of the virtual audience area are to be simplified, but the present invention is not limited to this, and as a modified example 5-1, as shown in Fig. 80, the viewer user may select how many viewer avatars participating in the virtual live are to be simplified by their operation. Note that when the number of viewer avatars to be simplified is selected in this manner, the selected number of viewer avatars may be set as the upper limit, and the viewer avatars closest to the viewer avatar corresponding to the viewer terminal 300 may be displayed as simplified avatars, or any viewer avatar among all the viewer avatars participating in the virtual live may be displayed as a simplified avatar.

By doing this, the viewer user can set the number of viewer avatars as simplified avatars, which further reduces the loss of interest in the virtual live performance caused by an unintended number of viewer avatars being set as simplified avatars.

In addition, in the above-mentioned embodiment 5, as shown in Figs. 76 and 77, by executing the simplification target area designation operation process, it is possible to select whether or not to set the third and fourth areas of the virtual audience area as simplification target areas, but the present invention is not limited to this, and as shown in Fig. 81 as modified example 5-2, the viewer user may be able to select the ratio of the number of viewer avatars to be simplified avatars to the number of viewer avatars participating in the virtual live show by a selection operation. Note that when the ratio of viewer avatars to be simplified is selected in this way, the viewer avatars may be displayed as simplified avatars in order from the viewer avatar closest to the viewer avatar corresponding to the viewer terminal 300 so as to achieve the selected ratio, or any viewer avatar among all the viewer avatars participating in the virtual live show may be displayed as a simplified avatar.

By doing this, the viewer user himself can set the ratio of viewer avatars that will be simplified avatars to the total number of viewer avatars participating in the virtual live, further preventing a decrease in interest in the virtual live from occurring due to a ratio of viewer avatars that is not intended by the viewer user becoming simplified avatars.

In addition, in the above-mentioned Example 5, as shown in FIG. 75, the live virtual space participation period is set as the simplified target area designation period, and if the viewer user does not designate a simplified target area during the live virtual space participation period, both the third and fourth areas of the virtual audience area become areas not subject to simplification, and neither of the viewer avatars are displayed as simplified avatars. However, the present invention is not limited to this, and if the viewer user does not designate a simplified target area during the live virtual space participation period, the viewer terminal 300 may automatically set both the third and fourth areas of the virtual audience area as simplified target areas at the end of the live virtual space participation period.

By doing this, both the third and fourth areas of the virtual audience area are automatically set as areas to be simplified, and the processing load related to the display output of the viewer avatar can be reduced compared to when only one of the third and fourth areas is set as the area to be simplified.

Furthermore, in this modified example 5-3, as shown in FIG. 84, only one of the third and fourth areas of the virtual audience area can be set as the area to be simplified, and by setting both the third and fourth areas as the area to be simplified, it is possible to reduce the processing load related to the display output compared to the case where only one area is set as the area to be simplified.

In addition, in the above-mentioned Example 5, an example was given of a form in which the simplification target area is determined by the viewer user's operation regardless of the processing power (specs) of the viewer terminal 300, but the present invention is not limited to this. As shown in Figures 83 and 84 as a modified example 5-4, the viewer terminal 300 may identify the processing power of the viewer terminal 300 and determine the simplification target area according to the identified processing power without receiving the viewer user's operation.

For example, as shown in FIG. 83, during the live virtual section participation period, the viewer terminal 300 executes a terminal spec identification process to identify the processing capabilities (specs) of the viewer terminal 300, such as the clock speed and number of cores of the CPU 101 of the viewer terminal 300 and the memory capacity of the RAM 102. The terminal spec identification process also classifies the processing capabilities of the identified viewer terminal 300 into three categories: "medium" which is average processing capabilities, "low" which is below average processing capabilities, and "high" which is above average processing capabilities.

Next, the viewer terminal 300 executes a process for determining the number of areas to be simplified from the third and fourth areas of the virtual audience area. In the process for determining the area to be simplified, as shown in FIG. 84, if the processing capacity of the identified viewer terminal 300 is "low", the number of areas to be simplified is determined to be two (the third and fourth areas), and if the processing capacity of the identified viewer terminal 300 is "medium", the number of areas to be simplified is determined to be one (the third or fourth area). If the processing capacity of the identified viewer terminal 300 is "medium", the area to be simplified may be determined randomly from the third and fourth areas, or the area to be simplified may be determined from the third and fourth areas depending on whether the viewer avatar corresponding to the viewer terminal 300 is participating in the third or fourth area, or whether the number of viewer avatars participating in the third area or the number of viewer avatars participating in the fourth area is greater.

After the viewer terminal 300 determines the area to be simplified in the simplification target area determination process, it transmits simplification target area information, including the viewer user's participant ID (viewer user ID), to the distribution server computer 100.

When the distribution server computer 100 receives the simplification target area information from the viewer terminal 300, it identifies the participant ID (viewer user ID) and which area is to be the simplification target area from the simplification target area information, and executes a participant user table update process to associate the data of the identified simplification target area with the participant ID and store it in the participant user table, as shown in FIG. 72.

The distribution server computer 100 also executes a participant user table update completion information transmission process that transmits participant user table update completion information to the viewer terminal 300 based on the execution of the participant user table update process. Then, based on receiving the participant user table update completion information, the viewer terminal 300 executes a participant user table update completion notification process that notifies the display device 310 that the update of the participant user table has been completed, i.e., that the viewer terminal 300 has completed registration of whether the third area or the fourth area is to be the simplification target area.

In this way, the viewer terminal 300 itself identifies the processing capacity of the viewer terminal 300 and sets the simplification target area based on the identified processing capacity without relying on the operation of the viewer user, thereby increasing the viewer user's freedom in setting the simplification target area and allowing the viewer user to set an appropriate simplification target area that corresponds to the processing capacity of the viewer terminal 300 that he or she is using without performing any operation himself or herself.

In addition, in the above-mentioned Example 5, as shown in FIG. 79, an example was given in which each viewer avatar generated in the viewer viewpoint video generation process (each viewer avatar generated in the viewer terminal 300) can be an image of a simplified avatar according to the setting of the simplification target area selected by the viewer user in the simplification target area designation operation process shown in FIG. 76. However, the present invention is not limited to this. As a modified example 5-5, as shown in FIG. 85(A) to FIG. 85(C), each viewer avatar generated in the viewer viewpoint video generation process can be an image of a simplified avatar according to the setting of the simplification target area selected by the viewer user in the simplification target area designation operation process shown in FIG. 76, while each viewer avatar generated in the virtual camera viewpoint video generation process (each viewer avatar generated in the distribution server computer 100) can all be an image of a simplified avatar regardless of the setting of the simplification target area selected by the viewer user in the simplification target area designation operation process.

By doing this, the processing load associated with the display output of the viewer avatar viewpoint video on the viewer terminal 300 can be reduced, and by generating the virtual camera viewpoint video on the distribution server computer 100 with all viewer avatars as simplified avatars, the processing load associated with the display output of the virtual camera viewpoint video can also be reduced.

Although Example 5 above exemplified a form in which a viewer user can operate the viewer terminal 300 to set a simplified target area in which viewer avatars participating from the third and fourth areas of the virtual audience area are made into simplified avatars, the present invention is not limited to this, and as shown in Figures 86(A) and 86(B) as Modification Example 5-6, when the viewer user operates the viewer terminal 300, a message explaining what settings are possible on the viewer terminal 300 may be displayed on the display device 310 as an auxiliary function for setting the simplified target area. In this way, the viewer user can accurately perform operations for setting the viewer avatar to be a simplified avatar by visually checking the message.

In the example shown in FIG. 86, a message is displayed by the viewer user operating the "HELP" area located in the window menu 310WA, but the present invention is not limited to this, and the message may be displayed without the viewer user operating the "HELP" area.

In addition, in this modified example 5-6, as an auxiliary function for setting the simplification target area, a form in which a message is displayed on the display device 310 explaining what settings are possible on the viewer terminal 300 is exemplified, but the present invention is not limited to this, and as an auxiliary function for setting the simplification target area, operation instructions may be given to the viewer user by displaying an image on the display device 310 or outputting sound from a speaker, etc.

Next, the virtual space content distribution system of Example 6 will be described below with reference to Figures 87 to 89. Note that the same components as those of Examples 1 and 2 will be given the same reference numerals and descriptions thereof will be omitted.

As shown in Figures 87(A) and 87(B), in the virtual space content distribution system of Example 6, a transparent object B1 is placed in boundary area B, which includes the boundary line between the virtual audience area and the virtual stage G. The transparent object B1 is a rectangular wall-like body formed on the virtual audience area side of boundary area B, which covers almost the entire virtual stage G when viewed from the front from the virtual audience area. Furthermore, the transparent object B1 has a transparency that makes it invisible or difficult to see from the viewpoint of a viewer avatar. This allows each viewer user to watch a virtual live performance through the transparent object B1 from the viewpoint of a viewer avatar in the virtual audience area.

In this embodiment 6, collision detection is performed between the transparent object B1 and the viewer avatar, making it impossible for the viewer avatar to enter the virtual stage G.

In this manner, in the virtual space update process (see FIG. 10) when a transparent object B1 is placed on the boundary between the virtual audience area and the virtual stage G, as shown in FIG. 88, the distribution server computer 100 first determines whether or not viewer avatar information has been received (step S701).

If viewer avatar information has been received (step S701; Yes), the position of the viewer avatar in the virtual audience area corresponding to the viewer avatar information is identified from the received viewer avatar information (step S702), and objects whose positions overlap with the viewer avatar (other viewer avatars, performer avatars, transparent object B1) are identified (step S703).

Then, the distribution server computer 100 determines whether the position of the viewer avatar overlaps with that of another viewer avatar (step S704). If the position of the viewer avatar does not overlap with that of another viewer avatar (step S704; No), it determines whether the position of the viewer avatar overlaps with that of a performer avatar (step S705). If the position of the viewer avatar does not overlap with that of a performer avatar (step S705; No), it determines whether the position of the viewer avatar overlaps with that of a transparent object B1 (step S706).

If the distribution server computer 100 determines that the positions of the viewer avatar and the transparent object overlap (step S706; Yes), it executes collision notification information transmission settings for the viewer terminal 300 according to the received viewer avatar information (step S707). The collision notification information is information for notifying that a collision determination has been made between the viewer avatar and the transparent object B1.

On the other hand, if the position of the viewer avatar overlaps with another viewer avatar (step S704; Yes), if the position of the viewer avatar overlaps with the performer avatar (step S705; Yes), or if the position of the viewer avatar does not overlap with the transparent object B1 (step S706; No), the distribution server computer 100 updates and stores the position of the viewer avatar to the position identified in step S702.

If the distribution server computer 100 determines that it has not received viewer avatar information (step S701; No) or after executing the processes of steps S707 and S708, it determines whether it has received performer avatar information (step S709). If it has received performer avatar information (step S709; Yes), it updates and stores the position of the performer avatar based on the received viewer avatar information (step S710).

After executing step S710 or if it is determined in step S709 that performer avatar information has not been received (step S707; No), the distribution server computer 100 executes other processing corresponding to the virtual space update processing (step S711) and terminates the virtual space update processing.

In this manner, in this Example 6, as shown in FIG. 89, when a viewer avatar attempts to move to a position where it overlaps with transparent object B1, collision detection is performed, whereas when a viewer avatar attempts to move to a position where it overlaps with another viewer avatar or a performer avatar, collision detection is not performed, and the viewer avatars and some or all of the viewer avatar and performer avatar will overlap in the same position. On the other hand, when a performer avatar attempts to move to a position where it overlaps with a viewer avatar, another performer avatar, or transparent object B1, collision detection is not performed, and the performer avatars and the viewer avatar, or the performer avatar and some or all of the transparent object B1 will overlap in the same position.

By doing this, in this embodiment 6, the viewer avatar is prevented from moving to the transparent object B1 located on the boundary between the virtual audience area and the virtual stage G, making it possible to prevent the viewer avatar from entering the virtual stage G.

As described above, in this embodiment 6, an increase in the processing load in the system is suppressed by not performing collision detection between viewer avatars in the virtual audience area, while viewer avatars are prevented from entering the virtual stage G by performing collision detection between the viewer avatar and transparent object B1.

In addition, transparent objects B1 are placed across the entire boundary area between the virtual audience area and the virtual stage G, thereby reliably preventing viewer avatars from entering the virtual stage G.

In addition, because no collision detection is performed between the performer avatar and the transparent object B1, it is possible to prevent inconveniences such as the performer avatar accidentally colliding with the transparent object B1, making it impossible to perform the performance.

In addition, no collision detection is performed between performer avatars and viewer avatars, which prevents inconveniences such as the performer avatar accidentally colliding with a viewer avatar, making it impossible to perform the performance.

<Modification of the Sixth Embodiment>
In this embodiment 6, an example is given of a form in which a viewer avatar is prevented from entering the virtual stage G by performing collision detection between the viewer avatar and a transparent object B1, but the present invention is not limited to this, and as a variant example 6-1, as a special case, a viewer avatar set on the administrator terminal 150 may be allowed to enter the virtual stage G without performing collision detection with the transparent object B1.

In such a case, for example, as shown in FIG. 90, during the live virtual space participation period, the administrator terminal 150 accepts an operation from the administrator of the administrator terminal 150 and executes a participating viewer information request process for transmitting participating viewer information request information to the distribution server computer 100 to request information about the viewer users participating in the live virtual space.

When the distribution server computer 100 receives the participating viewer information request information, it executes a participating viewer information identification process to identify the participants currently participating in the live virtual space from the participant user table shown in FIG. 23. It then executes a participating viewer information transmission process to return information about the identified participating viewer (participating viewer information) to the administrator terminal 150.

When the administrator terminal 150 receives the participating viewer information, it displays a list of viewer users currently participating in the live virtual space on an image display device (not shown) or the like, and executes a special viewer selection reception process that receives an operation to select a special viewer user from among these viewer users. Then, when a special viewer user is selected from the list of viewer users by the administrator's operation, a special viewer information transmission process is executed to transmit information about the selected special viewer user (special viewer information) to the distribution server computer 100.

Based on receiving the special viewer information, the distribution server computer 100 executes a special viewer information storage process to store the special viewer user indicated by the special viewer information.

Thereafter, when the distribution server computer 100 receives viewer avatar information based on the viewer terminal 300 executing the avatar motion operation process, it executes a virtual space update process, a virtual space update data generation process, and a virtual space update data distribution process, as shown in FIG. 90.

As shown in FIG. 91, in the virtual space update process, the distribution server computer 100 first executes the processes of steps S701 to S706 in the same manner as in FIG. 88. Then, if the distribution server computer 100 determines that the positions of the viewer avatar and the transparent object B1 overlap (step S706; Yes), it further determines whether the viewer avatar is an exceptional viewer avatar (step S706a).

Here, if the viewer avatar is not a special viewer, step S707 is executed, and then steps S709 to S711 are executed as in FIG. 88. Also, if the distribution server computer 100 determines that the viewer avatar is not a specific viewer avatar (step S706a; Yes), step S708 is executed as in the case where the viewer avatar and another viewer avatar overlap in position (step S704; Yes), the viewer avatar and the performer avatar overlap in position (step S705; Yes), or the viewer avatar and the transparent object B1 do not overlap in position (step S706; No), and then steps S709 to S711 are executed as in FIG. 88.

As described above, in this modified example 6-1, special viewer avatars are excluded from viewer avatars that are subjected to collision detection with transparent object B1. For this reason, as shown in FIG. 92, collision detection is performed between a general viewer avatar (non-special viewer avatar) and transparent object B1, preventing the viewer avatar from passing over the transparent object B1 and entering virtual stage G, whereas a viewer avatar that is set as a special viewer avatar by the administrator terminal 150 is not subjected to collision detection with transparent object B1, and is therefore able to pass over the transparent object B1 and enter virtual stage G.

In addition, in the above-mentioned Example 6, a form was exemplified in which a viewer avatar was prevented from entering the virtual stage G by placing a transparent object B1 on the boundary area B between the virtual stage G and the virtual audience area, but the present invention is not limited to this. As shown in FIG. 93 as a modified example 6-2, when a viewer user operates the viewer terminal 300 to cause the viewer avatar to enter the virtual stage G, a warning image may be displayed on the display device 310 informing the viewer that it is impossible to enter the virtual stage G beyond the transparent object B1, based on the viewer terminal 300 receiving collision notification information from the distribution server computer 100.

In this way, when a collision determination is made between the viewer avatar and the transparent object B1, a warning image is displayed on the display device 310, allowing the viewer user to recognize that a collision determination has been made between the viewer avatar and the transparent object B1.

In addition, in the above-mentioned Example 6, an example was given of a form in which a viewer avatar corresponding to each viewer user is placed in the virtual audience area, but the present invention is not limited to this. As a modified example 6-3, as shown in FIG. 94(A), each viewer avatar may be provided with a full avatar, which requires a relatively heavy processing load to display, and a simplified avatar, which requires a lighter processing load to display than the full avatar.

The full avatar and simple avatar may be switched automatically by the distribution server computer 100 depending on the processing load on the viewer terminal 300, or the viewer user may switch between them by operating the viewer terminal 300.

In addition, in this manner, when full avatars and simplified avatars are provided as viewer avatars, as shown in FIG. 94 (B), collision detection may not be performed between the viewer avatar including these full avatars and simplified avatars and the performer avatar, while collision detection may be performed between the viewer avatar and transparent object B1 regardless of whether the viewer avatar is a full avatar or a simplified avatar.

In this way, regardless of whether the viewer avatar is a full avatar or a simplified avatar that has a lower processing load than a full avatar, collision detection is performed with the transparent object B1, making it possible to prevent the viewer avatar from entering the virtual stage G while preventing an excessive increase in the display processing load.

In addition, while Example 6 above exemplified a form in which collision detection is not performed between the viewer avatar and the performer avatar, the present invention is not limited to this, and as shown in FIG. 95, when multiple performer avatars are placed on the virtual stage G, collision detection may not be performed between these multiple performer avatars. In this way, it is possible to prevent inconveniences such as the inability to perform a performance due to a performer avatar accidentally colliding with another performer avatar.

In addition, in the sixth embodiment, the transparent object B1 is exemplified as a rectangular wall that covers almost the entire virtual stage G when viewed from the front from the virtual audience area, but the present invention is not limited to this, and the shape and number of the transparent objects B1 are not important as long as it is possible to prevent the viewer avatar from entering the virtual stage G. For example, the transparent object B1 may be a thin film rather than a wall, and multiple columnar transparent objects may be arranged in a grid pattern on the boundary area between the virtual audience area and the virtual stage G.

In addition, in the above-mentioned Example 6, a form in which a viewer avatar is prevented from entering the virtual stage G by placing a transparent object B1 in the boundary area B between the virtual audience area and the virtual stage G was exemplified, but the present invention is not limited to this, and an object visible from the viewpoint of the viewer avatar may be placed in the boundary area B between the virtual audience area and the virtual stage G, and the viewer avatar may be prevented from entering the virtual stage G by the object. In this case, it is desirable that the object has a height that does not obscure the performer avatar from the viewpoint of the viewer avatar.

Next, the virtual space content distribution system of the seventh embodiment will be described below with reference to Figs. 96 to 104. Note that the same components as those of the first and second embodiments will be given the same reference numerals and the description will be omitted.

As shown in FIG. 96, the virtual space content distribution system of Example 7 has, in addition to the distribution server computer 100, multiple sub-venue server computers 400 (sub-venue server computers 400a, 400b, etc.) as server computers that perform virtual space processing, and these sub-venue server computers 400 are connected to the distribution server computer 100, administrator terminal 150, performer terminal 200, and each viewer terminal 300 via the Internet network N.

These sub-venue server computers 400, like the distribution server computer 100, are capable of executing various processes related to the virtual live venue and various processes related to the distribution of virtual live performances, and provide a new virtual live venue (hereinafter referred to as a sub-virtual live venue) on the condition that the number of viewer avatars participating in the virtual live venue provided by the distribution server computer 100 reaches a predetermined number, and are used to present performances using performer avatars in the same way as the virtual live venue of the distribution server computer 100 (hereinafter referred to as the main virtual live venue).

In addition, as shown in FIG. 97, a sub-virtual live venue control program 140 is stored in the storage 103 of the distribution server computer 100 in this embodiment 7. The distribution server computer 100 uses the sub-virtual live venue control program 140 to determine whether the number of viewer avatars participating in the main virtual live venue and sub-virtual live venues has reached a predetermined constant, and if the specified number has been reached, restricts the participation of new viewer avatars in the virtual live venue where the number of viewer avatars has reached the specified number, and controls the allocation of new viewer avatars to participate in other sub-virtual live venues.

The virtual space content distribution system of this embodiment 7 also has a chat information server computer 170, as shown in FIG. 96, and this chat information server computer 170 is connected to the distribution server computer 100, the administrator terminal 150, the performer terminal 200, each viewer terminal 300, and each sub-venue server computer 400 via the Internet network N.

The chat information server computer 170 is a computer that transmits to the performer terminal 200 and each viewer terminal 300 integrated chat information that is a chronologically rearranged version of chat information input by viewer users operating input devices such as keyboards and touch panels provided on each viewer terminal 300.

In addition, in this embodiment 7, the performer terminal 200 and each viewer terminal 300 receive integrated chat information from the chat information server computer 170 and display the same chat information on the display screen.

Next, the flow of how a viewer user virtually participates in a virtual live venue in this embodiment 7 will be explained using FIG. 98.

First, the distribution server computer 100 executes a participant authentication process when it receives a participation request from the viewer terminal 300, as in FIG. 22. Then, it executes a participating virtual live venue determination process using the sub virtual live venue control program 140. At this time, the distribution server computer 100 determines the live venue in which the viewer avatar will participate.

After that, similar to FIG. 22, the distribution server computer 100 updates the participant user table and transmits the advance data, the viewer terminal 300 identifies the advance data, transmits a request to download the advance data, and receives and stores the advance data.

When updating the participant user table after the participating virtual venue determination process, the distribution server computer 100 stores data indicating the participating virtual venue determined in the participating virtual venue determination process in association with the participant ID corresponding to the viewer terminal 300 that sent the participation request.

The specific process of the participating virtual live venue determination process will be explained using FIG. 100(A). Note that in the following explanation, for simplicity, the virtual space content distribution system is described as having two sub-venue server computers (sub-venue server computers 400a, 400b), but the present invention is not limited to this, and the virtual space content distribution system may have only one sub-venue server computer, or three or more sub-venue server computers.

In the virtual live participation determination process, the distribution server computer 100 first determines whether or not a second sub virtual live venue participation permission flag is set, indicating that the viewer avatar is permitted to participate in the second sub virtual live venue provided by the sub venue server computer 400b (step S811). If the second sub virtual live venue participation permission flag is not set (step S811; No), the distribution server computer 100 determines whether or not a first sub virtual live venue participation permission flag is set, indicating that the viewer avatar is permitted to participate in the first sub virtual live venue provided by the sub venue server computer 400a (step S812). Note that the first sub virtual live venue and the second sub virtual live venue are virtual live venues that have a virtual audience area and a virtual stage G, simulating the main virtual live venue.

If the first sub-virtual live venue participation permission flag is not set (step S812; No), the number of viewer avatars participating in the main virtual live venue provided by the distribution server computer 100 is identified by referring to the participant user table shown in FIG. 99 (step S813), and it is determined whether the number of viewer avatars participating in the main virtual live venue is equal to or greater than the predetermined participating virtual live venue change provision number (step S814).

As shown in FIG. 100(B), in this embodiment 7, the upper limit number of viewer avatars that can participate in the main virtual live venue is set to 200. On the other hand, the number of participating virtual live venue changes in the main virtual live venue is set to 150, which is 75% of the upper limit number of viewer avatars that can participate in the main virtual live venue.

In other words, in step S814, the distribution server computer 100 determines whether the number of viewer avatars participating in the main virtual live venue is 150 or more.

If the number of viewer avatars participating in the main virtual live venue is less than the predetermined participating virtual live venue change stipulation number (step S814; No), the distribution server computer 100 determines the participating virtual live venue to be the main virtual live venue and ends the participating virtual live venue determination process (step S815). Also, if the number of viewer avatars participating in the main virtual live venue is equal to or greater than the predetermined participating virtual live venue change stipulation number (step S814; Yes), the distribution server computer 100 sets the first sub-virtual live venue participation permission flag (step S816), determines the participating virtual live venue to be the first sub-virtual live venue, and ends the participating virtual live venue determination process (step S817).

In addition, if the first sub-virtual live venue participation permission flag is set (step S812; Yes), the distribution server computer 100 identifies the number of viewer avatars participating in the first sub-virtual live venue provided by the sub venue server computer 400a by referring to the participant user table shown in FIG. 99 (step S818), and determines whether the number of viewer avatars participating in the first sub-virtual live venue is equal to or greater than the predetermined participating virtual live venue change provision number (step S819).

As shown in FIG. 100(B), in this embodiment 7, the upper limit number of viewer avatars that can participate in the first sub virtual live venue is set to 100. On the other hand, the number of participating virtual live venue changes in the first sub virtual live venue is set to 75, which is 75% of the upper limit number of viewer avatars that can participate in the first sub virtual live venue.

In other words, in step S819, the distribution server computer 100 determines whether the number of viewer avatars participating in the first sub virtual live venue is 75 or more.

As shown in FIG. 100(B), the upper limit number of viewer avatars that can participate in the second sub-virtual live venue is set to 100, which is the same as the upper limit number of viewer avatars that can participate in the first sub-virtual live venue.

If the number of viewer avatars participating in the first sub virtual live venue is less than the predetermined participating virtual live venue change number (step S819; No), the distribution server computer 100 determines the participating virtual live venue to be the first virtual live venue and ends the participating virtual live venue determination process (step S820). Also, if the number of viewer avatars participating in the first sub virtual live venue is equal to or greater than the predetermined participating virtual live venue change number (step S819; Yes), the distribution server computer 100 sets the second sub virtual live venue participation permission flag (step S821), determines the participating virtual live venue to be the second sub virtual live venue, and ends the participating virtual live venue determination process (step S822).

After the participating virtual live venue determination process, the distribution server computer 100 updates the participant table by linking the data indicating the virtual live venue determined in steps S815, S817, S820, and S822 to the participant ID corresponding to the viewer terminal 300 that sent the participation request and storing it.

In this embodiment 7, it is possible for viewer avatars to participate in each virtual live venue up to the upper limit. In this case, however, when the viewer avatars move all at once, when the performer avatars move vigorously, when a light-emitting effect is performed in which a specific object is illuminated as part of the virtual live performance, etc., the processing load related to the display of these viewer avatars and performer avatars becomes enormous, and there is a risk of disrupting the distribution of the virtual live. For this reason, in this embodiment 7, before the number of viewer avatars participating in each virtual live venue reaches the upper limit, control is performed to allocate a new viewer avatar participating in the virtual live to another virtual live venue, thereby suppressing the processing load related to the display of the viewer avatars and performer avatars in each server computer in preparation for the above-mentioned cases when the viewer avatars move all at once, when the performer avatars move vigorously, when a light-emitting effect is performed in which a specific object is illuminated as part of the virtual live performance, etc., and enabling the virtual live to be distributed without disruption.

In addition, in this embodiment 7, the upper limit number (200) of viewer avatars that can participate in the main virtual live venue provided by the distribution server computer 100 is greater than the upper limit number (100) of viewer avatars that can participate in the first sub virtual live venue provided by the first sub venue server computer 400a and the second sub virtual live venue provided by the second sub venue server computer 400b. In other words, in the virtual space content distribution system of the present invention, when adding a new sub venue server computer according to the number of viewer avatars participating in the virtual live, it is sufficient to connect a server computer that has lower processing power for distributing the virtual live and is less expensive than the distribution server computer 100 each time, so there is no need to prepare a server computer with processing power for distributing the virtual live equivalent to that of the distribution server computer 100, and the virtual space content distribution system can be constructed inexpensively.

In this embodiment 7, the upper limit number of viewer avatars that can participate in the main virtual live venue is set to be greater than the upper limit number of viewer avatars that can participate in the first sub virtual live venue and the second sub virtual live venue, but the present invention is not limited to this, and the upper limit number of viewer avatars that can participate in the main virtual live venue may be the same as the upper limit number of viewer avatars that can participate in the first sub virtual live venue and the second sub virtual live venue.

In addition, in this embodiment 7, as shown in FIG. 100 (B), the number of participating virtual live venue changes for the main virtual live venue and the first sub virtual live venue are both set to 75% of the maximum number of participating viewer avatars for each virtual live venue, but the present invention is not limited to this, and the ratio of the number of participating virtual live venue changes to the maximum number of participating viewer avatars in the main virtual live venue and the ratio of the number of participating virtual live venue changes to the maximum number of participating viewer avatars in the first sub virtual live venue may be different.

By executing the participating virtual live venue determination process as described above, after the live virtual space participation period ends, the distribution server computer 100 executes the virtual space update process, virtual space update data generation process, and virtual space update data distribution process as in FIG. 10, and transmits the virtual space update data to the viewer terminal 300 and performer terminal 200 of the viewer avatar participating in the main virtual live venue. The distribution server computer 100 also executes the music playback process, distribution audio generation process, and generated audio distribution process as in FIG. 10.

On the other hand, as shown in FIG. 101, when the distribution server computer 100 receives performer avatar information from the performer terminal 200 after the live virtual space participation period has ended, it executes a virtual stage area space update process that updates only the data in the area of the virtual stage G out of the virtual space data. This reflects the movement (operation) of the performer avatar on the virtual stage G. Then, it generates virtual stage area update data based on the data in the area of the virtual stage G before the update and the data in the area of the virtual stage G after the update, and transfers the generated virtual stage area update data to the sub-venue server computers 400a, 400b (virtual stage area space update data generation and transmission process).

In addition, when the distribution server computer 100 receives viewer avatar information from a viewer terminal 300 whose viewer avatar is participating in the first or second sub-virtual live venue, it refers to the participant user table to identify the sub-virtual live venue in which the viewer avatar of the viewer terminal 300 is participating from the participant ID included in the received viewer avatar information, and executes a viewer avatar information transfer process to transfer the viewer avatar information to the sub-venue server computer 400a, 400b corresponding to the identified sub-virtual live venue.

In other words, the distribution server computer 100 always receives viewer avatar information from each viewer terminal 300, regardless of the virtual live venue in which the viewer avatar is participating. Then, if the received viewer avatar information is from a viewer terminal 300 whose viewer avatar is participating in the first sub-virtual live venue or the second sub-virtual live venue, the distribution server computer 100 transfers the viewer avatar information to the sub-venue server computer 400a, 400b corresponding to the sub-virtual live venue in which the viewer avatar is participating. Therefore, in this embodiment 7, the distribution server computer 100 is able to grasp the total number of viewer avatars (total number of viewer users) participating in the virtual live and the actions of each viewer avatar.

When the sub-venue server computers 400a, 400b receive virtual stage area space update data from the distribution server computer 100, they execute a virtual stage area space reflection process to reflect the contents of the virtual stage area space update data in the movements (operations) of the performer avatars on the virtual stage G of the first sub-virtual live venue and the second sub-virtual live venue.

In addition, when the sub-venue server computers 400a, 400b receive viewer avatar information from the distribution server computer 100, they execute a viewer area space update process to reflect the movements of each viewer avatar in the virtual audience area (viewer area space) based on the viewer avatar information.

Then, after executing the above-mentioned virtual stage area space reflection process and viewer area space update process, the sub-venue server computers 400a, 400b execute a virtual space update data generation process to generate virtual space update data based on the data of the area of the virtual stage G before the update and the data of the area of the virtual stage G after the update, and the data of the virtual audience area (viewer area space) before the update and the data of the virtual audience area (viewer area space) after the update, and then execute a virtual space update data distribution process to distribute the generated virtual space update data to the viewer terminal 300 and the performer terminal 200.

In the viewer terminal 300 to which the virtual space update data is delivered from the sub-venue server computers 400a, 400b, similar to FIG. 10, the pre-stored virtual space data is updated based on the delivered virtual space update data, and a performer perspective video generation process and a viewer perspective video generation process are executed using the updated virtual space data, thereby generating performer perspective video and viewer perspective video based on the updated virtual space data, and the generated performer perspective video and viewer perspective video of the first sub-virtual live venue and the second sub-virtual live venue are displayed on the performer terminal 200 and the viewer terminal 300 (viewer perspective video output process).

In addition, virtual space update data is not distributed from the sub-venue server computers 400a, 400b to the performer terminal 200 in this embodiment 7. Therefore, while an image of the main virtual live venue is displayed on the performer terminal 200, an image of the sub virtual live venue is not displayed.

In the example shown in FIG. 101, similarly to FIG. 10, the distribution server computer 100 executes the music playback process, the distribution audio generation process, and the generated audio distribution process, and the performer terminal 200 executes the playback music output process, the performer audio input process, the monitor audio generation process, and the monitor audio output process, so that the viewer terminal 300 executes the audio output process, and thus these are not shown in the figure.

Next, we will explain the exchange of chat information and integrated chat information between the performer terminal 200, each viewer terminal 300, and the chat information server computer 170.

As shown in FIG. 102, a viewer terminal 300 whose viewer avatar is participating in the main virtual live venue and a viewer terminal 300 whose viewer avatar is participating in the sub virtual live venue execute a chat information input acceptance process that accepts input of chat information from viewer users using input devices such as a keyboard or touch panel provided on each viewer terminal 300. At this time, when the input of chat information is accepted, the viewer terminal 300 executes a chat information transmission process that transmits the entered chat information to the chat information server computer 170.

When the chat information server computer 170 receives chat information from each viewer terminal 300, it executes an integrated chat information generation process to generate integrated chat information that arranges the received chat information in chronological order, and transmits the generated integrated chat information to the performer terminal 200 and each viewer terminal 300.

Then, as shown in FIG. 103, the performer terminal 200 and each viewer terminal 300 execute an integrated chat information display process to display the same integrated chat information in a chat display area CW provided in a predetermined position on the screen of the display device of each terminal (the lower right corner of the screen in this embodiment 7) based on the reception of the integrated chat information.

In summary, in this embodiment 7, as shown in FIG. 104, the viewer terminal 300 and the performer terminal 200 in which the viewer avatar is participating in the main virtual live venue display integrated chat information and images of the main virtual live venue as performer's perspective video and viewer's perspective video, and output music and performer audio. In addition, the viewer terminal 300 in which the viewer avatar is participating in the first sub virtual live venue displays integrated chat information and images of the first sub virtual live venue as performer's perspective video and viewer's perspective video, and output music and performer audio, and the viewer terminal 300 in which the viewer avatar is participating in the second sub virtual live venue displays integrated chat information and images of the second sub virtual live venue as performer's perspective video and viewer's perspective video, and output music and performer audio.

As described above, in this embodiment 7, when the number of viewer avatars participating in the main virtual live venue reaches 75% of the maximum number of viewer avatars that can participate due to viewer avatars participating in the main virtual live venue, the participation of new viewer avatars in the main virtual live venue is restricted, while the participation of new viewer avatars in the first sub-virtual live venue is permitted, so it is possible to prevent the virtual live from becoming impossible to stream due to the participation of viewer avatars in the main virtual live venue exceeding the maximum number of viewer avatars that can participate, which is 200.

In addition, common integrated chat information is displayed on both the viewer terminal 300 corresponding to the viewer avatar participating in the main virtual live venue and the viewer terminal 300 corresponding to the viewer avatar participating in the sub virtual live venue, so that the viewer user can view the common integrated chat information regardless of which virtual live venue the viewer avatar is participating in. This makes it difficult for a viewer user whose viewer avatar is participating in a sub virtual live venue to recognize that the viewer avatar is participating in the sub virtual live venue, and prevents a decrease in interest in the virtual live due to the viewer avatar's participation in the sub virtual live venue.

In addition to the main virtual live venue and the first sub-virtual live venue, which have a maximum number of viewer avatars that can participate of 200, a second virtual live venue is also provided, which also has a maximum number of viewer avatars that can participate of 200. When the number of viewer avatars participating in the first virtual live venue reaches 150, the distribution server computer 100 restricts the participation of new viewer avatars in the first sub-virtual live venue, while allowing new viewer avatars to participate in the second sub-virtual live venue. This prevents a decline in interest in the virtual live, while eliminating the risk of the virtual live becoming impossible to distribute due to more than 150 viewer avatars participating in the first virtual live venue.

<Modification of the Seventh Example>
In the seventh embodiment, the viewer terminal 300 and the performer terminal 200 in which the viewer avatar is participating in the main virtual live venue display an image of the main virtual live venue as a performer's perspective video or a viewer's perspective video, but do not display images of the first sub virtual live venue or the second sub virtual live venue; the viewer terminal 300 in which the viewer avatar is participating in the first virtual live venue displays an image of the first sub virtual live venue as a performer's perspective video or a viewer's perspective video, but do not display images of the main virtual live venue or the second sub virtual live venue; and the viewer terminal 300 in which the viewer avatar is participating in the second virtual live venue displays an image of the second sub virtual live venue as a performer's perspective video or a viewer's perspective video, but do not display images of the main virtual live venue or the first sub virtual live venue. However, the present invention is not limited to this, and the performer terminal 200 and the viewer terminal 300 may be capable of displaying all of the images of the main virtual live venue, the first sub virtual live venue, and the second sub virtual live venue as performer's perspective video or a viewer's perspective video. In particular, in such a case, as shown in Figure 105 as variant example 7-1, each performer terminal 200 and viewer terminal 300 may display an image of the main virtual live venue, an image of the first sub virtual live venue, and an image of the second sub virtual live venue in chronological order.

In this way, by switching between images of the main virtual live venue, the first sub-virtual live venue, and the second sub-virtual live venue on the viewer terminal 300 in chronological order, the viewer user is able to view the viewer's perspective image of the virtual live space in which the viewer avatar corresponding to his or her own viewer terminal 300 is participating, making it difficult for a viewer user whose viewer avatar is participating in the sub-virtual live venue to recognize that the viewer avatar is participating in the sub-virtual live venue, and preventing a decrease in interest in the virtual live due to the viewer avatar's participation in the sub-virtual live venue.

In addition, in the above-mentioned Example 7, an example was given in which only an image of the main virtual live venue was displayed as the performer's viewpoint image on the performer terminal 200 (see FIG. 104), but in this modified example 7-1, an image of the main virtual live venue, an image of the first sub virtual live venue, and an image of the second sub virtual live venue are displayed in chronological order as the performer's viewpoint image, allowing the performer to provide content using a performer avatar while being aware of the reactions of the viewer avatars in all virtual live venues.

In addition, in the modified example 7-1, the image of the main virtual live venue, the image of the first sub virtual live venue, and the image of the second sub virtual live venue are displayed on the viewer terminal 300 in a time-series switching manner, thereby allowing the viewer user to view the viewer's viewpoint video of the virtual live space in which the viewer avatar corresponding to his/her own viewer terminal 300 is participating, making it difficult for the viewer user whose viewer avatar is participating in the sub virtual live venue to recognize that the viewer avatar is participating in the sub virtual live venue, and thus preventing a decrease in interest due to the viewer avatar participating in the sub virtual live venue. However, the present invention is not limited to this, and as modified example 7-2, as shown in Figures 106 (A) and 106 (B), a virtual display 40 provided on virtual stage G of the main virtual live venue may display a virtual camera C2 perspective image of the main virtual live venue, a virtual display 40 provided on virtual stage G of the first sub virtual live venue may display a virtual camera C2 perspective image of the first sub virtual live venue, and a virtual display 40 provided on virtual stage G of the second sub virtual live venue may display a virtual camera C2 perspective image of the second sub virtual live venue.

In this way, it is possible to create a sense of realism for the virtual live performance in the main virtual live venue and the sub virtual live venue, while making it difficult for viewer users who are participating in the sub virtual live venue with a viewer avatar to recognize that the viewer avatar is participating in the sub virtual live venue, further preventing a decrease in interest in the virtual live performance due to the viewer avatar's participation in the sub virtual live venue.

In addition, in the seventh embodiment, a viewer avatar corresponding to the viewer terminal 300 operated by the viewer user is able to participate in a virtual live venue, but the present invention is not limited to this, and the viewer user may be able to register other viewer users as their own friends by operating the viewer terminal 300.

In this case, for example, the viewer user can operate the viewer terminal 300 in the same manner as in Figures 39 and 40 of the third embodiment described above to store the viewer user ID and viewer avatar data file name of the viewer to be registered as a friend in the friend user table.

Furthermore, when friend registration is made possible in this manner, as variant example 7-3, it may be possible to notify a user that a friend's viewer avatar has participated in the same virtual live space as the user's own viewer avatar during the period during which the user can participate in the live virtual space.

For example, as shown in FIG. 107, during the period when participation in the live virtual space is possible, the viewer terminal 300 executes a video output process, then refers to the friend user data table described above to identify the viewer user ID of the friend, and executes a virtual live participation friend inquiry process that transmits virtual live participation friend inquiry information including the identified viewer user ID and the viewer's own (the viewer operating the viewer terminal 300) viewer user ID to the distribution server computer 100.

When the distribution server computer 100 receives virtual live participation friend inquiry information from the viewer terminal 300, it identifies the friends participating in the virtual live by referring to the participant user table shown in FIG. 99 and the viewer user IDs of the friends included in the received virtual live participation friend inquiry information.

Furthermore, the distribution server computer 100 identifies friends who are participating in the same virtual live venue as the viewer user who made the virtual live participation friend inquiry, and transmits identification result information to the viewer terminal 300.

Based on receiving the specific result information, the viewer terminal 300 determines whether the viewer operating the viewer terminal 300 has a friend who has newly participated in the viewer avatar in the same virtual live venue as the viewer avatar corresponding to the viewer terminal 300, and if there is a friend who has newly participated in the viewer avatar in the same virtual live venue as the viewer avatar corresponding to the viewer terminal 300, it executes a friend participation notification process to notify that fact.

By doing this, as shown in Figure 108 (A), if the viewer operating the viewer terminal 300 has a friend who has newly participated in the virtual live venue with a viewer avatar corresponding to the viewer terminal 300, a notification that a friend has newly participated in the virtual live space is displayed in the chat display area CW on the screen. On the other hand, as shown in Figure 108 (B), if the viewer operating the viewer terminal 300 has a friend who has newly participated in the virtual live venue with a viewer avatar corresponding to the viewer terminal 300, a notification that a friend has newly participated in the virtual live space is not displayed in the chat display area CW on the screen.

This can create a sense of unity between friends whose viewer avatars are participating in the same virtual live venue, making the virtual live event more interesting.

In addition, in the seventh embodiment, when the number of viewer avatars participating in the main virtual live venue exceeds a predetermined participating virtual live venue change provision number, the participation of further viewer avatars in the main virtual live venue is restricted, while viewer avatars are permitted to participate in sub virtual live venues. However, the present invention is not limited to this, and viewer avatars participating in each virtual live venue may be allowed to participate in other virtual live venues.

In this case, as shown in FIG. 109 as variant example 7-4, a viewer avatar participating in the main virtual live venue may be permitted to move to the first sub virtual live venue or the second sub virtual live venue, a viewer avatar participating in the first sub virtual live venue may be permitted to move to the second sub virtual live venue, and a viewer avatar participating in the second sub virtual live venue may be permitted to move to the first sub virtual live venue, while a viewer avatar participating in the first sub virtual live venue or the second sub virtual live venue may be permitted to restrict movement to the main virtual live venue.

By doing this, it is possible to prevent more viewer avatars than the maximum number of viewer avatars that can participate, which is 200, from participating in the main virtual live venue, where the number of participating viewer avatars has reached 75% (150) of the maximum number of viewer avatars that can participate.

In the seventh embodiment, as shown in FIG. 103, a viewer terminal 300 whose viewer avatar is participating in the main virtual live venue and a viewer terminal 300 whose viewer avatar is participating in a sub virtual live venue display common integrated chat information, but the present invention is not limited to this. The viewer terminal 300 whose viewer avatar is participating in the main virtual live venue may display only integrated chat information for the main virtual live venue that integrates chat information input from the viewer terminals 300 whose viewer avatar is participating in the main virtual live venue, and the viewer terminal 300 whose viewer avatar is participating in the sub virtual live venue may display only chat information for the sub virtual live venue that integrates chat information input from the viewer terminals 300 whose viewer avatar is participating in the sub virtual live venue.

In such a case, the chat information server computer 170 may not be provided, and the distribution server computer 100 may generate the integrated chat information for the main virtual live venue based on chat information input from a viewer terminal 300 whose viewer avatar is participating in the main virtual live venue, and transmit the generated integrated chat information for the main virtual live venue to each viewer terminal 300. Similarly, the sub venue server computers 400a, 400b may generate the integrated chat information for the sub virtual live venue based on chat information input from a viewer terminal 300 whose viewer avatar is participating in the sub virtual live venue, and transmit the integrated chat information for the sub virtual live venue to each viewer terminal 300.

In addition, in the seventh embodiment, as shown in FIG. 101, in order to reflect the actions (operations) of the performer user and the viewer user in each avatar virtually participating in the sub virtual live venue, the virtual stage area space update data generated by the distribution server computer 100 and the viewer avatar information received by the distribution server computer 100 from the viewer terminal 300 are transferred to the sub venue server computers 400a, 400b, and the sub venue server computers 400a, 400b generate virtual space update data using the virtual stage area space update data and the viewer avatar information, but the present invention is not limited to this. The performer avatar information received from the performer terminal 200 and the viewer avatar information received from the viewer terminal 300 in the distribution server computer 100 may be directly transferred from the distribution server computer 100 to the sub venue server computers 400a, 400b, and the sub venue server computers 400a, 400b may generate virtual space update data from the performer avatar information and viewer avatar information transferred.

In addition, in the above embodiments and modified examples, the viewer avatar viewpoint video is generated and displayed on the viewer terminal 300 side, but the present invention is not limited to this. For example, in the case of a live event with few virtual participants, the viewer avatar viewpoint video may also be generated and distributed on the distribution server computer 100 side. In addition, the viewer user may be able to select whether the viewer avatar viewpoint video is generated on the viewer terminal 300 side or on the distribution server computer 100 side depending on the processing capacity of the terminal he or she owns and the available data communication environment, or the distribution server computer 100 may identify the processing capacity of the viewer terminal 300 with which it is connected for communication, and based on the identified processing capacity, determine whether to generate the viewer avatar viewpoint video on the viewer terminal 300 side or on the distribution server computer 100 side.

In addition, in each of the above embodiments and variants, the distribution server computer 100 is illustrated as being installed in a studio operated by the event management organization that hosts the virtual live show, but the present invention is not limited to this. As the distribution server computer 100, a server computer owned by the company that rents out the server computer in a data center or the like may be used, or as described above, a server computer provided by a cloud service may be used. The installation form of these server computers may be any form that provides the functions of the distribution server computer 100.

In addition, in each of the above embodiments and variants, a performer terminal 200 is provided in a one-to-one correspondence with a performer user, but the present invention is not limited to this, and a single performer terminal 200 may be shared by multiple people. If the performer user is a unit or group consisting of multiple members, for example, a viewpoint tracking target designation process may be executed in at least one of the distribution server computer 100 and the viewer terminal 300, allowing the viewer user to select a member to follow the viewpoint.

In addition, in each of the above embodiments and modified examples, a performer user sings, dances, and performs other performance actions alone, but the present invention is not limited to this, and the singer who sings and the actor who dances may be separate persons. In this case, the singer's performer terminal and the actor's performer terminal may be separate, and the singer may sing in a location other than the studio where the actor performs the performance actions.

In addition, in each of the above embodiments and variations, the Internet network is used as an example of a computer communication network connecting the distribution server computer 100, the administrator terminal 150, the performer terminal 200, and the viewer terminal 300, but the present invention is not limited to this, and these communication networks may be configured as local area networks that are capable of data communication only within a specific area, or may include a local area network or local data communication as part of the network.

In addition, in each of the above embodiments and variants, the viewer terminal 300 is exemplified as a smartphone P or a computer (PC), but the present invention is not limited to this, and these computers may be portable notebook computers, tablet terminals, e-book readers, wearable computers, game consoles, and various other information processing devices capable of outputting at least 2D video and audio.

In addition, in each of the above embodiments and variants, a performer user performs in a studio, including singing, but the present invention is not limited to this. For example, a performer user may use a performer terminal 200 installed in a home or the like to perform in a virtual live performance from a home or the like.

In addition, in the above embodiments and variants, a virtual live performance has been given as an example of virtual space content, but the present invention is not limited to this, and such virtual space content may be any content that is beneficial to viewer users, such as a fan meeting where performer users gather with their fans, or a talk session, and is related to a virtual space in which performer avatar 1 and viewer avatars virtually participate.

In addition, in each of the above embodiments and modified examples, the viewpoint switching of the four virtual cameras C1 to C4 as fixedly positioned (set) virtual cameras and the airborne moving virtual camera is exemplified as a form in which the viewpoint switching can be performed in advance by setting and operating the administrator terminal 150, but the present invention is not limited to this. For example, the event manager M may perform these viewpoint switching in real time on the administrator terminal 150 using a viewpoint switching controller or the like constituting the operation input device 161.

In addition, in each of the above embodiments and modified examples, virtual cameras C1 to C4 fixedly placed (set) in the virtual space are used as virtual cameras, but the present invention is not limited to this. For example, in addition to or instead of the virtual cameras C1 to C4, a movable virtual moving camera may be placed in the virtual live venue, and the position and imaging direction of the virtual moving camera in the virtual space may be controlled by the event manager M or a cameraman through the operation of the manager terminal 150 to produce the viewpoint image of the virtual moving camera.

In addition, in each of the above embodiments and variations, an example is given of a form in which the viewpoint image of the virtual camera C2 is displayed on the screen S so that the facial expressions of the performer user can be detected and reflected in the facial expressions of the performer avatar, but the present invention is not limited to this. For example, if the performer user is a high-performance head-mounted display that has a sensor function for detecting the facial expressions of the performer user, the performer user may wear this high-performance head-mounted display while performing the performance, which may improve the performer user's sense of immersion in the virtual live performance. In addition, if the facial expressions of the performer user do not need to be reflected in detail in the facial expressions of the performer avatar, the performer user may wear a normal head-mounted display while performing the performance.

In addition, while the virtual space content distribution system of the present invention has been exemplified by Examples 1 to 3 and each of the modified examples, the present invention is not limited to these, and the virtual space content distribution system of the present invention may be implemented by implementing only one of the examples and modified examples, or may be implemented by combining multiple examples and modified examples.

In addition, instead of the performer user's singing or singing actions (performance), the singing or singing actions (performance actions) of a virtual performer created using computer graphics or the like can be recorded as performer avatar information on a distribution server computer, and a virtual live performance can be held by playing back the recorded singing voice and singing actions (performance actions) as the live performance progresses.

In this way, when singing or singing actions (performance actions) are performed in advance to record and play back performer avatar information on the distribution server computer 100 to distribute a virtual live performance, the configuration of the virtual space content distribution system at the time of distribution will be made up of the distribution server computer 100, the administrator terminal 150, and the viewer terminal 300, and will not include the performer terminal 200, which is a performer user terminal that can be used by a performer user to operate the performer avatar 1 in the virtual space.

In addition, in the present invention, "available for use by the performer" does not mean that the performer user operates the virtual space, but also includes use by the performer user simply to view virtual space content.

In addition, although Examples 1 to 7 have been disclosed as examples of the present invention, and multiple modified examples of these examples have been disclosed, the present invention may be implemented by each of these examples and modified examples alone, or multiple examples and modified examples may be implemented in appropriate combination.

1 Performer avatar 100 Distribution server computer 150 Administrator terminal 200 Performer terminal 300 Viewer terminal

Claims (11)

A virtual space content distribution system capable of distributing virtual space content including at least content of a performance performed by a performer avatar in a virtual space,
a viewer user terminal capable of outputting an image of the virtual space content, the viewer user being able to view the image;
a server computer connected to the viewer user terminal via a communication network and capable of executing at least processes related to the virtual space and processes related to the distribution of the virtual space content;
At least
The viewer users are multiple, and the viewer user terminal is assigned to each viewer user;
The virtual space content may include a viewer avatar by allowing the viewer user to participate in a virtual space that is the subject of the virtual space content by using the viewer user terminal;
the viewer avatars include a first viewer avatar and a second viewer avatar that requires a smaller processing load for display than the first viewer avatar;
the viewer user terminal is capable of outputting, as the virtual space content, content of a performance by a first performer avatar and content of a performance by a second performer avatar different from the first performer avatar;
A virtual space content distribution system characterized in that the appearance of a second viewer avatar corresponding to a first performer , which is the second viewer avatar included in the content when the content of a performance by the first performer avatar is output on the viewer user terminal, is different from the appearance of a second viewer avatar corresponding to a second performer , which is the second viewer avatar included in the content when the content of a performance by the second performer avatar is output on the viewer user terminal. The virtual space content distribution system described in claim 1, characterized in that the appearance of the first viewer avatar is common when the virtual space content output on the viewer user terminal is content of a performance by the first performer avatar and when it is content of a performance by the second performer avatar. The viewer user terminal is capable of outputting, as the virtual space content, collaboration content in which the first performer avatar and the second performer avatar simultaneously perform a performance in the same virtual space,
The virtual space content distribution system described in claim 1 or 2, characterized in that when the collaboration content is output, the viewer user can select on the viewer user terminal whether the appearance of the second viewer avatar will be that of the first performer avatar or the second performer avatar. The viewer user terminal is capable of outputting, as the virtual space content, collaboration content in which the first performer avatar and the second performer avatar simultaneously perform a performance in the same virtual space,
A virtual space content distribution system as described in claim 1 or 2, characterized in that the state of the second viewer avatar when the virtual space content output on the viewer user terminal is the collaboration content is a special state different from the state when the performance of the first performer avatar is output and the state when the performance of the second performer avatar is output. the virtual space content includes a first content portion of a performance by the first performer avatar and a second content portion of a performance by the second performer avatar;
The virtual space content distribution system described in claim 1, characterized in that the appearance of the second viewer avatar when the first content portion is output on the viewer user terminal corresponds to the first performer avatar, and the appearance of the second viewer avatar when the second content portion is output on the viewer user terminal corresponds to the second performer avatar. The viewer avatar is changeable from the first viewer avatar to the second viewer avatar,
2. The virtual space content delivery system according to claim 1, wherein the plurality of viewer avatars participating in the virtual space change into the second viewer avatar at a common timing. The virtual space content distribution system described in claim 6, characterized in that when the viewer avatar changes to the second viewer avatar, the performer avatar does not change to a simplified performer avatar that requires a small processing load for display. the state of the second viewer avatar when the content of the program by the first performer avatar is outputted in the viewer user terminal includes an element corresponding to an attribute of the first performer avatar;
The virtual space content distribution system described in claim 1, characterized in that the appearance of the second viewer avatar when content of a performance by the second performer avatar is output on the viewer user terminal includes elements corresponding to attributes of the second performer avatar. In a case where the first performer avatar performs a performance in a first virtual space and the second performer avatar performs a performance in a second virtual space different from the first virtual space,
The virtual space content distribution system according to claim 1, characterized in that the state of the second viewer avatar participating in the first virtual space is different from the state of the second viewer avatar participating in the second virtual space. A virtual space content distribution program for distributing virtual space content including at least content of a performance performed by a performer avatar in a virtual space,
The virtual space content includes:
A content including a video that can be viewed by a viewer user at a viewer user terminal assigned to each of a plurality of viewer users,
The viewer may include a viewer avatar by being able to participate in the virtual space that is the subject of the virtual space content by using the viewer user terminal connected via a communication network to a server computer that is capable of executing at least processes related to the virtual space and processes related to the distribution of the virtual space content,
the viewer avatars include a first viewer avatar and a second viewer avatar that requires a smaller processing load for display than the first viewer avatar;
a virtual space content output process for outputting, as the virtual space content, content of a performance by a first performer avatar and content of a performance by a second performer avatar different from the first performer avatar at the viewer user terminal;
a viewer avatar behavior process in which a behavior of a second viewer avatar corresponding to a first performer , which is the second viewer avatar included in the content when the content of a program performed by the first performer avatar is output on the viewer user terminal, is different from a behavior of a second viewer avatar corresponding to a second performer, which is the second viewer avatar included in the content when the content of a program performed by the second performer avatar is output on the viewer user terminal;
A virtual space content delivery program comprising: A virtual space content distribution method for distributing virtual space content including at least content of a performance performed by a performer avatar in a virtual space, comprising:
The virtual space content includes:
A content including a video that can be viewed by a viewer user at a viewer user terminal assigned to each of a plurality of viewer users,
The viewer may include a viewer avatar by being able to participate in the virtual space that is the subject of the virtual space content by using the viewer user terminal connected via a communication network to a server computer that is capable of executing at least processes related to the virtual space and processes related to the distribution of the virtual space content,
the viewer avatars include a first viewer avatar and a second viewer avatar that requires a smaller processing load for display than the first viewer avatar;
a virtual space content output step of outputting, as the virtual space content, content of a performance by a first performer avatar and content of a performance by a second performer avatar different from the first performer avatar at the viewer user terminal;
a viewer avatar behavior procedure for setting different behaviors of a second viewer avatar corresponding to a first performer , which is the second viewer avatar included in the content when the content of a program performed by the first performer avatar is being output on the viewer user terminal, and a second viewer avatar corresponding to a second performer , which is the second viewer avatar included in the content when the content of a program performed by the second performer avatar is being output on the viewer user terminal;
A virtual space content distribution method comprising:

Images