JP2020102782A - Content distribution system, distribution device, reception device, and program - Google Patents

Abstract

To provide a suitable viewing experience while ensuring presence of a large number of participants.SOLUTION: In a content distribution system for distributing content presenting a main character and a plurality of secondary characters representing a viewing user to a plurality of reception devices, the distribution device determines a behavior to be taken by a plurality of secondary characters based on an analysis result of information indicating operation input involved in viewing the content on a plurality of reception devices and transmits information, including behavioral information indicating the behavior, necessary to reproduce the content to the reception device. The behavior information includes information on a ratio of the secondary characters that are made to behave in a certain way among a plurality of secondary characters.SELECTED DRAWING: Figure 7

Description

The present invention relates to a content distribution system, a distribution device, a reception device, and a program, and more particularly to a technology for distributing binocular stereoscopic content.

Devices that enable binocular stereoscopic vision, such as television receivers and head mounted displays (HMDs), are generally distributed, and users can easily enjoy binocular stereoscopic content by using such devices. Is becoming possible. Patent Document 1 discloses a content distribution system that distributes three-dimensional image content that can be viewed with such a device.

JP, 2016-212622, A

By the way, in recent years, so-called live distribution, which distributes in real time a moving image captured by a camera and a moving image capturing a screen on which computer graphics (CG) such as a game is drawn, is performed using various platforms. ing. Among the contents provided by such live distribution, the facial expressions and behaviors of the performers are detected as motions and reflected in the CGs of the characters, etc., to provide a viewing experience as if the characters actually existed. Some do. In addition, the information transmitted in the live distribution is not limited to a mode in which the information is a moving image stream, and the motion and audio data applied to the CG is transmitted to the device (client) for viewing, and the client draws the CG for viewing. A similar viewing experience can also be provided in a mode in which a screen for use is generated and presented.

Here, for example, a mode of live distribution of highly realistic content in which a plurality of persons such as performers and viewers participate is presented, that is, a scene in which characters to which the behaviors of a plurality of persons are applied are arranged in a three-dimensional space is presented. Consider a mode of performing live distribution. When it is attempted to realize such live distribution of content by a content distribution system as disclosed in Patent Document 1, information on behavior (motion) detected for each person is aggregated in a device such as a server, and the information is further viewed. It is necessary to transmit the data to each device and reflect it in the managed state information, and further draw and generate a screen for the left eye and a screen for the right eye. On the other hand, if it is attempted to perform such a series of processing while ensuring a favorable viewing experience with binocular stereoscopic vision, the computational load on the entire system, server, and client becomes high, which may cause failure.

The present invention has been made in view of the above problems, and a content distribution system, a distribution device, a reception device, and a content distribution system that realize a suitable viewing experience while ensuring the presence of a large number of participants. The purpose is to provide the program.

In order to achieve the above-mentioned object, a content distribution system according to the present invention includes a distribution device that distributes content that presents a main character, and a plurality of receptions that receive the distributed content and provide a viewing experience of the content. In the content distribution system having a device, the content is, in addition to the main character, a sub character different from the main character, and a plurality of sub characters indicating a viewing user who is viewing using the receiving device. The receiving device is configured to present, the first communication means for communicating information with the distribution device, a reproducing means for performing a process relating to reproduction of the content including drawing of the main character and the sub character, and the reproduced content. Presenting means for presenting, detecting means for detecting an operation input related to viewing the content, and first control for controlling the first communication means so as to transmit information indicating the detected operation input to the distribution device. And a second communication means for communicating information with a plurality of receiving devices, an analyzing means for analyzing information indicating an operation input received from the plurality of receiving devices, and an analysis result by the analyzing means. Based on the above, a behavior control means for deciding a behavior to be taken by a plurality of sub-characters, and a behavior information indicating a behavior to be taken by the plurality of sub-characters decided are transmitted to the receiving device, which is necessary for reproducing the content. And a second control means for controlling the second communication means, wherein the behavior control means determines a proportion of the sub-characters that cause a predetermined behavior among the plurality of sub-characters. To do.

With such a configuration, according to the present invention, it is possible to provide a suitable viewing experience while ensuring a sense of realism in which a large number of people participate.

The figure which showed the structure of the content delivery system which concerns on embodiment of this invention. FIG. 3 is a block diagram showing the functional configurations of the distribution client 100 and the motion capture system 400 according to the embodiment of the present invention. FIG. 3 is a block diagram showing a functional configuration of the receiving client 200 according to the embodiment of the present invention. Block diagram showing the functional configuration of the connection server 300 according to an embodiment of the present invention The figure which illustrated the three-dimensional space which concerns on the content delivered by the content delivery system which concerns on embodiment of this invention. The figure which illustrated the appreciation aspect of the content delivered by the content delivery system which concerns on embodiment of this invention. The flowchart which illustrated the behavior control process performed by the delivery client 100 which concerns on embodiment of this invention. The figure which showed the data structural example of some information exchanged in the content delivery system which concerns on embodiment of this invention.

[Embodiment]
Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the invention according to the claims, and all combinations of the features described in the embodiments are not necessarily essential to the invention. Two or more of the plurality of features described in the embodiments may be arbitrarily combined. Further, the same or similar configurations are denoted by the same reference numerals, and duplicated description will be omitted.

It should be noted that one embodiment described below is capable of performing live distribution of binocular stereoscopic content, which presents a main character that is interlocked with the performer's behavior, as an example of a distribution device that constitutes a content distribution system. An example will be described in which the present invention is applied to a distribution client and a reception client configured to be able to view the live-distributed binocular stereoscopic content as an example of a reception device. However, the present invention can be applied to any device group and system capable of providing a viewing experience of content configured so that a large number of people can participate and be viewed.

<Structure of content distribution system>
FIG. 1 is a diagram showing a system configuration of a content distribution system according to an embodiment of the present invention. As shown in the figure, the content distribution system is configured such that the distribution client 100 and the connection server 300 can communicate with each other, and the connection server 300 and the plurality of reception clients 200 can communicate with each other via the network 500. The distribution client 100 is also configured to be able to acquire information from the motion capture system 400.

In the content distribution system, the distribution client 100 provides, as a live distribution, a content viewing experience including presentation of a main character and presentation of a voice uttered by the performer, which is linked to the behavior of the performer. The behavior of the performer and the acquisition of the voice are assumed to be performed by the motion capture system 400, and the distribution client 100 acquires the motion data regarding the behavior of the performer and the voice data regarding the voice from the motion capture system 400. ..

In the present embodiment, the performer whose behavior is being measured in the motion capture system 400 will be described as being in charge of both the behavior of the main character relating to the content to be distributed and the voice uttered, but these are different performers. May be done by. Alternatively, it is needless to say that the performer does not need to perform it in real time, but may use motion data or voice data that is preconfigured.

In the present embodiment, the reception client 200 is assumed to be a HMD (Head Mounted Display) type device configured so that the three-dimensional space in which the main character is placed can be viewed with stereoscopic vision, and the content delivered live is It is assumed that the content is binocular stereoscopic content in which the presentation mode changes in association with the head movement of the user (viewing user) who uses the reception client 200. For this reason, in the content distribution system of the present embodiment, in order to improve the processing efficiency and reduce the calculation load of the distribution client 100, the live distribution is not performed by the video streaming method, but the reception client 200 generates the frame and It is performed by a method of distributing information necessary for outputting audio (reproduction of content).

The information required to reproduce the content includes information that requires real-time processing (motion data, audio data, etc. caused by the performer) and other information that is stored in the reception client 200 in advance and used during reproduction. (3D model data, music data, performance data, fixed motion data not attributed to the performer, etc.). In the content distribution system of the present embodiment, the latter information is downloaded to the receiving client 200 in advance in response to the viewing user using the receiving client 200 to perform a selection operation or a payment operation of the content desired to be viewed. Shall be stored.

A controller 240 is wirelessly connected to the reception client 200, and an operation input relating to such content selection, an operation of changing a viewing position (a viewpoint position used for drawing) in a three-dimensional space, and other operations by a viewing user. It is assumed that the operation input can be detected. The content provided by the distribution client 100 in the content distribution system of the present embodiment is configured so that various operation inputs can be performed using the controller 240 during the viewing experience. Is an operation input device that has an operation detection sensor 241 and a physical button 242 capable of detecting the. However, in the implementation of the present invention, the connection method between the reception client 200 and the controller 240 and the presence or absence of the controller 240 are not limited to the aspect of the present embodiment.

In the present embodiment, the receiving client 200 is described as a so-called eyeglass-type device including a display device (display unit 220) capable of providing a binocular stereoscopic viewing experience. It is not limited to this. That is, the receiving client 200 may be any device configured to execute processing related to the reproduction of the content and output a presentation screen or sound in order to present the viewing experience of the distributed content. The device for presenting these screens and sounds may be a device connected to the outside of the receiving client 200.

The connection server 300 is a device provided to provide stable live distribution of content to a plurality of connected reception clients 200. The connection server 300 is used to realize management of the receiving client 200 being connected, information transmission from the distribution client 100 to the receiving client 200, and management of information transmission from the receiving client 200 to the distribution client 100. Therefore, in the present embodiment, an example will be described in which the connection server 300 is introduced so as not to impose the processing load related to these processes on the distribution client 100, but the present invention is not limited to this. That is, the function performed by the connection server 300 may be performed by the distribution client 100, and in this case, the distribution client 100 and the reception client 200 are configured to directly exchange information. In other words, in the implementation of the present invention, whether or not the information communication between the distribution client 100 and the reception client 200 is performed via a device is not an essential item in the invention.

The network 500 may realize information communication between devices in a public communication network such as the Internet or a local area network (LAN). In this embodiment, the network 500 describes a mode in which information transmission/reception between the distribution client 100 and the connection server 300 and information transmission/reception between the reception client 200 and the connection server 300 are realized through one network 500. However, for example, the distribution client 100 and the connection server 300 are connected by a LAN, the reception client 200 and the connection server 300 are connected by a public communication network, and information communication is performed using different networks 500. May be.

<Functional configuration of distribution client>
Next, the functional configurations of the distribution client 100 and the motion capture system 400 according to the content distribution system of this embodiment will be described using the block diagram of FIG.

The control unit 101 is, for example, a CPU, and controls the operation of each block included in the distribution client 100. Specifically, the control unit 101 reads out, for example, an operation program of each block recorded in the recording medium 102 or a program of an application (distribution application) that performs live distribution of content, expands it in the memory 103, and executes it. Controls the operation of each block.

The recording medium 102 is a recording device such as a non-volatile memory or an HDD that can hold data permanently. The recording medium 102 stores information such as parameters necessary for the operation of each block, in addition to the operation program of each block of the distribution client 100 and the program of the distribution application. The memory 103 is a storage device used for temporary data storage, such as a volatile memory. The memory 103 is used not only as an expansion area for the operation program of each block, but also as a storage area for temporarily storing the data output in the operation of each block.

The acquisition unit 104 acquires motion data and audio data related to the main character for the content for which the viewing experience is provided in the content distribution system of this embodiment. As described above, the motion data and voice data related to the main character are acquired by the motion capture system 400 and output to the distribution client 100 using a sensor or a microphone (not shown) worn by the performer. The functional configuration of the motion capture system 400 is briefly shown in FIG. 2, but includes a detection unit 401 that detects the behavior of the performer (change of each part of the performer's body) based on a sensor attached to the performer, and a microphone. The audio acquisition unit 402 for acquiring the audio produced by the performer collected by the above, and the conversion unit 403 for converting this information into a data format usable by the distribution client 100 and the reception client 200. The motion capture system 400 also includes an output unit 404 that outputs, to the distribution client 100, motion data and voice data that are generated by being converted by the conversion unit 403 and that correspond to the behavior of the performer and the uttered voice. In the present embodiment, for the sake of simplicity, the motion capture system 400 is assumed to function as an interface of the distribution client 100 and does not have a unique control device or the like. However, the motion capture system 400 has a control device and the like as an independent device, It goes without saying that the operation may be controlled.

The distribution unit 105 controls live distribution of content for which a viewing experience is provided in the content distribution system of this embodiment. More specifically, the distribution unit 105 appropriately configures and sends information to be transmitted to the reception client 200 connected to the connection server 300 by making a viewing request for the content in a time period during which live distribution is performed. Perform processing. Further, the distribution unit 105 controls to send a start command or the like at a necessary timing for processing that needs to be executed in the reception client 200 for viewing the content.

The analysis unit 106 collects information on the operation input made while the content is being viewed in each of the receiving clients 200 that is viewing the content, and analyzes the distribution of the operation input made in the same period. .. As will be described later in detail, in the content distribution system of this embodiment, in order to provide interactivity in the content viewing experience, it is possible to see how the viewing users who are viewing the content at the same time react. The display mode of a plurality of mob (crowd) characters visibly arranged in the three-dimensional space is the same as that of the main character. Therefore, the information of the operation input made by the viewing user at each receiving client 200 for controlling the states of the plurality of mob characters is collected by the distribution client 100 via the connection server 300 and analyzed by the analysis unit 106. It

The behavior control unit 107 controls the behavior (motion) to be applied to the plurality of mob characters based on the analysis result by the analysis unit 106. More specifically, the behavior control unit 107 determines a motion to be performed by a plurality of mob characters based on an operation input made by each viewing user, and distributes the motion information as information that requires real-timeness. To the distribution unit 105.

The communication unit 108 is a communication interface with the external device of the distribution client 100. The communication unit 108 can be connected to an external device via a network 500 (whether wired or wireless) and can transmit/receive data. The communication unit 108 converts, for example, information input as a transmission target into data in a predetermined format, and transmits the data to an external device such as the connection server 300 via the network 500. Further, when the communication unit 108 receives information from an external device via the network 500, for example, the communication unit 108 decodes the information and stores it in the memory 103.

<Functional configuration of receiving client>
Next, the functional configuration of the reception client 200 according to the content distribution system of the present embodiment will be described using the block diagram of FIG. In the description of the functional configuration of the receiving client 200, the configuration that realizes the same function as the configuration of the distribution client 100 is prefixed with “reception” to distinguish it from the configuration of the distribution client 100. Shall be indicated. Here, “reception” added as a prefix indicates that the reception client 200 is simply a component, and is not used for limiting the functions of the blocks to be described.

The reception control unit 201 is, for example, a CPU, and controls the operation of each block included in the reception client 200. Specifically, the reception control unit 201 reads, for example, an operation program of each block recorded in the reception recording medium 202 or a program of an application (a viewing application) that provides a viewing experience of live-distributed content, and a reception memory The operation of each block is controlled by expanding it into 203 and executing it.

The reception recording medium 202 is a recording device capable of holding data permanently, such as a non-volatile memory or an HDD. The reception recording medium 202 stores, in addition to the operation program of each block and the viewing application program of the reception client 200, information such as parameters necessary for the operation of each block. The reception memory 203 is a storage device used for temporary data storage, such as a volatile memory. The reception memory 203 is used not only as an expansion area for the operation program of each block, but also as a storage area for temporarily storing the data output in the operation of each block.

The setting unit 204 sets a viewing position for viewing the corresponding three-dimensional space in viewing the live-delivered content. The viewing position corresponds to a viewpoint position that serves as a reference when an image group for binocular stereoscopic vision is generated by drawing the three-dimensional space. In the present embodiment, as shown in FIG. 5, a viewing user selects a desired viewing position from a plurality of predetermined viewing positions 501a to 501f in a three-dimensional space 510 and sets it. However, the implementation of the present invention is not limited to this. The viewing position does not have to be set according to the selection operation by the viewing user, and may be set and allocated by the setting unit 204, for example, on a first-come-first-served basis or by lottery when viewing the content. Further, in the present embodiment, description will be made assuming that one is selected from the plurality of viewing positions as shown in FIG. 5. However, in carrying out the present invention, the viewing position is, for example, a viewpoint moving operation or a body movement by a viewing user. It may be configured to be changeable to any position according to the operation.

The reproduction control unit 205 uses the information necessary for reproducing the content, including the information received from the distribution client 100 via the connection server 300, and executes the process related to the reproduction of the content. The process related to content reproduction includes generation of a binocular stereoscopic image group for one frame of live distribution and display on the display unit 220, reproduction of corresponding audio data, and output to the audio output unit 230. Including. The display unit 220 is a display device such as an LCD, for example, and the audio output unit 230 is an audio output device such as a speaker or earphone.

Here, the generation of the binocular stereoscopic image group (the image for the left eye and the image for the right eye) is performed by the drawing unit 206. The drawing unit 206 is, for example, a drawing device such as a GPU, and performs a predetermined drawing process to generate a left-eye image and a right-eye image to be displayed on the display unit 220. Specifically, when the viewing application starts providing the content viewing experience in the viewing application, the drawing unit 206 performs necessary calculations on the drawing objects (including the three-dimensional model) of the main character and the mob character to generate motion. By applying it and arranging it in a three-dimensional space, and drawing the three-dimensional space for the viewpoint for the left eye and the viewpoint for the right eye determined according to the set viewing position, Generate an image group.

At this time, the viewpoint for the left eye and the viewpoint for the right eye reflect the movement of the head of the viewing user who wears the receiving client 200, and thus are incorporated in the receiving client 200 such as an acceleration sensor or a gyro sensor. The sensor output from the viewpoint control sensor 210 is reflected. The viewing position set by the setting unit 204 includes a region in the three-dimensional space associated with the viewing position (a region where viewing is performed) and a binocular stereoscopic image group predetermined for the region. The reference viewpoint position and the line-of-sight direction at the time of generation are specified. In order to provide a viewing experience of binocular stereoscopic content with a high immersive feeling, the reference viewpoint position and the line-of-sight direction of the viewing user are set. It is preferable to change it according to the movement of the head. Therefore, after applying the translation and rotation based on the sensor output from the viewpoint control sensor 210 to the reference viewpoint position and the line-of-sight direction, the viewpoint position after application is set as the middle point and the eye-to-eye distances equivalent separation is performed. The two viewpoints having the line-of-sight direction after application are the viewpoint for the left eye and the viewpoint for the right eye.

For example, in a mode in which the display area of the display unit 220 is divided to display left-eye and right-eye images and the images are viewed through a predetermined optical system, stereoscopic vision is realized. 206 generates a final screen by combining the generated binocular stereoscopic image groups in parallel to form one screen. The reproduction control unit 205 realizes the presentation of the binocular stereoscopic content by sequentially displaying, on the display unit 220, the image of the screen corresponding to the frame generated in this way.

The operation detection unit 207 is a user interface included in the reception client 200, such as a power button and a menu button. When the operation detection unit 207 detects that an operation input has been made to the operation member, the operation detection unit 207 outputs a control signal corresponding to the operation input to the reception control unit 201. In addition, in the present embodiment, the reception client 200 includes a controller 240 wirelessly connected, and operates the physical button 242 that is a physical operation member of the controller 240 and shakes the controller 240 itself. The operation detection unit 207 also detects an operation input made by the above. Here, an operation input such as shaking the controller 240 itself is detected by the operation detection sensor 241 which may be, for example, a gyro sensor, and transmitted to the operation detection unit 207.

The reception communication unit 208 is a communication interface with the external device of the reception client 200. The reception communication unit 208 can be connected to an external device via a network 500 (whether wired or wireless) to enable data transmission/reception. The reception communication unit 208 converts, for example, the information input as a transmission target into data of a predetermined format, and transmits the data to an external device such as the connection server 300 via the network 500. Further, the reception communication unit 208, when receiving information from an external device via the network 500, for example, decodes the information and stores it in the reception memory 203.

<Functional configuration of connection server>
Next, the functional configuration of the connection server 300 according to the content distribution system of the present embodiment will be described using the block diagram of FIG. In the description of the functional configuration of the connection server 300, a configuration that realizes the same function as the configuration of the distribution client 100 and the reception client 200 has a prefix “connection” in order to distinguish it from the configuration of these devices. It shall be shown with letters. Here, the "connection" added as a prefix indicates merely one configuration of the connection server 300, and is not used for limiting the functions of the blocks to be described.

The connection control unit 301 is, for example, a CPU and controls the operation of each block included in the connection server 300. Specifically, the connection control unit 301 controls the operation of each block, for example, by reading out the operation program of each block recorded in the connection recording medium 302, expanding it in the connection memory 303, and executing it.

The connection recording medium 302 is a recording device capable of holding data permanently, such as a non-volatile memory or an HDD. The connection recording medium 302 stores information such as parameters necessary for the operation of each block, in addition to the operation program of each block included in the connection server 300. The connection memory 303 is a storage device used for temporary data storage, such as a volatile memory. The connection memory 303 is used not only as an expansion area for the operation program of each block, but also as a storage area for temporarily storing the data output in the operation of each block.

The communication control unit 304 controls to which device the information received from the external device is transmitted via the connection communication unit 305, which will be described later, and management of the receiving client 200 currently viewing the content. In order to provide a content viewing experience in the content distribution system, the necessary processing related to communication is performed. Note that, for the sake of simplicity, in the present embodiment, description will be made assuming that there is only one type of content distributed by the content distribution system, but the present invention is not limited to this, and is provided via the connection server 300. There may be multiple types of content. In this case, a combination of the distribution client 100 and the motion capture system 400 may be provided for each content.

The connection communication unit 305 is a communication interface with an external device included in the connection server 300. The connection communication unit 305 can connect to an external device via a network 500 (whether wired or wireless) and can transmit/receive data. The connection communication unit 305 converts, for example, information input as a transmission target into data in a predetermined format, and transmits the data to an external device such as the distribution client 100 and the reception client 200 via the network 500. Further, when the connection communication unit 305 receives information from an external device via the network 500, for example, the connection communication unit 305 decodes the information and stores it in the connection memory 303.

<Summary of provided contents>
Next, the experience of viewing the content provided by the receiving client 200 in the content distribution system of this embodiment having such a configuration will be specifically described.

In the content described as an example in the present embodiment, as shown in FIG. 5, a main character 502 that is interlocked with a performer's behavior and voice is arranged mainly on a stage 503 in a three-dimensional space, and a singing performance or a microphone performance is performed. This is a binocular stereoscopic content that enables the viewing user to view the content. The viewing position 501 assigned to the receiving client 200 is provided on the floor (passenger seat) 504 provided around the stage 503 as shown in the figure, and the viewing user is basically more than the stage 503. The content is configured so that the main character 502 can be viewed from a lower position.

That is, in the three-dimensional space configured for the contents, various drawing objects are arranged in a modeled space such as a so-called live house or concert hall as shown in the figure, and the viewing user can see the stereoscopic image with both eyes. By viewing the three-dimensional space, it is possible to obtain a viewing experience as if one were visiting a concert live in which the main character 502 appears. In the content of the present embodiment, a mob character 505 indicating the presence of a concert live is similarly arranged on the floor 504 in order to further enhance the realism of the viewing experience and as a lively element of the content. Further, the viewing user uses the controller 240 while viewing the content, and applauds (pressing some physical buttons 242), shaking hands or penlights (vibrating operation of the controller 240 itself), and cheering (one Multiple types of actions, such as pressing the physical button 242 of the section or voice input to a microphone (not shown), and presenting the action in the receiving client 200 in a state in which the viewing user can confirm the action, so that the user simply looks around. It is possible to provide a more immersive viewing experience.

Here, the mob character 505 may give an impression to the viewing user that a heterogeneous existence of repeating a predetermined motion is present on the floor 504 in a mode in which a predetermined motion is applied to the music, On the contrary, the interest in the content can be reduced. Therefore, the viewing user can sympathize with each other's reactions and feel more immersed in the world view of the content while feeling the presence of other viewing users who are viewing the live distribution of the same content through the viewing application at the same time. As described above, it is preferable to associate the mob character 505 with another viewing user from the viewpoint of improving the enjoyment of the viewing experience. That is, the viewing user who uses the receiving client 200 to view the content is viewing the content by himself in the real world, but in the virtual space provided by using the three-dimensional space, the mob character 505 is displayed. With the appearance of, the viewing experience of the concert live can be received together with the viewing user who is similarly interested in the content and the main character 502. Therefore, in order to provide a highly entertaining viewing experience, it is preferable that the mob character 505 reflect the reaction of the viewing user who is similarly viewing.

By the way, in the content distribution system as in the present embodiment, the number of receiving clients 200 connected to the connection server 300 at the same time depends on the scale of the connection server 300 in order to view the content distributed live in real time. However, it is assumed that the unit will be in the unit of hundreds to thousands. That is, the distribution client 100 provides each of the plurality of reception clients 200 via the connection server 300 with at least the motion data and the sound data applied to the main character in the live distribution of the content.

During the live distribution of the content, the information transmitted to the device of the receiving client 200 is limited to the information that should be applied to the drawing objects arranged in the three-dimensional space in real time, thereby reducing the traffic and providing a suitable viewing experience. Can be provided.

On the other hand, as described above, for each of the mob characters 505 arranged in the three-dimensional space, the operation performed by the viewing user using the reception client 200 associated with the mob character 505 using the controller 240. In order to reflect the input in real time, the following processing is required. That is, such operation input information is transmitted from each receiving client 200 to the distribution client 100 via the connection server 300, and the operation input information received from all the receiving clients 200 that are viewing the content is received by the distribution client 100. In summary, as information to be applied to the corresponding mob character 505 in real time, it is necessary to perform a process of transmitting together with the information to be applied to the main character 502 in real time.

In such an aspect, not only is the delivery client 100 controlled to acquire and transmit the motion data and voice data of the main character 502, but which mob character 505 is associated with each receiving client 200. , And the state of the mob character 505, and it is necessary to control the state information to be transmitted to the receiving client 200 that arranges the mob character 505 in the three-dimensional space. That is, when the operation input information is received from the receiving client 200 associated with each mob character 505, the information of the state of the mob character 505 (motion application state of the mob character 505 in the frame) is updated based on this information. It is necessary to perform processing so that the updated state is reflected in content viewing in another receiving client 200.

However, not only the main character 502 but also the mob character 505 cannot be compensated by application of fixed motion data, and the behavior control that strictly reflects the information of the operation input of the associated viewing user is not performed by the distribution client. Not only can the processing load on 100 be increased, but the traffic related to live distribution can also be increased, resulting in delays and dropped frames, which can hinder the provision of a suitable viewing experience.

Therefore, regarding the behavior of the mob character 505, the content provided by the content distribution system of the present embodiment does not associate the receiving client 200 with each mob character 505, and each type in the receiving client 200 during the content viewing. By controlling according to the frequency of the operation input, the occurrence of the above-mentioned harmful effects is reduced. More specifically, the distribution client 100 receives the information of the operation input made in each of the receiving clients 200, and to what extent the analyzing unit 106 is currently receiving content for each type. If the operation input of the type is performed in the reception client 200, that is, the ratio of the reception clients 200 to which the operation input of the type is performed in all the reception clients 200 is analyzed. Then, the behavior control unit 107, based on the information of the ratio analyzed for each type of operation input by the analysis unit 106, sets the percentage of the mob characters 505 among all the mob characters 505 arranged in the three-dimensional space. Information on whether to apply motion data corresponding to the operation input is transmitted to each reception client 200.

By doing so, it is possible to simply provide a concert live viewing experience that reflects the behavior of a viewing user who is viewing live-delivered content at the same time. That is, in the receiving client 200, the behavior of the mob character 505 in the three-dimensional space 510 constructed for drawing is based on the information of the ratio of the motion type (operation input type) supplied from the distribution client 100. The mob character 505 selected based on the proportion of the character 505 is controlled to apply the corresponding motion data, and an image group for binocular stereoscopic vision is generated. In other words, in the content distribution system of the present embodiment, each mob character 505 is not controlled to be in a state in which the operation input made in the associated receiving client 200 is strictly reflected, but at what ratio as a whole. A similar viewing experience is realized by analyzing whether or not the operation input is made and controlling so that the corresponding behavior is applied only to the number of mob characters 505 corresponding to the ratio.

It should be noted that the mob character 505 to which the behavior corresponding to the operation input is applied based on the supplied information on the ratio of the receiving client 200 to which the predetermined operation input is performed is a mob character arranged in the three-dimensional space 510. It may be randomly selected from the characters 505 by a lottery process or the like. Alternatively, it is arranged in front of the set viewing position (a region near the stage), that is, in a region included in the reference viewpoint position defined for the viewing position and the visual field range in the line-of-sight direction so that the viewing user can easily recognize it. The mob character 505 may be controlled to be preferentially selected. Further, with respect to the mob character 505 in which the application of the behavior corresponding to the relevant operation input has already been started in the previous frame or the like, it is preferentially selected as the application target of the behavior corresponding to the same operation input. And At this time, with respect to the motion that has already been applied to the mob character 505, the reception control unit 201 controls so that the motion is continued.

Further, as described above, in the content distribution system according to the present embodiment, the mob character 505 does not aim at presenting which viewing user and what operation input is performed in a distinguishable manner. It is not necessary for the appearance of the mob character 505 to be a mode in which the viewing user can be identified. Therefore, as shown in FIG. 6, for example, the drawing object 601 corresponding to the viewing user, which is provided for notifying the contents of the operation input that the viewing user is currently performing, is excluded, and the drawing object 601 is arranged on the floor 504. The mob characters 505 may all have the same appearance. That is, the viewing application downloads and stores information necessary for playing back the content (three-dimensional model, texture, music data, effect data, fixed motion, etc.) that is downloaded and stored in advance when an operation input for viewing the content is made. The amount of data (such as data) can also be reduced.

<Behavior control processing>
Hereinafter, with respect to the behavior control processing executed by the distribution client 100 of the content distribution system according to the present embodiment to determine the motion to be applied to the mob character 505 while watching the content, specific processing will be described with reference to the flowchart of FIG. 7. To do. The process corresponding to the flowchart can be realized by the control unit 101 reading out a corresponding process program of the distribution application stored in the recording medium 102, loading the program in the memory 103, and executing the program. .. The behavior control process will be described as being started when, for example, the distribution application is executed in the distribution client 100 and a content viewing experience providing sequence is reached. Here, as for the content viewing experience, the receiving client 200 executes the viewing application during a predetermined period provided immediately before the live distribution by the distribution application, or the target content. It is assumed that the viewing request is provided during the live distribution period, provided that the viewing request is transmitted from the receiving client 200 to the connection server 300. At this time, the communication control unit 304 adds the identification information (and destination information) of the receiving client 200 that has made the viewing request to the list of the receiving clients 200 that are viewing the corresponding content.

In step S<b>701, the control unit 101 acquires information about an operation input made by the receiving client 200 that is currently viewing content until the transmission of information necessary for the next reproduction related to content distribution. For the sake of simplicity, the behavior control processing of the present embodiment will be described on the assumption that the operation input information is acquired for each frame of the content to be live-distributed and the behavior control processing is performed. It is not limited to. The operation input information may be acquired for a period provided at a predetermined time interval or for a predetermined reproduction section defined for the music data reproduced in providing the content viewing experience. .. For example, as shown in FIG. 8A, the operation input information is associated with the device ID 801 that identifies the receiving client 200 that is the transmission source of the information, the content ID 802 that identifies the content being viewed, and the enumeration type. It may be configured to include operation type information 803 indicating the type of operation input that may be made.

In step S<b>702, the analysis unit 106 analyzes, under the control of the control unit 101, the information of the operation input performed in the reception client 200 currently viewing the content, which is acquired in step S<b>701. The ratio information indicating the ratio of the receiving clients 200 to which the operation input has been made is derived from all the receiving clients 200 currently undergoing content viewing.

In step S703, under the control of the control unit 101, the behavior control unit 107 configures the behavior information related to the behavior control of the mob character 505 group in viewing the content, based on the ratio information derived for each type of operation input in S702. To do. For example, as shown in FIG. 8B, the behavior information is configured to include ratio information 813 for each operation type information 812 indicating the type of operation input in association with the content ID 811 for identifying the corresponding content. You can In the present embodiment, even if a plurality of types of operation inputs are made in the receiving clients 200 that are currently viewing content, a motion corresponding to each type is applied to the mob characters 505 arranged in the three-dimensional space 510. It is assumed that the behavior information is configured to present the mob character 505 to which is applied based on the ratio. That is, the presented mob character 505 group is in a state in which the distribution of the types of operation inputs made in all the reception clients 200 is reflected, and the mob characters 505 performing different types of motions are arranged on the floor 504. To do. However, the implementation of the present invention is not limited to this, and for example, in order to create a sense of unity on the floor 504, the type of operation that is most frequently performed in the reception client 200 group that is currently viewing content is performed. Only the motion corresponding to the input may be applied to the mob character 505 group at a corresponding ratio.

In step S<b>704, the distribution unit 105 transmits, to the communication unit 108, under the control of the control unit 101, the motion data and voice data related to the main character 502 acquired from the motion capture system 400, and the behavior information configured in S<b>703. , To the connection server 300. When the communication control unit 304 receives the information necessary for reproducing the content transmitted in this way, the communication control unit 304 transfers the information to the receiving client 200 registered in the list where the content is currently viewed. .. Then, when the reception control unit 201 receives the information necessary for the reproduction, the reception control unit 201 causes the reproduction control unit 205 to perform the processing related to the reproduction, and realizes the information presentation regarding the content appreciation.

In step S705, the control unit 101 determines whether the live distribution period has ended. The control unit 101 completes this behavior control process when it determines that the period for performing live distribution has ended, and returns the process to S701 when it determines that the period for performing live distribution has not ended.

By doing so, according to the content distribution system of the present embodiment, it is possible to provide a suitable viewing experience while ensuring a sense of presence in which a large number of people are participating. More specifically, by providing content that reflects the actual behavior of the viewing user who is viewing the content while reducing the bandwidth used for live distribution, providing a more entertaining viewing experience. Can be realized.

It should be noted that in the present embodiment, each type of operation is performed without determining whether the operation input made in the reception client 200 is made by the viewing user who is viewing in which area corresponding to the viewing position. Although it is described that the ratio of the input receiving clients 200 is derived, the implementation of the present invention is not limited to this. For example, the operation input information is configured to include viewing position information, and the analysis unit 106 derives ratio information for each viewing position for each type of operation input, and behavior information is also provided for each region corresponding to the viewing position. The information may be configured to specify the proportion of the mob character 505 to which the behavior is applied. By doing so, for example, the viewing user can view the atmosphere of the surrounding mob character 505 and then select the viewing position of the desired atmosphere, or the viewing user who positively inputs an operation for exciting the content. Since the behavior of the mob character 505 is activated near the stage 503 where it is assumed that the user concentrates, it is expected that the viewing experience can be provided in which the viewing user's sense of unity in the concert live is more emphasized.

In addition, if the distribution client 100 is configured to be able to present the ratio information for each viewing position in this way, the performer obtains an index of which viewing position and direction the performance should attract. As a result, it becomes easier to guide the content viewing experience to a suitable one, and the opportunity for monetization can be expanded.

[Other Embodiments]
The present invention is not limited to the above-mentioned embodiments, and various modifications and variations can be made within the scope of the gist of the present invention. The content distribution system according to the present invention can also be realized by a program that causes one or more computers to function as a distribution device and a reception device of the system. The program can be provided/distributed by being recorded in a computer-readable recording medium or through an electric communication line.

100: distribution client, 101: control unit, 102: recording medium, 103: memory, 104: acquisition unit, 105: distribution unit, 106: analysis unit, 107: behavior control unit, 108: communication unit, 200: reception client, 201: reception control unit, 202: reception recording medium, 203: reception memory, 204: setting unit, 205: reproduction control unit, 206: drawing unit, 207: operation detection unit, 208: reception communication unit, 210: for viewpoint control Sensor, 220: Display unit, 230: Audio output unit, 240: Controller, 241: Operation detection sensor, 242: Physical button, 300: Connection server, 301: Connection control unit, 302: Connection recording medium, 303: Connection memory , 304: communication control unit, 305: connection communication unit, 400: motion capture system, 401: detection unit, 402: voice acquisition unit, 403: conversion unit, 404: output unit, 500: network

Claims (12)

A content distribution system comprising: a distribution device that distributes a content that presents a main character; and a plurality of reception devices that receive the distributed content and provide a viewing experience of the content,
In addition to the main character, the content is configured to present a plurality of sub characters different from the main character, the sub characters indicating a viewing user who is viewing using the receiving device.
The receiving device,
First communication means for communicating information with the distribution device;
Reproduction means for performing processing relating to reproduction of the content, including drawing of the main character and the sub character,
Presentation means for presenting the reproduced content;
Detection means for detecting an operation input related to viewing the content;
First control means for controlling the first communication means to cause the distribution device to transmit information indicating the detected operation input;
Have
The delivery device is
Second communication means for communicating information with the plurality of receiving devices;
Analysis means for analyzing information indicating the operation input received from the plurality of receiving devices;
Based on the analysis result by the analysis unit, a behavior control unit that determines a behavior to be taken by the plurality of sub characters,
Second control means for controlling the second communication means so as to cause the receiving device to transmit information necessary for reproducing the content, including behavior information indicating behavior determined to be caused by the plurality of sub-characters. When,
Have
The content distribution system, wherein the behavior control means determines a proportion of the sub-character having a predetermined behavior among the plurality of sub-characters. The analyzing unit specifies a ratio of the receiving device to which the operation input is made, among a plurality of receiving devices that have experienced viewing the content,
The content distribution system according to claim 1, wherein the behavior control unit determines a proportion of the sub-character that causes a behavior corresponding to the operation input, based on the identified proportion. The operation input that can be detected by the detection unit is provided in a plurality of types,
The content distribution system according to claim 1, wherein the analysis unit analyzes the information indicating the operation input for each type of the operation input. The reproduction device further includes a selection unit that selects an area to be viewed in a three-dimensional space in which the main character and the sub character are arranged when reproducing the content.
The information indicating the operation input includes information on the selected viewing area,
The content distribution system according to any one of claims 1 to 3, wherein the behavior control unit determines a behavior to be taken by the sub-character for each area where the viewing is performed. The content distribution system according to any one of claims 1 to 4, wherein the number of the plurality of sub characters in the content is the number of the viewing users or a predetermined number. The content distribution system according to claim 1, wherein the sub character does not have an appearance that identifies the corresponding viewing user. The receiving device further includes an acquisition unit that acquires motion information and audio information related to the main character,
7. The content distribution system according to claim 1, wherein the information necessary for reproducing the content includes the acquired motion information and the acquired audio information. The content distribution system according to any one of claims 1 to 7, wherein the content is binocular stereoscopic content. A distribution device that distributes content presenting a main character to a plurality of receiving devices,
In addition to the main character, the content is configured to present a plurality of sub characters different from the main character, the sub characters indicating a viewing user who is viewing using the receiving device.
The delivery device is
Communication means for communicating information with the plurality of receiving devices;
Analysis means for analyzing information received by the communication means and indicating operation input related to viewing the content in the plurality of receiving devices;
Based on the analysis result by the analysis unit, a behavior control unit that determines a behavior to be taken by the plurality of sub characters,
Control means for controlling the communication means so as to cause the receiving device to transmit information necessary for reproducing the content, including behavior information indicating behavior determined to be taken by the plurality of sub characters.
Have
The distribution device, wherein the behavior control means determines a proportion of the sub-character having a predetermined behavior among the plurality of sub-characters. A program for causing a computer to function as each unit of the distribution device according to claim 9. A receiving device that receives a content that presents a main character distributed by a distribution device and provides a viewing experience of the content,
In addition to the main character, the content is configured to present a plurality of sub characters different from the main character, the sub characters indicating a viewing user who is viewing using the receiving device.
The receiving device,
Communication means for communicating information with the distribution device;
Reproduction means for performing processing relating to reproduction of the content, including drawing of the main character and the sub-character, based on information necessary for reproduction of the content received by the communication means,
Presentation means for presenting the reproduced content;
Detection means for detecting an operation input related to viewing the content;
Control means for controlling the communication means to cause the distribution device to transmit information indicating the detected operation input,
Have
The information necessary for reproducing the content indicates a predetermined behavior among the plurality of sub-characters determined based on the operation input performed by the plurality of receiving devices providing the experience of viewing the content. A receiving device including behavior information indicating a ratio of the sub-character to be taken. A program for causing a computer to function as each unit of the receiving device according to claim 11.

Images