JP6937803B2 - Distribution A video distribution system, video distribution method, and video distribution program that delivers live video including animation of character objects generated based on the movement of the user. - Google Patents

Description

The disclosure in the present specification relates to a video distribution system, a video distribution method, and a video distribution program for live distribution of a video including an animation of a character object generated based on the movement of a distribution user.

Conventionally, there has been known a video distribution system that generates an animation of a character object based on the movement of a distribution user and delivers a video including the animation of the character object live. Such a video distribution system is described in, for example, Japanese Patent Application Laid-Open No. 2015-184689 (Patent Document 1) and "Yusuke Hasegawa, the dawn of a new era of virtual idols PROJECT Maria, CG WORLD, Japan, Bone Digital Co., Ltd., November 2017. 10th, Vol. 231 pp. 74-79 ”(Non-Patent Document 1).

A video distribution service that distributes a video containing a character object generated based on the movement of a distribution user is provided. As such a service, "Virtual Cast" provided by Dwango Co., Ltd. is known. In the virtual cast, the character object of the viewing user is included in the video distributed by the distribution user in response to the co-starring request from the viewing user who is viewing the video. As a result, the distribution user and the viewing user virtually co-star in the moving image via their respective character objects (see Non-Patent Document 2).

Yusuke Hasegawa, Dawn of a new era of virtual idols PROJECT MariA, CG WORLD, Japan, Bone Digital Co., Ltd., November 10, 2017, Volume 231 and pp. 74-79 Dwango Co., Ltd., [Virtual Cast] Convex ″, [online], [Search on May 21, 2019], Internet <URL: https://qa.nicovideo.jp/faq/show/10740?category_id=718&site_domain = default

When the number of viewing users who watch the distributed video increases, there is a possibility that the viewing user sends a large number of co-starring requests to the distribution user. In order to maintain the quality of the delivered video, what kind of viewing user is the co-starring partner is a serious concern for the delivered user. When a large number of co-starring requests are made, it becomes difficult for the distribution user to select a viewing user suitable as a co-starring partner from the large number of co-starring requests.

When determining whether or not the viewing user can co-star based on the co-starring request from the viewing user and sending a notification indicating the determined co-starring or not to the viewing user, the notification is to be generated and transmitted. There is a problem that the processing load and communication traffic increase. The above problem becomes more serious as the number of viewing users increases. Since popular distribution users are expected to obtain tens of thousands or more viewing users, technical measures are required to solve or alleviate the above problems.

An object of the present disclosure is to provide technical improvements that solve or alleviate at least some of the problems of the prior art described above. One of the more specific purposes of the present disclosure is to provide a video distribution system, a video distribution method, and a video distribution program that can reduce the processing load related to the processing required by the co-starring request from the viewing user. ..

One aspect of the present invention relates to a video distribution system comprising one or more computer processors. By executing a computer-readable instruction, the one or more computer processors distribute a moving image including a first character object generated based on the movement of the distribution user to each client device of the plurality of viewing users. , A plurality of participating users are selected from the plurality of viewing users in response to receiving requests for participation in the moving image from the plurality of viewing users, and a plurality of firsts corresponding to each of the plurality of participating users. Two character objects are included in the video, and at least a part of the plurality of participating users accepts a co-starring request for co-starring with the distribution user, and one or more guests from the participating users who have made the co-starring request. Select a user, send a notification regarding the selection of the one or more guest users to each of the one or more guest users, and correspond to at least a part of the one or more guest users in the second character object. The guest character object to be used is included in the moving image.

In one aspect, the one or more computer processors give commands to each client device of the plurality of participating users to activate a function for making the co-starring request in each client device of the plurality of participating users. Send.

In one aspect, the co-starring request from one of the plurality of participating users includes a message from the one participating user.

In one aspect, the co-starring request from one of the plurality of participating users includes motion data from the one participating user. The one or more computer processors causes the second character object corresponding to the one participating user to perform an operation corresponding to the motion data in the moving image.

In one aspect, the selection of the plurality of participating users based on the participation request is performed based on a predetermined algorithm.

In one aspect, the selection of the one or more guest users based on the co-starring request is made according to the selection of the distribution user.

The one or more computer processors do not send a notification regarding the selection of the plurality of participating users to the plurality of viewing users other than the plurality of participating users.

One aspect of the present invention relates to a video distribution method executed by one or more computer processors executing computer-readable instructions. The video distribution method includes a step of distributing a video including a first character object generated based on the movement of the distribution user to each client device of a plurality of viewing users, and a request for participation in the video from the plurality of viewing users. A step of selecting a plurality of participating users from the plurality of viewing users according to the acceptance of the above, and a step of including a plurality of second character objects corresponding to each of the plurality of participating users in the moving image. The process of receiving a co-starring request for co-starring with the distribution user from at least a part of the plurality of participating users and selecting one or a plurality of guest users from the participating users who have made the co-starring request, and the above-mentioned one. Alternatively, a step of transmitting a notification regarding selection of a plurality of guest users to each of the one or a plurality of guest users, and a guest character object corresponding to at least a part of the one or a plurality of guest users among the second character objects. It includes a step of including it in the moving image.

One aspect of the present invention relates to a video distribution program. This video distribution program includes a step of distributing a video including a first character object generated based on a movement of a distribution user to one or a plurality of computer processors to each client device of a plurality of viewing users, and the plurality of steps. A process of selecting a plurality of participating users from the plurality of viewing users in response to receiving a request for participation in the video from the viewing user, and a plurality of second characters corresponding to each of the plurality of participating users. The process of including the object in the moving image and one or more guests from among the participating users who have received the co-starring request for co-starring with the distribution user from at least a part of the plurality of participating users and made the co-starring request. A step of selecting a user, a step of transmitting a notification regarding selection of the one or more guest users to each of the one or a plurality of guest users, and a step of transmitting at least one or a plurality of the guest users of the second character object. The step of including the guest character object corresponding to a part in the moving image and the step of including the guest character object are executed.

According to the embodiment of the present invention, it is possible to reduce the processing load related to the processing required by the co-starring request from the viewing user.

It is a block diagram which shows the outline of the moving image distribution system by one Embodiment of this invention. It is a block diagram which shows the moving image distribution system of FIG. 1 in more detail. It is a schematic diagram which shows typically the equipment of the studio which produces the moving image distributed by the moving image distribution system of FIG. It is a figure explaining the user data stored in the moving image distribution system of FIG. It is a figure explaining the avatar data stored in the moving image distribution system of FIG. It is a figure for demonstrating the information included in the co-starring request. It is a figure which shows the example of the moving image which is displayed on the client apparatus 10a in one Embodiment. It is a figure which shows the example of the moving image which is displayed on the client apparatus 10a in one Embodiment. It is a figure which shows the example of the moving image which is displayed on the client apparatus 10a in one Embodiment. It is a figure which shows the example of the moving image which is displayed on the client apparatus 10a in one Embodiment. It is a figure which shows the example of the co-starring moving image which is displayed on the client apparatus 10a in one Embodiment. It is a flow figure which shows the flow of the moving image distribution processing in one Embodiment. It is a flow chart which shows the flow of processing a participation request in one Embodiment. It is a flow figure which shows the flow of the co-starring processing in one Embodiment.

Hereinafter, various embodiments of the present invention will be described with reference to the drawings as appropriate. The same or similar components are designated by the same reference numerals in a plurality of drawings.

An outline of a video distribution system according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing an outline of a moving image distribution system 1 according to an embodiment of the present invention. As shown in the figure, the moving image distribution system 1 includes a client device 10 and a server device 20. Although only one client device 10 is shown in FIG. 1 for simplification of the illustration, the moving image distribution system 1 can include a plurality of client devices 10. The server device 20 is configured to deliver a moving image. The client device 10 is configured to play a moving image distributed from the server device 20. The viewing user can view the moving image distributed from the server device 20 by using the client device 10. As will be described later, the server device 20 may be configured to distribute a moving image including an animation of a character object of a distribution user.

The viewing user can send a participation request for displaying his / her character object on the moving image to the server device 20 while viewing the distributed moving image on the client device 10. The server device 20 determines whether or not to display the character object of the viewing user in the moving image based on the participation request. In response to this determination, the server device 20 can generate a moving image so as to include the character object of the viewing user, and can distribute the moving image including the character object of the viewing user. In this way, the viewing user can participate in the moving image distributed from the server device 20 via his / her character object. Of a plurality of viewing users, a viewing user whose character object is displayed in the video being distributed, that is, a viewing user who participates in the video through his / her character object may be referred to as a participating user. There may be an upper limit on the number of participating users who are allowed to participate in the video. The server device 20 can select a maximum number of participating users or less from a plurality of viewing users who have requested participation.

Participating users can use the client device 10 to send a co-starring request for applying for co-starring with the distribution user to the server device 20. Upon receiving the co-starring request, the server device 20 determines whether or not to allow the co-starring of the participating user and the distribution user who sent the co-starring request. When a decision is made to allow the participating user and the distribution user to co-star, the server device 20 generates a co-starring video including the character object of the distribution user and the character object of the participating user, and distributes the co-starring video. be able to. In this way, in the video distribution system 1, the distribution user and the viewing user can virtually co-star in the video via their respective character objects. When the distribution user and the viewing user virtually co-star in the video via their respective character objects, the video may be referred to as a co-star video. In addition, the viewing user who is co-starring with the distribution user in the co-starring video may be called a guest user. The guest user may be one person or a plurality of people. When a plurality of guest users are determined, the distribution user can co-star with the plurality of guest users in sequence.

In one embodiment, the moving image distribution system 1 is configured to accept a co-starring request only from the participating users who are allowed to participate in the moving image among the viewing users who watch the moving image. For example, the function for making a co-starring request is activated in the client device 10 of the participating user whose participation request is accepted. As a result, the co-starring request can be accepted only from the participating users among the viewing users. Even if the number of viewing users increases, the number of co-starring requests received from the client device 10 can be suppressed.

The server device 20 determines whether or not the viewing user can participate based on the participation request from the viewing user, and then notifies all the viewing users who have made the participation request whether or not the participation in the video is permitted. May be configured not to send. Since the delivered video contains the character object of the viewing user who is allowed to participate, the viewing user himself / herself in the video to be viewed without receiving a notification indicating whether or not participation in the video is permitted. It is possible to know whether or not one's participation request is permitted based on whether or not the character object of is included. As described above, in one embodiment, since the information indicating whether or not the viewing user is allowed to participate in the video is included in the video, all the viewing users who have requested the participation other than the video are included. No notification is sent to indicate whether or not the viewing user is allowed to participate in the video. As a result, in the video distribution system 1, it is not necessary to send a notification indicating whether or not the viewing user is allowed to participate in the video to all the viewing users who have requested participation, separately from the distribution of the video. A notification indicating that participation may be permitted may be transmitted to the client device 10 of the participating user who is permitted to participate in the moving image among the viewing users. The client device 10 of the participating user may activate the function for the participating user to make a co-starring request in response to receiving the notification indicating that the participation is approved.

The server device 20 determines whether or not the participating user and the distribution user can co-star based on the co-starring request from the participating user, and then the co-starring is permitted to the participating user (guest user) who is permitted to co-star. May be configured to send a permission notice to notify. By receiving the permission notification, the guest user can know that he / she can co-star with the character object of the distribution user in the video. The guest user can prepare for the co-starring after being notified that the co-starring is permitted. Preparations for co-starring may include, for example, the purchase of additional content (eg, avatar parts) to enrich the depiction of one's character object (eg, avatar) displayed in the video. The server device 20 may be configured to send a disapproval notice notifying that the co-starring is not permitted to the participating users who have made the co-starring request and the co-starring is not permitted.

According to the video distribution system 1, a notification indicating whether or not the participation request is permitted is not transmitted to all the viewing users who have made the participation request other than the video to be distributed. As a result, the processing load on the server device 20 can be reduced, and an increase in communication traffic can be suppressed. On the other hand, in one embodiment, since the guest user is notified of the permission notification indicating that the co-starring is permitted based on the co-starring request, the guest user can prepare for the co-starring with the distribution user. Since the guest user is a user who has been granted a participation request and a co-starring request among all the viewing users, the number is smaller than the total number of viewing users who make a participation request. The processing load required for generation and transmission is less than when a notification is transmitted to the entire viewing user.

A video distribution system according to an embodiment will be described with reference to FIGS. 2 to 6. FIG. 2 is a block diagram showing a video distribution system 1 according to an embodiment, and FIG. 3 is a schematic diagram schematically showing equipment of a studio in which a video distributed by the video distribution system 1 is produced. 4 and 5 are diagrams for explaining the information stored in the moving image distribution system 1, and FIG. 6 is a diagram for explaining the information included in the co-starring request.

As shown in FIG. 2, the moving image distribution system 1 can include a studio unit 30 and a storage 25 in addition to the client device 10 and the server device 20 shown in FIG. In the embodiment shown in FIG. 2, three client devices 10a to 10c are shown as the client devices 10. The client devices 10a to 10c, the server device 20, and the storage 25 are connected to each other via the network 15 so as to be able to communicate with each other.

The moving image can be distributed from the server device 20 to each of the client devices 10a to 10c. The first viewing user who is the user of the client device 10a, the second viewing user who is the user of the client device 10b, and the third viewing user who is the user of the client device 10c watch the delivered video on each client device. Can be done. The video distribution system 1 may include a number of client devices less than three, or may include more than three client devices. Users who view moving images distributed from the server device 20 by using the client devices 10a to 10c and other client devices are collectively referred to as viewing users. In order to view the moving image delivered from the server device 20, application software having an instruction set that realizes a function for viewing the moving image may be downloaded and installed in the client devices 10a to 10c. Application software having a function of viewing a moving image distributed from such a server device 20 may be referred to as "viewing application software" or "viewing application". Viewing of the moving image delivered from the server device 20 may be performed by executing the function of the viewing application. In one embodiment, the video delivered from the server device 20 is viewed by the client device of the client devices 10a to 10c in which the viewing application is installed, and from the client device in which the viewing application is not installed. It does not have to be possible. A viewing user who watches a moving image distributed from the server device 20 can display his / her own character object on the moving image. In other words, the viewing user can participate in the video being distributed via his or her character object. An example of this character object is the avatar. The viewing user can display his / her own avatar on the distributed video and interact with the distribution user (or the character in the distributed video) or another viewing user through the avatar.

The client devices 10a to 10c are information processing devices such as smartphones. In addition to the smartphone, the client devices 10a to 10c may be a mobile phone, a tablet terminal, a personal computer, an electronic book reader, a wearable computer, a game console, and various information processing devices capable of reproducing moving images other than these. Each of the client devices 10a to 10c includes a computer processor, a memory, a communication I / F, a display, a sensor unit including various sensors such as a gyro sensor, a sound collecting device such as a microphone, and a storage for storing various information. May be good.

The display of the client devices 10a to 10c includes a display panel and a touch panel. The touch panel is configured to be able to detect a player's touch operation (contact operation). The touch panel can detect various touch operations such as tapping, double tapping, and dragging of the player. The touch panel may include a capacitive proximity sensor and may be configured to be capable of detecting a player's non-contact operation.

The viewing user can input a message related to the delivered moving image or other messages via the touch panel of the client devices 10a to 10c. This message may be, for example, a text format message, a still image format message, a moving image format message, a voice message, or an electronic message in any other format. Each of the client devices 10a to 10c may be configured to post (send) a message input by the viewing user to the server device 20.

In the illustrated embodiment, the server device 20 includes a computer processor 21, a communication I / F 22, and a storage 23.

The computer processor 21 is an arithmetic unit that loads various programs that realize an operating system and various functions from the storage 23 or other storage into a memory and executes instructions included in the loaded programs. The computer processor 21 is, for example, a CPU, an MPU, a DSP, a GPU, various arithmetic units other than these, or a combination thereof. The computer processor 21 may be realized by an integrated circuit such as an ASIC, PLD, FPGA, or MCU. Although the computer processor 21 is illustrated as a single component in FIG. 2, the computer processor 21 may be a collection of a plurality of physically separate computer processors. In the present specification, the program described as being executed by the computer processor 21 or the instructions contained in the program may be executed by a single computer processor or may be executed by a plurality of computer processors in a distributed manner. May be good. Further, the program executed by the computer processor 21 or the instructions included in the program may be executed by a plurality of virtual computer processors. The functions realized by the computer processor 21 will be described later.

The communication I / F 22 is implemented as hardware, firmware, communication software such as a TCP / IP driver or PPP driver, or a combination thereof. The server device 20 can send and receive data to and from other devices via the communication I / F 22.

The storage 23 is a storage device accessed by the computer processor 21. The storage 23 is, for example, a magnetic disk, an optical disk, a semiconductor memory, or various storage devices other than those capable of storing data. Various programs can be stored in the storage 23. At least a part of the program and various data that can be stored in the storage 23 may be stored in a storage (for example, a storage 25) that is physically separate from the server device 20. Details of the data stored in the storage 23 will be described later.

Many of the components of the studio unit 30 are located, for example, in the studio room R shown in FIG. As shown in the figure, actor A1 is performing in studio room R. The studio unit 30 is configured to detect the movement and facial expression of the actor A1 and output the detected information to the server device 20.

The actor A1 is a target whose movement and facial expression are captured by a group of sensors described later provided in the studio unit 30. In the illustrated embodiment, the actor A1 is a person who takes actions to advance a talk, a performance, and a program distributed as a moving image other than these. In addition to humans, actor A1 may be an animal or any other moving object (abiotic). The actor A1 may be, for example, a robot capable of independent movement. The number of actors in the studio room R may be 1 or 3 or more.

The studio unit 30 includes six motion sensors 31a to 31f mounted on the actor A1, a controller 33a held by the left hand of the actor A1, a controller 33b held by the right hand of the actor A1, and a fitting 37b. It has a camera 37a attached to the head of the actor A1 via a wristwatch-type actor device 91 attached to the left arm of the actor A1. A microphone for acquiring voice data may be attached to the fitting 37b. This microphone can acquire the utterance of the actor A1 as voice data. The microphone may be a wearable microphone mounted on the actor A1 via the mounting tool 37b, or may be a mounting type microphone installed on the floor, wall, or ceiling of the studio room R. In addition to the above components, the studio unit 30 includes a base station 35a, a base station 35b, a digital camera 36a, and a display 39. A supporter computer 40 is installed in the adjacent room separated from the studio room R by a glass window. The server device 20 may be installed in the same room where the supporter computer 40 is installed.

The motion sensors 31a to 31f cooperate with the base station 35a and the base station 35b to detect their own positions and orientations. In one embodiment, the base station 35a and the base station 35b are multi-axis laser emitters. After emitting the flashing light for synchronization, the base station 35a scans the laser light, for example, around the vertical axis. The base station 35b scans the laser beam, for example, around a horizontal axis. The motion sensors 31a to 31f may each include a plurality of optical sensors that detect the incident of blinking light and laser light from the base station 35a and the base station 35b, respectively. The motion sensors 31a to 31f have a time difference between the incident timing of the blinking light and the incident timing of the laser light, the light receiving time of each optical sensor, the incident angle of the laser light detected by each optical sensor, and other than these if necessary. You may detect your own position and orientation based on the information of. The motion sensors 31a to 31f may be, for example, the Vive Tracker provided by HTC CORPORATION. The base station 35a and the base station 35b may be, for example, a base station provided by HTC CORPORATION.

The detection information indicating the position and orientation of each motion sensor calculated by each of the motion sensors 31a to 31f is transmitted to the server device 20. This detection information may be wirelessly transmitted from each of the motion sensors 31a to 31f to the server device 20. Since the base station 35a and the base station 35b emit flashing light and scan the laser light at regular intervals, the detection information of each motion sensor is updated at each interval. The number of base stations may be three or more. The position of the base station may be changed as appropriate. For example, in addition to or in place of the base station located in the upper corner of the space to be detected by the tracking sensor shown in FIG. An arranged base station may be provided. For example, there may be four base stations.

In the illustrated embodiment, the actor A1 is equipped with six motion sensors 31a to 31f. The motion sensors 31a, 31b, 31c, 31d, 31e, and 31f are attached to the left back, right back, left foot, right back, hip, and top of the head of the actor A1, respectively. The motion sensors 31a to 31f may be attached to the actor A1 via an attachment. The motion sensors 31a to 31f shown in FIG. 2 are examples. The motion sensors 31a to 31f can be attached to various parts of the actor A1. The number of motion sensors mounted on the actor A1 may be 5 or less, or 7 or more. In this way, by detecting the positions and orientations of the motion sensors 31a to 31f attached to each part of the body of the actor A1, the movement of the body of the actor A1 can be detected. The body movement of actor A1 may be detected by an optical motion capture system.

In one embodiment, a large number of infrared LEDs are mounted on each of the plurality of motion sensors mounted on the actor A1, and the light from the infrared LEDs is emitted by an infrared camera provided on the floor or wall of the studio room R. By detecting, the position and orientation of each of the motion sensors may be detected. A visible light LED may be used instead of the infrared LED, and the position and orientation of each of the motion sensors may be detected by detecting the light from the visible light LED with a visible light camera. In this way, each of the plurality of motion sensors mounted on the actor is provided with a light emitting unit (for example, an infrared LED or a visible light LED), and the light from the light emitting unit is provided in the studio room R as a light receiving unit (for example). For example, the position and orientation of each of the motion sensors may be detected by detecting with an infrared camera or a visible light camera).

In one embodiment, a plurality of reflection markers can be used instead of the motion sensors 31a to 31f. The reflective marker is attached to each of the actors A1 with an adhesive tape or the like. The position and orientation of the reflection marker can be detected by photographing the actor A1 to which the reflection marker is attached, generating shooting data, and performing image processing on the shooting data. In one embodiment, the detection of body movement in the actor A1 may be performed using a suit with a built-in inertial sensor. The movement of the actor A1 can be detected by analyzing the sensor signal output from the inertial sensor of the suit with the built-in inertia sensor worn on the actor A1. As a suit with a built-in inertia sensor, a suit with a built-in inertia sensor compatible with a motion capture system called MVN developed by Xsens.com can be used.

The controller 33a and the controller 33b output a control signal corresponding to the operation of the actor A1 to the server device 20.

The digital camera 36a captures the actor A1 and generates imaging data. The position, imaging direction, and angle of view of the digital camera 36a are set according to the operation of the operator. The digital camera 36a transmits image setting information indicating its position, image pickup direction, and angle of view to the server device 20. Similarly, the position, imaging direction, and angle of view of the digital camera 36b are set according to the operation of the operator. In the illustrated embodiment, the digital camera 36a is provided on the left front side of the actor A1.

The camera 37a is attached to the head of the actor A1 as described above. For example, the camera 37a is arranged so that the face of the actor A1 can be imaged. The camera 37a continuously images the face of the actor A1 and acquires the image data of the face of the actor A1. The camera 37a transmits the image capture data of the face of the actor A1 to the server device 20. The camera 37a may be a 3D camera capable of detecting the depth of a person's face. The camera 37a may be, for example, a lidar device provided on the iPhone X provided by Apple Inc. The iPhone X is attached to, for example, a headgear mounted on the head of the actor A1. The camera 37a may be a fixed type mounted on a camera stand installed in the studio room R instead of being mounted on the actor A1.

The display 39 is configured to display information received from the supporter computer 40. The information transmitted from the supporter computer 40 to the display 39 may include, for example, text information, image information, and various other information. The display 39 is arranged at a position visible to the actor A1.

The actor device 91 is configured to generate a stimulus other than a perceptible sound by the actor A1 in response to a drive signal from the server device 20. When the actor device 91 generates a sound, the sound may be synthesized with the distributed content. Therefore, the actor device 91 is configured to generate a stimulus other than the sound. The actor device 91 can generate a stimulus other than the auditory stimulus (typically sound), for example, a tactile stimulus, a visual stimulus, or a stimulus perceptible by the actor A1 different from the other auditory stimuli. The actor device 91 may include an actuator 91a. The actuator 91a is configured to operate by a drive signal from the server device 20. The actuator 91a may be configured to vibrate when a drive signal is input. As the actor device 91, for example, Apple Watch provided by Apple Inc. can be used. The actor device 91 can take various shapes other than the wristwatch type. The actor A1 can be equipped with a plurality of actor devices 91 of the same type or different types from each other. The actor device 91 may be configured to operate in response to a drive signal from other than the server device 20. The plurality of actor devices 91 may be associated with an actor ID that identifies the actor A1. The actor device 91 can transmit a tactile stimulus to the actor A1 via the actuator 91a. The actor device 91 may be an LED (Light Emitting Diode) or a perceptual display device. LEDs and perceptual display devices can generate visual stimuli. The perceptual display device may be an electrical tactile display that drives the nerve axons under the skin with an electric current flowing from the electrodes.

In the illustrated embodiment, the supporter computer 40 is installed in a room adjacent to the studio room R. Since the room in which the supporter computer 40 is installed and the studio room R are separated by a glass window, the operator of the supporter computer 40 (sometimes referred to as "supporter" in the present specification) can visually recognize the actor A1. .. In the illustrated embodiment, supporter B1 and supporter B2 are present as operators of the supporter computer 40.

The supporter computer 40 may be configured so that the settings of the components of the studio unit 30 can be changed according to the operations of the supporter B1 and the supporter B2. The supporter computer 40 can, for example, set the scanning interval by the base station 35a and the base station 35b, set the position or orientation of the digital camera 36a and the tracking sensor 36b, and change various settings of various other devices. .. The switching of the active sensor, which will be described later, can be executed based on the operation of the supporter computer 40 by the supporter B1 and the supporter B2. At least one of the supporter B1 and the supporter B2 can input a message to the supporter computer 40 and display the input message on the display 39.

The components and functions of the studio unit 30 shown in FIG. 3 are exemplary. The studio unit 30 applicable to the present invention may include various components (not shown). The studio unit 30 may include a projector. The projector can project a moving image delivered to the client device 10a or other client devices onto the screen S.

On the other hand, the shooting for generating the delivered moving image may be performed by a simpler facility. For example, an information processing terminal such as a smartphone generates face motion data, which is a digital representation of an actor's facial movement, and based on the face motion data, the facial expression moves in synchronization with the movement of the actor's facial expression, but the body You may generate an animation of a character object that does not move. Such face motion data generation may be performed by the client device 10a to the client device 10c. In order to generate face motion data, application software having an instruction set that realizes a function for generating face motion data may be downloaded and installed in the client devices 10a to 10c. Application software having a function of generating such face motion data may be referred to as "distribution application software" or "distribution application". The distribution application and the above-mentioned viewing application may be separate application software, or a single application software may have both the function of the distribution application and the function of the described viewing application. The face motion data can include tracking data of the posture of the head of the actor, feature points which are digital representations of the feature points of the face of the actor, and blended face data expressing the deformation rate thereof. An information processing terminal such as a smartphone may create an image that exaggerates the facial expression of the actor based on the face motion data, and superimpose the image on the face of the character object. When generating an animation of a character object based on face motion data, motion sensors 31a, 31b, 31c, 31d, 31e, and 31f, base stations 35a, 35b, and digital camera 36a for detecting the movement of the actor's body. Is not required, so it is possible to generate an animation of a character object with simpler equipment. If the motion sensors 31a, 31b, 31c, 31d, 31e, and 31f, the base stations 35a, 35b, and the digital camera 36a are unnecessary, the character object can be generated at a place other than the studio room R, for example, at the actor's home. Because it can be done, convenience is improved.

Next, in one aspect, the information stored in the storage 23 will be described. In the illustrated embodiment, the storage 23 stores model data 23a, object data 23b, user data 23c, avatar data 23d, and various information necessary for generation and distribution of distribution moving images other than the above.

The model data 23a is model data for generating a character animation. The model data 23a may be three-dimensional model data for generating a three-dimensional animation, or may be two-dimensional model data for generating two-dimensional model data. The model data 23a includes, for example, rig data indicating the skeleton of the character (sometimes referred to as “skeleton data”) and surface data indicating the shape and texture of the surface of the character. The model data 23a can include a plurality of model data different from each other. The plurality of model data may have different rig data from each other, or may have the same rig data. The plurality of model data may have different surface data from each other, or may have the same surface data.

The object data 23b includes asset data for constructing a virtual space constituting a moving image. The object data 23b is data for drawing the background of the virtual space constituting the moving image, data for drawing various objects displayed in the moving image, and various objects displayed in the moving image other than these. Contains data. The object data 23b may include object position information indicating the position of the object in the virtual space.

In addition to the above, the object data 23b may include a gift object to be displayed in the moving image based on a display request from the viewing user of the client devices 10a to 10c. Gift objects can include effect objects, regular objects, and decoration objects. The viewing user can purchase the desired gift object.

The effect object is an object that affects the impression of the entire viewing screen of the distributed moving image, and is, for example, an object that imitates confetti. The object imitating confetti may be displayed on the entire viewing screen, whereby the impression of the entire viewing screen before and after the display can be changed. Effect objects may appear to overlap with character objects, but differ from decorative objects in that they are not displayed in association with a particular part of the character object.

A normal object is an object that becomes a digital gift from a viewing user to an actor (for example, actor A1) or a character in a moving image, for example, an object that imitates a stuffed animal or a bouquet. In one aspect, the normal object is displayed on the screen of the moving image so as not to come into contact with the character object. In one aspect, the normal object is displayed on the screen of the moving image so as not to overlap with the character object. Ordinary objects may appear to overlap objects other than character objects in virtual space. Ordinary objects may appear to overlap with character objects, but differ from decorative objects in that they are not displayed in association with a particular part of the character object. In one aspect, when the normal object is displayed overlapping with the character object, the normal object overlaps with a part other than the head including the face of the character object and does not overlap with the head of the character object. Is displayed.

A decoration object is an object that is displayed on the screen in association with a specific part of the character object. In one aspect, the decorative object displayed on the screen in association with a specific part of the character object is displayed on the screen so as to be in contact with the specific part of the character object. In one aspect, the decorative object displayed on the screen in association with a particular part of the character object is displayed on the screen so as to cover part or all of the particular part of the character object.

The decorative object is, for example, an object that can be attached to accessories (alice band, necklace, earrings, etc.), clothes (T-shirt, etc.), costumes, and other character objects attached to the character object. The object data 23b corresponding to the decoration object may include mounting position information indicating which part of the character object the decoration object is associated with. The mounting position information of a certain decorative object can indicate to which part of the character object the decorative object is mounted. For example, when the decoration object is a Katyusha, the mounting position information of the decoration object may indicate that the decoration object is mounted on the "head" of the character object. When the decorative object is a T-shirt, the mounting position information of the decorative object may indicate that the decorative object is mounted on the "body" of the character object.

A display time may be set for each of the gift objects according to the type of the gift object. In one aspect, the display time of the decorative object may be set longer than the display time of the effect object and the display time of the normal object. For example, the display time of the decorative object may be set to 60 seconds, the display time of the effect object may be set to 5 seconds, and the display time of the normal object may be set to 10 seconds.

The object data 23b may include a list showing gift objects owned by the user of the video distribution system 1. In the possession list, an object ID that identifies the gift object possessed by the viewing user may be stored in association with the user's account information. The user's account information is, for example, a user ID that identifies the user. This user may include, for example, first viewing users to third viewing users of the client devices 10a to 10c.

As shown in FIG. 4, the user data 23c can include account information of each user (for example, a viewing user) of the video distribution system 1 and avatar information regarding the avatar used by each user. In other words, the storage 23 can store avatar information about the avatar used by each user and information about other users in association with the account information of each user of the video distribution system 1. The account information of a user is, for example, a user ID that identifies the user. The user ID of the user may be issued when the user registers the use of the moving image service provided by the server device 20 by using the client device 10a to the client device 10c. A common user ID may be used when using the service of the server device 20 by using the viewing application and when using the service of the server device 20 by using the distribution application, or by using the viewing application. A different user ID may be used depending on whether the service of the server device 20 is used or the service of the server device 20 is used by using the distribution application. When different user IDs are used when using the service of the server device 20 using the viewing application and when using the service of the server device 20 using the distribution application, the viewing application of the same user The user ID for using the service (viewing user ID) and the user ID for using the distribution application (distribution user ID) may be stored in association with each other. The service of the server device 20 that can be received by using the viewing application is, for example, a video viewing service distributed from the server device 20. The service of the server device 20 that can be received by using the distribution application is, for example, a service that distributes a moving image including its own character object.

The avatar information of a certain user is, for example, an avatar ID that identifies the avatar that the user uses in the video distribution system 1. The avatar ID is assigned to the user when the user registers the avatar. The registration of the avatar is performed by the user selecting the head, hairstyle, clothes, accessories, and other parts that make up the avatar. The registration of the avatar may be performed by the client devices 10a to 10c in which the distribution application is installed. That is, the registration of the avatar may be performed by executing the function of the distribution application. In one embodiment, the avatar registration is performed by the client device in which the distribution application is installed among the client devices 10a to 10c, and may not be executed from the client device in which the distribution application is not installed. The registration of the avatar may be performed by application software (hereinafter, may be referred to as an "avatar creation application") having a function of creating, editing, and / or registering an avatar different from the distribution application. When registering an avatar by the distribution application and / or the avatar creation application, as described with reference to FIG. 3, an avatar ID that identifies the avatar is assigned, and the avatar ID is associated with the user ID of the user. It may be managed as user information. After registering the avatar, the user can change the avatar. For example, after the avatar is registered, some or all of the parts of the avatar can be changed. For example, even after registration, the avatar's hairstyle, possessed items, and other parts that make up the avatar can be changed.

In one aspect, it is distributed from the viewing history data indicating the viewing history of the user, the evaluation data indicating the evaluation of the user by another user, and the server device 20 other than these, in association with the user ID of the user. Data related to the video may be included.

As shown in FIG. 5, the avatar data 23d can include avatar identification information of the avatar used by the user in the moving image distribution system 1 and avatar display information for displaying the avatar in the moving image. In other words, the storage 23 can store the avatar display information for displaying the avatar in association with the avatar identification information of the avatar. The avatar identification information is, for example, an avatar ID that identifies the avatar. Avata is displayed in a moving image as, for example, an image imitating a human or an animal. The avatar display information is information used to display the avatar in the moving image. The avatar information includes, for example, parts information showing images of the head, hairstyle, facial parts (eyes, nose, mouth, etc.), torso, clothes, accessories, items, and other parts that make up the avatar. included. The user can register his / her own avatar by selecting a favorite part image. The avatar display information may include 2D display information for displaying the avatar in the moving image in 2D and 3D display information for displaying the avatar in the moving image in 3D. The 3D display information includes parts information that shows the image of the part for displaying the avatar three-dimensionally in the video, rig data for expressing the movement of the avatar in three dimensions, and other avatars that are displayed three-dimensionally. Contains known information used to do this.

Next, the functions realized by the computer processor 21 will be described more specifically. The computer processor 21 executes a computer-readable instruction included in the distribution program to execute a video generation unit 21a, a video distribution unit 21b, a participation request processing unit 21c, an avatar display unit 21d, an avatar control unit 21e, and a co-starring request. It functions as a processing unit 21f. At least a part of the functions realized by the computer processor 21 may be realized by a computer processor other than the computer processor 21 of the moving image distribution system 1. At least a part of the functions realized by the computer processor 21 may be realized by, for example, a computer processor mounted on the supporter computer 40 or another computer processor (not shown).

The moving image generation unit 21a constructs a virtual space and generates moving image information (three-dimensional moving image data) for drawing a drawing area of the virtual space determined based on the setting information of the virtual camera. The setting information of the virtual camera may include, for example, the position of the virtual camera, the gaze position, the gaze direction, and the angle of view in the virtual space. The moving image information can be generated based on the object data 23b and other data as needed. The moving image generation unit 21a renders the three-dimensional moving image data to generate a plurality of moving image frames. This moving image frame is generated at a predetermined frame rate. The virtual space constructed by the moving image generation unit 21a is defined as, for example, an omnidirectional three-dimensional space that surrounds the center position thereof. The position in the virtual space can be specified by the coordinate values in the three-dimensional global coordinate system set in the virtual space.

The moving image information generated by the moving image generation unit 21a may include information about the character object of the actor A1 in the drawing area of the virtual space. The moving image information may be generated so as to reflect the movement of the actor A1 in real time. The information about the character object of the actor A1 may include body motion data of each part of the body of the actor A1 and face motion data which is a digital representation of the facial movement of the actor A1. The body motion data is generated based on, for example, the detection information of each of the motion sensors 31a to 31f. Body motion data is generated at any time over time. Body motion data may be generated at predetermined sampling time intervals. In this way, the body motion data can represent the body movement of the actor A1 as digital data in chronological order. In the illustrated embodiment, since the motion sensors 31a to 31f are attached to the left and right limbs, hips, and head of the actor A1, the motion sensors 31a to 31f of the actor A1 are based on the detection information of the motion sensors 31a to 31f. It is possible to digitally represent the position and orientation of the whole body over time. The body motion data can define, for example, the position and rotation angle of the bones constituting the rig data included in the model data 23a. The face motion data is generated based on, for example, the image pickup data of the camera 37a. Face motion data is generated at any time with the passage of time. For example, face motion data may be generated at predetermined sampling time intervals. In this way, the face motion data can digitally represent the facial movement (change in facial expression) of the actor A1 in time series. By applying the body motion data and the face motion data to the predetermined model data included in the model data 23a, an animation of a character object that moves in the virtual space and whose facial expression changes may be generated. In this way, the moving image information may include information representing the animation of the character object that moves in synchronization with the movement of the body and facial expression of the actor A1 based on the body motion data and the face motion data related to the actor A1. The moving image generation unit 21a can generate a sequence of moving image frames representing an animation of a character object that moves in synchronization with the movement of the body and facial expression of the actor A1 based on the moving image information obtained as described above. The moving image generation unit 21a may synthesize the sound of the actor A1 acquired from the microphone of the studio unit 30 with the generated moving image.

The moving image generation unit 21a can load the avatar display information necessary for displaying the avatar for the avatars that may participate in the moving image before the moving image is distributed. Among the avatar display information, the moving image generation unit 21a can load the 3D display information that takes a long time to load and has a large processing load for loading. This loaded avatar display information can be referred to when the avatar is displayed in the moving image based on the participation request from the user. The use of the loaded avatar display information when displaying the avatar in the moving image will be described later.

The moving image distribution unit 21b distributes a sequence of moving image frames generated by the moving image generating unit 21a. The moving image frame may be compressed and encoded by a moving image distribution unit 21b by a predetermined coding method such as MPEG-4, or the compressed and encoded moving image frame may be distributed. In the present specification, when referring to the distribution of a moving image, it may mean the distribution of a sequence of moving image frames generated as described above.

The delivered moving image is received by the client devices 10a to 10c via the network 15 and played back in the client devices 10a to 10c. This moving image may be distributed to a client device (not shown) installed in the studio room R, and may be projected onto the screen S from this client device via a short focus projector. Further, this moving image may be distributed to the supporter computer 40. As a result, the supporter B1 and the supporter B2 can confirm the viewing screen of the distributed moving image.

A user who is involved in the generation and / or distribution of the moving image distributed from the moving image distribution unit 21b may be referred to as a distribution user. In the above example, the moving image delivered from the server device 20 includes a character object generated based on the body motion data and the face motion data of the actor A1, and the supporters B1 and the supporters B2 cooperate as necessary. Obtained and delivered. Therefore, not only the actor A1 but also the actor A1, the supporter B1, and the supporter B2 may be collectively referred to as a “delivery user”.

7a and 7b show an example of displaying a moving image generated by the moving image generation unit 21a and distributed by the moving image distribution unit 21b. 7a and 7b show a display example of a moving image distributed by the moving image distribution unit 21b and reproduced by the client devices 10a to 10c. Hereinafter, the images of FIGS. 7a and 7b will be described on the assumption that they are displayed on the client device 10a used by the first viewing user. The image shown in FIG. 7a may be displayed on the display of the client device 10a by executing the viewing application on the client device 10a and playing back the moving image distributed from the server device 20.

As shown, the image 60a includes a character object 51, a floor object 54a on which the character object 51 stands, and a screen object 54b that defines the rear edge of the stage. The image 60a also includes a display object 65 on which comments and system notifications from viewing users are displayed. The character object 51 is depicted so as to move in the moving image in synchronization with the movement of the body and facial expression of the actor A1. In one embodiment, the character object 51, the floor object 54a, and the screen object 54b may be objects included in the moving image frame, and the display object 65 may be an object not included in the moving image frame. The display object 65 may be an object generated by the function of the viewing application executed by the client device 10 without being based on the data included in the moving image frame transmitted from the server device 20.

In the image 60a, in addition to each object arranged in the virtual space 50, an operation button for accepting a user's operation may be displayed. In the example shown in FIG. 7a, the image 60a includes a gift button 61 for performing gifting, an evaluation button 62 for providing an evaluation, and a participation request button 63 for allowing the avatar to participate. It has been. Operation buttons other than these may be displayed on the image 60a. These operation buttons may be objects generated by the function of the viewing application executed by the client device 10 without being based on the data included in the moving image frame transmitted from the server device 20.

The gift button 61 is displayed on the image 60a so as to be selectable by operating the client device 10a on which the image 60a is displayed. The gift button 61 can be selected, for example, by tapping the area of the touch panel of the client device 10a where the gift button 61 is displayed. In one embodiment, when the gift button 61 is selected, the image 60a displays a window for selecting a gift to be given (gifted) to the distributor who is distributing the video being viewed. The user of the client device 10a can purchase and select a gift to be gifted from the gifts displayed in the window. In another embodiment, in response to the gift button 61 being selected, the image 60a displays a window containing a list of purchased gifts and / or a list of selectable gifts. In this case, the user who is watching the video can select the gift to be gifted from the gifts displayed in the window. In one embodiment, a gifting request may be sent to the server device 60 depending on the selection of the gift button 61 on the client device 10. The gifting request may include gift identification information (gift ID) that identifies the gift object corresponding to the gift to be gifted. In this way, the viewing user who is viewing the video can give a desired gift to the distribution user who is distributing the video by selecting the gift button 61.

The server device 20 receives a gifting request from the user's client device 10a and performs processing according to the request. As described above, the user who is watching the moving image can send the gifting request to the server device 20 by operating his / her own client device. In one aspect, when the server device 20 receives the gifting request, the server device 20 identifies the gift object to be gifted based on the gifting request, and performs display processing according to the specified gift object. As mentioned above, the gift object may include an effect object, a normal object, and a decoration object. When the gift object specified by the gifting request is an effect object, the server device 20 performs a process for including the effect object in the moving image frame. For example, when a gifting request is made for an effect object corresponding to confetti, the server device 20 includes an effect object indicating confetti in the moving image frame based on the gifting request. As a result, the confetti is depicted in the image 60a. In one aspect, when the server device 20 receives a gifting request for a specific normal object, the server device 20 performs a process for including the normal object specified by the gifting request in the moving image frame. For example, when a gifting request is made for a normal object indicating a teddy bear, the server device 20 includes the normal object indicating the teddy bear in the moving image frame. As a result, a teddy bear is depicted in image 60a.

In one aspect, when the server device 20 receives a gifting request for a specific decoration object from the viewing user, the server device 20 adds the decoration object specified by the gifting request to the candidate list. This candidate list is displayed, for example, on the display of the supporter computer 40. Supporter B1 and supporter B2 can select a decoration object to be displayed in the moving image from the candidate list displayed on the display of the supporter computer 40. The server device 20 includes the selected decorative object in the moving image frame. As a result, the decorative object is depicted in the image 60a. The selected decorative object is displayed in the image 60a in association with a specific part of the character object 51. For example, when the decoration object is an object indicating an alice band, the decoration object may be attached to the head of the character object 51. The decorative object may be displayed in the image 60a so as to move with the movement of the specific portion of the character object. For example, when the head of the character object 51 equipped with the alice band moves, a selection decoration object indicating the alice band is also attached to the head of the character object 51 as if the alice band is attached to the head of the character object 51. It works.

The evaluation button 62 is displayed on the image 60a so as to be selectable by the user who uses the client device 10a. The evaluation button 62 can be selected, for example, by tapping the area of the touch panel of the client device 10a where the evaluation button 62 is displayed. When the evaluation button 62 is selected by the user who is watching the video, a positive evaluation is given to the distribution user (for example, actor A1) who distributes the video, or a positive evaluation is made to the distributed video. Evaluation update information indicating that may be transmitted to the server device 60. By selecting the evaluation button 62, evaluation information indicating that the moving image has been negatively evaluated may be transmitted to the server device 60.

The participation request button 63 is displayed on the image 60a so as to be selectable by the viewing user who uses the client device 10a. The participation request button 63 can be selected, for example, by tapping the area of the touch panel of the client device 10a where the participation request button 63 is displayed. When the participation request button 63 is selected by the viewing user who is viewing the video, a participation request requesting that the viewing user's avatar participate in the video is transmitted to the server device 20. A user who has sent a participation request by selecting the participation request button 63 may be referred to as a "participation request user". The participation request from the participation request user may include the user ID of the participation request user and the avatar ID of the avatar of the participation request user. When the participation request button 63 is selected by a viewing user who has not registered his / her own avatar, the client device 10a may display a screen prompting the registration of the avatar. In this case, the viewing user can register the avatar according to the guidance on the screen. The participation request button 63 may be displayed on the client device 10 of all viewing users.

The participation request processing unit 21c receives a participation request for the moving image being viewed from the client device 10a of the participation request user, and determines whether or not the avatar of the participation request user can be displayed based on the participation request. There may be a maximum number of avatars that can participate in videos for which participation is requested. In this case, the participation request processing unit 21c can determine whether or not to participate based on the participation request so that the number is equal to or less than the upper limit. The participation request processing unit 21c receives, for example, a participation request during the reception period before the video is distributed, and among the plurality of participation request users who have made the participation request, the users who are less than or equal to the maximum number can participate in the video. Can be allowed. That is, the participation request processing unit 21c can select a plurality of participating users of the upper limit or less from the plurality of viewing users (participation request users) who have made the participation request. When the number of participation request users exceeds the upper limit, the participation request processing unit 21c may select a user equal to or less than the upper limit of the number of participation request users from a plurality of participation request users according to a predetermined algorithm. The participation request processing unit 21c can select, for example, users who are allowed to participate on a first-come, first-served basis or randomly. The participation request processing unit 21c may determine whether or not the participation request user can participate based on the viewing history data, the evaluation data, and other data stored as the user data 23c. For example, based on the viewing history data, the participation requesting user who has a long viewing time may be more likely to be permitted to participate. Based on the viewing history data, the participation requesting user who has a large number of viewings may be more likely to be permitted to participate. In the present specification, the viewing user who is permitted to participate in the moving image by the participation request processing unit 21c may be referred to as a "participating user".

In one embodiment, the participation request processing unit 21c determines whether or not the viewing user can participate based on the participation request from the viewing user, and then permits all the viewing users who have made the participation request to participate in the video. It may be configured not to send a notification indicating whether or not it has been done. The participation request processing unit 21c may send a command for activating the co-starring function described later to the client device 10 of the participating user who is permitted to participate in the moving image. In one embodiment, the participation request processing unit 21c determines whether or not the viewing user can participate based on the participation request from the viewing user, and then refers to the client device 10 of the participating user who is permitted to participate in the video. While transmitting the above command, the client device 10 of the participating user who is permitted to participate in the video is configured not to send a notification indicating a decision as to whether or not to participate in the video and neither of the above commands. Will be done.

When the participation requesting user is permitted to participate in the video, the avatar display unit 21d performs a process for including the avatar of the participating user who is permitted to participate in the video. For example, when the avatar of the first viewing user is included in the moving image, the avatar display unit 21d identifies the avatar ID associated with the user ID of the first viewing user by referring to the user data 23c, and then identifies the avatar ID associated with the user ID of the first viewing user. , The part information associated with the avatar ID is specified with reference to the avatar data 23d. The avatar display unit 21d constructs an avatar object representing the avatar of the first viewing user based on the specified part information. Further, the avatar display unit 21d determines the display position of the avatar object in the virtual space. The moving image generation unit 21a generates moving image information so that the avatar object constructed by the avatar display unit 21d is arranged at a determined position in the virtual space, and generates a moving image frame based on the moving image information. The moving image frame including the avatar object of the participating user generated in this way is transmitted to the client device 10 by the moving image distribution unit 21b.

The client device 10 can reproduce a moving image frame including an avatar object of a participating user. FIG. 7b shows a display example of a moving image including an avatar object of a participating user. As shown in FIG. 7b, image 60b contains avatar objects 56a-56c and 57a-57g of participating users. In the following description, it is assumed that the avatar 56a is the avatar (avatar object) of the first viewing user among the plurality of avatar objects included in the image 60b. The virtual camera used for drawing the image 60b shown in FIG. 7b is oriented in the direction of the character object 51 at a position farther from the character object 51 than the virtual camera used for drawing the image of FIG. 7a. It is arranged so that it faces. Therefore, in the image 60b, a wider range of the virtual space including the character object 51 is drawn than in the image 60a.

As shown in FIG. 7b, the emotion button 68 may be displayed superimposed on the image 60b. The emotion button 68 is a button for the viewing user to express emotions when viewing the moving image. The emotion button 68 may be superimposed and displayed on the moving image frame distributed from the server device 20. That is, the emotion button 68 does not have to be an object arranged in the virtual space. By selecting the emotion button 68, the viewing user can select that he / she was impressed by the moving image, that he / she could not sympathize with the moving image, and the emotions of the viewing user other than these. The emotion button 68 includes an applause button for selecting the feeling of being impressed by the video, a question mark button for indicating that the video could not be sympathized, a surprise button for indicating that the video was surprised, and other buttons. Can include buttons. Each of the emotion buttons 68 may include a symbol or graphic display representing the emotion selected by the emotion button 68. For example, the applause button may contain a graphic representation of the clapping hand, the question mark button may contain a question mark display, and the surprise button may contain an exclamation mark. A display may be included. The emotion button 68 can be selected, for example, by tapping the area of the touch panel of the client device 10a where the emotion button 68 is displayed. When the emotion button 68 is selected by the first viewing user who is viewing the moving image, the emotion display request is transmitted to the server device 20. The emotion display request may include emotion identification information indicating the type of button (applause button, question mark button, surprise button, etc.) selected by the first viewing user among the emotion buttons 68. By selecting the emotion button 68, a plurality of buttons (applause button, surprise button, etc.) associated with the emotion button 68 may be displayed in a pop-up. When a plurality of buttons associated with the emotion button 68 are displayed in a pop-up, the user can select a button that matches his / her emotion from the plurality of buttons displayed in the pop-up. The emotion button 68 is displayed when the user's avatar is participating in the video, and may not be displayed when the user's avatar is not participating in the video.

The avatar control unit 21e may receive an emotion display request from the first viewing user and control the movement of the first viewing user in the virtual space based on the emotion display request. For example, when the emotion display request is transmitted by the first viewing user selecting the applause button, which is a kind of emotion button, the avatar control unit 21e is based on the emotion display request of the first viewing user. The avatar 56a can be made to perform an action of applauding. When control is performed to cause the avatar 56a to applaud based on this emotion display request, if the avatar 56a is included in the drawing area, the animation in which the avatar 56a is clapping is expressed in the moving image. You may. On the contrary, when the avatar 56a is not included in the drawing area, the sound of the avatar 56a clapping may be expressed in the moving image, but the clapping animation may not be displayed. In one embodiment, the emotion button 68 may be displayed in the video being viewed by the user only when the user's avatar is displayed in the video. For example, when the image 60a shown in FIG. 7a is displayed on the client device 10a of the first viewing user, the avatar 56a of the first viewing user is not displayed in the image 60a, so this image. The emotion button 68 may not be included in the 60b. On the contrary, when the client device 10a of the first viewing user includes the image 60b shown in FIG. 7b, the avatar 56a of the first viewing user is included in the image 60b. Therefore, the emotion button 68 may be included in this image 60b.

In this way, based on the emotion display request from the user, the user's avatar makes a movement such as applause in the moving image. The emotion display request is a kind of operation instruction for instructing the operation of the avatar. In another embodiment, in addition to the emotion button, an operation instruction button for instructing the operation of the avatar may be included in the image. The operation instruction button is a button for instructing the operation of the avatar. The operation instruction button can be used, for example, to instruct an operation of jumping, an operation of waving a hand, an operation of waving a psyllium, and other operations. When the operation instruction button is selected, the operation instruction request is transmitted to the server device 20. In the server device 20, the avatar can be made to perform an operation based on the operation instruction request.

Next, a co-starring function in which the viewing user and the distribution user co-star in the moving image via their respective virtual characters will be described. As shown in FIG. 7b, when the first viewing user is permitted to participate in the moving image, the co-star application button 64 is displayed on the client device 10 used by the first viewing user. The co-starring application button 64 requests that the viewing user's avatar be displayed together with the character object 51, that is, the viewing user and the distribution user co-star through the viewing user's avatar and the distribution user's character object 51. It is a button for. The co-star application button 64 is superimposed and displayed on the image 60b so as to be selectable by the first viewing user who uses the client device 10a. The co-star application button 64 can be selected, for example, by tapping the area of the touch panel of the client device 10a where the co-star application button 64 is displayed. When the co-starring application button 64 is selected by the first viewing user who is viewing the moving image, a co-starring request for co-starring the avatar 56a of the first viewing user with the character object 51 is transmitted to the server device 20. The client device 10a of the first viewing user displays a message 66a indicating that the co-starring request has been made, as shown in FIG. 7c, in response to transmitting the co-starring request to the server device 20. May be good.

In one embodiment, the client device 10 may display the co-star application button 64 in response to receiving a command for activating the co-star function from the participation request processing unit 21c. As described above, this command is transmitted from the server device 20 to the client device 10 of the participating user who is permitted to participate in the moving image. The client device 10 does not have to display the co-star application button 64 until it receives a command for activating the co-star function from the participation request processing unit 21c. The co-star application button 64 is displayed in a state in which it cannot be selected by the user until a command for activating the co-star function is received from the participation request processing unit 21c, and can be selected by the user according to the reception of the command. May be done.

When the co-starring request is received by the server device 20, the co-starring request processing unit 21f determines whether or not to allow the co-starring between the viewing user (participating user) who made the co-starring request and the distribution user. The co-starring request processing unit 21f can determine whether or not co-starring is possible by using a predetermined algorithm based on various information about the viewing user who has made the co-starring request. The co-starring request processing unit 21f determines, for example, whether or not the evaluation of the viewing user who made the co-starring request by another user is higher than a predetermined criterion by referring to the user data 23c, and the evaluation of the viewing user is evaluated. If it is higher than the standard, the co-starring request from the viewing user can be permitted. On the contrary, when the evaluation of the viewing user by another user is lower than the predetermined criterion, the co-starring request from the viewing user can be rejected. A viewing user (participating user) who is permitted to request co-starring may be referred to as a "guest user".

In response to the reception of the co-starring request from the viewing user, the co-starring request processing unit 21f inquires the distribution user whether or not to allow the co-starring application from the viewing user, and determines whether or not the co-starring is possible according to the answer. You may. When the co-starring request processing unit 21f receives the co-starring request from the first viewing user, the co-starring request processing unit 21f may generate a display request user list including the first viewing user and present the display request user list to the distribution user. The display request user list for a certain video is a list format of information (for example, user ID, user name, nickname, avatar ID, or other information) that identifies the user (or the avatar of the user) who made the co-starring request. May be indicated by. The display request user list does not have to include all the users who sent the co-star request. For example, the display request user list may include only the users who have sent the co-starring request and the 3D display information of the avatar of the user has already been loaded. Such a display request user list does not include information that identifies a user who has sent a co-starring request and for which the 3D display information of the avatar of the user has not been loaded. 3D display information is not loaded In order to perform 3D display of the avatar, it is necessary to load the 3D display information at runtime, which consumes a lot of processing resources of the server device 20. The runtime load can be avoided by including only the user who uses the loaded avatar in the 3D display information in the co-starring request.

The co-starring request processing unit 21f can transmit the display request user list to the supporter computer 40. The supporter B1 and the supporter B2 can select an avatar user who is allowed to be displayed together with the character object from the display request user list displayed on the display of the supporter computer 40. The display request user list may be displayed on the display 39 of the studio room R. The actor A1 may select an avatar user to be displayed together with the character object 51 from the display request user list by operating the controller 33a or the controller 33b.

In one embodiment, the user may be deleted from the display request user list in response to the selection of a user from the display request user list. In another embodiment, the user may be deleted from the display request user list when the co-starring display of the avatar of a user is completed. As a result, it is possible to prevent the avatar that has been co-starred once from being repeatedly co-starred.

In one embodiment, even if a supporter or a distribution user instructs to select a user to be co-starred from the display request list while a user is co-starred, the co-starred display of the selected user's avatar is immediately displayed. It does not have to be done in. Further, the selected user may not be deleted from the display request list until the co-starring display is performed.

When the co-starring request processing unit 21f receives co-starring requests from a plurality of participating users, the co-starring request processing unit 21f may select a plurality of guest users having a predetermined upper limit or less from the plurality of participating users.

In one embodiment, the co-starring request processing unit 21f is a client device 10 of the participating user (that is, a guest user) when co-starring between the participating user and the distribution user is permitted based on the co-starring request from the participating user. You may send a permission notice notifying that the co-starring is permitted. Upon receiving the permission notification, the client device 10 may display a message 66b indicating that the co-starring is permitted based on the permission notification on the display, as shown in FIG. 7d. The message 66b displayed based on this permission notification allows the guest user to know that his / her avatar is included in the co-starring video and delivered. In addition to the message 66b, the client device 10 can notify the guest user that an icon display, a voice notification, a vibration notification, and a permission notification have been received by a method other than these.

Upon receiving the permission notification from the server device 20, the client device 10 of the guest user can urge the guest user to purchase the avatar parts. For example, the client device 10 purchases avatar parts in response to receiving a permission notification from the server device 20.

When the permission notification is received from the server device 20 in the client device 10 of the guest user, the client device 10 can display a display prompting the purchase of the avatar parts. For example, the client device 10 can display a window for purchasing the avatar body or its parts on its display in response to receiving the permission notification from the server device 20. The guest user can select the purchase target from the avatar or its parts displayed in the window and actually purchase it. The window for purchasing the avatar or its parts may be displayed only on the client device 10 that has received the permission notification from the server device 20, or may be displayed on the other client devices 10. Upon receiving the permission notice, the client device 10 may display a window in which a different type of avatar or a part thereof can be purchased than before the permission notice was purchased. The client device 10 that has received the permission notification can display, for example, a window for purchasing an avatar or a part suitable for the co-starring video.

In one embodiment, when the client device 10 of the guest user receives the permission notification from the server device 20, the client device 10 may make the so-called gacha function available to the guest user. By using this gacha function, the guest user transmits an acquisition request for the avatar or its parts to the server device 20, and the avatar or the guest user is given to the guest user by a predetermined algorithm based on the acquisition request in the server device. The part is selected. This selected avatar or its part is given to the guest user who made the acquisition request. In this way, the guest user can acquire a new avatar or its parts by using the gacha function. The gacha function for acquiring the avatar or its parts may be available only in the client device 10 that has received the permission notification from the server device 20, or may be available in other client devices 10. .. The client device 10 that has received the permission notification may be able to acquire a different type of avatar or a part thereof from before the purchase of the permission notification by the gacha function.

In this way, the guest user can update his / her own avatar displayed in the co-starring video before the start of the co-starring by purchasing a new avatar or its parts or acquiring it by the gacha function.

In one embodiment, when the co-starring user and the distribution user are permitted to co-star based on the co-starring request from the participating user, the co-starring request processing unit 21f and the participating user (that is, the guest user) and the distribution user Generates a co-starring video that virtually co-stars. For example, the co-starring request processing unit 21f displays the guest user's avatar together with the character object 51 in the moving image. The co-starring request processing unit 21f can construct the guest user's avatar by the same processing as the avatar control unit 21e, and can arrange the guest user's avatar in the virtual space. In one embodiment, in the co-starring moving image including the guest user's avatar for which the co-starring request is permitted and the character object 51 of the distribution user, the avatar of the viewing user other than the guest user's avatar is not displayed. In this case, the co-starring request processing unit 21f arranges only the guest user's avatar among the plurality of viewing user's avatars in the virtual space. In the present specification, displaying the avatar of the guest user for whom the co-starring request is permitted together with the character object 51 of the distribution user is referred to as "co-starring display", and the avatar of the viewing user and the character of the distribution user for which the co-starring request is permitted are displayed. A moving image including the object 51 may be referred to as a "co-starring moving image".

A display example of the co-starring moving image is shown in FIG. 7e. The image 60e shown in FIG. 7e includes an avatar 56a of the first viewing user and a character object 51 that have requested co-starring. In the image 60e shown in FIG. 7e, the avatar 56a of the first viewing user is displayed next to the character object 51. The image 60d may display a message 66c for notifying that the co-starring display has been performed.

When the avatar 56a of the first viewing user is displayed together with the character object 51 based on the co-starring request from the first viewing user, the avatar 56a may be displayed in 3D. As described above, the avatar 56a may be displayed in 2D in the image (for example, the image 60b) before the co-starring request is made. The display mode of the avatar 56a that was displayed in 2D before the co-starring request may be changed to 3D display in the co-starring display. Since the avatar 56a is displayed in 3D in the co-star display, it is displayed on the client device 10a of the first viewing user only when the 3D display information about the avatar 56a is loaded before the time when the co-star request is received. The co-star application button 64 may be displayed in the image 60b.

In one embodiment, the co-star application button 64 is displayed on the image for the user who uses the avatar to which the 3D display information is loaded, and the co-star application is applied for the image for the user who uses the avatar to which the 3D display information is not loaded. By not displaying the button 64, it is possible to reduce the processing load of the server device 20 by preventing the runtime load of the 3D display information from occurring. In one embodiment, in order to avoid the occurrence of the processing load required for the runtime load, when the user accepts the co-starring display of the avatar, it is determined whether or not the 3D display information of the avatar has been loaded, and the avatar has been loaded. The 3D display of the avatar may be performed only in the case of. In this case, the co-star application button 64 is displayed in the image for the user who uses the avatar to which the 3D display information is loaded, while the co-star application button 64 is displayed in the image for the user who uses the avatar to which the 3D display information is not loaded. By not displaying 64, it is possible to prevent unnecessary transmission of the co-starring request and occurrence of processing based on the co-starring request.

When the avatar is co-starred in the moving image, the effect related to the co-starring display of the avatar may be displayed. For example, it is possible to produce an effect in which the avatar is in the spotlight so as to emphasize the avatar that is co-starred. A similar or different effect can be displayed when the co-starring display of the co-starring display is terminated.

The co-starring request processing unit 21f may measure the elapsed time since the first viewing user's avatar 56a is displayed in the moving image together with the character object 51. The co-starring request processing unit 21f may display the elapsed time on the image 60d. In one embodiment, an upper limit of the time during which the character object 51 and the user's avatar are displayed together (the time during which the co-starring is displayed for a certain avatar) may be set based on the co-starring request. The co-starring request processing unit 21f may display the upper limit time on the image 60d. The co-starring request processing unit 21f calculates the remaining time during which the avatar 56a can be displayed based on the upper limit time and the elapsed time since the first viewing user's avatar 56a is displayed in the moving image together with the character object 51. Then, the remaining time may be displayed on the image 60d. The elapsed time since the avatar 56a of the first viewing user is displayed in the moving image together with the character object 51, the upper limit of the display time set in the avatar 56a, and the remaining time during which the avatar 56a can be displayed are the client devices of the first viewing user. It may be included only in the image 60d displayed on the 10a, or may be included in the image 60d displayed on the client device of the first viewing user and other users. By setting an upper limit of the time during which the co-starring display is performed, it is possible to provide more viewing users with an opportunity to co-star with the distribution user. In addition, by displaying the remaining time of the co-starring display during execution, it is possible to inform the viewing user of the time until the next avatar is co-starring displayed.

The co-starring request processing unit 21f can exclude the avatars participating in the moving image from the moving image based on the instruction from the moving image distribution user or other operators. The excluded avatar will not be displayed on the screen of the video. Similarly, the co-starring request processing unit 21f can exclude the co-starring avatar from the co-starring display based on an instruction from the video distribution user or another operator. The excluded avatar will not be displayed on the screen of the co-starring display. The distribution user, for example, avatars who made undesired comments in the video, avatars who took undesired actions in the video, and avatars who are not desirable to participate in other videos are and / or co-starred from the video. It can be excluded from the display screen.

In one embodiment, the avatar control unit 21e can control the movement of the guest user's avatar based on an instruction from the user. For example, the avatar control unit 21e can receive an emotion display request from a user of the avatar who is co-starring and display, and can control the action of the avatar based on the emotion display request. The avatar control unit 21e controls the movement of the avatar by using the face motion data and the body motion data transmitted from the client device 10 of the guest user who is co-starring with the distribution user in the co-starring video during the distribution of the co-starring video. You may do it. The avatar control unit 21e may make the avatar utter using the voice data transmitted from the client device 10 of the guest user who is co-starring with the distribution user in the co-starring video during the distribution of the co-starring video.

As shown in FIG. 6, the co-starring request may include at least one of message data and motion data in association with a user ID that identifies a participating user who makes a co-starring request. The avatar control unit 21e may cause the guest user avatar to utter a message corresponding to the message data in the moving image based on the message data included in the co-starring request. As this message, for example, a greeting such as "Hello" can be considered. A text message may be generated based on the message data, and the text message may be displayed in association with the avatar of the participating user who made the co-starring request. The message data may be data corresponding to a plurality of messages.

The motion data can include at least one of face motion data and body motion data. The motion data included in the co-starring request may be acquired by the sensor unit included in the client device 10, or may be acquired by a sensing device different from the client device 10. The avatar control unit 21e may control the movement of the guest user's avatar based on the motion data included in the co-starring request. The motion data is data representing, for example, the movement of the avatar in which the avatar waving his right hand to greet him. The motion data may be data representing a set of avatar movements. A set of avatar movements is, for example, a set of avatar waving and then the avatar bowing.

In one embodiment, for a participating user who has transmitted a co-starring request with the distribution user but the co-starring is rejected, an image (two-shot image) including the avatar of the participating user and the character object 51 of the distribution user is provided. May be given. For example, when the co-starring request from the first viewing user is rejected, the co-starring request processing unit 21f generates the same video frame as the video frame generated when the co-starring request is permitted, and this video frame is generated. The moving image file representing the above may be transmitted to the client device 10 of the first viewing user whose co-starring is refused. This moving image frame is transmitted only to the client device 10 of the first viewing user whose co-starring request is rejected, and is not distributed to the client device 10 of another viewing user. This video frame corresponds to, for example, the image 60e shown in FIG. 7e (however, the message 66c and the operation buttons 61, 62, 68 are not included, so that the participation with which the co-starring with the distribution user is refused is rejected. It is possible to provide the user with content for feeling the connection with the distribution user. Although the two-shot image with the character object 51 of the distribution user is requested to participate in the video, the participation request is made. It may be sent to the rejected viewing user.

Subsequently, with reference to FIG. 8, a moving image distribution process according to one aspect will be described. FIG. 8 is a flow chart showing the flow of the moving image distribution process in one embodiment. In the video distribution process, it is assumed that the actor A1 is present in the studio room R.

First, in step S11, body motion data that is a digital representation of the body movement of the actor A1 and face motion data that is a digital representation of the face movement (facial expression) of the actor A1 are generated. The body motion data and face motion data are set in, for example, the detection information of the motion sensors 31a to 31f mounted on the actor A1, the imaging information captured by the digital camera 36a, and the studio room R other than these as necessary. It is generated based on the information detected by various devices.

Next, in step S12, by applying the body motion data and face motion data of the actor A1 to the model data for the actor A1, an animation of a character object that moves in synchronization with the movement of the body and facial expression of the actor A1 is generated. Will be done.

Next, in step S13, three-dimensional moving image data including an image of the virtual space and an animation of the character object corresponding to the actor A1 is generated. This moving image information may include the sound of the actor A1. Further, the three-dimensional moving image data is rendered to generate a moving image. This moving image may be a sequence of moving image frames. This moving image is generated, for example, by the moving image generation unit 21a described above.

Next, the process proceeds to step S14, and the moving image generated in step S13 is distributed. This moving image is distributed to the client devices 10a to 10c and other client devices via the network 15. This moving image may be distributed to the supporter computer 40 and projected on the screen S in the studio room R. The video is continuously distributed over a predetermined distribution period. The video distribution period can be set to, for example, 30 seconds, 1 minute, 5 minutes, 10 minutes, 30 minutes, 60 minutes, 120 minutes, and any other time.

Next, the process proceeds to step S15, and it is determined whether or not the termination condition for terminating the distribution of the moving image is satisfied. The end condition is, for example, that the end time of the distribution has been reached, that the supporter computer 40 has given an instruction to end the distribution, or other conditions. If the end condition is not satisfied, the processes of steps S11 to S14 are repeatedly executed, and the distribution of the moving image including the animation synchronized with the movement of the actor A1 is continued. If it is determined that the end condition for the video is satisfied, the distribution process of the video is terminated.

Subsequently, with reference to FIG. 9, the participation request processing according to one aspect will be described. FIG. 9 is a flow chart showing a flow of participation request processing in one embodiment. In the participation request processing of FIG. 9, it is assumed that the first viewing user who is viewing the moving image distributed from the server device 20 by the client device 10a requests participation in the moving image. The participation request processing may be performed in parallel with the video distribution processing shown in FIG.

First, in step S16, a participation request requesting participation in the moving image being viewed is received from the client device 10 of the first viewing user. The participation request is transmitted from the client device 10a to the server device 20 in response to the selection of the participation request button 63 displayed on the client device 10a by the first viewing user.

Next, in step S16, it is determined whether or not the participation request from the first viewing user is permitted. This determination is made, for example, by the participation request processing unit 21c described above.

If the participation request is approved, the participation request processing proceeds to step S18. In step S18, the avatar 56a of the first viewing user is arranged at a predetermined position in the virtual space 50. As a result, when the position where the avatar 56a is arranged is included in the drawing area of the virtual camera, the avatar 56a of the first viewing user is included in the delivered moving image. The process for including the avatar 56a in the moving image is performed by, for example, the above-mentioned avatar display unit 21d.

If the participation request is not permitted (rejected), the participation request processing ends without displaying the avatar 56a of the first viewing user.

Although the participation request processing has been described by taking the case where the first viewing user makes a participation request as an example, the server device 20 can receive the participation request from a plurality of viewing users at any time during the distribution of the moving image. Therefore, the server device 20 can execute the above-mentioned participation request processing for each of the plurality of participation requests. In this way, the server device 20 can select a plurality of participating users from a plurality of viewing users who have requested participation in the moving image.

Next, with reference to FIG. 10, the distribution process of the co-starring moving image in one embodiment will be described. FIG. 10 is a flow chart showing a flow of processing for distributing a co-starring moving image in one embodiment. In the distribution process of the co-starring video described with reference to FIG. 10, it is assumed that the viewing user who is viewing the video including the character object of the distribution user makes a co-starring request to the distribution user. .. The co-starring moving image distribution process may be performed in parallel with the moving image distribution process shown in FIG. 8 and the participation request processing shown in FIG.

First, in step S21, it is determined whether or not a co-starring request has been made by a viewing user (participating user) who is permitted to participate in the video during distribution of the video including the character object of the distribution user. The co-starring request is made, for example, by the first viewing user. When the co-starring request is made by the first viewing user, the process proceeds to step S22.

In step S22, it is determined whether or not to allow the co-starring of the first viewing user and the distribution user based on the co-starring request from the first viewing user who applied for the co-starring. For example, the co-starring request processing unit 21f determines whether or not to allow the co-starring request. If the co-starring of the first viewing user and the distribution user is permitted, the process proceeds to step S23.

In step S23, a permission notification notifying that the co-starring is permitted is transmitted to the client device 10 of the first viewing user who is permitted to co-star with the distribution user. The permission notification is transmitted, for example, by the co-starring request processing unit 21f. The client device 10 of the first viewing user may display a message 66b indicating that the co-starring is permitted based on the permission notification on the display. The message 66b displayed based on this permission notification allows the first viewing user to know that his / her avatar is included in the co-starring video, and to prepare for co-starring such as updating the avatar. ..

Subsequently, in step S24, a co-starring moving image including the character object of the distribution user and the avatar of the first viewing user is generated. The generation of the co-starring moving image in step S24 is performed, for example, by the co-starring request processing unit 21f described above.

Next, the process proceeds to step S25. In step S25, the co-starring moving image generated in step S24 is distributed. This co-starring moving image is distributed to the client device 10 and other client devices and the distribution user device via the network 50. The co-starring moving image includes, for example, the character object 56a of the first viewing user and the character object 51 of the distribution user, as shown in FIG. 7e.

Next, the process proceeds to step S26. In step S26, it is determined whether or not the co-starring end condition is satisfied. For example, when a predetermined time or more has passed from the start of distribution of the co-starring video, it is determined that the co-starring end condition is satisfied. If the co-starring end condition is satisfied, the process proceeds to step S27.

In step S27, a process for terminating the distribution of the co-starring moving image is performed. For example, by excluding the character object 51 of the first viewing user who is the guest user from the co-starring video, the co-starring of the first viewing user and the distribution user is terminated. The moving image delivered after the co-starring is finished does not include the character object 51 of the first viewing user. When a plurality of guest users are selected, the co-starring of the guest user other than the first viewing user and the distribution user may be started after the co-starring of the first viewing user and the distribution user is completed. Even if the co-starring with all the guest users is finished, the video may be continuously distributed. The determination of the co-starring end condition and the co-starring end processing are performed by, for example, the co-starring end processing unit 61f.

The action and effect produced by the above embodiments will be described. According to the video distribution system 1 in the above embodiment, the video distribution system 1 does not accept co-starring requests from all the viewing users who are watching the distributed video, but requests participation in the video to the video. It is configured to accept co-starring requests only from participating users whose participation requests have been approved. For example, the client device 10 of the viewing user (that is, the participating user) who is permitted to participate in the video activates the function for making a co-starring request, and the client device 10 of the viewing user who is not permitted to participate in the video is activated. By not activating the function for making a co-star request, the co-star request can be accepted only from the participating users. As a result, even if the number of viewing users increases, the number of co-starring requests received from the client device 10 can be suppressed.

In the above embodiment, the server device 20 notifies the viewing users other than the participating users who are permitted to participate in the video among the plurality of viewing users who view the video, regarding the selection of the participating users. Is not sent. In one embodiment, the notification regarding the selection of the participating user is transmitted only to the client device 10 of the participating user who is permitted to participate in the moving image among the plurality of viewing users who view the moving image. As a result, the processing load of generating and transmitting the notification regarding the result of determining whether or not to allow participation in the server device 220 can be reduced, and the communication traffic can be reduced.

In the above embodiment, the user who requests to participate in the video and is refused is not notified of the refusal. In this case, the viewing user who made the participation request knows whether or not his / her participation request is permitted by confirming whether or not his / her character object (avatar) is included in the delivered video. Can be done. In other words, if the delivered video contains your own avatar, you know that you are allowed to participate, and if the delivered video does not contain your own avatar, you know that you were not allowed to participate. As described above, in the above embodiment, by distributing the moving image including the avatar of the participating user from the server device 20, the participation request is rejected without additionally transmitting the notification other than the moving image. It is possible to notify all viewing users including the viewing user whether or not the participation request is permitted. That is, in the above embodiment, it is possible to notify the viewing user whose participation request is rejected whether or not the participation request is permitted without transmitting a notification other than the moving image. In addition, by notifying whether or not the participation request is permitted through the video including the avatar of the participating user, it is possible to attract the attention and interest of the viewing user who made the participation request to the distributed video. For videos of popular distribution users, the number of viewing users can be enormous. In this case, the effect of reducing the processing load of the server device 20 is increased by not notifying all the viewing users regarding the selection of the participating users.

In the above embodiment, the server device 20 transmits a command to the client device 10 of the participating user to activate the function for the participating user to make a co-starring request in the client device 10. As a result, it is possible to make a co-starring request on the client device 10 of the participating user among all the viewing users, and it is not possible to make a co-starring request on the client device 10 of the viewing user who is not permitted to participate. Therefore, since the number of users who can make the co-starring request is smaller than that of the entire viewing user, the processing load for processing the co-starring request in the server device 20 can be reduced. When the decision on whether or not to allow co-starring based on the co-starring request is left to the selection by the distribution user, the time and effort of the decision by the distribution user can be reduced.

In one embodiment described above, the co-starring request may include message data indicating a message from a participating user who makes the co-starring request. The co-starring request may include motion data that specifies the movement of the avatar of the participating user who makes the co-starring request. By including the message data and motion data in the co-starring request, the guest user does not have to perform any special operation during the distribution of the co-starring video, based on the data provided before the start of the co-starring in the form of message data or motion data. The movement and voice of the guest user can be controlled.

The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist thereof. For example, the delivered moving image may be shot and generated in a place other than the studio room R. For example, the shooting for generating the delivered video may be performed at the home of the actor or supporter. The client device 10a may have a function of transmitting an image (for example, an image 60e) in which both the avatar 56a and the character object 51 are displayed to another communication device. For example, the first viewing user uses the function of the client device 10a to display an image in which both the avatar 56a and the character object 51 are displayed on the social networking service or the server device 20 in the video service provided by the server device 20. It can be distributed on a social networking service that is different from the video service provided by.

As described above, at least a part of the functions realized by the computer processor 21 may be realized by a computer processor other than the computer processor 21 of the moving image distribution system 1. At least a part of the functions realized by the computer processor 21 may be realized by the computer processor provided in any of the client devices 10a to 10c, for example. For example, some or all of the functions of the moving image generation unit 21a, the moving image distribution unit 21b, the participation request processing unit 21c, the avatar display unit 21d, the avatar control unit 21e, and the co-starring request processing unit 21f are executed by the client devices 10a to 10c. May be done. Specifically, the moving image generated by any of the client devices 10a to 10c may be transmitted to the server device 20 and distributed from the server device 20 to the client devices 10a to 10c.

When a viewing user who is viewing a moving image including the character object 51 of the distribution user on the client device 10a applies for co-starring with the character object 51 (or its distribution user), the viewing user who has applied for co-starring The animation of the avatar 56a may be generated by either the server device 20 or the client device 10a. Further, the generation of the co-starring moving image including the character object 51 and the viewing user's avatar 56a may be performed by either the server device 20 or the client device 10a. The co-starring moving image may be shared and generated by the server device 20, the client device 10a, and other devices. When the co-starring moving image including the character object 51 of the distribution user and the avatar 56a of the viewing user is generated in the client device 10a, the motion data and audio of the distribution user are generated from the server device 20 or the studio unit 30 to the client device 10a. The data may be transmitted to the client device 10a. In the client device 10a, the animation of the character object 51 is generated by applying the received motion data of the distribution user to the model data for the distribution user. The model data of the distribution user may be transmitted from the server device 20 or the studio unit 30 to the client device 10a. When a gift object is included in the co-starring video generated by the client device 10a, data for displaying the gift object is transmitted from the server device 20 or the client device of the viewing user who performed the gifting to the client device 10a. May be done. In this way, the co-starring moving image can be generated by any device included in the moving image distribution system 1. Animations of character objects may be generated by devices not explicitly shown in FIG. Motion data and audio data of the distribution user corresponding to the character object 51 and the viewing user corresponding to the avatar 56a are transmitted to the device for generating the co-starring moving image including the character object 51 and the avatar 56a.

In the processing procedure described in the present specification, particularly the processing procedure described using the flow chart, a part of the steps (steps) constituting the processing procedure is omitted, as a step constituting the processing procedure. It is possible to add steps that are not specified and / or to change the order of the steps, and the processing procedure in which such omissions, additions, and changes in order are made does not deviate from the gist of the present invention. It is included in the scope of the invention.

1 Video distribution system 10 Client device 20 Server device

Claims (9)

Equipped with one or more computer processors
The one or more computer processors may execute computer-readable instructions.
A video containing a first character object generated based on the movement of a distribution user is distributed to each client device of a plurality of viewing users.
In response to receiving requests for participation in the video from the plurality of viewing users, a plurality of participating users are selected from the plurality of viewing users.
A plurality of second character objects corresponding to each of the plurality of participating users are included in the moving image.
A co-star request for co-starring with the distribution user is received from at least a part of the plurality of participating users, and one or more guest users are selected from the participating users who have made the co-star request.
Send a notification regarding the selection of the one or more guest users to each of the one or more guest users.
A co-star movie including the guest character object corresponding to at least a part of the one or more guest users and the first character object among the second character objects is generated .
The co-starring video does not include a character object that has not been selected as the one or more guest users among the plurality of participating users.
Video distribution system. The one or more computer processors transmit a command for activating the function for making the co-starring request in each client device of the plurality of participating users to each client device of the plurality of participating users.
The video distribution system according to claim 1. The co-starring request from one of the plurality of participating users includes a message from the one participating user.
The video distribution system according to claim 1 or 2. The co-starring request from one of the plurality of participating users includes motion data from the one participating user.
The one or more computer processors causes the second character object corresponding to the one participating user to perform an operation corresponding to the motion data in the moving image.
The video distribution system according to any one of claims 1 to 3. The selection of the plurality of participating users based on the participation request is performed based on a predetermined algorithm.
The video distribution system according to any one of claims 1 to 4. The selection of the one or more guest users based on the co-starring request is performed according to the selection of the distribution user.
The video distribution system according to any one of claims 1 to 5. The one or more computer processors do not send a notification regarding the selection of the plurality of participating users to the plurality of viewing users other than the plurality of participating users.
The video distribution system according to any one of claims 1 to 6. A video distribution method executed by one or more computer processors executing computer-readable instructions.
A process of distributing a video including a first character object generated based on a movement of a distribution user to each client device of a plurality of viewing users, and a process of distributing the video.
A process of selecting a plurality of participating users from the plurality of viewing users in response to receiving requests for participation in the video from the plurality of viewing users, and a process of selecting a plurality of participating users from the plurality of viewing users.
A step of including a plurality of second character objects corresponding to each of the plurality of participating users in the moving image, and
A process of receiving a co-starring request for co-starring with the distribution user from at least a part of the plurality of participating users and selecting one or more guest users from the participating users who have made the co-starring request.
The step of transmitting a notification regarding the selection of the one or more guest users to each of the one or more guest users, and
A step of generating a guest character object corresponding to at least a part of the one or more guest users among the second character objects and a co-starring moving image including the first character object.
Equipped with a,
The co-starring video does not include a character object that has not been selected as the one or more guest users among the plurality of participating users.
Video distribution method. For one or more computer processors
A process of distributing a video including a first character object generated based on a movement of a distribution user to each client device of a plurality of viewing users, and a process of distributing the video.
A process of selecting a plurality of participating users from the plurality of viewing users in response to receiving requests for participation in the video from the plurality of viewing users, and a process of selecting a plurality of participating users from the plurality of viewing users.
A step of including a plurality of second character objects corresponding to each of the plurality of participating users in the moving image, and
A process of receiving a co-starring request for co-starring with the distribution user from at least a part of the plurality of participating users and selecting one or more guest users from the participating users who have made the co-starring request.
The step of transmitting a notification regarding the selection of the one or more guest users to each of the one or more guest users, and
A step of generating a guest character object corresponding to at least a part of the one or more guest users among the second character objects and a co-starring moving image including the first character object.
To run ,
The co-starring video does not include a character object that has not been selected as the one or more guest users among the plurality of participating users.
Video distribution program.

Images