JP2019204244A - System for animated cartoon distribution, method, and program - Google Patents

Abstract

To promote activation of animation distribution.SOLUTION: An animation distribution server 10 according to an embodiment provides an animation distribution service for distributing an animated cartoon provided by a distribution user to multiple viewing users via a user terminal 30. Since the sever 10 renders each avatar on the basis of each three-dimensional model for multiple avatars corresponding to multiple users who conduct joint distribution of an animated cartoon as well as controlling each motion of the multiple avatars on the basis of information inputted from each of the multiple users, respectively, and generates an animated cartoon corresponding to a virtual space in which the multiple avatars are included, for example, even when a user who feels resistance to publish his/her natural face is included in, it is possible to conduct joint distribution of an animated cartoon by multiple users easily, as a result, activation of animation distribution can be promoted.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a system, method, and program for distributing moving images.

  2. Description of the Related Art Conventionally, a system for distributing a moving image by a user has been provided, and a moving image distributed through such a system includes a moving image (self-shooting moving image) in which the user himself / herself appears as a subject (for example, a patent) Reference 1). For example, the user can shoot a moving image including the user's own image and sound through a camera and a microphone included in a smartphone, a personal computer, and the like, and can distribute the captured moving image to a plurality of viewers.

JP 2017-121036 A

  However, among users who wish to distribute such moving images, there may be users who feel resistance to opening their faces to an unspecified number of viewers. For example, collaborative distribution in which general viewers participate in videos of users who are accustomed to distributing self-shooting videos can promote the activation of video distribution services. It is thought that resistance is easily felt. Therefore, for example, instead of the user himself / herself, it is considered useful if the collaborative distribution of the moving image in which the user's avatar appears can be easily performed.

  An object of the embodiment of the present invention is to promote activation of moving image distribution. Other objects of the embodiments of the present invention will become apparent by referring to the entire specification.

  A system according to an embodiment of the present invention is a system for distributing moving images, and includes at least one storage device that stores at least image information of each avatar of a plurality of users, and one or more computer processors. And the one or more computer processors include first and second avatars corresponding to the first and second users who jointly distribute the video in response to execution of a readable instruction. The process of generating a moving image corresponding to the virtual space and the process of providing the generated moving image to each of a plurality of viewing users and generating the moving image are the first and second avatars. And drawing the first and second avatars based on at least the image information of the first and second avatars stored in the one or more storage devices. Comprises operating in said virtual space based on at least the first and second first and second input information input through the respective user terminal by the user.

  A method according to an embodiment of the present invention is performed by one or more computers accessible to one or more storage devices that store at least image information of avatars of each of a plurality of users, for delivering moving images. A method for generating a video corresponding to a virtual space including first and second avatars corresponding to first and second users who perform joint video distribution, and viewing the generated video in a plurality of ways Providing to each of the users, wherein the step of generating the moving image includes the first and second stored in the one or more storage devices, the first and second avatars. And drawing the first and second avatars via the first and second user terminals by the first and second users. Te comprises operating in said virtual space based on at least the input information inputted respectively.

  The program which concerns on one Embodiment of this invention is a program for delivering a moving image, Comprising: One or several which can access one or several memory | storage devices which memorize | store at least the image information of each avatar of several users A video corresponding to a virtual space including first and second avatars corresponding to the first and second users who perform joint distribution of the video on the one or a plurality of computers according to execution on the computer. A process of generating and a process of providing the generated moving image to each of a plurality of viewing users, and the process of generating the moving image includes: the first and second avatars; Rendering based on at least the image information of the first and second avatars stored in the storage device; and the first and second avatars are rendered as the first and second users. Based at least on the input information input through the respective first and second user terminals by comprises operating in said virtual space.

  Various embodiments of the present invention may facilitate activation of video distribution.

The block diagram which shows schematically the structure of the network containing the moving image delivery server 10 which concerns on one Embodiment of this invention. The block diagram which shows roughly the function which the moving image delivery server 10 has. The figure which illustrates the information managed in the user information table 411. The figure which illustrates the information managed in the delivery information table 412. The flowchart which illustrates the process which the moving image delivery server 10 performs when individual moving image delivery is performed. The figure for demonstrating the communication performed during the delivery of a live moving image. The figure which illustrates viewing screen 60. The figure which illustrates typically the input image 50 input via the camera of the delivery user terminal 30. FIG. The figure which illustrates screen 70 for distribution. The flowchart which illustrates the process which the video delivery server 10 performs according to the participating request by a viewing-and-listening user. The figure which illustrates the participation request notification object 80. The figure for demonstrating the communication performed during the period of collaboration delivery. The figure which illustrates the screen 60 for viewing during the period of collaboration delivery. The figure for demonstrating the communication performed during the period of the collaboration delivery of other embodiment. The figure for demonstrating the communication performed during the period of the collaboration delivery of other embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a configuration diagram schematically showing a configuration of a network including a moving image distribution server 10 according to an embodiment of the present invention. As shown in the figure, the moving image distribution server 10 is connected to a user terminal 30 through a communication network 20 such as the Internet so as to be communicable. Although only one user terminal 30 is illustrated in FIG. 1, the server 10 is connected to be able to communicate with a plurality of user terminals 30. The server 10 provides a moving image distribution service for distributing a moving image provided by a distribution user to a plurality of viewing users via the user terminal 30. In the present embodiment, the user of the user terminal 30 can distribute a moving image as a distribution user, and can also view a moving image distributed by another user as a viewing user.

  The moving image distribution server 10 is configured as a general computer, and as shown in FIG. 1, a computer processor 11, a main memory 12, an input / output I / F 13, a communication I / F 14, a storage (storage device) These components are electrically connected via a bus (not shown) or the like.

  The computer processor 11 is configured as a CPU, a GPU, or the like, reads various programs stored in the storage 15 or the like into the main memory 12, and executes various instructions included in the programs. The main memory 12 is configured by, for example, a DRAM.

  The input / output I / F 13 includes various input / output devices for exchanging information with a user or the like. The input / output I / F 13 includes, for example, an information input device such as a keyboard and a pointing device (for example, a mouse and a touch panel), a voice input device such as a microphone, and an image input device such as a camera. The input / output I / F 13 includes an image output device such as a display and an audio output device such as a speaker.

  The communication I / F 14 is implemented as hardware such as a network adapter, various types of communication software, and combinations thereof, and is configured to realize wired or wireless communication via the communication network 20 or the like.

  The storage 15 is configured by, for example, a magnetic disk, a flash memory, or the like. The storage 15 stores various programs including an operating system, various data, and the like.

  In the present embodiment, the moving image distribution server 10 can be configured using a plurality of computers each having the hardware configuration described above. For example, the moving image distribution server 10 may be configured by one or a plurality of server devices.

  The moving image distribution server 10 configured as described above has functions as a web server and an application server, and is provided from a web browser and other applications (for example, an application for moving image distribution service) installed in the user terminal 30. Various processes are executed in response to the request, and screen data (for example, HTML data or control data) corresponding to the result of the process is transmitted to the user terminal 30. On the user terminal 30, a web page or other screen based on the received data is displayed.

  The user terminal 30 is configured as a general computer, and as shown in FIG. 1, a computer processor 31, a main memory 32, an input / output I / F 33, a communication I / F 34, and a storage (storage device). 35, and these components are electrically connected via a bus or the like (not shown).

  The computer processor 31 is configured as a CPU, GPU, or the like, reads various programs stored in the storage 35 or the like into the main memory 32, and executes various instructions included in the programs. The main memory 32 is configured by, for example, a DRAM.

  The input / output I / F 33 is various input / output devices for exchanging information with a user or the like. The input / output I / F 33 includes, for example, an information input device such as a keyboard and a pointing device (for example, a mouse and a touch panel), a voice input device such as a microphone, and an image input device such as a camera. The input / output I / F 33 includes an image output device such as a display and an audio output device such as a speaker.

  The communication I / F 34 is implemented as hardware such as a network adapter, various types of communication software, and combinations thereof, and is configured to realize wired or wireless communication via the communication network 20 or the like.

  The storage 35 is configured by, for example, a magnetic disk or a flash memory. The storage 35 stores various programs including the operating system, various data, and the like. The program stored in the storage 35 can be downloaded from an application market or the like and installed.

  In the present embodiment, the user terminal 30 can be configured as a smartphone, a tablet terminal, a wearable device, a personal computer, or the like.

  The user of the user terminal 30 configured as described above performs video distribution provided by the server 10 by executing communication with the video distribution server 10 via a web browser or other application installed in the storage 35 or the like. You can use the service.

  Next, functions that the moving image distribution server 10 of this embodiment has will be described. FIG. 2 is a block diagram schematically showing functions of the moving image distribution server 10. As illustrated, the server 10 includes an information storage management unit 41 that stores and manages various types of information, a basic function control unit 43 that controls basic functions of the video distribution service, and a video distribution control unit that controls video distribution. 45. These functions are realized by the cooperation of hardware such as the computer processor 11 and the main memory 12 and various programs and data stored in the storage 15, for example, in the main memory 12. This is realized by the computer processor 11 executing instructions included in the read program. 2 may be realized by the cooperation of the server 10 and the user terminal 30, or may be realized by the user terminal 30.

  The information storage management unit 41 of the moving image distribution server 10 stores and manages various information in the storage 15 and the like. For example, as shown in FIG. 2, the information storage management unit 41 includes a user information table 411 that manages information related to a user of the video distribution service, and a distribution information table 412 that manages information related to individual video distribution. Can be configured.

  Moreover, in this embodiment, the information storage management part 41 has 3D model DB413, as shown in FIG. The DB 413 stores a plurality of 3D models (image information) for drawing (rendering) a user's avatar in a three-dimensional virtual space.

  The basic function control unit 43 of the video distribution server 10 executes various processes related to control of the basic functions of the video distribution service. For example, the basic function control unit 43 transmits HTML data or control data of various screens related to the basic function to the user terminal 30, and in response to an operation input by the user via the screen displayed on the user terminal 30. This process is executed, and HTML data or control data corresponding to the result of the process is transmitted to the user terminal 30. The basic functions controlled by the basic function control unit 43 include, for example, login authentication (user authentication), billing control, and user management (for example, update of the user information table 411).

  The moving image distribution control unit 45 of the moving image distribution server 10 executes various processes related to moving image distribution control. In the present embodiment, the moving image distribution control unit 45 generates a moving image corresponding to a virtual space including a plurality of avatars corresponding to a plurality of users who perform joint distribution (collaborative distribution) of moving images. It is comprised so that it may provide with respect to each viewing user. For example, the moving image distribution control unit 45 is configured to transmit the generated moving image to the user terminals 30 of a plurality of viewing users.

  In the present embodiment, the moving image distribution control unit 45 is configured to draw each of a plurality of avatars corresponding to a plurality of users who perform joint distribution of moving images based on the 3D model stored in the 3D model DB 413. ing. For example, when starting the joint distribution of the moving image, the moving image distribution control unit 45 identifies each of a plurality of users performing the joint distribution, identifies a 3D model corresponding to each identified user's avatar, and identifies the identified 3D. The model is configured to be acquired from the 3D model DB 413. Information for specifying the 3D model corresponding to each user's avatar is managed in the user information table 411, for example.

  In addition, the moving image distribution control unit 45 is configured to operate each of the plurality of avatars in the virtual space based at least on input information input via the user terminal 30 by the corresponding user.

  As described above, the moving image distribution server 10 according to the present embodiment draws each avatar based on each 3D model of a plurality of avatars corresponding to a plurality of users who perform joint distribution of moving images, and the plurality of users respectively Since each of the plurality of avatars is controlled based on the input information to be input, and a moving image corresponding to a virtual space including the plurality of avatars is generated and distributed, for example, the real face is disclosed. Even when a user who feels resistance is included, it is possible to easily perform joint distribution of moving images by a plurality of users. As a result, activation of moving image distribution can be promoted.

  In this embodiment, the input information input in order to operate an avatar contains various information, for example, the operation information with respect to the touch panel of the user terminal 30, the operation information with respect to a physical controller, etc. Further, for example, the input information includes an image input via the camera of the user terminal 30. In this case, the moving image distribution control unit 45 selects an avatar based on the posture of the user included in the input image. Can be configured to operate. For example, the moving image distribution control unit 45 receives an image input (captured) via the camera of the user terminal 30 from each of the user terminals 30 of a plurality of users who perform joint distribution of moving images, and converts the received image into the received image. The user's posture (bone) included is detected, and the operation of the corresponding avatar is controlled based on the detected user's posture (for example, in conjunction with the user's posture). Detection of the user's posture included in the image can be realized, for example, by applying a known human posture estimation technique. In addition, the moving image distribution control unit 45 is configured to operate the avatar based at least on the arrangement in the image of one or more predetermined parts (for example, the face and both hands) of the user's body included in the input image. Can be done. Such a configuration makes it possible to operate the avatar based on the actual movement of the user.

  In addition, the moving image distribution control unit 45 may be configured to generate a moving image including sound that is input via the user terminal 30 by each of a plurality of users who perform joint distribution of moving images. For example, the moving image distribution control unit 45 receives sound input from the user terminals 30 of each of a plurality of users who perform joint distribution of moving images through the microphone of the user terminal 30, and includes the received sound. Is configured to generate Such a configuration enables distribution of moving images including sound input by each user who performs joint distribution of moving images.

  In this case, the moving image distribution control unit 45 may be configured to control the movement of the corresponding avatar's mouth based on the sound input by each of a plurality of users who perform the joint distribution of moving images. For example, the moving image distribution control unit 45 operates the avatar's mouth of the specific user according to a predetermined rule during a period when the voice is input by the specific user, while the period when there is no voice input by the specific user. The mouth of the specific user's avatar is configured to be stationary. Such a configuration helps to bring the movement of the avatar's mouth closer to the movement of the actual user's mouth.

  Moreover, the moving image delivery control part 45 may be comprised so that the volume of the audio | voice input by the corresponding user may be controlled based at least on the position in the virtual space of the avatar. For example, as the distance between the virtual camera and the avatar in the virtual space is closer (or farther), the moving image distribution control unit 45 increases (or decreases) the volume of the voice input by the corresponding user. To produce a video. In addition, for example, the moving image distribution control unit 45 controls the volume of each of the left speaker and the right speaker for the sound input by the corresponding user based on the positional relationship between the virtual camera and the avatar in the virtual space. (For example, when the avatar is positioned on the left side of the virtual camera, the volume for the left side of the corresponding voice is increased as compared with the volume for the right side). Such a configuration makes it possible to output sound at a volume corresponding to the position of the avatar in the virtual space.

  Next, a specific example of the moving image distribution server 10 of this embodiment having such a function will be described. In this example, the server 10 provides a live video distribution service for a user to distribute a video in real time. FIG. 3 illustrates information managed in the user information table 411 in this example. The user information table 411 manages information related to the user of the live video distribution service. In this example, the user information table 411 is information related to the user's avatar in association with a “user account” that identifies an individual user as illustrated. Information such as “avatar information”, “distribution history information” that is information related to the distribution history of live video, and “viewing history information” that is information related to the viewing history of live video is managed.

  The avatar information includes information for specifying a 3D model selected in advance by the user from among a plurality of 3D models stored in the 3D model DB 413. The plurality of 3D models stored in the 3D model DB 413 correspond to a plurality of avatars having different appearances from each other, and the user selects a desired 3D model from among the plurality of 3D models. The 3D model can be selected in advance.

  FIG. 4 illustrates information managed in the distribution information table 412 in this example. The distribution information table 412 manages information related to individual video distribution, and in this example, as illustrated, the distribution information table 412 identifies a distribution user who distributes a video in association with a “distribution ID” that identifies individual video distribution. Information such as “distribution user account”, “participating user account” for identifying participating users participating in the video distribution, “distribution status information”, and the like are managed. The distribution status information includes information such as the number of simultaneous viewers, the total number of viewers, and the elapsed time since the start of distribution.

  FIG. 5 is a flowchart illustrating the process executed by the moving image distribution server 10 when individual moving image distribution is performed in this example. For example, the user can instruct the start of moving image distribution via a screen displayed on the user terminal 30. First, as illustrated, the server 10 acquires a 3D model of the distribution user's avatar from the 3D model DB 413 (step S100). Information for specifying the 3D model of the distribution user avatar is managed in the avatar information of the user information table 411.

  When the 3D model of the distribution user is acquired, the server 10 is from the distribution user terminal 30 (hereinafter, referred to as “distribution user terminal 30”) until the distribution ends (NO in step S130). The input image input through the camera and the input sound input through the microphone are received, and the process of generating and transmitting the moving image based on the received input image and input sound is repeated (steps S110 and S120). ).

  6 is performed between the distribution user terminal 30, the video distribution server 10, and the user terminals 30 of a plurality of viewing users (hereinafter may be referred to as “viewing user terminal 30”) during the distribution of the live video. It is a figure for demonstrating said communication. As illustrated, an input image input via the camera of the distribution user terminal 30 and an input sound input via a microphone are transmitted from the distribution user terminal 30 to the moving image distribution server 10. The moving image distribution server 10 generates a moving image based on the received input image and input sound, and transmits the generated moving image to each of the distribution user terminal 30 and the plurality of viewing user terminals 30. Each of the distribution user terminal 30 and the plurality of viewing user terminals 30 displays the received moving image.

  FIG. 7 illustrates a viewing screen 60 displayed on the viewing user terminal 30 that receives the moving image generated by the moving image distribution server 10. As shown in the figure, the screen 60 displays the virtual space 100 on the entire screen, a comment input area 62 positioned on the left side of the lower end of the screen, a comment display area 63 positioned on the upper side of the area 62, and a lower end of the screen. The participation button 64 located on the right side of the virtual space 100 is arranged so as to overlap the virtual space 100. When the viewing user inputs a comment via the comment input area 62, the input comment is displayed in the comment display area 63. When the viewing user selects the participation button 64, a request to participate in the live video being viewed is made. Details of the request for participation in the live video will be described later.

  In this example, the moving image generated by the moving image distribution server 10 corresponds to the virtual space 100. As shown in FIG. 7, the virtual space 100 has a stage 102 at a substantially center, and a distribution user's avatar 110 is arranged on the stage 102. The avatar 110 is drawn based on the 3D model of the distribution user's avatar. The operation of the avatar 110 is controlled based on the input image received from the distribution user terminal 30. Specifically, the operation of the avatar 110 is controlled based on the distribution user's face and the arrangement of both hands in the input image. Be controlled.

  FIG. 8 schematically illustrates an input image 50 received from the distribution user terminal 30. As shown in the figure, in this example, the palm of the right hand RH of the distribution user is provided with a circular marker MK1 of the first color (for example, red), and the palm of the palm of the left hand LH of the distribution user is A circular marker MK2 of two colors (for example, yellow) is provided. These markers MK1 and MK2, for example, are configured as stickers to be attached to the palm of the hand, and the stickers are provided to the user by, for example, a provider of a live video distribution service. The markers MK1 and MK2 are directly drawn with ink or the like on the palm of the hand, for example. In this case, for example, the user draws the markers MK1 and MK2 on the palms of both hands according to the instructions distributed by the provider of the live video distribution service.

  As shown in FIG. 8, in this example, the user's face FC is detected and recognized as a rectangular detection area DA1 surrounding the outline of the face FC. In addition, the user's hands RH and LH are detected and recognized as rectangular detection areas DA2 and DA3 surrounding the contours of the markers MK1 and MK2 provided on the palms of the hands RH and LH, respectively. Such recognition of the face FC and both hands RH and LH (markers MK1 and MK2) is realized using a known object tracking technique, for example, using a learned model generated through machine learning. .

  As described above, when the user's face and both hands in the input image received from the distribution user terminal 30 are recognized, the operation of the avatar 110 is controlled so as to be interlocked with the arrangement of these faces and both hands. For example, when the distribution user takes the ancestry attitude, the arrangement of the distribution user's face and both hands included in the input image is an arrangement corresponding to the ancestry attitude, and the avatar 110 in the virtual space 100 becomes the ancestry attitude. Its operation is controlled.

  In this example, the moving image generated by the moving image distribution server 10 includes input sound (distribution user's voice) received from the distribution user terminal 30. In this example, the movement of the mouth of the avatar 110 may be controlled based on the input voice.

  FIG. 9 illustrates a distribution screen 70 displayed on the distribution user terminal 30. As shown in the drawing, the screen 70 displays the virtual space 100 on the entire screen, like the viewing screen 60 described above, and a comment input area 72 located on the left side of the lower end of the screen and an upper side of the area 72. The comment display area 73 is arranged so as to overlap. In addition, on the right side of the lower end of the distribution screen 70, a distribution stop button 74 is arranged in the virtual space 100 in place of the participation button 64 on the viewing screen 60. The distribution stop button 74 is an object for stopping the distribution of the live video.

  For example, the distribution user moves the body in front of the in-camera of the user terminal 30 configured as a smartphone, and operates the avatar 110 via an input image input (photographed) through the in-camera. A moving image corresponding to the virtual space 100 including the avatar 110 can be viewed through the distribution screen 70 displayed on the user terminal 30.

  Here, an operation when a viewing user participates in a live video will be described. In this example, the viewing user can make a participation request for the live video being viewed by selecting the participation button 64 on the viewing screen 60. FIG. 10 is a flowchart illustrating processing executed by the video distribution server 10 in response to a participation request from a viewing user. When the participation button 64 is selected, the server 10 first notifies the distribution user of a participation request by the viewing user, as shown (step S200).

  FIG. 11 illustrates the participation request notification object 80 displayed on the distribution user terminal 30 in response to the selection of the participation button 64 by the viewing user. The object 80 notifies that there has been a participation request from a specific user (user A in the example of FIG. 11), and rejects the participation request and a permission button 82 for permitting the participation request. Refusal button 84 is arranged. The participation request notification object 80 is displayed so as to be superimposed on the distribution screen 70.

  Then, when the permission button 82 is selected by the distribution user (YES in step S210), the server 10 acquires a 3D model of the viewing user (participating user) avatar who has requested to participate from the 3D model DB 413 ( Step S220), the distribution (collaboration distribution) of the moving image in which the participating user has participated is started (Step S230). If reject button 84 is selected by the distribution user (NO in step S210), the participation request is rejected, so the processes in steps S220 and S230 are skipped.

  FIG. 12 shows a distribution user terminal 30, a user terminal 30 of a participating user (hereinafter also referred to as “participating user terminal 30”), a video distribution server 10, and a plurality of viewing user terminals during a collaboration distribution period. FIG. 3 is a diagram for explaining communication performed between 30; As shown in the figure, during the collaboration distribution period, an input image and an input sound are transmitted from the distribution user terminal 30 to the video distribution server 10, and from the participating user terminal 30 to the video distribution server 10. An input image input via the camera of the participating user terminal 30 and an input sound input via the microphone are transmitted. The moving image distribution server 10 generates a moving image based on the input image and the input sound received from the distribution user terminal 30 and the participating user terminal 30, respectively, and the generated moving image is distributed to the distribution user terminal 30, the participating user terminal 30, and a plurality of viewing users. It transmits to each of the terminals 30. Each of the distribution user terminal 30, the participating user terminal 30, and the plurality of viewing user terminals 30 displays the received moving image.

  In this example, the avatar of the participating user is added to the moving image generated by the moving image distribution server 10 during the collaboration distribution period in the virtual space 100. FIG. 13 illustrates a viewing screen 60 during the collaboration distribution period. As shown in the figure, during the collaboration distribution period, the position of the distribution user's avatar 110 on the stage 102 slightly slides to the left, and the participating user's avatar 112 is newly aligned to the right side of the avatar 110. Arranged (added). The avatar 112 is drawn based on the 3D model of the participating user's avatar, and its operation is controlled based on the input image received from the participating user terminal 30 in the same manner as the avatar 110 of the distribution user.

  In addition, during the collaborative distribution period, the moving image generated by the moving image distribution server 10 includes the input voice of the participating user received from the participating user terminal 30 (participation) in addition to the input voice of the distributing user received from the distributing user terminal 30. User voice). In this example, based on the positions of the avatars 110 and 112 in the virtual space 100, the volume of the input voice of each of the distribution user and the participating user may be controlled. For example, the input voice of the distribution user whose avatar 110 is located on the left side on the stage 102 is set to have a louder left speaker volume compared to the right speaker volume. The input voice of the participating user whose avatar 112 is located on the right side in the upper side may have a volume for the right speaker that is higher than the volume for the left speaker.

  As described above, in this example, it is possible to easily generate a moving image corresponding to a virtual space including the avatars of the distribution user and the participating user and distribute it to a plurality of viewing users. In this example, the participation button 64 on the viewing screen 60 cannot be selected during the collaboration distribution period, and the number of participating users is limited to one. However, in another example of the present embodiment, collaboration distribution in which two or more participating users participate may be possible.

  The video delivery server 10 according to the present embodiment described above draws each avatar based on each 3D model of a plurality of avatars corresponding to a plurality of users who perform joint delivery of the videos, and the plurality of users respectively. The operation of each of the plurality of avatars is controlled based on input information (for example, an input image input through the camera of the user terminal 30), and corresponds to a virtual space including the plurality of avatars. Since a video is generated and distributed, for example, even when a user who feels resistance to publishing a face is included, it is possible to easily perform joint distribution of a video by a plurality of users, As a result, activation of video distribution can be promoted.

  In the embodiment described above, the moving image distribution server 10 generates the moving image. However, in another embodiment of the present invention, the moving image generation can be performed in the distribution user terminal 30. FIG. 14 illustrates communication performed between the distribution user terminal 30, the participating user terminal 30, the video distribution server 10, and the plurality of viewing user terminals 30 during the collaboration distribution period in the other embodiment. FIG. As shown in the figure, in this embodiment, only the input image is transmitted from the distribution user terminal 30 to the video distribution server 10, and the input image and the input are input from the participating user terminal 30 to the video distribution server 10. Audio is transmitted. The moving image distribution server 10 analyzes each input image received from each of the distribution user terminal 30 and the participating user terminal 30, and acquires arrangement information regarding the user's face and both hands in each input image. Then, the moving image distribution server 10 transmits the acquired arrangement information (distribution user) and arrangement information (participating user) and the input voice received from the participating user terminal 30 to the distribution user terminal 30. The distribution user terminal 30 generates a moving image based on the arrangement information (distribution user), arrangement information (participating user), and input voice (participating user) received from the moving picture distribution server 10 and the input voice in the distribution user terminal 30. Then, the generated moving image is transmitted to each of the participating user terminals 30 and the plurality of viewing user terminals 30 via the moving image distribution server 10. The distribution user terminal 30 acquires the 3D model of the distribution user and the participating user from the 3D model DB 413 before the collaboration distribution is started.

  Further, in the above-described embodiment in which the generation of the moving image is performed in the distribution user terminal 30, the analysis of each input image and the acquisition of the arrangement information can be performed in each of the distribution user terminal 30 and the participating user terminal 30. In this case, arrangement information and input sound are transmitted from the participating user terminals 30 to the moving image distribution server 10. The moving image distribution server 10 transmits the arrangement information and the input sound received from the participating user terminal 30 to the distribution user terminal 30. The distribution user terminal 30 is based on the arrangement information (participating user) and the input voice (participating user) received from the video distribution server 10, and the arrangement information (distribution user) and the input voice (distribution user) in the distribution user terminal 30. A moving image is generated, and the generated moving image is transmitted to each of the plurality of viewing user terminals 30 via the moving image distribution server 10.

  In still another embodiment of the present invention, the generation of the moving image can be performed in each of the distribution user terminal 30, the participating user terminal 30, and the plurality of viewing user terminals 30. FIG. 15 is a diagram for explaining communication performed between the distribution user terminal 30, the participating user terminal 30, the video distribution server 10, and the plurality of viewing user terminals 30 during the collaboration distribution period in the embodiment. It is. As shown in the figure, in this embodiment, the input image and the input sound are transmitted from the distribution user terminal 30 and the participating user terminal 30 to the moving image distribution server 10. The moving image distribution server 10 analyzes each input image received from each of the distribution user terminal 30 and the participating user terminal 30, and acquires arrangement information regarding the user's face and both hands in each input image. Then, the moving image distribution server 10 transmits the acquired arrangement information (distribution user) and arrangement information (participating user) and the input voice received from the participating user terminal 30 to the distribution user terminal 30, and the distribution user terminal 30 A moving image is generated based on the arrangement information (distribution user), the arrangement information (participating user), the input sound (participating user), and the input sound in the distribution user terminal 30 received from the moving image distribution server 10. In addition, the moving image distribution server 10 transmits the acquired arrangement information (distribution user) and arrangement information (participating user) and the input voice received from the distribution user terminal 30 to the participating user terminal 30. A moving image is generated based on the arrangement information (distribution user), the arrangement information (participating user), the input voice (delivery user), and the input voice in the participating user terminal 30 received from the moving picture distribution server 10. Furthermore, the moving image distribution server 10 receives the acquired arrangement information (distribution user) and arrangement information (participating user), and the input audio received from the distribution user terminal 30 and the participating user terminal 30, respectively. Each viewing user terminal 30 transmits video based on arrangement information (distribution user), arrangement information (participating user), input voice (distribution user), and input voice (participating user) received from the video distribution server 10. Is generated. In addition, each of the distribution user terminal 30, the participating user terminal 30, and the plurality of viewing user terminals 30 acquires a 3D model of each avatar of the distribution user and the participating user from the 3D model DB 413 before the start of the collaboration distribution.

  In the embodiment described above, the image information (3D model) of a plurality of users' avatars is managed in the video distribution server 10 (3D model DB 413). However, in other embodiments of the present invention, such avatar images are managed. At least a part of the information is managed by a device other than the server 10, and can be managed by the user terminal 30, for example.

  The processes and procedures described in the present specification are implemented by software, hardware, or any combination thereof other than those explicitly described. For example, the processes and procedures described in this specification are realized by mounting logic corresponding to the processes and procedures on a medium such as an integrated circuit, a volatile memory, a nonvolatile memory, and a magnetic disk. The processing and procedure described in this specification can be implemented as a computer program corresponding to the processing / procedure and executed by various computers.

  Even if the processes and procedures described herein are described as being performed by a single device, software, component, or module, such processes or procedures may be performed by multiple devices, multiple software, multiple Component and / or multiple modules. Also, the software and hardware elements described herein can be implemented by integrating them into fewer components or by disassembling them into more components.

  In the present specification, when the constituent elements of the invention are described as one or a plurality, or when they are described without being limited to one or a plurality of cases, they should be understood separately in context. The component may be either singular or plural.

DESCRIPTION OF SYMBOLS 10 Movie delivery server 11 Computer processor 30 User terminal 41 Information storage management part 411 User information table 412 Delivery information table 413 3D model DB
43 Basic Function Control Unit 45 Video Distribution Control Unit 50 Input Image 60 Viewing Screen 70 Distribution Screen 100 Virtual Space 110 Distribution User Avatar 112 Participating User Avatar

Claims (10)

A system for delivering video,
One or more storage devices that store at least image information of each avatar of a plurality of users, and one or more computer processors,
The one or more computer processors are responsive to execution of readable instructions,
A process of generating a moving image corresponding to a virtual space including the first and second avatars corresponding to the first and second users performing joint distribution of the moving image;
Providing the generated video to each of a plurality of viewing users,
The process of generating the moving image includes drawing the first and second avatars based on at least image information of the first and second avatars stored in the one or more storage devices, and , Operating the first and second avatars in the virtual space based at least on input information respectively input by the first and second users via the first and second user terminals, respectively. ,
system. The input information includes an image input via a camera,
The process of generating the moving image includes operating the first or second avatar based at least on the posture of the first or second user included in the input image.
The system of claim 1. The processing for generating the moving image is based on at least the arrangement of one or a plurality of predetermined parts of the body of the first or second user included in the input image in the input image. Including operating a second avatar,
The system of claim 2. The process of generating the moving image includes generating a moving image including sound input by the first or second user through the microphone of the first or second user terminal.
The system according to claim 1. The process of generating the moving image includes controlling movement of the mouth of the first or second avatar based at least on the input voice.
The system of claim 4. The process of generating the moving image includes controlling the volume of audio input by the first or second user based at least on the position of the first or second avatar in the virtual space.
The system according to any one of claims 1 to 5. The process of generating the moving image is executed in the first or second user terminal.
The system according to claim 1. The process of generating the moving image is executed in each user terminal of the plurality of viewing users.
The system according to claim 1. A method for delivering a moving image, executed by one or more computers accessible to one or more storage devices that store at least image information of each avatar of a plurality of users,
Generating a video corresponding to a virtual space including the first and second avatars corresponding to the first and second users performing joint video distribution;
Providing the generated video to each of a plurality of viewing users, and
The step of generating the moving image includes drawing the first and second avatars based on at least image information of the first and second avatars stored in the one or more storage devices, and , Operating the first and second avatars in the virtual space based at least on input information respectively input by the first and second users via the first and second user terminals, respectively. ,
Method. A program for distributing videos,
In response to execution on one or more computers accessible to one or more storage devices that store at least the image information of each avatar of the plurality of users, the one or more computers,
A process of generating a moving image corresponding to a virtual space including the first and second avatars corresponding to the first and second users performing joint distribution of the moving image;
Providing the generated video to each of a plurality of viewing users, and
The process of generating the moving image includes drawing the first and second avatars based on at least image information of the first and second avatars stored in the one or more storage devices, and , Operating the first and second avatars in the virtual space based at least on input information respectively input by the first and second users via the first and second user terminals, respectively. ,
program.

Images