JP2019092146A - Distribution server, distribution program and terminal - Google Patents

Abstract

To format distribution visual data for live broadcast by automatically adding a presentation effect thereto.SOLUTION: A distribution server comprises a receiving section, a determination section, a composite section and a distribution section. The receiving section receives visual data for live broadcast from a distributor terminal. The determination section determines whether or not a presentation effect condition is satisfied. The composite section generates composite visual data by combining the visual data and presentation effect data when the presentation effect condition is determined to be satisfied. The distribution section distributes the composite visual data to the distributor terminal and an audience terminal.SELECTED DRAWING: Figure 2

Description

  The present invention relates to live delivery of video in a video sharing system.

  In recent years, in some moving image sharing systems, it is possible to live distribute a moving image shot by a user (hereinafter simply referred to as a distributor) to a large number of spectators via the Internet. In such moving pictures lively distributed via the Internet (hereinafter simply referred to as live programs), available time frames are allocated to distributors, as in the case of television programs.

  Television programs are not limited to regular recorded programs, and even live broadcast programs are often given opening, ending, and other intermediate effects. However, since the technologies and human resources that can be used by live program distributors are usually limited compared to television program production companies, the presentation of opening and ending etc. has been added to the videos taken by the distributors. Real-life delivery is difficult. For this reason, the live program tends to have the main part of the live program suddenly start and end suddenly according to the assigned time frame, and the audience familiar with the program structure of the television program feels unnatural It may be.

  Also, immediately after the start of live delivery, often only a limited number of spectators, centered on the followers of the distributor, gather. Since the distributor wants to get in as much of the audience as possible before entering the main project, there is a sense of lengthening, such as waiting in a state of being idle for a while, conducting a microphone test, or leaving the seat in some cases. The scene may continue. When the audience encounters such a scene, they may feel bored and stop viewing.

  According to Patent Document 1, when the distribution request from the viewer is received (S100) ([0027]), the program content for personal use is extracted from the program content system stored in the program content database (S106). It is described that content is organized (S108) ([0028]) and distributed to a viewer terminal ([0030]). Also, it is described that the organized program content includes a title and an ending ([0027, [0029]).

  In addition, in Patent Document 2 (in particular, [0022], [0057], [0059], abstracts), the background portion is cut out from the photographed moving image data and synthesized with the background image or moving image data prepared in advance. A technique is disclosed for creating synthetic moving picture data and further editing the synthetic moving picture data to generate integrated moving picture data including opening title and end role data, and sharing the integrated moving picture data via a network.

Unexamined-Japanese-Patent No. 2006-339780 JP, 2016-154021, A

  Although Patent Document 1 describes that program content is distributed in a state where real-time content is included at any time, this real-time content includes, for example, scheduled news, national news, local limited local news, and weather forecast information. Etc. ([0021]), and it is configured, for example, by combining content 824a composed of images and narration of several news photos and content 826a etc. by coverage animation ( [0029]). Here, since the coverage moving image generally refers to a recorded moving image, the real-time content described in Patent Document 1 is different from a live program. Also, the title 810 and the ending 850 included in the personalized program content 800 are treated as contents separate from the real-time content 820 (FIG. 3, [0029]).

  The technology described in Patent Document 2 generates integrated motion picture data based on a motion picture that has been fully shot and shares the motion picture data, and does not assume application to a live program. In addition, the opening title and the end role are data independent of the synthetic moving image, and it is not assumed to combine the two.

  Therefore, even if these techniques are applied to delivery of live programs, titles, openings, endings, end rolls, etc. independent of the main program are added. Therefore, it is easy to improve the distinctiveness at the beginning and the end, the sense of extension immediately after the start, the lack of sense of sharpness due to the limited range of available expressions, etc. specific to user-initiated live programs. Not

  An object of the present invention is to automatically add an effect to video data for live distribution to adjust the appearance.

  The distribution server according to the first aspect of the present invention includes a receiving unit, a determining unit, a combining unit, and a distributing unit. The receiving unit receives video data for live distribution from the distributor terminal. The determination unit determines whether the rendering condition is satisfied. The synthesizing unit synthesizes the effect data with the video data when the effect condition is satisfied, and generates synthesized video data. The distribution unit distributes the composite video data to the distributor terminal and the audience terminal.

  A distribution server according to a second aspect of the present invention includes a receiving unit, a determination unit, a processing unit, and a distribution unit. The receiving unit receives video data for live distribution from the distributor terminal. The determination unit determines whether the rendering condition is satisfied. The processing unit performs predetermined processing on the video data when the rendering condition is satisfied, and generates processed video data. The distribution unit distributes the processed video data to the distributor terminal and the audience terminal.

  The distribution server according to the third aspect of the present invention includes a determination unit, a reception unit, a generation unit, and a distribution unit. The determination unit determines whether the rendering condition is satisfied. The receiving unit receives video data for live distribution from the distributor terminal, and when the rendering condition is satisfied, the receiver unit is further from at least one of the distributor terminal and the audience terminal to at least one of the distributor terminal and the audience terminal Receive an operation request for dynamic presentation data to be combined with video data. The generation unit generates operation data of dynamic effect data based on the operation request. The distribution unit distributes the video data and the operation data to at least one of the distributor terminal and the audience terminal.

  A terminal according to a fourth aspect of the present invention includes a determination unit, a generation unit, a transmission unit, a reception unit, a control unit, and a combination unit. The determination unit determines whether the rendering condition is satisfied. The generation unit generates an operation request for dynamic presentation data according to a user input when the presentation condition is satisfied. The transmission unit transmits the operation request to the distribution server. The receiving unit receives, from the distribution server, live delivery video data and operation data of dynamic presentation data generated based on the operation request when the presentation condition is satisfied. The control unit controls dynamic presentation data based on the operation request. The combining unit combines dynamic presentation data with video data.

  According to the present invention, it is possible to automatically add a rendition to video data for live distribution and adjust the appearance.

BRIEF DESCRIPTION OF THE DRAWINGS The block diagram which illustrates the live delivery system of the animation concerning a 1st embodiment. FIG. 2 is a block diagram illustrating a distribution server according to the first embodiment. The figure which illustrates video data. FIG. 4 is a view exemplifying composite video data obtained by combining effect data with the video data of FIG. 3; FIG. 4 is a view exemplifying composite video data obtained by combining effect data with the video data of FIG. 3; FIG. 4 is a view exemplifying composite video data obtained by combining effect data with the video data of FIG. 3; FIG. 4 is a view exemplifying composite video data obtained by combining effect data with the video data of FIG. 3; 3 is a flowchart illustrating the operation of the distribution server of FIG. The block diagram which illustrates the delivery server concerning a 2nd embodiment. The block diagram which shows the modification of the terminal concerning 1st Embodiment and 2nd Embodiment. The block diagram which illustrates the delivery server concerning a 3rd embodiment. The figure which illustrates video data. The figure which illustrates the processed video data which processed the video data of FIG. The figure which illustrates the processed video data which processed the video data of FIG. The block diagram which shows the modification of the terminal concerning a 3rd embodiment. The block diagram which illustrates the delivery server concerning a 4th embodiment. FIG. 4 is a view exemplifying composite video data obtained by combining dynamic presentation data with the video data of FIG. 3; FIG. 4 is a view exemplifying composite video data obtained by combining dynamic presentation data with the video data of FIG. 3; The block diagram which illustrates the terminal concerning a 4th embodiment. FIG. 17 is a block diagram showing a modification of FIG. 16;

  Hereinafter, the description of the embodiment will be described with reference to the drawings. Hereinafter, elements which are the same as or similar to the already described elements are denoted by the same or similar reference numerals, and redundant descriptions will be basically omitted. For example, when there are a plurality of identical or similar elements, a common code may be used to explain each element without distinction, and the common code may be used to distinguish and describe each element. In addition, branch numbers may be used.

First Embodiment
The distribution server according to the first embodiment can be incorporated into the moving image live distribution system illustrated in FIG. The live distribution system includes a distributor terminal 100, a distribution server 200, and audience terminals 300-1, 300-2, and so on.

  The distribution server 200 is connected to the distributor terminal 100 and the audience terminal 300 via a network, and can mutually transmit and receive data. In addition, the number of each apparatus shown in FIG. 1 is only an illustration. For example, the number of spectator terminals 300 may be zero, or may be hundreds or thousands. In addition, a Web server or a comment delivery server not shown in FIG. 1 may be further provided, or these functions may be incorporated into the delivery server 200.

  The distributor terminal 100 is, for example, an electronic device such as a computer connected to a moving image source such as a video camera, for example, a television receiver (including an internet television), a PC (Personal Computer), a mobile terminal (for example, a tablet, a smartphone, A laptop, a feature phone, a portable game console, a digital music player, an electronic book reader, etc., a VR (Virtual Reality) terminal, an AR (Augmented Reality) terminal, etc. may be used. The distributor terminal 100 sequentially transmits, to the distribution server 200, encoded video data output from the moving image source.

  The distribution server 200 receives encoded video data sequentially transmitted from the distributor terminal 100 over a time frame assigned to live distribution of video data. The distribution server 200 basically distributes the video data to the distributor terminal 100 and the audience terminal 300 as it is.

  However, when the effect condition described later is satisfied, the distribution server 200 combines the effect data described later with the image data, and the combined image data (hereinafter referred to as combined image data) is the distributor terminal 100. And distribute to the audience terminal 300. The determination of satisfaction / dissatisfaction of the rendering condition, creation of the rendering data, and composition of the rendering data may be realized, for example, by the delivery server 200 executing a script.

  The audience terminal 300 may be an electronic device similar to the distributor terminal 100, but unlike the distributor terminal 100, the audience terminal 300 need not be connected to a moving image source.

  By the distribution server 200 automatically synthesizing such effect data, the distributor can easily use, for example, an opening effect, an ending effect, or an intermediate effect. Since the addition of this effect is performed by the distribution server 200, there is no problem even if the distributor does not have advanced video editing skills, hardware or software for moving image editing, or a supporter such as a director. Therefore, the distributor can easily perform live presentation of a well-formatted video in the television program style.

  Hereinafter, the configuration and operation of distribution server 200 in FIG. 1 will be sequentially described with reference to the drawings.

  The distribution server 200 is a computer, and controls distribution of video data received from the distributor terminal 100, determination of rendering conditions, creation of rendering data, composition of rendering data, various types of distribution data received from the distributor terminal 100 and the audience terminal 300 A processor that interprets the request of the request and generation of a response to the request, a memory that temporarily stores a program executed by the processor to realize the processing, data used by the processor, and the like. It contains.

  The distribution server 200 can further use a communication device for connecting to a network and an auxiliary storage device for storing a large amount of data. The communication device and the auxiliary storage device may be incorporated in the distribution server 200 or may be externally attached to the distribution server 200.

  The communication device communicates with the distributor terminal 100 and the audience terminal 300 via the network. For example, the communication device receives video data from the distributor terminal 100 or transmits a response to a distribution request from the audience terminal 300.

  The auxiliary storage device stores, for example, data such as effect conditions and effect data. The auxiliary storage device is preferably, for example, a non-volatile storage medium such as a hard disk drive (HDD) or a solid state drive (SSD). The auxiliary storage device may be a file server connected to the distribution server 200 via a network.

  Next, the description of the configuration example of the distribution server 200 will be continued using FIG. The distribution server 200 of FIG. 2 includes a receiving unit 201, a clock unit 202, a distribution control unit 203, a rendering condition determination unit 204, a rendering condition storage unit 205, a rendering data generation unit 206, and a rendering data combining unit 207. And a video distribution unit 208.

The receiving unit 201 receives various data via the network. The receiving unit 201 may be the above-described communication device or an interface with the communication device.
Specifically, the receiving unit 201 receives video data for live distribution from the distributor terminal 100. The receiving unit 201 sends this video data to the rendering condition determining unit 204. Also, the receiving unit 201 receives the distribution request of the video data from the audience terminal 300. The receiving unit 201 sends a distribution request to the distribution control unit 203.

  Furthermore, the receiving unit 201 may receive information indicating that a predetermined user input has been made on the distributor terminal 100 or the audience terminal 300, and may send this to the presentation condition determining unit 204. Also, the receiving unit 201 may receive a comment posted on the video data from a comment distribution server not shown in FIG. 1 and send this to the presentation condition judging unit 204. When the distribution server 200 has the function of a comment distribution server, the receiving unit 201 can receive a comment directly from the distributor terminal 100 or the audience terminal 300 and can send it to the effect condition determination unit 204.

  The clock unit 202 instructs a date and time. The clock unit 202 includes, for example, a crystal oscillator that vibrates at a fixed frequency, a divider circuit that divides its output to obtain a 1 Hz signal, and a counter that counts this signal to obtain a serial value indicating date and time. . The delivery control unit 203 and the effect condition determination unit 204 can refer to the date and time instructed by the clock unit 202.

The delivery control unit 203 controls live delivery of video data. The distribution control unit 203 may be the processor and memory described above.
Specifically, the delivery control unit 203 compares the date and time instructed by the clock unit 202 with the time frame assigned to live delivery of video data, and manages the start and end of live delivery of video data. Further, the distribution control unit 203 receives a distribution request from the audience terminal 300 from the reception unit 201, and controls the video distribution unit 208 so that the video data is distributed to the audience terminal 300. The distribution control unit 203 may manage the number of spectators based on the received distribution request. Information such as the time frame managed by the distribution control unit 203 and the number of spectators can be referred to by the effect condition determination unit 204 or the effect data creation unit 206.

  The effect condition determination unit 204 receives the video data from the reception unit 201, reads the effect condition for the video data from the effect condition storage unit 205, and determines whether the effect condition is satisfied. If the effect condition determination unit 204 determines that the effect condition is satisfied, the effect condition determination unit 204 sends the video data to the effect data combining unit 207; otherwise, it sends the video data to the video distribution unit 208. The effect condition determination unit 204 may be the processor and memory described above.

  In order to determine whether the rendering condition is satisfied, the rendering condition determining unit 204 determines the date and time instructed by the clock unit 202, the time frame managed by the distribution control unit 203, the number of spectators, and the receiving unit as necessary. The comment received by 201, information indicating that a predetermined user input has been made on the distributor terminal 100 or the audience terminal 300, reproduced video or reproduced audio of video data, or the like can be referred to.

  Here, the rendering condition is, for example, a reference for determining whether to add an opening performance, an ending effect, or an intermediate effect. Specifically, the presentation condition (opening presentation condition) for adding the opening presentation is the clock at the beginning of the time frame assigned to the video data, or at the presentation start point set a predetermined time after the beginning It may be determined to be satisfied after the date and time instructed by the unit 202 has arrived and until the effect end condition is satisfied.

  The opening presentation condition may be determined so as to be satisfied between the number of audiences of the video data satisfying the predetermined condition and the satisfaction of the presentation end condition. The condition here may be, for example, that the number of spectators has reached a predetermined number.

  The opening presentation condition may be determined so as to be satisfied between the presence of a predetermined user input on the distributor terminal 100 or the audience terminal 300 and the satisfaction of the presentation end condition. The predetermined user input may be, for example, selecting a graphical user interface (GUI) component, such as a button for instructing start of presentation, displayed on the distributor terminal 100.

  The opening presentation condition may be determined so as to be satisfied between the comment posted on the video data satisfying the predetermined condition and the satisfaction of the presentation end condition. The conditions here may be, for example, that the number of comments has reached a predetermined number, that the number of comments including a specific keyword has reached a predetermined number, or that there is a comment that includes a rare word.

  The opening effect condition may be determined to be satisfied between the time when the reproduced video or reproduced sound of the video data satisfies the predetermined condition and the satisfied condition of the effect. The conditions here are, for example, that the distributor has recognized that the distributor has taken a predetermined gesture or expression, for example, and that the reproduction voice has recognized that the distributor has uttered a predetermined phrase, for example. possible.

  The presentation end condition of the opening presentation is, for example, when a predetermined time has elapsed from the start of presentation, when the number of spectators or the number of comments has reached the required number, or the user terminal 100 or spectator terminal 300 has a predetermined user input At the time, for example, when it is recognized from the reproduced video of the video data that the distributor took a predetermined gesture or expression, for example, when the reproduced voice recognizes that the distributor uttered a predetermined phrase, etc. It may be determined to

  By starting the opening production from a point shifted from the beginning of the time frame, the video data will be distributed as it is immediately after the start of the live distribution, and the opening production will start when the audience gathers to some extent, and the audience for the main program of the live program Can be motivated to continue watching and raising expectations. On the other hand, it is possible to prevent the first stage audience from leaving without waiting for the opening presentation by starting the opening presentation from a point coincident with or near the beginning of the time frame.

  In addition, as the effect condition (ending effect condition) for adding the ending effect, the date and time indicated by the clock unit 202 reaches the effect start point set a predetermined time before the end of the time frame assigned to the video data Then, it may be determined to be satisfied before the end condition of presentation is satisfied. In addition, the ending effect condition may be determined in the same manner as the opening effect condition described above.

  The ending condition of the ending effect may be the same as the ending condition of the opening effect, and is determined to be satisfied when the date and time indicated by the clock unit 202 reaches the end of the time frame assigned to the video data. It may be done.

  The intermediate rendering may be realized, for example, by combining a presentation video such as a fanfare, video data with a replay video or a slow motion playback video, in the middle of a time frame assigned to the video data. The replay video or the slow motion playback video may be, but not limited to, the previous video. In addition, when a live program is shot by a multi-camera, images of different viewpoints may be combined or the viewpoint may be switched.

  The effect condition (intermediate effect condition) for adding the intermediate effect is after the date and time indicated by the clock unit 202 reaches the effect start point set a predetermined time after the start of the time frame assigned to the video data , And may be determined to be satisfied before the effect termination condition is satisfied. In addition, the intermediate rendering condition is that the distributor or the audience clicks the advertisement or the product introduction given to the live program, or the distributor or the audience purchases the product targeted for the advertisement or the product introduction. It may be determined to be satisfied after a predetermined time of purchase until the end condition of presentation is satisfied. Here, the advertisement and the product introduction may be displayed, for example, in the display page of the live program or in a page to which transition can be made from the display page, at the distributor terminal 100 or the audience terminal 300. In addition, the intermediate rendering condition may be determined in the same manner as the opening rendering condition described above. Further, the end condition of the intermediate effect may be the same as the end condition of the opening effect.

  For example, when the number of spectators of the video data or the number of comments posted to the video data suddenly rises, an intermediate effect of synthesizing the video of the fanfare may be started. Also, for example, when a comment including a specific keyword such as "replay", "slow", "one more time", etc. is posted to video data for a predetermined number or more within a predetermined time, replay video or slow motion playback video An intermediate effect to be synthesized may be started.

  The effect condition storage unit 205 stores the various effect conditions described above. The presentation condition stored in the presentation condition storage unit 205 is read out by the presentation condition determination unit 204 and the presentation data creation unit 206 as necessary. The effect condition storage unit 205 may be the above-described auxiliary storage device or an interface with the auxiliary storage device.

The effect data creation unit 206 creates effect data and sends it to the effect data combining unit 207. The effect data creation unit 206 may be the processor and memory described above.
The presentation data can be roughly divided into dynamic presentation data and static presentation data. The static effect data may be fixed moving image data or still image data that is not influenced by the content of the video data. The fixed moving image data may be, for example, various animations or live-shot images having an area for inserting a reproduced image of the video data to be delivered live. The fixed still image data may be, for example, a frame for decorating the screen end of the reproduced video of the video data, an object covered by the reproduced video of the video data, or the like. In addition, when not using the dynamic presentation data mentioned later, the presentation data creation part 206 can be abbreviate | omitted by presetting static presentation data.

  The dynamic rendering data may be, for example, moving image data or still image data including information dynamically indicating a necessary condition for satisfying the rendering end condition, or after live distribution of video data is started It may be moving image data or still image data including information about an event that has occurred, or it may be a replay video or slow motion playback video of video data to be subjected to live distribution, or a video of another viewpoint For example, it may be moving image data or still image data including a moving image title, a credit title, a distributor comment or an explanatory note. Such information may be extracted from the information input by the distributor, the audience or the service operator before the start of the live distribution, or the information automatically added by the distribution server 200 or another server. For example, the effect data may be information automatically generated from metadata such as a moving image tag which is a keyword assigned to a live program, or a sponsor name assigned to a live program, for example, a program category, It may be moving image data or still image data including a URL (Uniform Resource Locator) and the like. Here, the moving image tag and the sponsor name may be manually provided by the distributor, the audience or the service operator, or may be automatically provided by the distribution server 200 or another server.

  Furthermore, dynamic presentation data may be automatically generated based on information displayed on a web page associated with the live program. For example, on a program introduction page such as a front page of a moving image sharing service, thumbnails of a large number of live programs currently being released may be displayed side by side. Then, on such a program introduction page, supplemental information on a live program such as "during popularity" may be displayed around the thumbnail. The live data corresponding to the thumbnail to which the supplementary information is added may be the supplementary information itself or presentation data including information extracted / processed from the supplementary information, or the presentation data to be applied may be applied. It may be switched depending on the supplementary information. For example, the program title may be automatically edited based on the information extracted from the supplementary information, the sponsor name extracted from the supplementary information, or the like.

  Specifically, the effect data creation unit 206 is, for example, information indicating the gap between the current number of spectators of the video data or the number of comments and the necessary number as a necessary condition for satisfying the effect end condition, for example, You may create presentation data including texts such as "after xxx people" and "after xxx comments". Information on the current number of spectators or the number of comments of the video data can be acquired from, for example, the distribution control unit 203. Further, information on the number of spectators or the number of comments necessary for the effect end condition to be satisfied can be acquired from, for example, the effect condition storage unit 205. By displaying such information, the spectator can grasp the standard time until the start of the main program of the live program, and can judge whether or not he / she can wait until then and whether he / she can leave the seat.

  Further, the effect data creation unit 206 may use, for example, the number of audiences of the video data, a part of comments posted on the video data (for example, a digest), as information related to an event that occurs after live distribution of the video data is started. The number of comments posted for video data, the audience name who posted the comments for video data, the number of clicks on advertisements and product introductions attached to live programs, and information on products purchased from advertisements and product introductions ( For example, effect data including information indicating the name of a product, the number of purchases, the total purchase amount, a photograph, purchaser information, and the like may be created. By using such information as, for example, an end roll, it is possible to perform an ending effect appropriate to the aspect of live distribution of individual video data.

  Also, the effect data creation unit 206 may create effect data that changes in response to a user input made to the distributor terminal 100 or the audience terminal 300 or a comment posted by the distributor terminal 100 or the audience terminal 300. Good. For example, the effect data is an object blocking the screen of the reproduced video of the video data (for example, a confetti, a symbol of “8” meaning applause, a bouquet, a heart mark, a cloud, a hedge, an icon set by a distributor, a moving image It may include the mascot character of the shared service.

  In this case, the effect data creation unit 206 creates effect data in which objects covering the entire screen are spread at the start of the effect, and a predetermined user input (for example, an object, a button, etc.) to the distributor terminal 100 or the audience terminal 300 When the GUI is selected or a comment is posted, the object may be deleted, moved, or shrunk, etc. to gradually make the reproduced video screen of the video data visible. Conversely, the effect data creation unit 206 creates effect data including zero or a small number of objects at the start of the effect, and performs predetermined user input to the distributor terminal 100 or the audience terminal 300 (for example, selection of GUI such as object or button) ), Or when a comment is posted, the object may be made to appear, moved, enlarged, etc. so that the reproduced video screen of the video data may gradually disappear.

  The effect of gradually making the reproduced video screen of video data visible is suitable for the opening, but is not limited to this. The effect of gradually losing sight of the reproduced video screen of the video data is suitable for ending, but is not limited to this.

  In addition, even if the state of the object is not completely synchronized between all the terminals, it does not affect the viewing experience of the main program of the live program, so the frequency of performing synchronization may be reduced to reduce the load on the server.

  Also, in this example, the effect end condition may be determined by the number, size, position, etc. of the objects. In this case, the distributor and the audience increase the number of objects required, for example, how many objects need to be reduced, from the state of the objects included in the playback video of the composite video data to the satisfaction of the rendering termination condition. You can figure out what you need. Therefore, such an object also corresponds to an example of information dynamically indicating the necessary condition for satisfying the effect end condition.

  The effect data synthesis unit 207 receives the video data from the effect condition determination unit 204 and receives the effect data from the effect data generation unit 206. The effect data synthesizing unit 207 synthesizes the effect data with the video data to generate synthesized video data. For example, the effect data combining unit 207 may combine the effect data with the video data using a known technique such as alpha blending and picture-in-picture. The effect data synthesis unit 207 sends the synthesized video data to the video distribution unit 208. The effect data synthesis unit 207 may be the processor and memory described above. The type of effect data used for synthesis may be automatically determined by a predetermined algorithm, may be determined by user input information of a distributor and / or an audience, or may be determined by a combination of both. May be For example, according to information on the number of clicks on advertisements and product introductions given to live programs, information on products purchased from advertisements and product introductions, such as the number of purchases of products and total purchase amount, the effect data used for ending effect is It may be switched.

  For example, the effect data synthesizing unit 207 may resize a part or all of the screen of the reproduced video of the video data, or may trim a part of the screen and combine it with the effect data, or the reproduced video of the video data The video data may be combined with the effect data so that a part of the screen of is exposed and the remaining part is hidden. Video data is illustrated in FIG. 3, and composite video data is illustrated in FIG. 4 to FIG.

  As exemplified in FIG. 4, the effect data synthesizing unit 207 resizes a part (for example, the face area of the distributor) and the whole of the screen of the reproduced video of the image data of FIG. May be embedded in moving image data or still image data. Further, in the example of FIG. 4, the text of “cast xxx” is included in the effect data as a credit title. As described above, the entire screen of the reproduced video of the video data is resized to be small and then combined with the effect data, so that the screen can be prevented from being masked by the effect data. Therefore, it is possible to prevent a happening where the part that the distributor wants to show to the audience is hidden by the effect data. On the other hand, if not all but a part of the reproduced video screen of the video data is resized small or large and then combined with the effect data, the part that the distributor wants to show to the audience is not excessively reduced or emphasized can do. For example, a part of the screen may be a static area such as an area near the center of the screen, or may be a dynamic area such as an area with the face of the distributor or an area with large motion. In the case of extracting a dynamic region, for example, known face recognition technology, moving object detection technology, etc. may be used.

  As described above, by resizing the screen of the reproduced video of the video data and combining it with the effect data, the distributor and the audience are made aware that a special effect different from the main program of the live program is added. be able to.

  The effect data synthesizing unit 207 synthesizes the effect data with the image data of FIG. 3 as illustrated in FIGS. 5 and 6, and generates synthesized image data in which an object blocking the screen of the reproduced image of the image data is synthesized. You may In the example of FIG. 5, at the beginning of the rendering, the object of the silhouette of the mascot character with the motif of the television is spread all over the screen, and the spectator (and the distributor) taps this object, for example. It fades away gradually and the production is complete. Further, in the example of FIG. 6, at the start of rendering, the symbol “8” is slightly accumulated or not accumulated at the bottom of the screen, and the spectators (and distributors) tap the screen, or “8”, for example. By posting a comment, this object gradually accumulates and the rendering is complete. Note that, in the effects shown in FIGS. 5 and 6, even when there is no user input, objects may disappear automatically or objects may be automatically stacked according to the passage of time.

  As exemplified in FIG. 7, the effect data combining unit 207 resizes the entire reproduced image screen of the image data of FIG. 3 to a smaller size, and then combines effect data including an area for displaying an end roll. It is also good. In the example of FIG. 7, information such as the number of spectators in the video data of FIG. 3 and the spectator name who posted a comment is displayed as part of the end roll. The example of FIG. 7 is not limited to the ending effect, and may be applied as an intermediate effect for closing the first half of the live delivery, for example.

  The video distribution unit 208 receives the video data from the effect condition determination unit 204 or the composite video data from the effect data synthesis unit 207, and distributes this to the distributor terminal 100 and the audience terminal 300. The video distribution unit 208 is controlled by the distribution control unit 203, for example, a distribution destination of video data. The video distribution unit 208 may be the above-described communication device or an interface with the communication device.

  Although the video distribution unit 208 may immediately discard the distributed video data (or composite video data), for example, buffering is performed for a predetermined period to enable the effects accompanied by the aforementioned replay or slow motion reproduction. Alternatively, reference by the effect data creation unit 206 may be permitted.

  Next, an operation example of the distribution server 200 will be described using FIG. The operation of FIG. 8 begins, for example, with the beginning of the time frame assigned to the video data. When the operation of FIG. 8 starts, the process proceeds to step S401.

  In step S 401, the receiving unit 201 receives video data for live distribution from the distributor terminal 100. The effect condition determination unit 204 determines whether a predetermined effect condition is satisfied for the video data received in step S401 (step S402). If the rendering condition is satisfied, the process proceeds to step S403, and if not, the process proceeds to step S405.

  In step S403, the effect data combining unit 207 combines the video data received in step S401 with the effect data. The effect data combining unit 207 may perform processing such as resizing and trimming of the reproduced video of the image data as necessary, and then combine with the effect data. Then, the video distribution unit 208 distributes the composite video data generated in step S403 to the audience terminal 300 (step S404). On the other hand, in step S405, the video distribution unit 208 distributes the video data received in step S401 to the audience terminal 300.

  If the date and time instructed by the clock unit 202 has reached the end of the time frame assigned to the video data to be delivered live at the end of step S404 or step S405, the operation of FIG. 8 ends, otherwise The process returns to step S401 (step S406).

  As described above, the distribution server according to the first embodiment automatically combines effect data with video data for live distribution conditionally. As a result, the distributor can easily deliver live-styled videos in the style of a television program, for example, with the addition of opening effects, ending effects, intermediate effects, etc., without requiring advanced video editing skills and resources. Can.

  In particular, by adding an opening presentation, the distributor and the audience are made aware of the beginning of the live program, and the distributor prepares by taking a microphone test, taking a rest, etc., and comments before entering the main program of the live program. You can communicate with the audience, or wait for the start of the main program without disturbing the audience.

  Moreover, by adding an ending effect, the distributor and the audience are made aware of the end of the live program, and the distributor enters the summary, gives a notice for the next time, communicates with the audience via comments, etc. It will be easier to take time.

  Furthermore, as an intermediate performance, the width of the expression that can be used in live distribution is possible by adding video effects to inspire distributors such as fanfares in exciting scenes, and enabling replay of critical moments and slow motion reproduction. Can be extended.

Second Embodiment
The distribution server according to the second embodiment allows the distributor and / or the audience to specify how to combine video data and presentation data for live distribution. Specifically, it is specified in what layout (alignment) the two are combined, that is, how much one or both should be translated, or rotation, inversion, resizing and / or trimming for one or both. It is possible to specify settings.

  A distribution server 500 according to the second embodiment is illustrated in FIG. In the following description, differences between the distribution server 500 and the distribution server 200 will be mainly described. The distribution server 500 receives encoded video data sequentially transmitted from the distributor terminal 100 over a time frame assigned to live distribution of video data. When the rendering condition is satisfied, the distribution server 500 synthesizes the rendering data with the video data in accordance with the designation of the distributor and / or the audience, and distributes the composite video data to the distributor terminal 100 and the audience terminal 300. Here, the rendering condition may be the same as or similar to the rendering condition described in the first embodiment.

  The distribution server 500 performs automatic composition of such effect data according to the designation of the distributor and / or the audience, thereby further expanding the width of the opening effect, the ending effect, or the intermediate effect described in the first embodiment. be able to. For example, in the case of combining video data with presentation data, it is possible to make the image look more natural or to make it look unnatural.

  The functional configuration of distribution server 500 will be described below with reference to FIG. The hardware configuration of the distribution server 500 may be the same as or similar to that of the distribution server 200, and thus the description thereof will be omitted.

  The distribution server 500 includes a reception unit 501, a clock unit 502, a distribution control unit 503, an effect condition determination unit 504, an effect condition storage unit 505, an effect data generation unit 506, an effect data synthesis unit 507, and an image. And a delivery unit 508. Among them, the clock unit 502, the distribution control unit 503, the effect condition storage unit 505, the effect data creation unit 506, and the video distribution unit 508 may be the same as or similar to the elements of the same name in the distribution server 200, respectively. The other elements will be described below focusing on the differences from the elements of the distribution server 200 with the same name.

  The receiving unit 501 can receive specified data from the distributor terminal 100 and / or the audience terminal 300 in addition to the data received by the receiving unit 201. This designation data designates how to combine video data for live distribution and presentation data. Specifically, the designation data is a combination position of video data and effect data, enlargement / reduction ratio of video data, enlargement / reduction ratio of effect data, rotation amount of video data, rotation amount of effect data, identification in image data , And a specific area in the rendering data. The receiving unit 501 sends this designation data to the rendering condition determining unit 504.

  If the rendering condition is satisfied, the rendering condition determination unit 504 sends the designation data to the rendering data combining unit 507 in addition to the video data. The designated data may not exist unless the distributor and the audience designate in particular how to combine the video data for live distribution and the effect data.

  The effect data combining unit 507 receives the video data and the designated data from the effect condition determining unit 504, and receives the effect data from the effect data creating unit 506. The effect data combining unit 507 pre-processes the video data and / or the effect data according to the designated data, and then combines the two to generate combined video data.

  Specifically, when the designation data designates the combining position of the video data and the effect data, the effect data combining unit 507 can move the video data and / or the effect data in parallel to control the combining position of the two.

  Also, when the designated data designates the enlargement / reduction ratio of the video data and / or the enlargement / reduction ratio of the presentation data, the presentation data combining unit 507 may resize the video data and / or the presentation data. Furthermore, when the designated data designates the amount of rotation of the video data and / or the amount of rotation of the effect data, the effect data combining unit 507 may rotate the image data and / or the effect data. Here, the rotation may mean not only two-dimensional rotation on the image plane, but also three-dimensional rotation including vertical and horizontal flipping, and a combination thereof.

  Furthermore, when the designated data designates a specific area in the video data and / or a specific area in the effect data, the effect data combining unit 507 may trim the area specified in the video data and / or the effect data. .

  By performing the pre-processing as described above, the effect data combining unit 507 can combine the video data and the effect data according to the preference of the distributor and / or the audience. As an example, when the effect data is a face-out frame or the like, it is possible to combine the face of the video data and the inset part of the face-out frame so as to look natural or unnaturally.

  As described above, the distribution server according to the second embodiment receives, from the distributor terminal and / or the audience terminal, designation data for specifying how to combine the video data and the rendering data, and the video according to this. The data and / or presentation data are preprocessed and then combined. Therefore, according to this distribution server, it is possible to further widen the width of the opening effect, the ending effect, or the intermediate effect described in the first embodiment.

(Modification)
By the way, in the first embodiment and the present embodiment described above, the distribution server combines the video data and the effect data. As a result, there is an advantage that it is easy to synchronize synthesized video data that the distributor and the audience view, and the processing load on the terminal can be reduced. There is also an advantage of reducing the amount of communication from the distribution server to the terminal. However, it is also possible for the terminal to combine video data and presentation data. A brief description will be given below.

  A terminal 600 according to this modification is illustrated in FIG. In addition, since the terminal 600 may be either a distributor terminal or a spectator terminal, description of parts unique to the distributor terminal and the spectator terminal will be omitted.

  Terminal 600 may be the various electronic devices described with respect to distributor terminal 100. The terminal 600 includes, for example, a processor that performs input / output control, communication control, determination of effect conditions, creation of effect data, and synthesis of video data and effect data. Here, the processor is typically a central processing unit (CPU) and / or a graphics processing unit (GPU), but a microcomputer, a field programmable gate array (FPGA), or a digital signal processor (DSP). May be The terminal 600 also includes a memory that temporarily stores programs executed by the processor to realize such processing and data used by the processor.

  The terminal 600 may handle all data in the on-memory state, or part of the data may be saved in the auxiliary storage device. The auxiliary storage device may be, for example, an HDD, an SSD, a flash memory or the like built in or externally attached to the terminal 600, or may be a database server accessible from the terminal 600.

  Also, the terminal 600 may be a combination of multiple electronic devices. For example, different functional units in terminal 600 may be distributed and implemented on separate electronic devices.

  The terminal 600 can further use a communication I / F (interface) for connecting to a network, an output device for outputting content, and the like. The communication I / F and / or the display device may be incorporated in the terminal 600 or may be externally attached to the terminal 600.

  The communication I / F may be a module for communicating with the distribution server 500 (or distribution server 200) or another server not shown via a network. The output device may include a display device for displaying moving images, still images, text and the like, and / or a speaker for outputting sound, music and the like. The display device is, for example, a liquid crystal display, an organic EL (electroluminescence) display, a CRT (Cathode Ray Tube) display, or the like. Note that the display device may have a function of an input device like a touch screen.

  The terminal 600 includes a reception unit 601, an effect condition determination unit 604, an effect condition storage unit 605, an effect data creation unit 606, an effect data combining unit 607, and a display control unit 608.

  The receiving unit 601 receives various data, for example, from the distribution server 500 via the network, and sends the received data to, for example, the rendering condition determining unit 604. The receiving unit 601 may be the above-described communication I / F.

  Specifically, the receiving unit 601 receives video data for live distribution. This video data may be the video data itself transmitted (uploaded) from the distributor terminal 100 to the distribution server 500, or may be some processing such as re-encoding. In addition, the receiving unit 601 may receive the above-mentioned designated data. The designation data may be, for example, data collected from the distributor terminal 100 and / or the audience terminal 300 by the distribution server 500 and distributed to each terminal 600. Furthermore, the receiving unit 601 can receive, for example, data necessary for determining the rendering condition described in the first embodiment.

  The effect condition determination unit 604 may be the same as or similar to the effect condition determination unit 504 (or the effect condition determination unit 204). That is, the rendering condition determination unit 604 receives the video data (and the designated data) from the receiving unit 601, reads the rendering condition from the rendering condition storage unit 605, and determines whether the rendering condition is satisfied. If the effect condition determination unit 604 determines that the effect condition is satisfied, the effect condition determination unit 604 sends the video data and the designated data to the effect data combining unit 607, and sends the image data to the display control unit 608 otherwise. The effect condition determination unit 604 may be the processor and memory described above.

  The effect data creation unit 606 may be the same as or similar to the effect data creation unit 206. That is, the effect data creation unit 606 creates effect data and sends it to the effect data combining unit 607. The effect data creation unit 206 may be the processor and memory described above.

  The effect data combining unit 607 may be the same as or similar to the effect data combining unit 507 (or the effect data combining unit 207). That is, the effect data combining unit 607 receives the video data and the designated data from the effect condition determining unit 604, and receives the effect data from the effect data creating unit 606. The effect data combining unit 607 performs preprocessing on the video data and / or the effect data according to the designated data, and then combines the two to generate combined video data. The effect data combining unit 607 sends the combined video data to the display control unit 608. The effect data combining unit 607 may be the processor and memory described above.

  The display control unit 608 controls the above-described display device, and displays video data or composite video data on the screen of the display device. The display control unit 608 may be, for example, the processor and memory described above.

  As described above, the terminal according to this modification performs combining of the video data and the effect data described in the first embodiment and the second embodiment. Therefore, according to this terminal, it is possible to add the effect described in the first embodiment and the second embodiment to the video data for live distribution even when the distribution server can not use the function. Become. In the example of FIG. 10, the terminal determines the rendering condition, but the distribution server may be further modified to determine the rendering condition.

Third Embodiment
In the first and second embodiments described above, effects are realized by combining the effect data with the video data. The distribution server according to the third embodiment realizes effects by processing the video data itself for live distribution.

  A distribution server 700 according to the third embodiment is illustrated in FIG. In the following description, differences between the distribution server 700 and the distribution server 200 or the distribution server 500 will be mainly described. The distribution server 700 receives encoded video data sequentially transmitted from the distributor terminal 100 over a time frame assigned to live distribution of video data. When the rendering condition is satisfied, the distribution server 700 processes the video data, and distributes the processed video data to the distributor terminal 100 and the audience terminal 300. Here, the rendering condition may be the same as or similar to the rendering condition described in the first embodiment.

  By processing the video data as described above, the distribution server 700 can use an effect of an expression different from the effect described in the first embodiment and the second embodiment. For example, there may be processing for filtering a specific area (for example, a face area) or the entire area of video data. The filter can change the visual impression of the video data by adding various effects such as, for example, mosaic, sepia, monochrome, retro, blur, night vision-like effects, distortion, cartoon-like effects, etc. it can. Furthermore, there may be processing such that a duplicate of a specific area of video data is pasted to the video data.

  The functional configuration of distribution server 700 will be described below with reference to FIG. The hardware configuration of the distribution server 700 may be the same as or similar to the distribution server 200 or the distribution server 500, and thus the description thereof will be omitted.

  Distribution server 700 includes a receiving unit 701, a clock unit 702, a distribution control unit 703, a rendering condition determination unit 704, a rendering condition storage unit 705, a video processing unit 707, and a video distribution unit 708. Among them, elements other than the video processing unit 707 may be the same as or similar to the elements of the same name in the distribution server 200 or the distribution server 500, respectively. Therefore, the video processing unit 707 will be described below.

  The video processing unit 707 receives video data (and designated data) from the rendering condition determination unit 704. First, when receiving the designated data, the video processing unit 707 performs the pre-processing described in the second embodiment on the video data. Then, the video processing unit 707 performs predetermined processing on the video data from the rendering condition determination unit 704 or the video data on which the video data has been preprocessed to generate the processed video data. The video processing unit 707 sends the processed video data to the video distribution unit 708. The image processing unit 707 may be the processor and memory described above.

  Specifically, the video processing unit 707 may apply any of the aforementioned filters to video data. Alternatively, the video processing unit 707 may paste a duplicate of a specific area of video data to the video data. For example, the image processing unit 707 may perform the processing illustrated in FIG. 13 or FIG. 14 on the image data as shown in FIG.

  In the example of FIG. 13, an image obtained by horizontally reversing the duplication of the region on the left side of the central axis indicated by the broken line is pasted to the right of the central axis. Note that the number of target axes serving as a reference for inversion, the respective positions and angles may be automatically determined, or may be specified by the distributor and / or the audience, for example, through designated data.

  In the example of FIG. 14, a duplicate itself of the area around a person in the video data, or a video obtained by pre-processing the duplicate is pasted in the area indicated by the broken line. In addition, it is automatically determined which area in the video data is to be replicated, whether or not preprocessing is to be performed on the replication, what preprocessing is to be performed on the replication, and the position and number of pastes, etc. It may be possible for the distributor and / or the audience to designate, for example, through designated data.

  As described above, the distribution server according to the third embodiment realizes effects by processing video data. Therefore, according to this distribution server, it is possible to use an effect of an expression different from the effect described in the first embodiment and the second embodiment.

(Modification)
In the present embodiment, the distribution server processes video data. As a result, there is an advantage that it is easy to synchronize synthesized video data that the distributor and the audience view, and the processing load on the terminal can be reduced. There is also an advantage of reducing the amount of communication from the distribution server to the terminal. However, it is also possible for the terminal to process the video data. A brief description will be given below.

  A terminal 800 according to this modification is illustrated in FIG. The hardware configuration of the terminal 800 may be the same as or similar to that of the terminal 600, and thus the description thereof will be omitted. Further, since the terminal 800 may be either a distributor terminal or a spectator terminal, description of parts unique to the distributor terminal and the spectator terminal will be omitted.

  The terminal 800 includes a reception unit 801, an effect condition determination unit 804, an effect condition storage unit 805, an image processing unit 807, and a display control unit 808. Among them, elements other than the image processing unit 807 may be the same as or similar to the elements of the same name in the terminal 600. Therefore, the video processing unit 807 will be described below.

  The image processing unit 807 may be the same as or similar to the image processing unit 707. That is, the video processing unit 807 receives the video data (and designated data) from the rendering condition determination unit 804. First, when receiving the designated data, the video processing unit 807 performs the preprocessing described in the second embodiment on the video data. Then, the video processing unit 807 performs the predetermined processing described in the third embodiment on the video data from the rendering condition determination unit 804 or the video data on which the video data is preprocessed and processed. Generate The video processing unit 807 sends the processed video data to the video distribution unit 708. The image processing unit 807 may be the processor and memory described above.

  As described above, the terminal according to this modification processes the video data described in the third embodiment. Therefore, according to this terminal, even when the distribution server can not use such a function, it is possible to add the effect described in the third embodiment to video data for live distribution. In the example of FIG. 15, the terminal determines the rendering condition, but the distribution server may be further modified to determine the rendering condition.

Fourth Embodiment
In the first embodiment described above, it is possible to combine dynamic rendering data with video data to produce rendering. In the fourth embodiment, a highly interactive presentation is realized by controlling the dynamic presentation data according to the operation request from the distributor and / or the audience.

  A distribution server 900 according to the fourth embodiment is illustrated in FIG. In the following description, differences between the distribution server 900 and the distribution server 200, the distribution server 500, or the distribution server 700 will be mainly described.

  The distribution server 900 receives encoded video data sequentially transmitted from the distributor terminal 100 over a time frame assigned to live distribution of video data. The distribution server 900 receives, from the distributor terminal 100 and / or the audience terminal 300, an operation request for dynamic presentation data to be combined with video data when the presentation condition is satisfied, and based on the operation request, the distribution server 900 receives the operation request. Operation data of the effect data is generated. Then, distribution server 900 transmits the operation data to distributor terminal 100 and audience terminal 300 in addition to the video data. The distributor terminal 100 and the audience terminal 300 respectively control the dynamic effect data based on the received operation data, combine the received image data with the received video data, and display it on the screen of the display device. Here, the rendering condition may be the same as or similar to the rendering condition described in the first embodiment.

  The distribution server 900 generates and distributes operation data of such dynamic presentation data based on the operation request, and the distributor terminal 100 and the audience terminal 300 control the dynamic presentation data based on the operation data. By combining with video data, it is possible to produce highly interactive effects.

  The functional configuration of distribution server 900 will be described below with reference to FIG. The hardware configuration of the distribution server 900 may be the same as or similar to the distribution server 200, the distribution server 500, or the distribution server 700, and a description thereof will be omitted.

  Distribution server 900 includes a reception unit 901, a clock unit 902, a distribution control unit 903, a rendering condition determination unit 904, a rendering condition storage unit 905, a video distribution unit 908, and an operation data generation unit 909. Among them, the clock unit 902, the distribution control unit 903, and the effect condition storage unit 905 may be the same as or similar to the elements of the same name in the distribution server 200, the distribution server 500, or the distribution server 700, respectively. Hereinafter, the other elements will be described focusing on differences from the elements of the distribution server 200, the distribution server 500, or the distribution server 700 with the same name.

  In addition to the data received by the receiving unit 201, the receiving unit 901 can receive an operation request for dynamic presentation data from the distributor terminal 100 and / or the audience terminal 300. This operation request may be, for example, various tools for generating dynamic effect data, such as a drawing application, a handwriting input application, a stopwatch, a timer, a counter, dice, playing cards, fortunes, a bingo game, a rock game, a still image viewer , Etc., may correspond to input data provided to the corresponding program for operating the player of the content to be quoted during the live delivery including, for example, selection of the object. The receiving unit 901 sends this operation request to the rendering condition determining unit 904.

  The presentation condition determination unit 904 sends the video data to the video distribution unit 908 regardless of the determination result of the presentation condition. Furthermore, the rendering condition determination unit 904 sends an operation request to the operation data generation unit 909 when the rendering condition is satisfied. If the distributor and the spectator do not make an input for issuing the operation request, the operation request may not exist.

  The operation data generation unit 909 receives an operation request from the effect condition determination unit 904, and generates operation data of dynamic effect data based on the operation request. The operation data generation unit 909 sends the operation data to the video distribution unit 908. The operation data generation unit 909 may be the processor and memory described above. When a plurality of terminals transmit operation requests substantially simultaneously, collision of operation requests may occur. That is, although an operation request from a certain terminal may contradict an operation request from another terminal, the operation data generation unit 909 may generate such operation data after performing arbitration of such a collision.

  In addition, it is burdensome for the operation data generation unit 909 to generate operation data in consideration of all the operation requests by transmitting the operation requests substantially simultaneously from a large number of terminals. Therefore, although the operation data generation unit 909 is also associated with the arbitration of the collision, the operation data generation unit 909 may perform filtering to discard part of the operation request and generate operation data based on the remaining operation request. As an example, the operation data generation unit 909 maintains an operation request from a person who charges for the audience, a person who is a channel member of the distributor, a person who is in a specific area, a person who has won the lottery, etc. You may discard the operation request of.

  In the present embodiment, dynamic presentation data may be image data that causes screen transition, that is, appearance change in accordance with at least operation data. Specifically, as an example of the screen transition according to the operation data, update of the image of the hand according to the operation data to select the deck of the playing card, update of the result of the dice according to the operation data to select the dice, Update of the drawing image according to the operation data based on handwriting input, etc. are mentioned. In addition to the operation data, there may be screen transition according to the passage of time. For example, in response to operation data for selecting a start button, the image on the display of the stopwatch or timer starts to change to count down or count up. Subsequent images of the display unit will be updated as time passes.

  The screen transition may or may not be common to all terminals. As an example of the latter, the dynamic presentation data may be execution screen data of a rock-paper-scissors game illustrated in FIGS. 17 and 18. In this example, initially, as shown in FIG. 17, an image representing a selectable hand is combined with video data, and the operation data generation unit 909 generates an operation request from a distributor and / or a spectator until a predetermined deadline time. Standby, and generates operation data after the end of the deadline time. After the end of the deadline time, a screen transition to an image representing the result of rocking rock occurs. As a result, the distributor terminal 100, as exemplified in FIG. 18, as presentation data representing the result of the game with the audience, for example, the hand selected by the distributor and the number of persons selected for each of the three hands. Can be generated and synthesized into video data. Further, the audience terminal 300 generates effect data representing, for example, a hand selected by the distributor and a hand selected by the spectator who operates the audience terminal 300 as a result of the game with the distributor, and synthesizes the effect data into video data. be able to. Alternatively, the distributor terminal 100 and / or the audience terminal 300 may generate presentation data representing tallying of the outcome of the game like winner-name, loser-name, aiko-name, and combine it with video data.

  The video distribution unit 908 may receive video data from the rendering condition determination unit 904 and may further receive operation data from the operation data generation unit 909. The video distribution unit 908 distributes video data (and operation data) to the distributor terminal 100 and the audience terminal 300.

  A terminal 1000 according to the fourth embodiment is illustrated in FIG. In the following description, differences from the terminal 600 or 800 will be mainly described. The hardware configuration of the terminal 1000 may be the same as or similar to that of the terminal 600 or the terminal 800, and thus the description thereof will be omitted. Further, since terminal 1000 may be either a distributor terminal or a spectator terminal, the description of the parts unique to the distributor terminal and the spectator terminal will be omitted.

  The terminal 1000 includes a reception unit 1001, an effect condition determination unit 1004, an effect condition storage unit 1005, an effect data control unit 1006, an effect data combining unit 1007, a display control unit 1008, an input unit 1010, and an operation request. A generation unit 1011 and a transmission unit 1012 are included. Among them, the rendering condition storage unit 1005, the rendering data combining unit 1007, and the display control unit 1008 may be the same as or similar to the elements of the same name in the terminal 600 or 800, respectively. Hereinafter, the other elements will be described focusing on the differences from the elements of the same name in terminal 600 or terminal 800.

  The input unit 1010 receives user input and generates user input data. The input unit 1010 sends user input data to the presentation condition determination unit 1004. The input unit 1010 may be an input device built in or externally attached to the terminal 1000. The input device may be, for example, a keyboard, a mouse, a ten key, or the like, and may have a display function such as a touch screen. The user input is typically a tap, a click, pressing of a specific key, etc., and the user input data may be data representing the occurrence of these events. Besides, user input data may be, for example, voice captured by a microphone, biological data detected by a biological sensor (for example, temperature, facial expression, etc.), position data identified by GPS (Global Positioning System) or base station information, acceleration The action of the user estimated based on the acceleration data detected by a sensor (for example, the terminal 1000 was shaken) etc. can also be included.

  The rendering condition determination unit 1004 sends user input data to the operation request generation unit 1011 when the rendering condition is satisfied.

  The operation request generation unit 1011 receives user input data from the effect condition determination unit 1004, and generates an operation request according to the user input data. The operation request generation unit 1011 sends this operation request to the transmission unit 1012. The operation request generation unit 1011 may be the processor and memory described above.

  The transmission unit 1012 receives the operation request from the operation request generation unit 1011 and transmits the operation request to the distribution server 900. The transmission unit 1012 may be the above-described communication I / F.

  The receiving unit 1001 receives video data for live distribution. The video data may be the video data itself transmitted (uploaded) from the distributor terminal 100 to the distribution server 900 or may be processed in some way. In addition, the receiving unit 1001 can receive operation data when the rendering condition is satisfied. Furthermore, the receiving unit 1001 may receive the above-described designated data, data necessary for determining the rendering condition, and the like. The receiving unit 1001 sends the operation data to the effect data control unit 1006, sends the video data and the designation data to the effect data combining unit 1007, and sends data necessary for the determination of the effect condition to the effect condition determining unit 1004.

  The effect data control unit 1006 receives operation data from the receiving unit 1001, and controls dynamic effect data based on the operation data. Specifically, the effect data control unit 1006 executes programs (applications) corresponding to the various tools described above, and provides operation data to the programs. The operation data may include time data and operation input data corresponding to the time data. In this case, the effect data control unit 1006 can provide the program with operation input data corresponding to the time data at the time indicated by the time data. The program can generate an image transitioning the screen according to the given operation input data as an execution result, that is, dynamic effect data. The effect data control unit 1006 sends dynamic effect data to the effect data combining unit 1007. The effect data control unit 1006 may be the processor and memory described above. Here, the program may be incorporated in a player for reproducing video data for live distribution, or may be an external program viewed from the player.

  As described above, the distribution server according to the fourth embodiment collects operation requests for dynamic presentation data from the distributor terminal and / or the audience terminal, and generates operation data of the dynamic presentation data. To deliver. Then, the terminal according to the present embodiment controls dynamic effect data based on the operation data, and combines it into video data for live distribution. Therefore, according to this distribution server and terminal, for example, highly interactive presentation such as a distributor and a spectator playing together in real time becomes possible.

(Modification)
By the way, in the present embodiment, the terminal performs control of effect data that is dynamic and composition of image data and dynamic effect data. However, it is also possible that the distribution server controls the dynamic rendering data, and sequentially transmits the rendering data to be combined into the video data to the terminal, and the video data and the dynamic as in the first embodiment. It may even be combined with the typical effect data. As a result, there is an advantage that it is easy to synchronize synthesized video data that the distributor and the audience view, and the processing load on the terminal can be reduced. An example in which the distribution server performs control of dynamic rendering data and composition of video data and dynamic rendering data will be briefly described.

  A distribution server 1100 according to this modification is illustrated in FIG. The hardware configuration of the distribution server 1100 may be the same as or similar to the distribution server 200, the distribution server 500, the distribution server 700, or the distribution server 900, and thus the description thereof will be omitted.

  The distribution server 1100 includes a reception unit 1101, a clock unit 1102, a distribution control unit 1103, a presentation condition determination unit 1104, a presentation condition storage unit 1105, a presentation data control unit 1106, a presentation data synthesis unit 1107, and a video. A distribution unit 1108 and an operation data generation unit 1109 are included. Among them, the receiving unit 1101, the clock unit 1102, the distribution control unit 1103, the effect condition determination unit 1104, the effect condition storage unit 1105, the effect data combining unit 1107, the video distribution unit 1108, and the operation data generation unit 1109 , Delivery server 500, delivery server 700 or delivery server 900 may be the same as or similar to the element of the same name. Therefore, the effect data control unit 1106 will be described below.

  The effect data control unit 1106 may be the same as or similar to the effect data control unit 1006. That is, the effect data control unit 1106 receives the operation data from the operation data generation unit 1109, and controls dynamic effect data based on the operation data. The effect data control unit 1106 sends dynamic effect data to the effect data combining unit 1107. The effect data control unit 1106 may be the processor and memory described above.

  As described above, the distribution server according to this modification performs the effects described in the fourth embodiment, and distributes composite video data to the distributor terminal and the audience terminal. Therefore, according to this distribution server, there is an advantage that it is easy to synchronize synthesized video data that the distributor and the audience view, and it is possible to reduce the processing load of the terminal. There is also an advantage of reducing the amount of communication from the distribution server to the terminal.

  In the present embodiment, although both the terminal and the distribution server determine the rendering condition, only one of them may perform the determination. For example, only the terminal may determine the rendering condition and notify the distribution server that the rendering condition is satisfied. Conversely, only the distribution server may determine the rendering condition and notify the terminal that the rendering condition is satisfied.

  The embodiments described above are merely illustrative examples to assist in understanding the inventive concept, and are not intended to limit the scope of the present invention. Embodiments can add, delete, or convert various components without departing from the scope of the present invention.

  The various functional units described in the above embodiments may be realized by using a circuit. The circuit may be a dedicated circuit that implements a specific function or may be a general-purpose circuit such as a processor.

  At least a part of the processing in each of the above-described embodiments can also be realized by using a general-purpose computer as basic hardware. The program for realizing the above process may be provided by being stored in a computer readable recording medium. The program is stored in the recording medium as an installable file or an executable file. The recording medium may be a magnetic disk, an optical disk (CD-ROM, CD-R, DVD or the like), a magneto-optical disk (MO or the like), a semiconductor memory or the like. The recording medium may store the program and may be any computer readable one. Further, the program for realizing the above processing may be stored on a computer (server) connected to a network such as the Internet, and may be downloaded to the computer (client) via the network.

100 ... distributor terminal 200, 500, 700, 900, 1100 ... distribution server 201, 501, 601, 701, 801, 901, 1001, 1101 ... reception unit 202, 502, 702, 902, 1102 ... Clock unit 203, 503, 703, 903, 1103 ... Distribution control unit 204, 504, 604, 704, 804, 904, 1004, 1104 ... Performance condition determination unit 205, 505, 605, 705, 805, 905, 1005, 1105 ... effect condition storage unit 206, 506, 606 ... effect data creation unit 207, 507, 607, 1007, 1107 ... effect data combining unit 208, 508, 708, 908, 1108 ··· Video distribution unit 300 ··· Audience terminal 600, 800, 1000 · · · Terminal 60 , 808, 1008 ... display control unit 707, 807 ... image processing unit 909, 1109 ... operation data generation unit 1006, 1106 ... effect data control unit 1010 ... input unit 1011 ... operation Request generation unit 1012 ... Transmission unit

Claims (17)

A receiver for receiving video data for live distribution from the distributor terminal;
A determination unit that determines whether the rendering condition is satisfied;
A synthesizing unit which synthesizes effect data with the image data to generate synthesized image data when the effect condition is satisfied;
A distribution unit that distributes the composite video data to the distributor terminal and the audience terminal.   As the presentation condition, (1) the first presentation end condition is satisfied after the presentation start point set in the time frame assigned for the raw delivery of the video data reaches the date and time when the distribution server refers (2) There is a predetermined user input to the distributor terminal or the spectator terminal until (2) the audience number of the video data satisfies the predetermined condition and the second effect end condition is satisfied. Until the third effect termination condition is satisfied, (4) the comment posted to the video data satisfies the predetermined condition, and then the fourth effect termination condition is satisfied, and (5) The distribution server according to claim 1, wherein the distribution server is satisfied during at least one of the reproduction video or the reproduction sound of the video data until the fifth effect termination condition is satisfied after the predetermined condition is satisfied.   A necessary condition for satisfying any one of the first effect end condition, the second effect end condition, the third effect end condition, the fourth effect end condition and the fifth effect end condition The distribution server according to claim 2, further comprising a creation unit configured to create the effect data including information dynamically shown.   The distribution server according to claim 3, wherein the creation unit creates the effect data including information indicating a gap between the number of spectators of the video data or the number of comments and the required number.   The distribution server according to claim 1 or 2, further comprising a creation unit that creates the effect data including information on an event that occurs after live distribution of the video data is started.   The creation unit may include the number of spectators of the video data, a part of comments posted to the video data, the number of comments posted to the video data, and spectators who posted comments on the video data The distribution server according to claim 5, wherein the effect data including information indicating at least one of names is created.   (1) Changes in response to at least one of user input made to at least one of the distributor terminal and the audience terminal, and (2) a comment posted by at least one of the distributor terminal and the audience terminal The distribution server according to claim 1 or 2, further comprising a creation unit that creates presentation data to be played.   The apparatus further comprises a creation unit that creates the effect data including information generated based on at least one of metadata attached to the video data and information displayed on a web page related to the video data. The delivery server according to claim 1 or claim 2.   The composition unit according to any one of claims 1 to 7, wherein the combining unit resizes a part or all of a screen of a reproduced video of the video data and combines the resized data with the effect data to generate the combined video data. Distribution server described.   The synthesizing unit synthesizes the video data with the effect data such that a part of a screen of a reproduced video of the video data is exposed and the remaining part is hidden, thereby generating the synthesized video data. The delivery server according to any one of 7. Computer,
A means for receiving video data for live distribution from the distributor terminal;
Means for determining whether the rendering condition is satisfied,
A means for combining effect data with the image data to generate combined image data when the effect condition is satisfied;
A distribution program for functioning as means for distributing the composite video data to the distributor terminal and the audience terminal. The receiving unit is configured to combine the image data and the effect data from at least one of the distributor terminal and the spectator terminal, an enlargement / reduction ratio of the image data, an enlargement / reduction ratio of the effect data, and the image data And receiving designation data specifying at least one of a rotation amount of the effect data, a rotation amount of the effect data, a specific area in the video data, and a specific area in the effect data,
The synthesizing unit synthesizes the video data and the effect data after performing at least one of parallel movement, resizing, rotation, inversion, and trimming on at least one of the image data and the effect data according to the designation data. , Generating the composite video data,
The distribution server according to any one of claims 1 to 10. A receiver for receiving video data for live distribution from the distributor terminal;
A determination unit that determines whether the rendering condition is satisfied;
A processing unit that performs predetermined processing on the video data to generate processed video data when the rendering condition is satisfied;
A distribution unit that distributes the processed video data to the distributor terminal and the audience terminal.   The processing unit pastes the duplication of the specific area in the video data as it is or performs at least one of rotation, inversion, and resizing, when the rendering condition is satisfied, and then pastes it to the video data, The distribution server according to claim 13, which generates processed video data. A determination unit that determines whether the rendering condition is satisfied;
The video data for live distribution is received from the distributor terminal, and if the effect condition is satisfied, then at least one of the distributor terminal and the audience terminal from at least one of the distributor terminal and the audience terminal A reception unit for receiving an operation request for dynamic presentation data to be combined with the video data in
A generation unit that generates operation data of the dynamic effect data based on the operation request;
A distribution unit configured to distribute the video data and the operation data to at least one of the distributor terminal and the audience terminal.   The distribution server according to claim 15, wherein the dynamic effect data is video data that makes a screen transition according to the operation data. A determination unit that determines whether the rendering condition is satisfied;
A generation unit that generates an operation request for dynamic presentation data according to a user input when the presentation condition is satisfied;
A transmitter for transmitting the operation request to the distribution server;
A receiving unit that receives, from the distribution server, live distribution video data and operation data of the dynamic effect data generated based on the operation request, when the effect condition is satisfied;
A control unit that controls the dynamic effect data based on the operation request;
A synthesizing unit which synthesizes the dynamic effect data with the video data;
Equipped with a terminal.