JP2018101965A - System, method for distributing video, and program for use therein - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress deterioration in the speech quality of telephone conversation performed between a distributor terminal for distributing the video and a viewer terminal for viewing that video.SOLUTION: A video distributing system 1 includes a distribution server 10, and multiple user terminals 30 connected communicably with the distribution server 10 via a network 20, where the distribution server 10 provides video distribution service for distributing a distribution video, transmitted from the user terminals 30, to the multiple user terminals 30. This video distributing system 1 is configured so that in response to start of a prescribed communication, between a subscriber terminal 30 participating to the distribution video and the subscriber terminal 30, a voice received in the prescribed communication is outputted in place of the voice contained in the distribution video received from the distribution server 10.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a system, a method, and a program used for distributing a moving image.

  Conventionally, in a video distribution service that distributes a video transmitted from a distributor terminal to a plurality of viewer terminals, communication between the distributor and the viewer has been made possible. For example, Patent Document 1 below discloses that a content server that distributes live video transmitted from a distributor terminal distributes the contents of a voice call between the distributor and a specific viewer along with the live video. . Such a system allows viewers to participate in the live video of the distributor via a voice call.

JP 2011-172200 A

  However, in the conventional system described above, the content of the voice call between the participant and the distributor of the live video is transmitted to the participant's own terminal together with the live video through the content server. It can cause problems. Specifically, for example, when a participant's voice output via a speaker at a distributor terminal is inputted via a microphone, the participant's voice is a participant with a live video with a certain delay. It will be output at the terminal. Thus, the participant's own voice (echo) output with a delay causes deterioration in voice quality such as howling, and hinders smooth communication between the participant and the distributor.

  An embodiment of the present invention has an object of suppressing deterioration in voice quality of a call performed between a distributor terminal that distributes a moving image and a viewer terminal that views the moving image. Other objects of the embodiments of the present invention will become apparent by referring to the entire specification.

  A system according to an embodiment of the present invention includes a distributor terminal, a distribution server, and a plurality of viewer terminals, and is a system for distributing a moving image, wherein the distributor terminal is connected via a microphone. Transmitting a distribution video including at least real-time audio input to the distribution server, and transmitting the distribution video received from the distributor terminal to each of the plurality of viewer terminals. Each of the plurality of viewer terminals outputting an image and sound included in the distribution video received from the distribution server, a participant terminal included in the plurality of viewer terminals, and the distribution Starting predetermined communication capable of performing at least a call between the participant terminals, and the participant terminal starting the predetermined communication Flip and, in place of the speech contained in the distribution video received from the distribution server executes the steps of outputting a sound received at said predetermined communication.

  A method according to an embodiment of the present invention is a method for distributing a moving image by a system including a distributor terminal, a distribution server, and a plurality of viewer terminals, and the distributor terminal includes a microphone Transmitting a distribution video including at least real-time audio input via the distribution server, and the distribution server transmitting the distribution video received from the distributor terminal to each of the plurality of viewer terminals Each of the plurality of viewer terminals outputting an image and audio included in the distribution video received from the distribution server, a participant terminal included in the plurality of viewer terminals, and Starting at least a predetermined communication capable of executing a call between the distributor terminals; and the participant terminal opening the predetermined communication. Depending on, instead of the speech contained in the distribution video received from the distribution server, and a step of outputting a sound received at said predetermined communication.

  A first program according to an embodiment of the present invention includes a distributor terminal, a distribution server, and a plurality of viewer terminals, and is executed on the distributor terminal in a system for distributing moving images. An image included in the distribution video received from the distribution server, and a process of transmitting to the distribution server a distribution video including at least real-time audio input via a microphone to the distributor terminal. And a process for starting a predetermined communication capable of executing at least a call with the participant terminals included in the plurality of viewer terminals that output audio, and the distribution server according to the start of the predetermined communication In addition to real-time audio input via a microphone, the distribution video to be transmitted to And a process of adding the received voice, and the process of adding the received voice includes a process of removing an echo component of the voice received in the predetermined communication from the voice input through the microphone. Including performing.

  A second program according to an embodiment of the present invention includes a distributor terminal, a distribution server, and a plurality of viewer terminals, and is executed on the viewer terminal in a system for distributing moving images. A distribution video received from the distribution server to the viewer terminal, which is received by the distribution server from the distributor terminal and input via the microphone of the distributor terminal. A process for outputting an image and a sound included in the distribution video including at least sound, a process for starting a predetermined communication capable of executing at least a call with the distributor terminal, and a start of the predetermined communication. Accordingly, instead of the sound included in the distribution video received from the distribution server, the sound received in the predetermined communication And outputting the causes execution.

  Various embodiments of the present invention suppress the deterioration of the voice quality of a call performed between a distributor terminal that distributes a moving image and a viewer terminal that views the moving image.

1 is a configuration diagram schematically showing a network configuration of a moving image distribution system 1 according to an embodiment of the present invention. 1 is a block diagram schematically showing functions of a moving image distribution system 1. FIG. The figure which shows an example of the main screen. The figure which shows an example of the delivery preparation screen. The figure which shows an example of the delivery screen. The figure which shows an example of the viewing-and-listening screen 90. FIG. The sequence diagram which shows an example of the process performed between the distributor terminal 30 and the viewer terminal 30 when a viewer participates in a screen moving image. The figure which shows an example of the delivery screen 80 when the participation request from the viewer terminal 30 is received. The figure for demonstrating the input-output of the audio | voice in each of the distributor terminal 30 and the viewer terminal 30 at the normal time. The figure for demonstrating the input-output of the audio | voice in each of the distributor terminal 30, the participant terminal 30, and the other viewer terminal 30 in the period when the participant is participating in a screen moving image. The figure for demonstrating the detail of the input-output of the audio | voice in the communication control part 57 of the distributor terminal 30. FIG.

 Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a configuration diagram schematically showing a network configuration of a moving image distribution system 1 according to an embodiment of the present invention. As shown in the figure, the moving image distribution system 1 includes a distribution server 10 and a user terminal 30 that is communicably connected to the distribution server 10 via a network 20 such as the Internet. Although only one user terminal 30 is illustrated in FIG. 1, the moving image distribution system 1 includes a plurality of user terminals 30. The distribution server 10 refers to a distribution video transmitted by a distributor's user terminal 30 (hereinafter sometimes referred to as “distributor terminal 30”) as a viewer's user terminal 30 (hereinafter referred to as “viewer terminal 30”). To provide video distribution services. In the present embodiment, a user of a video distribution service can distribute a video as a distributor, and can also view a video distributed by another user as a viewer.

  The distribution server 10 is configured as a general computer, and as shown in FIG. 1, a CPU (computer processor) 11, a main memory 12, a user I / F 13, a communication I / F 14, and a storage (memory). Device) 15, and these components are electrically connected via a bus (not shown) or the like.

  The CPU 11 reads various programs stored in the storage 15 or the like into the main memory 12 and executes various instructions included in the programs. The main memory 12 is configured by, for example, a DRAM.

  The user I / F 13 includes various input / output devices for exchanging information with the user. The user I / F 13 includes, for example, an information input device such as a keyboard and a pointing device (for example, a mouse and a touch panel), a voice input device such as a microphone (microphone), and an image input device such as a camera. The user I / F 13 includes an image output device such as a display and an audio output device such as a speaker.

  The communication I / F 14 is implemented as hardware such as a network adapter, various types of communication software, and combinations thereof, and is configured to realize wired or wireless communication via the network 20 or the like.

  The storage 15 is configured by, for example, a magnetic disk, a flash memory, or the like. The storage 15 stores various programs including an operating system, various data, and the like.

  In the present embodiment, the distribution server 10 can be configured using a plurality of computers each having the hardware configuration described above. For example, the distribution server 10 can be configured by one or a plurality of server devices.

  The distribution server 10 configured as described above has functions as a web server and an application server, and requests from a web browser and other applications (for example, an application for a video distribution service) installed in the user terminal 30. In response to this, various processes are executed, and screen data (for example, HTML data) and control data corresponding to the result of the processes are transmitted to the user terminal 30. On the user terminal 30, a web page or other screen based on the received data is displayed.

  The user terminal 30 is configured as a general computer, and as shown in FIG. 1, a CPU (computer processor) 31, a main memory 32, a user I / F 33, a communication I / F 34, and a storage (memory). Device) 35, and these components are electrically connected via a bus or the like (not shown).

  The CPU 31 reads various programs stored in the storage 35 and the like into the main memory 32 and executes various instructions included in the programs. The main memory 32 is configured by, for example, a DRAM.

  The user I / F 33 is various input / output devices for exchanging information with the user. The user I / F 33 includes, for example, an information input device such as a keyboard and a pointing device (for example, a mouse and a touch panel), a voice input device such as a microphone (microphone), and an image input device such as a camera. The user I / F 33 includes an image output device such as a display and an audio output device such as a speaker.

  The communication I / F 34 is implemented as hardware such as a network adapter, various types of communication software, and combinations thereof, and is configured to realize wired or wireless communication via the network 20 or the like.

  The storage 35 is configured by, for example, a magnetic disk or a flash memory. The storage 35 stores various programs including the operating system, various data, and the like. The program stored in the storage 35 can be downloaded from an application market or the like and installed.

  In the present embodiment, the user terminal 30 can be configured as a smartphone, a tablet terminal, a wearable device, a personal computer, a game dedicated terminal, a VR (Virtual Reality) device (head mounted display or the like), and the like.

  The user of the user terminal 30 configured as described above provides the distribution server 10 by executing communication with the distribution server 10 via a web browser or an application for moving image distribution service installed in the storage 35 or the like. You can use the video distribution service. The application for moving image distribution service can be an example of a program that implements part or all of the program of the present invention.

  Next, functions of the moving image distribution system 1 according to the present embodiment will be described. FIG. 2 is a block diagram schematically showing functions of the distribution server 10 and the user terminal 30. As shown in the figure, the distribution server 10 includes an information storage management unit 41 that stores and manages various information, a basic function control unit 43 that controls basic functions of the video distribution service, and a video distribution control that controls video distribution. Part 45. These functions are realized by the cooperation of hardware such as the CPU 11 and the main memory 12 and various programs and data stored in the storage 15 and the like. For example, the functions are read into the main memory 12. This is realized by the CPU 11 executing instructions included in the program. 2 may be realized by the cooperation of the distribution server 10 and the user terminal 30, or may be realized by the user terminal 30.

  As shown in FIG. 2, the user terminal 30 has an information storage management unit 51 that stores and manages various information, a distribution function control unit 53 that controls a distribution function in the video distribution service, and a viewing function in the video distribution service. It has a viewing function control unit 55 that controls, and a communication control unit 57 that controls predetermined communication performed between the distributor and the viewer. These functions are realized by the cooperation of hardware such as the CPU 31 and the main memory 32, and various programs (for example, an application for moving image distribution service) and data stored in the storage 35 and the like. For example, it is realized by the CPU 31 executing an instruction included in the program read into the main memory 32. 2 may be realized by the cooperation of the server 10 and the user terminal 30, or may be realized by the server 10.

  The information storage management unit 41 of the distribution server 10 stores and manages various information in the storage 15 and the like. The basic function control unit 43 of the server 10 executes various processes related to control of the basic function of the moving image distribution service. For example, the basic function control unit 43 transmits HTML data or control data of various screens related to the basic function to the user terminal 30, and in response to an operation input by the user via the screen displayed on the user terminal 30. This process is executed, and HTML data or control data corresponding to the result of the process is transmitted to the user terminal 30. The basic functions controlled by the basic function control unit 43 include, for example, login authentication (user authentication), billing control, management of information related to users, and management of information related to individual video distribution. Information relating to the user and individual video distribution can be managed in the information storage management unit 41 (storage 15 or the like).

  The moving image distribution control unit 45 of the distribution server 10 executes various processes related to moving image distribution control. For example, the moving image distribution control unit 45 transmits a distributed moving image received from the distributor terminal 30 to each of the plurality of viewer terminals 30. Specifically, the moving image distribution control unit 45 receives, for example, moving image data of a distributed moving image transmitted from the distributor terminal 30 and transmits the received moving image data to the viewer terminal 30. Distribution of such a moving image is performed by a streaming method using a protocol such as RTMP (Real Time Messaging Protocol).

  The distribution video in the video distribution service of the present embodiment can include real-time audio input via the microphone of the distributor terminal 30. Moreover, the said delivery moving image contains the image corresponding to the real-time display screen displayed via the display of the distributor terminal 30, etc., for example. Hereinafter, a moving image including an image corresponding to the display screen may be referred to as a “screen moving image”. In addition, the distribution moving image includes, for example, an image captured (input) in real time via the camera of the distributor terminal 30. Furthermore, the distribution video can be configured by real-time audio input via a microphone and non-real-time video (for example, a video stored in the storage 35 of the distributor terminal 30).

  The information storage management unit 51 of the user terminal 30 stores and manages various information in the storage 35 and the like. The distribution function control unit 53 of the user terminal 30 executes various processes related to the control of the distribution function of the moving image distribution service. The distribution function is a function for the user of the user terminal 30 to distribute a moving image as a distributor. For example, the distribution function control unit 53 transmits the distribution video to the distribution server 10. Specifically, for example, the distribution function control unit 53 generates moving image data of a distribution video based at least on real-time audio input via a microphone, and transmits the generated video data to the distribution server 10. For example, when the distribution video is the screen video, the distribution function control unit 53 uses the image corresponding to the display screen displayed via the display of the user terminal 30 and the sound input via the microphone. The moving image data of the configured screen moving image is generated in real time, and the generated moving image data is transmitted to the distribution server 10.

  The viewing function control unit 55 of the user terminal 30 executes various processes related to the control of the viewing function of the moving image distribution service. The viewing function is a function for the user of the user terminal 30 to view a video distributed by another user as a viewer. For example, the viewing function control unit 55 outputs an image and sound included in the distribution moving image received from the distribution server 10. For example, the viewing function control unit 55 receives moving image data of a distributed moving image transmitted from the distribution server 10 and displays an image included in the moving image via a display or the like based on the received moving image data. Audio included in the video is output via a speaker or the like.

  The communication control unit 57 of the user terminal 30 performs predetermined communication between the distributor terminal 30 and the user terminal 30 of the participant in the distribution video (hereinafter, may be referred to as “participant terminal 30”). Various processes related to the control of are performed. In the present embodiment, the predetermined communication is communication that can execute at least a telephone call. For example, the communication control unit 57 executes processing related to establishment of communication (session) with the user terminal 30 of the other party that performs predetermined communication, establishes the communication session, and is input via a microphone. The audio data is transmitted through the session, and the audio corresponding to the audio data received through the session is output through a speaker or the like.

  In the present embodiment, the participant terminal 30 is configured to output the sound received in the predetermined communication instead of the sound included in the distribution video received from the distribution server 10 in response to the start of the predetermined communication. ing. For example, the viewing function control unit 55 is configured to mute the sound included in the distribution video received from the distribution server 10 in response to the start of predetermined communication. As a result, the participant terminal 30 Then, instead of the voice included in the distribution video, the voice received in the predetermined communication is output.

  As described above, the video distribution system 1 according to the present embodiment receives the predetermined communication in place of the audio included in the distribution video received by the participant terminal 30 from the distribution server 10 in response to the start of the predetermined communication. The sound to be output is output. Therefore, the content of the voice call between the participant terminal 30 and the distributor terminal 30 is transmitted to the participant terminal 30 via the distribution server 10 as sound included in the distribution video, and is output at the participant terminal 30. Compared with the case where the voice signal of the participant in the participant terminal 30 is delayed (occurrence of echo) is suppressed, and as a result, the deterioration of the voice quality of the call is suppressed.

  In the present embodiment, the distributor terminal 30 performs predetermined communication in addition to real-time audio input via a microphone for a distribution video to be transmitted to the distribution server 10 in response to the start of predetermined communication. It may be configured to append received audio. For example, the communication control unit 57 is configured to synthesize voice data input through a microphone and voice data received in a predetermined communication, and the distribution function control unit 53 performs the synthesis. The moving image data of the distribution moving image is generated based on the audio data. In such a configuration, since the sound received in the predetermined communication is added to the distribution video without passing through the microphone of the distributor terminal 30, the audio included in the distribution video (the audio is not transmitted to the other viewer terminals 30). Output quality). In addition, since the sound received through the predetermined communication is added to the distribution video without passing through the microphone of the distributor terminal 30, the participant terminal can be used even when using headphones or the like in the distributor terminal 30. The distribution video including the contents of the voice call between the terminal 30 and the distributor terminal 30 is distributed to the other viewer terminals 30.

  Further, the distributor terminal 30 can be configured to execute processing for removing echo components of audio received in predetermined communication from real-time audio input via a microphone. For example, the communication control unit 57 applies a general echo cancellation technique, and based on audio data of audio received in predetermined communication, from the audio data of audio input through a microphone, the communication control unit 57 A process for removing the echo component is performed. In this case, the distribution function control unit 53 generates and transmits moving image data of the distribution moving image based on the audio data on which the process of removing the echo component is executed, and the communication control unit 57 executes the process of removing the echo component. The transmitted voice data is transmitted in a predetermined communication. Such a configuration suppresses deterioration in quality due to echo components of the voice included in the distribution video and the call voice in the predetermined communication.

  In the present embodiment, the predetermined communication is performed without going through the distribution server 10, and may be configured to be performed using, for example, P2P (Peer to Peer) communication. In this case, the communication control unit 57 of each of the distributor terminal 30 and the participant terminal 30 starts predetermined communication using P2P communication after performing signaling via a signaling server (not shown), for example. Communication using P2P communication can be realized by applying, for example, WebRTC (Web Real-Time Communication). Here, in the present embodiment, the predetermined communication is not limited to that performed using P2P communication, and includes, for example, that performed using client-server type communication.

  In the present embodiment, the predetermined communication includes a communication capable of transmitting / receiving other information in addition to a call (sending / receiving voice). For example, the predetermined communication can execute image transmission / reception. In this case, the distributor terminal 30 performs the predetermined communication on the distribution video to be transmitted to the distribution server 10 in response to the start of the predetermined communication. It may be configured to add an image to be received. For example, the distribution function control unit 53 is configured to acquire an image received in a predetermined communication via the communication control unit 57, and add the acquired image to a part of the display area of the distribution video. The Here, the image received in the predetermined communication is, for example, an image corresponding to a real-time display screen displayed via the display or the like of the participant terminal 30, or taken in real time via the camera or the like of the participant terminal 30 ( Input) and various other images. Such a configuration enables participation in a distribution video by image in addition to participation in a distribution video by voice.

  In the present embodiment, the participant terminal 30 is configured to output the audio included in the distribution video received from the distribution server 10 instead of the audio received in the predetermined communication in response to the end of the predetermined communication. obtain. For example, the viewing function control unit 55 is configured to cancel the mute of the audio included in the distribution video received from the distribution server 10 in response to the end of predetermined communication.

  In the present embodiment, the predetermined communication is performed according to the specific viewer terminal 30 (participant) in accordance with the consent in the distributor terminal 30 of the participation request from the specific viewer terminal 30 included in the plurality of viewer terminals 30. Terminal 30) and distributor terminal 30 may be configured to start. For example, the viewing function control unit 55 of the viewer terminal 30 accepts a participation request from a viewer via a screen displayed on the viewer terminal 30, and sends the participation request to the distributor terminal 30 via the distribution server 10. And the distribution function control unit 53 of the distributor terminal 30 accepts the consent of the participation request by the distributor via the screen displayed on the distributor terminal 30, and distributes according to the acceptance of the acceptance of the participation request. The communication control unit 57 of each of the participant terminal 30 and the participant terminal 30 is configured to start predetermined communication.

  Next, a specific example of the moving image distribution system 1 of this embodiment having such a function will be described. In the moving image distribution service in this specific example, a real-time screen moving image including an image corresponding to the display screen of the distributor terminal 30 is distributed to a plurality of viewer terminals 30 as a distributed moving image. Such screen video distribution is sometimes referred to as “screen live distribution”.

  FIG. 3 shows an example of the main screen 60 that is a starting point when the user of the video distribution service uses the service in this example. The main screen 60 is displayed when, for example, an application for a video distribution service is started on the user terminal 30, or the user terminal 30 connects to a video distribution service website provided by the distribution server 10 via a web browser. Displayed when accessed.

  As shown in FIG. 3, the main screen 60 has a recommended area 61 for displaying recommended videos and a moving picture list area 62 for displaying a list of videos being distributed. A basic menu area 100 is arranged at the lower end. Yes. Information related to the moving image displayed in the recommended area 61 and the moving image list area 62 includes a moving image title (distributed title, which is displayed as “YYY”, “XXX”, etc. in the example of FIG. 3), and a distributor. Contains information. The user can start viewing the moving image as a viewer by selecting any of the moving images displayed in the recommended area 61 and the moving image list area 62.

  The basic menu area 100 is composed of menus that serve as a basis for using the moving image distribution service, and is similarly arranged on main screens other than the main screen 60. Specifically, the basic menu area 100 includes a main menu 102 for displaying the main screen 60, a search menu 104 for searching for users and moving images, a distribution menu 106 for starting distribution of moving images, A notification menu 108 for displaying a notification to the user and a my page menu 109 for displaying its own user page (my page) are configured.

  When the user selects the search menu 104, for example, a search screen for searching for another user or a moving image using a keyword or the like is displayed. In addition, when the user selects the My Page menu 109, a My Page screen that displays the user's own user page (My Page) is displayed, and the user can view and edit his / her basic information, for example, via the screen. It is also possible to browse the distribution history and viewing history.

  When the user selects the distribution menu 106, a distribution preparation screen 70 illustrated in FIG. 4 is displayed on the user terminal 30. As shown in the figure, the distribution preparation screen 70 includes a title input area 72 for inputting a distribution title and a distribution start button 74 displayed as “Start distribution!”, And a basic menu area 100 is arranged at the lower end. Has been. This distribution preparation screen 70 is a screen for the user to instruct the start of distribution of moving images.

  When the user selects a distribution start button 74 after inputting a desired distribution title (for example, “Live distribution of game X!” Etc.) in the title input area 72, the distribution preparation screen 70 is closed, and the screen animation is displayed. Distribution of is started. Specifically, the generation and distribution server 10 of the moving image data of the screen moving image constituted by the image corresponding to the display screen in the user terminal 30 (distributor terminal 30) of the distributor and the sound input through the microphone. Transmission to is started. When the distribution of the screen moving image is started, the moving image data of the screen moving image is transmitted from the distribution server 10 to the user terminal 30 (viewer terminal 30) of the viewer in response to the viewing request from the viewer. Become.

  FIG. 5 shows an example of a distribution screen 80 displayed on the distributor terminal 30 during the distribution of moving images. The distribution screen 80 is displayed when the distribution start button 74 is selected and the distribution preparation screen 70 is closed. In this specific example, during the distribution of a moving image, a program such as an application for moving image distribution service operates in the background, and the display area 210 corresponding to the entire screen of the distribution screen 80 has, for example, an OS home screen or , The screen of another application that is running is displayed.

  In the distribution screen 80, as shown in FIG. 5, a comment input area 82 at the upper end of the screen, a camera image display area 84 at the lower left side thereof, and a setting button 86 and an end button 88 at the lower right corner of the screen are displayed in the display area 210. It is arranged so as to overlap. The comment input area 82 is configured so that the distributor can input a comment such as a character string. The camera image display area 84 displays an image (specifically, for example, a video of a distributor) input via the front camera of the distributor terminal 30 or the like.

  When the distributor selects the setting button 86, various distribution settings can be performed via a setting screen (not shown). For example, settings such as microphone invalidation (mute), camera invalidation (including non-display of the camera image display area 84), comment viewing, comment column non-display (comment input area 82 non-display), etc. It can be carried out. Here, “view comment” is a setting for displaying a comment input by the distributor and a comment input by the viewer.

  Here, the comment input area 82, the camera image display area 84, the setting button 86, and the end button 88 are widget-type objects that are controlled by an application for a moving image distribution service that operates in the background. The distributor performs an operation (OS, OS, etc.) via the screen displayed in the display area 210 by tapping the area other than the objects 82, 84, 86 and 88 in the entire display area 210 of the distribution screen 80. Alternatively, an operation on another application being started can be executed. The distributor can also change the display position (move the object) by performing a slide operation or the like on the comment input area 82 and the camera image display area 84.

  FIG. 6 shows an example of a viewing screen 90 displayed on the user terminal 30 (viewer terminal 30) of the viewer who views the screen moving image. For example, the viewer selects any of the moving images displayed in the recommended area 61 and the content list area 62 of the main screen 60 or selects any of the moving pictures searched through the search menu 104. The video to be viewed can be selected by, for example, and the video data of the corresponding screen video is transmitted from the distribution server 10 to the viewer terminal 30 in response to the selection of such video, and the video corresponding to the video data is displayed. A viewing screen 90 to be displayed is displayed on the viewer terminal 30.

  As shown in FIG. 6, the viewing screen 90 includes a moving image display area 92 for displaying a screen moving image, a comment input area 93 located at the lower left corner of the screen, a setting button 94 and a participation request button 95 located at the lower right corner of the screen. Have As shown in the drawing, the moving image display area 92 displays a screen moving image including an image corresponding to the display screen (distribution screen 80) in the distributor terminal 30.

  The comment input area 93 is configured so that the viewer can input a comment such as a character string. When the viewer selects the setting button 94, various viewing settings can be made via a setting screen (not shown). For example, setting such as viewing a comment can be performed.

  The participation request button 95 on the viewing screen 90 is a button for the viewer to request participation in the screen video through a call with the distributor. Hereinafter, an operation when the viewer participates in the screen video will be described. FIG. 7 is a sequence diagram illustrating an example of processing executed between the distributor terminal 30 and the viewer terminal 30 when the viewer participates in the screen video. First, as shown in the figure, the viewer terminal 30 transmits a participation request to the distributor terminal 30 in accordance with the selection of the participation request button 95 by the viewer (step S100). The participation request includes information regarding the viewer who has transmitted the participation request, and is transmitted to the distributor terminal 30 via the distribution server 10.

  Then, the distributor terminal 30 that has received the participation request accepts consent from the distributor of the participation request (step S110). FIG. 8 illustrates the distribution screen 80 when a participation request is received. As shown in the drawing, the distribution screen 80 is configured to display a notification object 89 in response to the reception of the participation request. The notification object 89 is located on the right side of the camera image display area 84, displays information related to the viewer who has requested to participate, and has a consent button 891. The distributor can accept the participation request from the viewer by selecting the accept button 891. In this example, the notification object 89 disappears after being displayed for a predetermined period (for example, 10 seconds) unless the consent button 891 is selected by the distributor.

  When acceptance of the participation request is accepted in accordance with the selection of the acceptance button 891 of the notification object 89 in this way, communication (session) for a call is performed between the distributor terminal 30 and the viewer terminal 30 (participant terminal 30). It is established (step S120). Specifically, in this example, P2P communication was used between the distributor terminal 30 and the participant terminal 30 after the distributor terminal 30 and the participant terminal 30 performed signaling via a signaling server (not shown). A call session is established. According to the establishment of the session, a call (voice transmission / reception) between the distributor terminal 30 and the participant terminal 30 becomes possible.

  Here, input / output of audio in each of the distributor terminal 30, the viewer terminal 30, and the participant terminal 30 will be described. First, audio input / output during normal times (a period in which there are no participants participating in the screen video) will be described. FIG. 9 is a diagram for explaining input / output of audio in each of the distributor terminal 30 and the viewer terminal 30 at normal times. As shown in the figure, in the distributor terminal 30, the distribution function control unit 53 generates moving image data of a screen moving image constituted by sound (microphone sound) input via a microphone and an image corresponding to the display screen. In the viewer terminal 30, the viewing function control unit 55 outputs the screen moving image (image and sound) corresponding to the moving image data received via the distribution server 10 as it is.

  FIG. 10 shows the input of audio in each of the distributor terminal 30, the participant terminal 30, and the other viewer terminals 30 during the period in which the participant participates in the screen video (the period in which the call session is established). It is a figure for demonstrating an output. As shown in the figure, during the period when the participant is participating in the screen video, the communication control unit 57 of each of the distributor terminal 30 and the participant terminal 30 uses the microphone voice data input via the microphone as a call. A call voice corresponding to the voice data transmitted through the session and received through the session is output through a speaker or the like.

  FIG. 11 is a diagram for explaining the details of voice input / output in communication control unit 57 of distributor terminal 30 in FIG. As illustrated, in this example, the communication control unit 57 includes an echo canceller 572. The echo canceller 572 has a function of removing the echo component of the received voice data from the voice data of the microphone voice input via the microphone, based on the voice data received via the call session. The audio data transmitted by the communication control unit 57 via the call session is audio data that has been processed by the echo canceller 572 to remove the echo component. Similarly, the communication control unit 57 of the participant terminal 30 executes a process of removing an echo component by the echo canceller 572.

  As shown in FIG. 11, in this example, the communication control unit 57 includes a mixer 574. The mixer 574 has a function of synthesizing the sound data of the microphone sound that has been subjected to the process of removing the echo component by the echo canceller 572 and the sound data received through the call session. The distribution function control unit 53 acquires the voice data of the synthesized voice from the communication control unit 574, and generates and transmits moving image data including the voice data of the synthesized voice.

  Returning to FIG. 10, the distribution server 10 transmits the moving image data of the screen moving image transmitted from the distributor terminal 30 to a plurality of viewer terminals 30 including the participant terminals 30. In a viewer terminal 30 other than the participant terminal 30 (viewer terminal 30 not participating in the screen moving image), the image and sound included in the screen moving image corresponding to the received moving image data are output as they are. On the other hand, during the period in which the participant terminal 30 participates in the screen moving image, the sound included in the screen moving image is muted (silenced), and only the image included in the screen moving image is output (displayed). As a result, the participant terminal 30 displays an image included in the screen moving image and outputs a call voice through the call session.

  In this way, during the period in which the participant is participating in the screen video, the participant terminal 30 does not output the audio included in the screen video distributed via the distribution server 10 and does not pass through the call session. The call voice is output. The call voice is suppressed in delay compared to the voice included in the screen moving image distributed via the distribution server 10. In addition, since the echo canceller 572 of the communication control unit 57 of each of the distributor terminal 30 and the participant terminal 30 performs a process of removing the echo component included in the microphone voice, the quality of the call voice is deteriorated due to the echo component. It is suppressed. Further, since the distributor terminal 30 generates and transmits the video data of the screen moving image including the synthesized sound of the microphone sound that has been subjected to the process of removing the echo component and the call sound through the call session, Deterioration of the quality of the audio output from the viewer terminal 30 that has not participated in the video is also suppressed.

  The distributor and the participant can instruct the end of the call (cancellation of the session) via the distribution screen 80 or the viewing screen 90. In response to the end of the call, the participant terminal 30 returns to a state in which it does not participate in the screen moving image, and specifically, the mute (mute) of the sound included in the screen moving image is released.

  In the example described above, the case where only one viewer participates in the screen video has been described, but a plurality of viewers can also participate in the screen video. In this case, a call session using the mesh-type P2P communication network is established between the distributor terminal 30 and the plurality of participant terminals 30, and the distributor terminal 30 and the plurality of participants are connected via the session. A call (group call) with the terminal 30 is performed.

  In the above-described example, a call (send / receive voice) is performed between the distributor terminal 30 and the participant terminal 30 via a call session. However, in addition to the call, image transmission / reception is performed. May be performed. For example, the WebRTC described above can be applied to communication involving transmission / reception of various information other than voice. In this case, the distributor terminal 30 may add an image received via the established session to a part of the display area of the screen moving image. Further, the participant terminal 30 may receive the screen moving image itself from the distributor terminal 30 via the established session. In this case, the participant terminal 30 outputs the screen moving image received via the session without outputting the screen moving image distributed by the distribution server 10.

  The moving image distribution system 1 according to the present embodiment described above has the participant terminal 30 in response to the start of predetermined communication (for example, establishment of a call session) performed between the viewer terminal 30 and the participant terminal 30. However, instead of the audio included in the distribution video received from the distribution server 10, the audio received in the predetermined communication is output. Therefore, the content of the voice call between the participant terminal 30 and the distributor terminal 30 is transmitted to the participant terminal 30 via the distribution server 10 as sound included in the distribution video, and is output at the participant terminal 30. Compared with the case where the voice signal of the participant in the participant terminal 30 is delayed (occurrence of echo) is suppressed, and as a result, the deterioration of the voice quality of the call is suppressed. That is, the embodiment of the present invention suppresses deterioration of the voice quality of a call performed between a distributor terminal that distributes a moving image and a viewer terminal that views the moving image.

  In the above-described embodiment, the user terminal 30 is configured to function as the distributor terminal 30 and the viewer terminal 30. However, in the embodiment of the present invention, each terminal may be configured as a dedicated terminal. For example, the terminal dedicated to the distributor may be configured not to include the viewing function control unit 55, and the terminal dedicated to the viewer may be configured not to include the distribution function control unit 53.

  The processes and procedures described in the present specification are implemented by software, hardware, or any combination thereof other than those explicitly described. For example, the processes and procedures described in this specification are realized by mounting logic corresponding to the processes and procedures on a medium such as an integrated circuit, a volatile memory, a nonvolatile memory, and a magnetic disk. The processing and procedure described in this specification can be implemented as a computer program corresponding to the processing / procedure and executed by various computers.

  Even if the processes and procedures described herein are described as being performed by a single device, software, component, or module, such processes or procedures may be performed by multiple devices, multiple software, multiple Component and / or multiple modules. Also, the software and hardware elements described herein can be implemented by integrating them into fewer components or by disassembling them into more components.

  In the present specification, when the constituent elements of the invention are described as one or a plurality, or when they are described without being limited to one or a plurality of cases, they should be understood separately in context. The component may be either singular or plural.

1 video distribution system 10 distribution server 20 network 30 user terminal (distributor terminal, viewer terminal, participant terminal)
41 Information Storage Management Unit 43 Basic Function Control Unit 45 Video Distribution Control Unit 51 Information Storage Management Unit 53 Distribution Function Control Unit 55 Viewing Function Control Unit 57 Communication Control Unit 60 Main Screen 70 Distribution Preparation Screen 80 Distribution Screen 90 Viewing Screen

Claims (11)

A system for delivering a video, comprising a distributor terminal, a distribution server, and a plurality of viewer terminals,
The distributor terminal transmitting a distribution video including at least real-time audio input via a microphone to the distribution server;
The distribution server transmitting the distribution video received from the distributor terminal to each of the plurality of viewer terminals;
Each of the plurality of viewer terminals outputting an image and sound included in the distribution video received from the distribution server;
Starting predetermined communication capable of at least making a call between the participant terminals included in the plurality of viewer terminals and the distributor terminal; and
The participant terminal, in response to the start of the predetermined communication, executes a step of outputting the audio received in the predetermined communication instead of the audio included in the distribution video received from the distribution server. ,
system.   In response to the start of the predetermined communication, the distributor terminal receives the distribution video to be transmitted to the distribution server in the predetermined communication in addition to the real-time sound input via a microphone. The system of claim 1, further comprising the step of adding speech.   The distributor terminal further executes a step of executing a process of removing an echo component of a voice received in the predetermined communication from a voice input via a microphone in response to the start of the predetermined communication. Item 1 or 2 system.   4. The system according to claim 1, wherein the predetermined communication is performed using P2P communication.   The said participant terminal further performs the step which outputs the audio | voice contained in the said delivery moving image received from the said delivery server instead of the audio | voice received in the said predetermined communication according to completion | finish of the said predetermined communication. Item 5. The system according to any one of Items 1 to 4. The system according to any one of claims 1 to 5,
The predetermined communication is capable of transmitting and receiving images,
The distributor terminal further executes a step of adding an image received in the predetermined communication to the distribution video to be transmitted to the distribution server in response to the start of the predetermined communication.
system.   The step of starting the predetermined communication includes the first viewer as the participant terminal in response to an approval in the distributor terminal of a participation request from one viewer terminal included in the plurality of viewer terminals. The system according to claim 1, comprising starting the predetermined communication between a terminal and the distributor terminal.   The system according to any one of claims 1 to 7, wherein the distribution moving image includes an image corresponding to a display screen of the distributor terminal and / or an image input via a camera of the distributor terminal. A system comprising a distributor terminal, a distribution server, and a plurality of viewer terminals is a method for distributing a video,
The distributor terminal transmitting a distribution video including at least real-time audio input via a microphone to the distribution server;
The distribution server transmitting the distribution video received from the distributor terminal to each of the plurality of viewer terminals;
Each of the plurality of viewer terminals outputting an image and sound included in the distribution video received from the distribution server;
Starting predetermined communication capable of at least making a call between the participant terminals included in the plurality of viewer terminals and the distributor terminal; and
The participant terminal includes a step of outputting a sound received in the predetermined communication instead of a sound included in the distribution moving image received from the distribution server in response to the start of the predetermined communication.
Method. In a system for delivering a video, comprising a distributor terminal, a distribution server, and a plurality of viewer terminals, a program executed on the distributor terminal,
Processing for transmitting a distribution video including at least real-time audio input via a microphone to the distribution server;
A process of starting a predetermined communication capable of executing at least a call with a participant terminal included in the plurality of viewer terminals that outputs an image and sound included in the distribution moving image received from the distribution server;
Processing for adding sound received in the predetermined communication to the distribution video to be transmitted to the distribution server in addition to real-time sound input via a microphone in response to the start of the predetermined communication; , Execute
The process of adding the received voice includes executing a process of removing an echo component of the voice received in the predetermined communication from the voice input through a microphone.
program. A system comprising a distributor terminal, a distribution server, and a plurality of viewer terminals, and is a program executed on the viewer terminal in a system for distributing moving images,
A distribution video received from the distribution server, the image included in the distribution video including at least real-time audio received by the distribution server from the distributor terminal and input via a microphone of the distributor terminal; Processing to output audio,
A process of starting a predetermined communication capable of executing at least a call with the distributor terminal;
In response to the start of the predetermined communication, in place of the audio included in the distribution video received from the distribution server, a process of outputting the audio received in the predetermined communication is executed.
program.

Images