JP2009089156A - Distribution system and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a distribution system and a distribution method, capable of distributing data with high quality and performing communication with high real-time performance by utilizing a distribution path thereof. <P>SOLUTION: This distribution system has, as a communication mode, a high-quality mode giving priority to the quality of video and audio while permitting a delay, and a low-delay mode giving priority to low delay while having quality inferior to that of the high-quality mode. In this case, in the high-quality mode, since the high-quality mode is uni-directional communication, there is no need to shorten a delay and each communication terminal can receive high-quality video and audio. In the low-delay mode, though communication using bi-directional communication is performed, high-quality video and audio are not required for the communication, smooth communication with high real-time performance can be achieved by shortening the delay without requiring high-quality image and audio for the communication, and a user can be free from the sense of incongruity. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a technique for distributing data to a distribution server and a communication terminal and communicating between users of the distribution server and the communication terminal.

In general, a system for distributing video and audio such as television broadcasting does not switch between the distribution side and the reception side, so that high-quality video and audio data can be distributed by allowing a delay. . On the other hand, in a system that allows such a delay, if communication is attempted between the distribution side and the reception side, even if high-quality video and audio communication can be performed, real-time performance cannot be secured due to the effect of the delay, The quality of communication will be very bad. In order not to be affected by such a delay, a technique for performing communication with a small delay by providing a communication path with a small delay different from the distribution path that allows the delay (for example, patents) is disclosed. Reference 1).
JP 2006-148741 A

  However, with such a method, a system that allows delay and a system with low delay are simply combined, and in order to realize communication using another route, it leads to an increase in system cost such as addition of hardware. It was.

  The present invention has been made in view of the above-described circumstances, and provides a distribution system and a distribution method capable of distributing high-quality data and performing high-real-time communication using the distribution route. The purpose is to do.

  In order to solve the above-described problem, the present invention provides a distribution system including a distribution server and a communication terminal that perform communication in a high-quality mode or a low-delay mode communication mode via a communication network. Alternatively, distribution data input means for inputting distribution data in a streaming format indicating at least one of audio, and high quality encoding means for encoding the distribution data input to the distribution data input means in a first processing time Low-delay encoding means for encoding the delivery data input to the delivery data input means in a processing time shorter than the first processing time, and the communication mode is a high-quality mode or a low-delay mode. When the selection means for selecting one of them and the high quality mode is selected as the communication mode, the high quality error is selected. The distribution data encoded by the code means is transmitted to the communication terminal, and the distribution data encoded by the low delay encoding means is transmitted to the communication terminal when the low delay mode is selected. Data transmitting means, terminal data receiving means for receiving terminal data transmitted from the communication terminal, and decoding means for decoding terminal data received by the terminal data receiving means, the communication terminal, Distribution data receiving means for receiving distribution data transmitted from a distribution server, high quality decoding means for decoding the distribution data received by the distribution data receiving means in a second processing time, and the distribution data For the distribution data received by the receiving means, the decoding time is shorter than the second processing time. A low-delay decoding means for performing a transmission, an identification means for identifying whether the communication mode is a high-quality mode or a low-delay mode, and the identification means has identified that the communication mode is a high-quality mode In this case, the delivery data decoded by the high-quality decoding unit is output. When the low-delay mode is identified, the delivery data decoded by the low-delay decoding unit is output. Output means, terminal data input means for inputting terminal data in a streaming format indicating at least one of video and audio, encoding means for encoding terminal data input to the terminal data input means, Terminal data transmitting means for transmitting the terminal data encoded by the encoding means to the distribution server; Provided is a distribution system characterized by comprising.

  In another preferable aspect, the distribution server further includes selection information transmission means for transmitting selection information indicating the high quality mode or the low delay mode selected by the selection means, and the identification means includes the distribution means. The selection information transmitted from the server may be received, and whether the communication mode is the high quality mode or the low delay mode may be identified based on the received selection information.

  In another preferable aspect, the communication terminal further includes an analysis unit that analyzes a method of encoding the distribution data received by the distribution data reception unit, and the identification unit is based on an analysis result of the analysis unit. Thus, it may be identified whether the communication mode is a high quality mode or a low delay mode.

  In another preferred embodiment, the communication terminal includes a plurality of communication terminals, and the encoding in the low delay encoding unit is performed by synthesizing the terminal data decoded by the decoding unit with the distribution data input to the distribution data input unit. It may be performed on the distribution data generated.

  In another preferable aspect, the communication terminal is plural, and the distribution server generates specific information for specifying one or more communication terminals out of the plurality of communication terminals, and transmits the specific information transmitting means. The communication terminal further includes specific information receiving means for receiving specific information transmitted from the distribution server, and the communication terminal indicated by the specific information among the plurality of communication terminals The terminal data may be transmitted to.

  Moreover, in another preferable aspect, when the communication mode is a low delay mode, transmission of terminal data from the communication terminal to the distribution server may be performed.

  In another preferred aspect, the high quality decoding means buffers the distribution data received by the distribution data receiving means with a predetermined amount of data, decodes the buffered data, and performs the low delay decoding. The means may buffer the distribution data received by the distribution data receiving means with a data amount smaller than the predetermined data amount, and decode the buffered data.

  Further, the present invention is a distribution method used in a distribution system having a distribution server and a communication terminal for performing communication in a communication mode of high quality mode or low delay mode via a communication network, and used in the distribution server. A distribution data input process in which distribution data in a streaming format indicating at least one of video and audio is input, and the distribution data input in the distribution data input process is encoded in a first processing time. A high-quality encoding process, a low-delay encoding process for encoding the distribution data input in the distribution data input process in a processing time shorter than the first processing time, and a high-quality mode as the communication mode. Or a selection process for selecting one of the low-delay modes and the communication mode as a high quality mode When selected, the distribution data encoded by the high quality encoding process is transmitted to the communication terminal, and when the low delay mode is selected, the encoding is performed by the low delay encoding process. A distribution data transmission process for transmitting distribution data to the communication terminal; a terminal data reception process for receiving terminal data transmitted from the communication terminal; and a decoding process for decoding terminal data received by the terminal data reception process; The method used in the communication terminal includes: a distribution data receiving process for receiving distribution data transmitted from the distribution server; and a second processing time for the distribution data received by the distribution data receiving process. And the distribution received by the distribution data reception process. A low-delay decoding process for decoding data with a processing time shorter than a second processing time; an identification process for identifying whether the communication mode is a high-quality mode or a low-delay mode; and the identification process When the communication mode is identified as the high quality mode, the delivery data decoded by the high quality decoding process is output, and when the communication mode is identified as the low delay mode, An output process for outputting distribution data decoded by a low delay decoding process, a terminal data input process for inputting terminal data in a streaming format indicating at least one of video and audio, and an input in the terminal data input process The terminal data is encoded, and the encoding is performed by the encoding process. And a terminal data transmission process for transmitting the terminal data to the distribution server.

  According to the present invention, it is possible to provide a distribution system and a distribution method capable of distributing high-quality data and performing highly real-time communication using the distribution route.

  Hereinafter, an embodiment of the present invention will be described.

<Embodiment>
As shown in FIG. 1, the distribution system according to the embodiment of the present invention includes a distribution server 1 and communication terminals 2-A and 2-B. Hereinafter, when the communication terminal 2-A and the communication terminal 2-B are not distinguished, they are simply referred to as the communication terminal 2. The distribution server 1 and the communication terminal 2 are connected via the communication network 1000 and transmit / receive various data. The distribution server 1 can perform multicast communication with the plurality of communication terminals 2.

  The hardware configuration of the distribution server 1 will be described with reference to FIG. FIG. 2 is a block diagram illustrating a hardware configuration of the distribution server 1. A CPU (Central Processing Unit) 11 loads a program stored in the storage unit 12 into a RAM (Random Access Memory) 13 and executes it. Thereby, the CPU 11 controls each part of the distribution server 1 via the bus 10. The RAM 13 functions as a work area when the CPU 11 processes each data.

  The storage unit 12 is a storage unit such as a hard disk, a ROM (Read Only Memory), and a nonvolatile memory, and stores the above-described program and various types of information.

  The operation unit 14 is, for example, a keyboard, a mouse, a remote controller, and the like. When a user of the distribution server 1 operates the operation unit 14, information representing the operation content is output to the CPU 11. In addition, the operation unit 14 is provided with a switch for selecting either the high quality mode or the low delay mode as the communication mode, and outputs selection information indicating the mode selected by the user. The communication mode will be described later. Further, as will be described later, when the communication mode is selected as the low delay mode, the operation unit 14 allows the user to operate the operation unit 14 from which communication terminal 2 video data and audio data are received. And specifying information indicating the specified communication terminal 2 is output.

  The display unit 15 is a display device such as a liquid crystal display, and performs display based on video data input by the CPU 11. Further, various screens such as a menu screen for operating the distribution server 1 are displayed based on the control of the CPU 11.

  The sound output unit 16 has sound emitting means such as a speaker, and emits sound based on input sound data.

  The video input unit 17 includes an image sensor such as a CCD (Charge Coupled Device), and generates and outputs video data (hereinafter referred to as input video data) based on the shooting content of the image sensor.

  The voice input unit 18 includes a microphone that collects sound, and generates and outputs audio data (hereinafter referred to as input voice data) based on the collected sound of the microphone.

  The communication unit 19 is a communication unit that transmits and receives data to and from the communication terminal 2 via the communication network 1000 by wire, wireless, or the like. In the present embodiment, data transmitted and received between the distribution server 1 and the communication terminal 2 are video data and audio data in a streaming format, and the communication unit 19 sends selection information and specific information to the communication terminal 2. Such information is also transmitted, and connection request information described later is received from the communication terminal 2. The above is the description of the hardware configuration of the distribution server 1.

  Next, the hardware configuration of the communication terminal 2 will be described. The communication terminal 2 has the same configuration as that of the hardware of the distribution server 1 shown in FIG. Here, in order to distinguish from the distribution server 1, the hardware configuration of the communication terminal 2 is as follows: bus 20, CPU 21, storage unit 22, RAM 23, operation unit 24, display unit 25, audio output unit 26, The video input unit 27, the audio input unit 28, and the communication unit 29 are referred to.

  The operation unit 24 is, for example, a keyboard, a mouse, a remote controller, and the like. When a user of the communication terminal 2 operates the operation unit 24, information representing the operation content is output to the CPU 21. Further, the operation unit 24 is operated by the user to generate and output connection request information indicating that video data and audio data are to be transmitted to the distribution server 1 and the own terminal.

  The communication unit 29 is a communication unit that transmits and receives data to and from the distribution server 1 via the communication network 1000 by wire, wireless, or the like. In the present embodiment, the data transmitted and received between the distribution server 1 and the communication terminal 2 is video data and audio data in a streaming format, and the communication unit 29 receives selection information, specific information, and the like from the distribution server. The connection request information is transmitted to the distribution server 1.

  The components other than the operation unit 24 and the communication unit 29 of the communication terminal 2 have the same functions as those of the distribution server 1, and thus the description thereof is omitted.

  Next, among functions realized by the CPU 11 of the distribution server 1 executing a program stored in the storage unit 12, functions related to transmission / reception of video data and audio data will be described with reference to FIG. FIG. 3 is a block diagram showing a software configuration showing the functions realized by the CPU 11.

  The video decoding unit 101 receives video data transmitted from the communication terminal 2 and received by the communication unit 19 (hereinafter referred to as terminal video data) from the communication unit 19. The terminal video data is video data that has been subjected to predetermined encoding in the communication terminal 2 as will be described later. The video decoding unit 101 decodes the input terminal video data to generate output video data, and outputs the generated video data to the display unit 15 to display the video related to the output video data on the display unit 15. The output video data is also output to the video composition unit 103. In this manner, the video decoding unit 101 generates and outputs output video data by performing decoding corresponding to a predetermined encoding on the terminal video data.

  The audio decoding unit 102 receives audio data (hereinafter referred to as terminal audio data) transmitted from the communication terminal 2 and received by the communication unit 19 from the communication unit 19. The terminal audio data is audio data that has been subjected to predetermined encoding in the communication terminal 2 as will be described later. The audio decoding unit 102 decodes the input terminal audio data to generate output audio data, and outputs the generated audio data to the audio output unit 16, thereby outputting sound related to the output audio data from the audio output unit 16. Let The output voice data is also output to the voice synthesizer 104. Thus, the audio decoding unit 102 generates and outputs output audio data by performing decoding corresponding to a predetermined encoding on the terminal audio data.

  The video composition unit 103 synthesizes the input video data output from the video input unit 17 and the output video data output from the video decoding unit 101 to generate composite video data, and outputs the composite video data to the video encoding unit 105. The synthesis of the input video data and the output video data may be, for example, a mode in which the video related to the output video data is superimposed on the video related to the input video data. When displayed on the display unit 15, the screen may be divided so that both videos are displayed. When output video data is not input from the video decoding unit 101, the input video data may be output to the video encoding unit 105 as it is.

  The voice synthesis unit 104 synthesizes the input voice data output from the voice input unit 18 and the output voice data output from the voice decoding unit 102 to generate synthesized voice data, and outputs the synthesized voice data to the voice encoding unit 106. The synthesis of the input voice data and the output voice data may be, for example, a mode in which the voice related to the output voice data is mixed with the voice related to the input voice data, or the synthesized voice data has a plurality of channels. Thus, the input audio data and the output audio data may be assigned to different channels. When output audio data is not input from the audio decoding unit 102, the input audio data may be output to the audio encoding unit 106 as it is.

  The video encoding unit 105 performs predetermined encoding on the composite video data input from the video synthesis unit 103 based on the selection information output from the operation unit 14 to generate distribution video data, and outputs the distribution video data to the communication unit 19 To do.

  The voice encoding unit 106 performs predetermined encoding on the synthesized voice data input from the voice synthesis unit 104 based on the selection information output from the operation unit 14 to generate distribution voice data, and outputs the generated distribution voice data to the communication unit 19. To do. Note that the selection information output from the operation unit 14 is also output to the communication unit 19.

  Here, detailed configurations of the video encoding unit 105 and the audio encoding unit 106 will be described with reference to FIG. FIG. 4 is a block diagram illustrating configurations of the video encoding unit 105 and the audio encoding unit 106.

  The video encoding unit 105 includes a high quality video encoding unit 1051 and a low delay video encoding unit 1052. The high-quality video encoding unit 1051 and the low-delay video encoding unit 1052 receive the composite video data from the video synthesis unit 103 and the selection information.

  The high quality video encoding unit 1051 operates when the communication mode indicated by the selection information is the high quality mode. The high quality video encoding unit 1051 encodes the input composite video data according to standards such as MPEG2 and MPEG4, and distributes the distribution video data. Generate and output to the communication unit 19. On the other hand, the low-delay video encoding unit 1052 operates when the communication mode indicated by the selection information is the low-delay mode. Similar to the high-quality video encoding unit 1051, MPEG2 and MPEG4 The distribution video data is generated by performing encoding according to a standard such as, and is output to the communication unit 19.

  The high-quality video encoding unit 1051 and the low-delay video encoding unit 1052 are different in encoding processing time for the input composite video data, and the low-delay video encoding unit 1052 is shorter than the processing time in the high-quality video encoding unit 1051. The processing time is shortened. That is, the high-quality video encoding unit 1051 has a long processing time in order to perform high-quality encoding, and the low-delay video encoding unit 1052 has a lower quality than the encoding in the high-quality video encoding unit 1051, but the processing time It is getting shorter.

  This difference in quality is due to the difference in resolution, the number of frames per unit time, the bit rate, etc., for the video related to the composite video data. The higher the resolution, the larger the number of frames, the higher the bit rate. Encoding to maintain high quality video. By performing such encoding with different degrees of quality, the processing performed by the low-delay video encoding unit 1052 is more burdensome than the processing performed by the high-quality video encoding unit 1051. Lighter and shorter processing time.

  In this way, the video encoding unit 105 receives the synthesized video data output from the video synthesizing unit 103, and when the communication mode indicated by the selection information is the high quality mode, the video encoding unit 105 performs high quality on the synthesized video data. When the low-delay mode is performed, the distribution video data generated by encoding the synthesized video data with a lower quality but shorter processing time than the high-quality mode is output to the communication unit 19. .

  The audio encoding unit 106 includes a high quality audio encoding unit 1061 and a low delay audio encoding unit 1062. The high-quality speech encoding unit 1061 and the low-delay speech encoding unit 1062 receive the synthesized speech data from the speech synthesis unit 104 and the selection information.

  The high-quality audio encoding unit 1061 operates when the communication mode indicated by the selection information is the high-quality mode. The high-quality audio encoding unit 1061 encodes the input synthesized audio data according to standards such as MPEG2 and MPEG4, and distributes the distribution audio data. Generate and output to the communication unit 19. On the other hand, the low-delay audio encoding unit 1062 operates when the communication mode indicated by the selection information is the low-delay mode. Similar to the high-quality audio encoding unit 1061, the low-delay audio encoding unit 1062 applies MPEG2 and MPEG4 to input synthetic audio data. Encoding is performed according to a standard such as that to generate distribution audio data and output it to the communication unit 19.

  The high-quality audio encoding unit 1061 and the low-delay audio encoding unit 1062 are different in encoding processing time for the input synthesized audio data, and the low-delay audio encoding unit 1062 is shorter than the processing time in the high-quality audio encoding unit 1061. The processing time is shortened. That is, the high-quality audio encoding unit 1061 has a long processing time in order to perform high-quality encoding, and the low-delay audio encoding unit 1062 has a lower quality than the encoding in the high-quality audio encoding unit 1061, but the processing time It is getting shorter.

  This difference in quality is due to the difference in frequency band, sampling frequency, bit rate, etc. for the voice related to the synthesized voice data. The wider the frequency band, the higher the sampling frequency and bit rate, the higher the quality of the sound. It becomes the encoding which maintains. By performing such encoding with different levels of quality, the processing performed by the low-delay audio encoding unit 1062 is more burdensome than the processing performed by the high-quality audio encoding unit 1061. Lighter and shorter processing time.

  In this way, when the synthesized speech data output from the speech synthesizer 104 is input to the speech encoding unit 106 and the communication mode indicated by the selection information is the high quality mode, the speech encoding unit 106 performs high quality on the synthesized speech data. When the low-delay mode is performed, the distribution audio data generated by encoding the synthesized audio data with a lower quality but shorter processing time than in the high-quality mode is output to the communication unit 19. .

  Returning to the description of FIG. Under the control of the CPU 11, the communication unit 19 sets the distribution video data output from the video encoding unit 105 and the distribution audio data output from the audio encoding unit 106 to a state in which they can be synchronized and reproduced, for example, by adding a time code, It transmits to each communication terminal 2 in a streaming format. The above is the description of the software configuration of the distribution server 1.

  Next, among functions realized by the CPU 21 of the communication terminal 2 executing a program stored in the storage unit 22, functions related to transmission / reception of video data and audio data will be described with reference to FIG. FIG. 5 is a block diagram showing a software configuration showing functions realized by the CPU 21.

  The video decoding unit 201 receives distribution video data transmitted from the distribution server 1 and received by the communication unit 29 from the communication unit 29. The distribution video data is video data that has been subjected to predetermined encoding in the distribution server 1 as described above. The video decoding unit 201 performs predetermined decoding on the input distribution video data to generate output video data, and outputs the generated video data to the display unit 25 to display the video related to the output video data on the display unit 25. . In addition, the video decoding unit 201 also receives selection information transmitted from the distribution server 1 and received by the communication unit 29 from the communication unit 29.

  The audio decoding unit 202 receives the distribution audio data transmitted from the distribution server 1 and received by the communication unit 29 from the communication unit 29. This distributed audio data is audio data that has been subjected to predetermined encoding in the distribution server 1 as described above. The audio decoding unit 202 performs predetermined decoding on the input distribution audio data to generate output audio data, and outputs the generated audio data to the audio output unit 26, thereby outputting audio related to the output audio data from the audio output unit 26. Let it sound. The audio decoding unit 202 also receives selection information transmitted from the distribution server 1 and received by the communication unit 29 from the communication unit 29.

  Here, detailed configurations of the video decoding unit 201 and the audio decoding unit 202 will be described with reference to FIG. FIG. 6 is a block diagram illustrating the configuration of the video decoding unit 201 and the audio decoding unit 202.

  The video decoding unit 201 includes a high quality video decoding unit 2011 and a low delay video decoding unit 2012. The high-quality video decoding unit 2011 and the low-delay video decoding unit 2012 receive distribution video data from the communication unit 29 and selection information.

  The high quality video decoding unit 2011 operates when the communication mode indicated by the selection information is the high quality mode, and the input distribution video data is encoded by the high quality video encoding unit 1051 in the distribution server 1 described above. The output video data is generated by decoding according to a standard similar to the standard and output to the display unit 25. That is, when the communication mode indicated by the selection information is the high quality mode, the input distribution video data is recognized as the data encoded by the high quality video encoding unit 1051 of the distribution server 1, and the encoding is performed. The high-quality video decoding unit 2011 that performs decoding corresponding to is operated.

  On the other hand, the low-delay video decoding unit 2012 operates when the communication mode indicated by the selection information is the low-delay mode. Similar to the high-quality video decoding unit 2011, the low-delay video decoding unit 2012 operates on the distribution video data input above. Decoding is performed according to a standard similar to the standard encoded by the low-delay video encoding unit 1052 in the server 1 to generate output video data, which is output to the display unit 25. That is, when the communication mode indicated by the selection information is the low delay mode, the input distribution video data is recognized as data encoded by the low delay video encoding unit 1052 of the distribution server 1, and the encoding is performed. The low-delay video decoding unit 2012 that performs decoding corresponding to is operated.

  The high-quality video decoding unit 2011 and the low-delay video decoding unit 2012 are different in the decoding processing time for the input distribution video data, and the low-delay video decoding unit 2012 is longer than the processing time in the high-quality video decoding unit 2011. The processing time is shortened. That is, the high quality video decoding unit 2011 takes a long processing time to decode the high quality distribution video data encoded by the high quality video encoding unit 1051 of the distribution server 1. On the other hand, the low-delay video decoding unit 2012 performs distribution video data that is encoded by the low-delay video encoding unit 1052 of the distribution server 1 and has lower quality than the distribution video data encoded by the high-quality video encoding unit 1051. Therefore, the processing time in the high-quality video decoding unit 2011 is shorter.

  In this way, when the distribution video data output from the communication unit 29 is input to the video decoding unit 201 and the communication mode indicated by the selection information is the high quality mode, the distribution video data is encoded with a high quality. If it is recognized as data that has been processed and is in the low-delay mode, the distribution video data is recognized as data that has been encoded with a lower quality but shorter processing time than in the high quality mode. The output video data generated by decoding is output to the display unit 25.

  The audio decoding unit 202 includes a high quality audio decoding unit 2021 and a low delay audio decoding unit 2022. The high-quality audio decoding unit 2021 and the low-delay audio decoding unit 2022 receive the distribution audio data from the communication unit 29 and the selection information.

  The high quality audio decoding unit 2021 operates when the communication mode indicated by the selection information is the high quality mode, and the input distribution audio data is encoded by the high quality audio encoding unit 1061 in the distribution server 1 described above. The output audio data is generated by decoding according to the standard similar to the standard, and output to the audio output unit 26. That is, when the communication mode indicated by the selection information is the high quality mode, the input distribution audio data is recognized as data encoded by the high quality audio encoding unit 1061 of the distribution server 1, and the encoding is performed. The high-quality audio decoding unit 2021 that performs decoding corresponding to is operated.

  On the other hand, the low-delay audio decoding unit 2022 operates when the communication mode indicated by the selection information is the low-delay mode. Similar to the high-quality audio decoding unit 2021, the low-delay audio decoding unit 2022 performs the above-described distribution on the input audio data. Decoding is performed according to a standard similar to the standard encoded by the low-delay audio encoding unit 1062 in the server 1, and output audio data is generated and output to the audio output unit 26. That is, when the communication mode indicated by the selection information is the low-delay mode, the input distribution audio data is recognized as data encoded by the low-delay audio encoding unit 1062 of the distribution server 1, and the encoding is performed. The low-delay audio decoding unit 2022 that performs decoding corresponding to is operated.

  The high-quality audio decoding unit 2021 and the low-delay audio decoding unit 2022 are different in decoding processing time for the input distribution audio data, and the low-delay audio decoding unit 2022 is shorter than the processing time in the high-quality audio decoding unit 2021. The processing time is shortened. That is, the high-quality audio decoding unit 2021 takes a long processing time to decode the high-quality distribution audio data encoded by the high-quality audio encoding unit 1061 of the distribution server 1. On the other hand, the low-delay audio decoding unit 2022 is distributed audio data that is encoded by the low-delay audio encoding unit 1062 of the distribution server 1 and has lower quality than the distribution audio data encoded by the high-quality audio encoding unit 1061. Therefore, the processing time in the high quality audio decoding unit 2021 is shorter.

  In this way, the audio decoding unit 202 receives the distribution audio data output from the communication unit 29, and when the communication mode indicated by the selection information is the high quality mode, the audio decoding unit 202 encodes the distribution audio data with high quality. If it is recognized as data that has been processed and is in the low-delay mode, the distribution audio data is recognized as data that has been encoded with a lower quality but shorter processing time than in the high quality mode. Is output to the audio output unit 26.

  Returning to the description of FIG. The video encoding unit 205 receives input video data from the video input unit 27. Then, predetermined encoding is performed on the input video data to generate terminal video data, which is output to the communication unit 29. In this embodiment, the predetermined encoding is equivalent to the encoding in the low-delay video encoding unit 1052 of the distribution server 1, and performs encoding that realizes low delay although it is not high quality.

  The audio encoding unit 206 receives input audio data from the audio input unit 28. Then, the terminal audio data is generated by performing predetermined encoding on the input audio data, and output to the communication unit 29. Like the video encoding unit 205, this predetermined encoding is equivalent to the encoding in the low-delay audio encoding unit 1062 of the distribution server 1 in this embodiment, and is an encoding that realizes low delay although it is not high quality. I do.

  Here, when the CPU 21 recognizes that the communication terminal 2 indicated by the specific information received by the communication unit 29 indicates its own communication terminal 2, the communication unit 29 is controlled by the CPU 21 so that the communication unit 29 The terminal video data output from 205 and the terminal audio data output from the audio encoding unit 206 are set in a state where they can be synchronously reproduced, such as by giving a time code, and are transmitted to the distribution server 1 in a streaming format.

  On the other hand, when it is recognized that the specific information indicates another communication terminal 2, the terminal video data and the terminal audio data are not transmitted. In this case, since transmission is not performed, the CPU 21 may stop the operations of the video encoding unit 205 and the audio encoding unit 206, and further, the video input unit 27 generates the input video data and the audio input unit 28. The generation of the input voice data in may also be stopped. Further, since distribution server 1 does not have to receive terminal video data and terminal audio data from communication terminal 2, communication unit 19 of distribution server 1 receives terminal video from a communication terminal other than communication terminal 2 indicated by the specific information. The CPU 11 may perform control so that data and terminal voice data are not received. The above is the description of the software configuration of the communication terminal 2.

  Next, the operation of the distribution system according to the embodiment of the present invention will be described. In this description, the user of the communication terminal 2-A is called user A, the user of the communication terminal 2-B is called user B, and the user of the distribution server 1 is called user C.

  A user C performs a live performance of a musical instrument (hereinafter referred to as a live performance) in a music studio where the distribution server 1 is installed. At this time, the user C operates the operation unit 14 to select the communication mode as the high quality mode.

  The high-quality video encoding unit 1051 of the video encoding unit 105 encodes input video data indicating a video (hereinafter referred to as a live video) obtained by shooting the live performance generated by the video input unit 17. On the other hand, the high-quality audio encoding unit 1061 of the audio encoding unit 106 encodes the input audio data indicating the audio (hereinafter referred to as live audio) recorded from the live performance generated by the audio input unit 18.

  Then, the communication unit 19 transmits the distribution video data and the distribution audio data in the streaming format to the communication terminals 2-A and 2-B, and transmits selection information indicating that the communication mode is selected as the high quality mode. To do.

  The communication unit 29 of the communication terminals 2-A and 2-B receives selection information, distribution video data, and distribution audio data. When the CPU 21 recognizes that the communication mode indicated by the selection information is the high quality mode, the CPU 21 controls the high quality video decoding unit 2011 of the video decoding unit 201 to decode the distribution video data, and the audio decoding unit. Control is performed so that the high-quality audio decoding unit 2021 decodes the distributed audio data.

  Thereby, the user A and the user B can enjoy the live video displayed on the display unit 25 and the live audio emitted from the audio output unit 26. As described above, in the high quality mode, since the live performance of the user C is only one-way broadcast communication for distributing to the user A and the user B, the encoding in the distribution server 1, the communication terminal Even if a delay occurs due to the processing time required for decoding in 2-A and 2-B, the user A and the user B do not feel uncomfortable.

  Then, the user C ends the live performance and proposes to the users A and B through the video input unit 17 and the audio input unit 18 whether or not to take communication. In response to this, the user A operates the operation unit 24 of the communication terminal 2-A in order to inform the user C that he / she wants to communicate, and sends the terminal video data, A request to transmit terminal audio data and connection request information indicating the communication terminal 2-A are transmitted.

  On the other hand, when the distribution server 1 receives the connection request information, the CPU 11 recognizes that the communication terminal indicated by the connection request information is the communication terminal 2-A, and communicates with the display unit 15 from the communication terminal 2-A. A display is made to indicate that there has been an intention to do so.

  When the user C confirms the display and understands that communication is to be performed, the user C operates the operation unit 14 to generate specific information for identifying the communication terminal 2-A. The specific information is transmitted from the communication unit 19 to the communication terminal 2-A. At this time, the user C operates the operation unit 14 to select the low delay mode as the communication mode, and also generates selection information indicating that the low delay mode is selected. Are transmitted from the communication unit 19 to the communication terminals 2-A and 2-B. When the specific information is generated, selection information indicating that the low-delay mode is selected may be generated without operating the operation unit 14. Further, the specific information may be transmitted to the communication terminal 2-B.

  As a result, in the communication terminal 2-A which is the communication terminal indicated by the specific information, input video data indicating a video (hereinafter referred to as conversation video A) taken of the user A is input to the video encoding unit 205, and the input video Terminal video data generated by encoding the data is transmitted from the communication unit 29 to the distribution server 1. In addition, input voice data indicating the content of recording the voice of user A (hereinafter referred to as conversation voice A) is input to the voice encoding unit 206, and terminal voice data generated by encoding the input voice data. Is transmitted from the communication unit 29 to the distribution server 1.

  When the distribution server 1 receives the terminal video data and the terminal audio data, the distribution server 1 performs decoding in the video decoding unit 101 and the audio decoding unit 102, respectively, and displays the conversation video A on the display unit 15 based on the generated output video data. And the conversation voice A is emitted from the voice output unit 16 based on the generated output voice data.

  On the other hand, the video synthesizing unit 103 shows output video data and video shot of the user C (hereinafter, the video in the low delay mode is distinguished from the video in the high quality mode and is called the conversation video C). Input video data is input. As a result, composite video data indicating the conversation video AC in which the conversation video A is superimposed on a part of the conversation video C is generated. The voice synthesizer 104 also outputs input voice data and user C's voice (hereinafter, voice in the low delay mode is distinguished from voice in the high quality mode and is called the conversation voice C). Data is input. Thus, synthesized voice data indicating the conversation voice AC in which the conversation voice A and the conversation voice C are mixed is generated.

  When the communication mode is switched to the low delay mode, the encoding mode in the video encoding unit 105 and the audio encoding unit 106 is switched, and the distribution video data transmitted from the distribution server 1 is transmitted to the conversation video AC by the low delay video encoding unit 1052. The video data and the distribution audio data that have been encoded with respect to the synthesized video data indicating the voice become the audio data that has been encoded with respect to the synthesized voice data that indicates the conversation voice AC by the low-delay audio encoding unit 1062.

  On the other hand, also in the communication terminals 2-A and 2-B, the decoding mode in the video decoding unit 201 and the audio decoding unit 202 is switched by receiving the selection information. That is, the distribution video data transmitted from the distribution server 1 is decoded by the low-delay video decoding unit 2012, and the distribution audio data transmitted from the distribution server 1 is decoded by the low-delay audio decoding unit 2022. become. Thus, the conversation video AC is displayed on the display unit 25 in the communication terminals 2-A and 2-B, and the conversation voice AC is emitted from the voice output unit 26. Here, in the low delay mode, since the quality is lower than the quality in the high quality mode, the delay can be reduced, so that the conversation video AC and the conversation voice AC are compared with the live video and the live voice. While the voice quality is low, the delay can be reduced. For this reason, in the communication between the user A and the user C, it is possible to prevent a sense of incongruity due to delay. At this time, the quality required for communication is lower than the quality required for viewing live performances. Therefore, even if the quality of the conversation video AC and conversation voice AC is lower than that of the live video and live voice, there is a problem. There is no.

  When the communication between the user A and the user C ends, the user C operates the operation unit 14 and selects the high quality mode as the communication mode. Thereby, the video encoding unit 105 of the distribution server 1 performs encoding again in the high quality video encoding unit 1051, and the audio encoding unit 106 performs encoding in the high quality audio encoding unit 1061. On the other hand, the video decoding unit 201 of the communication terminals 2-A and 2-B performs decoding in the high quality video decoding unit 2011, and the audio decoding unit 202 performs decoding in the high quality audio decoding unit 2021.

  In addition, transmission of terminal video data and terminal audio data from the communication unit 29 of the communication terminal 2-A is stopped. As a result, it is possible to view live video and audio with high quality but large delay on the communication terminals 2-A and 2-B. At this time, since live video and live audio are one-way broadcast communication from the distribution server 1 to the communication terminals 2-A and 2-B, the users A and B may feel uncomfortable even if the delay is large. Without being able to enjoy live performances.

  As described above, the distribution system of the present invention has a high quality mode in which delay is allowed with priority on video and audio quality as a communication mode, a low delay in which priority is given to low delay, and a lower quality than the high quality mode. In the high quality mode, broadcast type one-way communication is performed from the distribution server 1 to each communication terminal 2 in the high quality mode, and in the low delay mode, bidirectional communication between the distribution server 1 and each communication terminal 2 is performed. Realize communication by communication. At this time, in the high quality mode, since it is a one-way communication, it is not necessary to shorten the delay, and each communication terminal 2 can receive high quality video and audio. In the low-delay mode, two-way communication is performed. However, high-quality video and audio are not necessary for communication, and smooth communication with high real-time characteristics is possible by shortening the delay. It is possible to prevent an uncomfortable feeling.

  As mentioned above, although embodiment of this invention was described, this invention can be implemented in various aspects as follows.

<Modification 1>
In the embodiment described above, the communication terminal 2 identifies whether the communication mode indicated by the received selection information is the high quality mode or the low delay mode for the decoding performed by the video decoding unit 201 and the audio decoding unit 202. The processing according to the communication mode is performed, but the communication mode may be identified by another method. For example, the data contents of the distribution video data and distribution audio data received by the communication unit 29 may be analyzed, and the communication mode may be identified from the analysis result. In this case, the video decoding unit 201 and the audio decoding unit 202 of the communication terminal may be configured to include a video data analysis unit 2013 and an audio data analysis unit 2023, respectively, as shown in FIG.

  The video data analysis unit 2013 receives the distribution video data received by the communication unit 29 and analyzes the distribution video data. The video data analysis unit 2013 obtains information indicating the encoding format attached to the distribution video data by this analysis, and is the distribution video data encoded in the high quality video encoding unit 1051 in the distribution server 1, It is identified whether or not the distribution video data is encoded by the low-delay video encoding unit 1052. When the video data analysis unit 2013 identifies that the distribution video data is the distribution video data encoded by the high quality video encoding unit 1051, the video data analysis unit 2013 outputs the distribution video data to the high quality video decoding unit 2011. On the other hand, if the low-delay video encoding unit 1052 identifies that the video data has been encoded, the distribution video data is output to the low-delay video decoding unit 2012.

  Similarly to the above, the audio data analysis unit 2023 receives the distribution audio data received by the communication unit 29 and analyzes the distribution audio data. The audio data analysis unit 2023 obtains information indicating the encoding format attached to the distribution audio data by this analysis, and is the distribution audio data encoded by the high quality audio encoding unit 1061 in the distribution server 1, Whether the low-delay audio encoding unit 1062 is distribution audio data encoded or not is identified. If the audio data analysis unit 2023 identifies that the distribution audio data is the distribution audio data encoded by the high quality audio encoding unit 1061, the audio data analysis unit 2023 outputs the distribution audio data to the high quality audio decoding unit 2021. On the other hand, if the low-delay audio encoding unit 1062 identifies that the audio data has been encoded, the distributed audio data is output to the low-delay audio decoding unit 2022.

  In this way, the communication terminal 2 can identify whether the communication mode is the high quality mode or the low delay mode without receiving the selection information. It should be noted that information indicating the encoding format attached to the distribution video data and the distribution audio data may be added when encoding is performed in the video encoding unit 105 and the audio encoding unit 106 of the distribution server 1, respectively. Further, the communication terminal 2 that receives the specific information may identify that the low-delay mode has been established by reception, and other communication terminals 2 may be identified as in the present modification.

<Modification 2>
In the embodiment described above, the output video data is input to the display unit 15 of the distribution server 1 and the conversation video A is displayed. However, the synthesized video data generated by the video synthesis unit 103 is displayed. It may be input. In this way, the display unit 15 can display a conversation video AC similar to the video displayed on the display unit 25 of the communication terminal 2.

<Modification 3>
In the embodiment described above, the bidirectional communication between the distribution server 1 and the communication terminal 2 performed when the communication mode is the low-delay mode is performed with one communication terminal 2 indicated by the specific information. There may be a plurality of communication terminals 2 indicated by the information. In this case, the terminal video data and the terminal audio data transmitted from the plurality of communication terminals 2 to the distribution server 1 are decoded by the video decoding unit 101 and the audio decoding unit 102, respectively. What is necessary is just to synthesize | combine in 104. FIG. As described above, the bidirectional communication performed in the low delay mode may be performed between the plurality of communication terminals 2 and the distribution server 1. Here, when the output video data to be input is data from a plurality of communication terminals, the display unit 15 may divide the screen and display the video related to each output video data. .

<Modification 4>
In the above-described embodiment, the communication terminal 2 that performs bidirectional communication with the distribution server 1 in the low delay mode is the communication terminal 2 indicated by the specific information, but the communication terminal 2 that communicates with the distribution server 1 is 1 If there are a plurality of communication terminals 2 and the two-way communication in the low delay mode is performed with all the communication terminals 2, the specific information may not be used. In this case, connection request information may not be transmitted from the communication terminal 2 to the distribution server 1.

<Modification 5>
In the above-described embodiment, the user has selected whether to select the high quality mode or the low delay mode as the communication mode by operating the operation unit 14 of the distribution server 1. May be selected. For example, the storage unit 12 stores time information that defines the time of the high quality mode and the time of the low delay mode, and selects the high quality mode or the low delay mode as the communication mode based on the time information. You can do it. As described above, the communication mode can be selected by various methods.

<Modification 6>
In the video decoding unit 201 of the communication terminal 2 in the above-described embodiment, the difference in processing time when the high-quality video decoding unit 2011 and the low-delay video decoding unit 2012 are decoded is the buffer amount of the input distribution video data The difference in processing time due to the difference between the two may be included. In this case, the high-quality video decoding unit 2011 buffers the input distribution video data with a predetermined amount of data and encodes the buffered distribution video data. The low-delay video decoding unit 2012 The distributed distribution video data may be buffered with a data amount smaller than that of the high quality video decoding unit 2011, and the buffered distribution video data may be encoded.

  By buffering a large amount of data, even if the communication speed between the distribution server 1 and the communication terminal 2 temporarily deteriorates, the distribution video data is buffered, so that the video is prevented from being interrupted and has high quality. Video. On the other hand, in the low delay mode, priority is given to the low delay over the quality, so even if the video is cut off slightly due to the deterioration of the communication speed and the quality deteriorates, the communication is smoothed by maintaining the low delay. It can be carried out. In this modification, the video decoding unit 201 has been described, but the same applies to the audio decoding unit 202.

<Modification 7>
In the above-described embodiment, by transmitting connection request information from the communication terminal 2 to the distribution server 1, the user of the communication terminal 2 displays an intention to communicate with the user of the distribution server 1. However, such intention expression may be performed by other methods. For example, it may be performed by transmitting terminal video data relating to a video captured by the video input unit 27 of the communication terminal 2 to the distribution server 1 and displaying the video on the display unit 15 of the distribution server 1. Further, the video input unit 27 shoots a user as a still image, transmits terminal still image data indicating the still image to the distribution server 1, and displays the still image on the display unit 15 of the distribution server 1. May be. As described above, since the user of the communication terminal 2 only needs to display an intention to communicate with the user of the distribution server 1, any method may be used. The specific information generated by the operation of the operation unit 14 of the distribution server 1 does not need to be performed after the connection request information is received. When the user of the distribution server 1 does not request an intention display, the connection request information is It may not be transmitted from the communication terminal 2.

It is explanatory drawing which shows the connection relation of the delivery server and communication terminal in the delivery system which concerns on embodiment. It is a block diagram which shows the structure of the hardware of the delivery server and communication terminal which concern on embodiment. It is a block diagram which shows the structure of the software of the delivery server which concerns on embodiment. It is a block diagram which shows the detailed structure of the video encoding part of the delivery server which concerns on embodiment, and an audio | voice encoding part. It is a block diagram which shows the software structure of the communication terminal which concerns on embodiment. It is a block diagram which shows the detailed structure of the video decoding part of the communication terminal which concerns on embodiment, and an audio decoding part. It is a block diagram which shows the detailed structure of the video decoding part of the communication terminal which concerns on the modification 1, and an audio decoding part.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Distribution server, 2, 2-A, 2-B ... Communication terminal 10, 20 ... Bus, 11, 21 ... CPU, 12, 22 ... Memory | storage part, 13, 23 ... RAM, 14, 24 ... Operation part, 15, 25 ... display unit, 16, 26 ... audio output unit, 17, 27 ... video input unit, 18, 28 ... audio input unit, 19, 29 ... communication unit, 101, 201 ... video decoding unit, 2011 ... high quality Video decoding unit 2012 ... Low delay video decoding unit 2013 ... Video data analysis unit 102, 202 ... Audio decoding unit 2021 ... High quality audio decoding unit 2022 ... Low delay audio decoding unit 2023 ... Audio data analysis unit DESCRIPTION OF SYMBOLS 103 ... Video synthesis part, 104 ... Audio | voice synthesis part, 105,205 ... Video encoding part, 1051 ... High quality video encoding part, 1052 ... Low delay video encoding part, 106,206 ... Audio | voice Nkodo unit, 1061 ... high quality voice encoding unit, 1062 ... low delay audio encoding unit, 1000 ... communication network

Claims (8)

In a distribution system having a distribution server and a communication terminal for performing communication in a communication mode of a high quality mode or a low delay mode via a communication network,
The distribution server
Distribution data input means for inputting distribution data in a streaming format indicating at least one of video and audio;
High-quality encoding means for encoding the delivery data input to the delivery data input means in a first processing time;
Low-delay encoding means for encoding the delivery data input to the delivery data input means with a processing time shorter than the first processing time;
Selection means for selecting either the high quality mode or the low delay mode as the communication mode;
When the high quality mode is selected as the communication mode, the delivery data encoded by the high quality encoding means is transmitted to the communication terminal, and when the low delay mode is selected, the low delay mode is selected. Distribution data transmission means for transmitting distribution data encoded by the encoding means to the communication terminal;
Terminal data receiving means for receiving terminal data transmitted from the communication terminal;
Decoding means for decoding terminal data received by the terminal data receiving means,
The communication terminal is
Distribution data receiving means for receiving distribution data transmitted from the distribution server;
High-quality decoding means for decoding the distribution data received by the distribution data receiving means in a second processing time;
Low-delay decoding means for decoding the delivery data received by the delivery data receiving means in a processing time shorter than a second processing time;
Identifying means for identifying whether the communication mode is a high quality mode or a low delay mode;
When the identification unit identifies that the communication mode is the high quality mode, the distribution data decoded by the high quality decoding unit is output, and when it is identified as the low delay mode. Output means for outputting the delivery data decoded by the low delay decoding means,
Terminal data input means for inputting terminal data in a streaming format indicating at least one of video and audio;
Encoding means for encoding the terminal data input to the terminal data input means;
Terminal data transmitting means for transmitting terminal data encoded by the encoding means to the distribution server. The distribution server further comprises selection information transmission means for transmitting selection information indicating the high quality mode or the low delay mode selected by the selection means,
The identification means receives selection information transmitted from the distribution server, and identifies whether the communication mode is a high quality mode or a low delay mode based on the received selection information. The distribution system according to claim 1. The communication terminal further comprises analysis means for analyzing a method of encoding the distribution data received by the distribution data receiving means,
The distribution system according to claim 1, wherein the identification unit identifies whether the communication mode is a high quality mode or a low delay mode based on an analysis result of the analysis unit. The communication terminal is plural,
The encoding in the low-delay encoding unit is performed on distribution data generated by combining terminal data decoded by the decoding unit with distribution data input to the distribution data input unit. The distribution system according to any one of claims 1 to 3. The communication terminal is plural,
The distribution server further includes specific information transmitting means for generating and transmitting specific information for specifying one or more communication terminals among the plurality of communication terminals,
The communication terminal further comprises specific information receiving means for receiving specific information transmitted from the distribution server,
The distribution system according to any one of claims 1 to 4, wherein terminal data is transmitted from the communication terminal indicated by the specific information to the distribution server among the plurality of communication terminals. The distribution system according to any one of claims 1 to 5, wherein terminal data is transmitted from the communication terminal to the distribution server when the communication mode is a low-delay mode. The high-quality decoding means buffers the distribution data received by the distribution data receiving means with a predetermined amount of data, decodes the buffered data,
The low-delay decoding unit buffers the distribution data received by the distribution data receiving unit with a data amount smaller than the predetermined data amount, and decodes the buffered data. The distribution system according to claim 6. A distribution method used in a distribution system having a distribution server and a communication terminal that perform communication in a communication mode of a high quality mode or a low delay mode via a communication network,
The method used in the distribution server is:
A distribution data input process in which distribution data in a streaming format indicating at least one of video and audio is input;
A high-quality encoding process for encoding the distribution data input in the distribution data input process in a first processing time;
A low-delay encoding process for encoding the distribution data input in the distribution data input process in a processing time shorter than the first processing time;
A selection process for selecting either the high quality mode or the low delay mode as the communication mode;
When the high quality mode is selected as the communication mode, the distribution data encoded by the high quality encoding process is transmitted to the communication terminal, and when the low delay mode is selected, the low delay mode is selected. A distribution data transmission process for transmitting distribution data encoded by an encoding process to the communication terminal;
A terminal data reception process for receiving terminal data transmitted from the communication terminal;
A decoding process for decoding terminal data received by the terminal data receiving process,
The method used in the communication terminal is:
A distribution data receiving process for receiving distribution data transmitted from the distribution server;
A high quality decoding process for decoding the distribution data received by the distribution data receiving process in a second processing time;
A low-delay decoding process for decoding the distribution data received by the distribution data receiving process in a processing time shorter than a second processing time;
An identification process for identifying whether the communication mode is a high quality mode or a low delay mode;
When the communication mode is identified as the high quality mode by the identification process, the delivery data decoded by the high quality decoding process is output, and when it is identified as the low delay mode. Is an output process for outputting the delivery data decoded by the low delay decoding process;
A terminal data input process in which terminal data in a streaming format indicating at least one of video and audio is input;
An encoding process for encoding the terminal data input in the terminal data input process;
A terminal data transmission process of transmitting terminal data encoded by the encoding process to the distribution server.

Images