JP2010021867A - Streaming playback apparatus, streaming distribution playback system, streaming playback method and streaming playback program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To effectively cope with delay that occurs on a transmission line, caused by network status degradation or the like during streaming distribution to which high real-time property is requested. <P>SOLUTION: A streaming signal receiving section 11 receives video/audio data and streaming data having a video presentation time stamp (PTSV) and an audio presentation time stamp (PTSA) via a network such as the Internet. A video buffer 13 and an audio buffer 14 buffer the streaming data. A video decoder 16 and an audio decoder 17 decode the encoded video/audio data. A reference clock section 18 starts counting a reference time from the first PTSV and PTSA. A control section 10 adjusts the reference time of the reference clock section 18 in accordance with a predetermined index value (e.g., the amount of stored data in the buffers) representing a communication status on the network. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a streaming reproduction apparatus, a streaming distribution reproduction system, a streaming reproduction method, and a streaming reproduction program that receive and reproduce a streaming signal composed of content data such as moving image data and audio multimedia data.

  2. Description of the Related Art Conventionally, as a technique for distributing multimedia data such as moving images and sounds, a distribution technique called streaming is known in which moving image / audio data encoded in advance is sequentially transmitted to a network, and is received and played back. It has been. A typical example of streaming distribution technology is RTP (Real-time Transport Protocol) formulated in RFC (Request for Comments) standard.

  Here, in the case of streaming distribution via a network such as the Internet, there is a known problem that a delay occurs on a transmission path due to, for example, deterioration of the communication state of the network. For this reason, when streaming signals such as moving images and audio files are distributed, the reception side smoothes the delay due to the transmission path by accumulating (buffering) a considerable amount of data before reproduction. Then, a countermeasure is taken such that the stored data is sequentially read and reproduced (decoded).

  In streaming distribution, a moving image that is actually captured may be distributed in real time. However, even for real-time streaming delivery, a relatively long time, such as several seconds to several minutes, is allowed for the delay between the actual shooting time on the transmission side and the display time on the reception side. For this reason, in the case of the real-time streaming distribution, as described above, on the receiving side, a considerable amount of data before buffering is buffered to smooth the delay due to the transmission path, and the accumulated data is sequentially reproduced. The method is adopted.

  As another method for dealing with the delay caused by the transmission path, the reception status is monitored on the reception side, the reception status is fed back to the transmission side, and the amount and contents of data to be transmitted are changed on the transmission side. Methods are also known. A typical example of such a countermeasure is RTCP (RTP Control Protocol) established by the RFC standard.

  In addition, for example, in Japanese Patent No. 3757933 (Patent Document 1), in a communication apparatus having a network device that transmits and receives packets by connecting to a network, the quality of streaming is maintained regardless of the state of the host-side control unit. The technology that has been made possible is disclosed. That is, in Patent Document 1, information data encoded on the streaming transmission side is made into a packet of a transport stream, and the packet is transmitted to the network. On the other hand, the reception side decodes the received packet. It is described that information data is reproduced.

Japanese Patent No. 3757933 (FIG. 1)

  By the way, streaming distribution is expected to become increasingly popular in the future, and its use for various purposes is considered. As an example, it is conceivable to use the remote control of the transmitting side from the receiving side while reproducing on the receiving side.

  That is, for example, the video and audio output from a remote television device, video device, or electronic game device are streamed and reproduced on the receiver side, and the remote devices are remotely controlled from the receiver side. Possible uses.

  In these applications, an allowable amount for a delay from distribution on the transmission side to reproduction on the reception side is required to be within, for example, about several hundred milliseconds to one second. In other words, in streaming distribution in this case, higher real-time characteristics, that is, low latency (short delay) and low latency (short waiting time) are required, in other words, higher interactivity is required. Become.

  Here, in such streaming delivery that requires high real-time performance, for example, if the above-described delay smoothing technique using buffering is used, the delay until the reproduction is actually performed on the receiving side is the buffer. It grows in proportion to the amount of rings. In other words, the conventional delay smoothing method using buffering is suitable for applications that require high real-time performance such that the delay tolerance from delivery on the transmission side to playback on the reception side is within several hundred milliseconds to one second. Is unsuitable.

  Further, the above-described method of feeding back the reception status to the transmission side requires time for returning the reception status to the transmission side and for the transmission side to reflect the feedback information on the transmission data. For this reason, the method of feeding back the reception status is not suitable for streaming delivery that requires high real-time performance, as in the case of the delay smoothing method using buffering.

  As described above, when streaming delivery requiring high real-time performance is performed, the existing technology cannot effectively cope with the problem that a delay occurs on the transmission path due to deterioration of the network condition. Therefore, there is a problem that when streaming delivery requiring high real-time performance is performed, if a network state is deteriorated even a little, it is impossible to reproduce a smooth moving image or sound.

  The present invention has been proposed in view of such a situation, and can effectively cope with a delay generated on a transmission path due to deterioration of a network situation or the like in streaming delivery requiring high real-time performance. An object of the present invention is to provide a streaming playback device, a streaming delivery playback system, a streaming playback method, and a streaming playback program.

  According to the present invention, a streaming signal having at least content data and its reproduction start instruction time information is received through a transmission path whose communication quality is not guaranteed, temporarily stored, and then read and reproduced. And the above-mentioned subject is solved by adjusting the reference time used in the case of the reproduction processing according to the predetermined index value showing the communication situation of a transmission line.

  That is, according to the present invention, the reference time that should not normally be varied is adjusted so as to be appropriately changed according to the communication status of the transmission path.

  In the present invention, a high real-time property is required by adjusting a reference time used for reproduction processing of a streaming signal received through a transmission line whose communication quality is not guaranteed according to a predetermined index value representing a communication state of the transmission line. In the streaming distribution, it is possible to effectively cope with the delay generated on the transmission path due to the deterioration of the network condition. In other words, in the present invention, in streaming distribution that requires reproduction with high real-time characteristics (low latency: short waiting time), it is possible to set an optimum latency according to the state of communication on the transmission path. Yes.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  In the following description, as an application example of the present invention, streaming video / audio data output from a remote video device or electronic game device is streamed, and the video / audio is transmitted to the mobile phone terminal on the receiving side. An example is given in which remote devices are remotely controlled while playing. Of course, what was mentioned by this embodiment is an example to the last, and it cannot be overemphasized that this invention is not limited to this example.

[System outline configuration]
FIG. 1 shows a schematic configuration of an entire streaming delivery / playback system according to an embodiment of the present invention.

  In FIG. 1, a streaming transmitter 32 and a streaming receiver 50 are connected through a network 40 such as a local area network (LAN), a wireless LAN standardized by 802.11b / g, or the Internet. In other words, it is assumed that the streaming transmitter 32 and the streaming receiver 50 are placed in places (remote places) separated from each other, for example, between different rooms in the same building or between separated buildings. Further, a moving image / audio data output device 31 and a television device 30 are provided on the streaming transmitter 32 side, and the moving image / audio data output device 31 is connected to the television device 30 and the streaming transmitter 32. ing.

  The moving image / sound data output device 31 is a device capable of outputting content data such as a moving image / sound, such as an electronic game device or a hard disk recorder. The moving image / sound data output device 31 is a device that is remotely controlled by a remote control signal such as infrared rays. In the case of the present embodiment, the moving image / audio data output device 31 can be remotely controlled by an infrared remote control signal emitted from a remote control signal transmitter 33 mounted on the streaming transmitter 32. .

  The television device 30 is a so-called television monitor device. When the moving image / sound data output from the moving image / sound data output device 31 is input, the moving image display and sound output are performed. .

  The streaming receiver 50 is, for example, a mobile phone terminal, and includes at least a key input unit 53, an image display unit 51, and an audio output unit 52 together with each component included in a general mobile phone terminal. The key input unit 53 includes a key operation signal generation circuit that converts a pressing operation of these keys into a pressing signal, such as a numeric keypad and a cross key for a user to input an operation. The image display unit 51 includes a display device that can display a moving image, a peripheral circuit thereof, an image processing circuit, and the like. The sound output unit 52 includes a speaker capable of outputting sound, a peripheral circuit thereof, a sound processing circuit, and the like. In addition, illustration and description of other components included in a general mobile phone terminal are omitted.

  Further, the streaming receiver 50 uses a key operation signal generated by the key input unit 53 in response to an operation by the user in accordance with a standard such as TCP / IP (Transmission Control Protocol / Internet Protocol), HTTP (HyperText Transfer Protocol), or the like. The transmission to the streaming transmitter 32 via the network 40 is possible.

  Here, as an example, the key operation signal generated by the key input unit 53 in response to an operation by the user is a remote operation signal when the moving image / audio data output device 31 is remotely controlled. That is, in this case, the remote operation signal generated by the key input unit 53 is sent to the streaming transmitter 32 via the network 40.

  Then, the streaming transmitter 32 that has received this remote operation signal causes the remote control signal transmitter 33 to transmit an infrared remote control signal corresponding to the remote operation signal to the moving image / audio data output device 31.

  In other words, according to the present embodiment, the streaming receiver 50 indirectly remotely controls the moving image / audio data output device 31 via the network 40 and the remote control signal light transmission unit 33 of the streaming transmitter 32. Made possible. That is, the user on the streaming receiver 50 side can remotely operate the moving image / audio data output device 31 on the streaming transmitter 32 side.

  The moving image / sound data output from the moving image / sound data output device 31 in response to a remote operation by the user of the streaming receiver 50 is output to the television device 30 and the streaming transmitter 32 as described above. Also sent to.

  At this time, the streaming transmitter 32 encodes the moving image / audio data according to a standard such as a so-called MPEG (Moving Picture Experts Group). In addition, the streaming transmitter 32 adds reproduction start instruction time information of each frame in accordance with a standard such as RTP for each frame of the encoded moving image / audio data. Then, the streaming transmitter 32 transmits (distributes) the encoded moving image / audio data and the streaming data including the reproduction start instruction time information to the streaming receiver 50 via the network 40 in units of predetermined packets. To do.

  When the streaming receiver 50 receives the streaming data, the streaming receiver 50 decodes the encoded moving image / audio data, and the decoded moving image / audio data at a time indicated by the reproduction start instruction time information. To play.

  As a result, the moving image sent from the moving image / sound data output device 31 is displayed on the image display unit 51 of the streaming receiver 50, and the moving image / sound data output from the sound output unit 52. The sound transmitted from the device 31 is output.

  As described above, according to the system of the present embodiment, the moving image / sound data output from the moving image / sound data output device 31 at a remote location is distributed to the streaming receiver 50 via the network 40. In the streaming receiver 50, it is possible to reproduce these moving images / sounds. At the same time, according to the system of this embodiment, by sending a remote operation signal from the key input unit 53 to the streaming transmitter 32 side via the network 40, the streaming receiver 50 can move the moving image / sound at a remote location. The data output device 31 can be remotely controlled. That is, according to the present embodiment, the streaming receiver 50 remotely controls the moving image / audio data output device 31 on the streaming transmitter 32 side in the remote location and is output from the device 31 according to the remote control. Thus, the moving image and sound data distributed from the streaming transmitter 32 can be reproduced.

[Time delay from remote operation to video / audio playback and video / audio playback quality]
By the way, as the remote operation by the user on the streaming receiver 50 side, for example, operations such as playback / stop / pause / fast-forward / fast-reverse of a video program recorded in a hard disk recorder, or a game character in an electronic game device Is assumed to be

  Then, in the case where streaming distribution and playback of moving image / audio data corresponding to these remote operations are performed, the latency from the input of the remote operation to the playback of moving images / sounds ( It is desirable to minimize (latency). That is, when streaming video / audio data according to such remote operation, the delay amount from remote operation input to video / audio reproduction is, for example, within a few hundred milliseconds to within one second. High real-time property (low delay) and low latency (short waiting time), in other words, high interactivity is required.

  Here, on the streaming transmitter 32 side, almost all of the output processing of moving image / sound data corresponding to the remote operation signal, the encoding of the moving image / sound data, and the sending processing to the network 40 are hard. It is done by wear. For this reason, moving image / audio data is output in response to the remote operation signal, and the time until the moving image / audio data is transmitted to the network 40 is about several tens of milliseconds to several hundreds of milliseconds. Conceivable. In addition, on the streaming receiver 50 side, almost all of the reception processing of moving image / sound data from the network 40 and the decoding / reproduction processing of these moving images / sounds are performed by hardware. For this reason, it is considered that the time from the reception and reproduction of the moving image / audio data is about several tens of milliseconds to several hundreds of milliseconds.

  On the other hand, the network 40 connecting the streaming transmitter 32 and the streaming receiver 50 is the Internet, a LAN, a wireless LAN, and the like, and QoS (network quality) is generally not guaranteed. Is. For this reason, on the transmission path of the network 40, there is a possibility that, for example, packet loss or delay may occur.

  And if these omissions or delays occur, if you play the video and audio without taking any action against them, the video or audio will be lost or the audio will be interrupted. Will be played.

  For this reason, in general, in order to prevent the deterioration of the moving image / sound quality due to the above-described omission or delay, on the streaming reception side, a certain amount of received data that arrives irregularly from the network, for example, several seconds to several minutes Minutes of data are always buffered. In other words, the streaming receiving side is configured to send the data after smoothing for the above-mentioned omissions and delays by the buffering to a moving image / sound reproduction circuit.

  However, if the received data is buffered for a long time of several seconds to several minutes as described above, the above-described high real-time property (low delay) cannot be realized.

  For this reason, in the present embodiment, at the same time as the realization of streaming that requires a high real-time property (low delay), for example, within a few hundred milliseconds to one second, the allowable amount for delay from distribution to reproduction is simultaneously achieved in the network. Even if the situation deteriorates, adjustments are made so that smooth video / audio reproduction is possible with a minimum delay. In other words, in the present embodiment, the delay is minimized when the network communication state is good, while the minimum delay is within a range where smooth video / audio reproduction is possible even when the network communication state is bad. Make adjustments so that As a result, in the present embodiment, it is possible to reproduce the moving image / sound with the best quality in the situation at that time, regardless of whether the communication state of the network is good or bad.

[Detailed configuration of streaming receiver]
FIG. 2 shows a detailed configuration example of the streaming receiver 50 according to the embodiment of the present invention, which can maintain the reproduction quality and adjust the delay as described above. 2 is the key input unit 53 in FIG. 1, the image display unit 19 in FIG. 2 is the image display unit 51 in FIG. 1, and the audio output unit 20 in FIG. 2 is the audio output unit in FIG. 52. FIG. 3 shows a schematic data structure of video data and audio to be distributed in streaming.

  In FIG. 2, the key input unit 22 includes a key device and a key operation signal generation circuit for a user on the streaming receiver 50 side to input an operation. When the user performs, for example, the above-described remote operation through the key input unit 22, the remote operation signal is transmitted from the key signal transmission unit 21 to the network 40 and transmitted to the streaming transmitter 32 side.

  The streaming signal receiver 11 of the streaming receiver 50 according to the present embodiment receives streaming data distributed from the streaming transmitter 32 via the network 40.

  The streaming data received by the streaming signal receiving unit 11 is data having video data and audio data as shown in FIG. The video data is data in which encoded video data for each frame is connected to PTSV (video presentation time stamp) corresponding to the one frame. Similarly, the audio data is data in which encoded audio data for each frame is connected to PTSA (audio presentation time stamp) corresponding to the one frame. The streaming data is distributed in units of video packets composed of a plurality of frames of the video data and audio packets composed of a plurality of frames of the audio data. Therefore, the streaming signal receiving unit 11 receives streaming data in units of packets. Note that PTSV is a reproduction start instruction time attached to each frame as a reproduction unit of encoded moving image data, and reproduction (display) of the moving image of the frame is started at that time. This is information that indicates what should be done. Similarly, PTSA is a playback start instruction time attached to each frame which is a playback unit of encoded audio data, and playback (output) of the audio of the frame should be started at that time. Is information for instructing.

  When the streaming signal receiving unit 11 receives the streaming data in units of packets, the streaming signal receiving unit 11 separates video data and audio data for each frame from the streaming data of the received packets. Then, the streaming signal receiving unit 11 outputs the video data of each frame to the video buffer 13 and the PTSV extracting unit 12, and outputs the audio data of each frame to the audio buffer 14 and the PTSA extracting unit 15.

  The PTSV extraction unit 12 extracts PTSV data of each frame from the input video data, and outputs the extracted PTSV data to the control unit 10. The PTSA extraction unit 15 extracts PTSA data of each frame from the input audio data, and outputs the extracted PTSA data to the control unit 10.

  The video buffer 13 accumulates video data under the control of the control unit 10 and outputs the video data in units of frames in response to a read request from the video decoder 16 at the subsequent stage. Further, the video buffer 13 always reports the data storage amount to the control unit 10. Similarly, the audio buffer 14 accumulates audio data under the control of the control unit 10 and outputs the audio data in frame units in response to a read request from the audio decoder 17 at the subsequent stage. The audio buffer 14 always reports the data accumulation amount to the control unit 10.

  The control unit 10 sets the initial time value of the reference clock (STC) unit 18 based on the PTSV of the video data and the PTSA of the audio data that have arrived first as streaming data. Moreover, although mentioned later for details, the control part 10 performs the process which adjusts the reference | standard time of the reference | standard timepiece part 18 based on a predetermined | prescribed index value with the streaming reproduction program concerning this invention.

  When the time initial value is set, the reference clock unit 18 starts to count up the reference time when the video decoder 16 and the audio decoder 17 reproduce each frame data at the PTSV and PTSA reproduction start instruction time, respectively. To do. Then, the reference clock unit 18 sends information on the reference time to the video decoder 16 and the audio decoder 17.

  The video decoder 16 stores the video data read from the video buffer 13 in the internal buffer, and when the reproduction start time indicated by the PTSV of the stored video data matches the reference time of the reference clock unit 18, The frame data of the PTSV is decoded so as to be displayed on the display screen of the image display unit 19. Similarly, the audio decoder 17 stores the audio data read from the audio buffer 14 in the internal buffer, and the reproduction start time indicated by the PTSA of the stored audio data matches the reference time of the reference clock unit 18. The data of the PTSA frame is decoded so that it is sometimes output from the speaker of the audio output unit 20.

  The moving image data decoded by the video decoder 16 is sent to the image display unit 19 and displayed on the display screen of the image display unit 19. The audio data decoded by the audio decoder 17 is sent to the audio output unit 20 and output from the speaker of the audio output unit 20.

  Here, for example, when the communication status of the network 40 is good and the video data and audio data packets arrive at the streaming signal receiving unit 11 in the correct order, the decoding of the video buffer 16 and the audio buffer 17 is performed smoothly. It will be. In this case, since the video decoder 16 can sequentially read and decode the video data from the video buffer 13, the amount of data stored in the video buffer 13 does not increase beyond a certain level. That is, since the input video data is output immediately in the video buffer 13, the data storage amount does not increase. Similarly, since the audio decoder 17 can sequentially read out audio data from the audio buffer 14, the amount of data stored in the audio buffer 14 does not increase beyond a certain level. That is, since the input audio data is output immediately in the audio buffer 14, the data storage amount does not increase.

  On the other hand, for example, when the communication status of the network 40 is poor and the video data and audio data packets do not arrive at the streaming signal receiving unit 11 in the correct order, the video decoder 16 and the audio decoder 17 perform the decoding smoothly. I will not be broken. In this case, the amount of data stored in the video buffer 13 and the audio buffer 14 gradually increases. That is, for example, when a packet of video data that should originally arrive in order arrives before a packet that should arrive first, the video buffer 13 receives a packet that should arrive first. Until then, data of packets that should arrive after the above will continue to be accumulated. For this reason, the amount of data stored in the video buffer 13 increases. The same applies to the audio buffer 14, and if the packet of audio data that should arrive later is arrived before the packet that should arrive first, the audio buffer 14 arrives first. Until the packet to be received arrives, the data of the packet that should arrive later will continue to be accumulated. In this case, the amount of data stored in the audio buffer 14 increases.

  As described above, when the data storage amount of the video buffer 13 and the audio buffer 14 increases due to the deterioration of the communication status of the network 40, the control unit 10 controls the reference clock unit 18 to perform the reference. Delay time.

  Thus, when the reference time is delayed, the video decoder 16 delays the playback timing of the decoded moving image. Similarly, the playback timing of the audio decoded by the audio decoder 17 is also delayed. When the video / audio playback timings in the video decoder 16 and the audio decoder 17 are thus delayed, the streaming receiver 50 can wait for more time for data from the network 40. That is, in this case, although the delay in moving image / sound reproduction increases to some extent, it becomes possible to wait for a packet that has arrived late, and as a result, more packets can be reproduced. .

  Thereafter, when the situation of the network 40 is improved again and the reproduction of moving images / sounds proceeds smoothly, the data storage amount of the video buffer 13 and the audio buffer 14 decreases. When the data storage amount decreases in this way, the control unit 10 controls the reference clock unit 18 to advance the reference time.

  As the reference time advances in this way, the video decoder 16 and the audio decoder 17 each accelerate the playback timing of the decoded moving image / sound. When the video / audio playback timing in the video decoder 16 and the audio decoder 17 is accelerated, the delay in video / audio playback is eliminated.

  According to the embodiment of the present invention, as described above, an optimal delay amount can be set by adjusting the reference time according to whether the communication status of the network is good or bad. In addition, according to the embodiment of the present invention, when the communication condition of the network is good, a higher real-time property (low delay) is required such that the allowable delay from distribution to reproduction is within several hundred milliseconds to one second. Streaming can be realized. On the other hand, according to the embodiment of the present invention, it is possible to obtain moving image / audio data with good reproduction quality with a minimum necessary delay even when the communication status of the network is poor.

[Reference time adjustment process flow]
FIG. 4 shows a schematic flow of processing when the control unit 10 controls the reference clock unit 18 by executing the streaming playback program according to the present invention.

  In FIG. 4, when the PTSV and PTSA of the first streaming data are supplied from the PTSV extraction unit 12 and the PTSA extraction unit 15, the control unit 10 uses the PTSV and PTSA to process the time of the reference clock unit 18 as a process of step S1. Set the initial value.

  Thereafter, the control unit 10 monitors the communication status of the network by using the data accumulation amount supplied from the video buffer 13 and the audio buffer 14 as a predetermined index value as the process of step S2.

  In addition, the control unit 10 determines whether the network communication status is good or bad as the process of step S3. That is, for example, when the decoding process of the video decoder 16 and the audio decoder 17 proceeds smoothly and the data accumulation amount of the video buffer 13 and the audio buffer 14 is smaller than a predetermined threshold amount, for example, If it is determined that the communication status of the network is good, the process proceeds to step S4. On the other hand, when the data accumulation amount of the video buffer 13 and the audio buffer 14 exceeds a predetermined threshold amount, the control unit 10 determines that the network communication state is bad and advances the process to step S5.

  When it is determined that the communication status of the network is good and the process proceeds to step S4, the control unit 10 determines whether or not the reference time of the reference clock unit 18 is the original reference time that has not been changed. If it is determined that the reference time is the original reference time, the control unit 10 returns the process to step S2. On the other hand, when determining that the reference time has been changed, the control unit 10 advances the processing to step S6.

  When the process proceeds to step S6, the control unit 10 controls the reference clock unit 18 to return to the original reference time so as to advance the reference time when the reference time has been changed to be delayed. . Note that the playback processing when the delayed reference time is returned to the original reference time may be performed, for example, so-called fast-forward playback, or so-called frame skip playback. It may be broken. Then, when the reference time reaches the original reference time, the control unit 10 returns the process to step S2.

  If it is determined in step S3 that the network communication status is poor and the process proceeds to step S5, the control unit 10 adjusts the reference clock unit 18 to delay the reference time, and then proceeds to step S2. return. Note that the playback processing when the fast reference time is returned to the original reference time may be performed, for example, so-called slow playback, or so-called frame advance processing. May be.

  When the control unit 10 executes the processing of the flowchart of FIG. 4, the adjustment of the reference time according to the quality of the network communication state as described above is realized.

[Examples of other index values]
In the above-described embodiment, the data accumulation amount of the buffer is cited as an index value when the reference clock is adjusted by judging whether the communication status of the network is good or bad. In the present invention, the index value is the data accumulation amount. It is not limited.

  As another index value, for example, information indicating the reception status of packets from the network can be cited. Here, as information indicating the reception status of packets, for example, the number of missing packets within a certain fixed time, the number of packets whose order has been changed within a certain fixed time, and each packet arriving at the streaming receiving side The amount of fluctuation in the arrival time interval can be listed.

  That is, when the number of missing packets within a certain time is used as an index value, the streaming signal receiving unit 11 counts the number of missing packets within the certain time as an example. Then, the count information of the number of missing packets within the certain time is reported to the control unit 10. The control unit 10 determines that the communication status of the network is good when the number of missing packets within the predetermined time is less than (or almost absent) a predetermined threshold number, for example. If the threshold number is exceeded (or a missing packet occurs), it is determined that the network communication status is bad. Note that the processing after determining whether the communication status of the network is good or not is the same as described above, and the description thereof is omitted.

  For example, when the number of packets whose order has been changed within a certain time is used as an index value, the streaming signal reception unit 11 counts the number of packets that have been exchanged within the certain time as an example. Then, the count information of the number of switched packets within the predetermined time is reported to the control unit 10. The control unit 10 determines that the communication status of the network is good when, for example, the number of switched packets within the predetermined time is less than (or almost absent) a predetermined threshold number, while the number of switched packets is equal to the threshold value. When the number is exceeded (or a replacement packet is generated), it is determined that the network communication status is bad.

  Further, for example, when the fluctuation amount of the arrival time interval of each arrived packet is used as the index value, the streaming signal reception unit 11 measures the fluctuation amount of the arrival time interval as an example. Then, information on the fluctuation amount of the arrival time interval is reported to the control unit 10. The control unit 10 determines that the communication status of the network is good when the fluctuation amount of the arrival time interval is less than (or hardly), for example, a predetermined threshold amount, while the fluctuation amount of the arrival time interval is When the threshold amount is exceeded (or fluctuation occurs), it is determined that the network communication status is bad.

[Select index value]
In the present invention, the index values as described above may have a plurality of values, or a plurality of types of index values themselves. The streaming receiver may appropriately select which of a plurality of index values or which of a plurality of types of index values is used. In addition, it is possible to determine which index value of a plurality of values or a plurality of types of index values is selected in accordance with, for example, the type of content data to be streamed.

  In FIG. 5, as an example of the type of content data, video and audio data of an electronic game, playback moving image and audio data of a hard disk recorder, and other data (standard) are listed. In FIG. 5, the control unit 10 controls the reference time of the reference clock unit 18 according to the type of each content data, and the control policy is selected according to the type of content data. An example of the index value to be set is given. Needless to say, FIG. 5 is merely an example.

  In FIG. 5, when the type of content data represents video and audio data of an electronic game, for example, the control unit 10 allows the video game and sound of the electronic game to be interrupted to some extent, while regarding latency. Select and set index values to minimize. That is, in this example, as an example, the control unit 10 uses the buffer data accumulation amount as the index value, and sets the data accumulation amount as the index value to a small value.

  In FIG. 5, when the type of content data represents, for example, playback moving image and audio data of a hard disk recorder, the control unit 10 provides a sufficient amount of data stored in the buffer, and the latency increases to some extent. However, the index value is selected and set so that the reproduced moving image and the sound are not interrupted. That is, in this case, as an example, the control unit 10 uses the buffer data accumulation amount and the reception status of the packet as an index value, sets the buffer data accumulation amount to a large value, and changes the number of missing packets and the packet order. The index value indicating the fluctuation amount of the packet arrival time interval is also set to such a value that it decreases.

  In addition, in FIG. 5, when the type of content data represents standard data, the control unit 10 selects and sets an index value so as to reduce latency as much as possible within a range in which video and audio are not interrupted. That is, in this example, as an example, the control unit 10 uses the buffer data accumulation amount as the index value, and uses the data accumulation amount as the index value as the video and audio of the electronic game and the playback moving image of the hard disk recorder. For example, it is set to a value that is medium compared to the case of audio content.

[Flow of index value selection / setting and reference time adjustment processing according to content type]
FIG. 6 shows an index value corresponding to the type of content data as described above as another embodiment of the process when the control unit 10 controls the reference clock unit 18 by executing the streaming playback program according to the present invention. A schematic flow of a reference time adjustment process when selecting and setting is shown.

  In FIG. 6, when the PTSV and PTSA of the first streaming data are supplied from the PTSV extraction unit 12 and the PTSA extraction unit 15, the control unit 10 uses the PTSV and PTSA to process the time of the reference clock unit 18 as a process of step S10. Set the initial value.

  Next, the control unit 10 determines the type of content data of the streaming data received by the streaming signal receiving unit 11 as the process of step S11. Note that the content data type determination in the control unit 11 is performed using, for example, information indicating the type of content data in the streaming data. For example, when the target device when the streaming receiver 50 is remotely operated is known, the control unit 10 can also determine the type of content data by the remote target device.

  Then, when the type of content data is determined in step S11, the control unit 10 selects and sets an index value corresponding to the type of content data as processing in step S12.

  Next, the control part 10 monitors the communication state of a network with the parameter | index value set in step S12 as a process of step S13.

  Further, the control unit 10 determines whether the network communication status is good or bad as the process of step S14. When the control unit 10 determines that the network communication status is good, the control unit 10 proceeds to step S15. When the control unit 10 determines that the network communication status is poor, the control unit 10 proceeds to step S16.

  When it is determined that the communication status of the network is good and the process proceeds to step S15, the control unit 10 determines whether the reference time of the reference clock unit 18 is the original reference time. The process returns to S13, while if the reference time has been changed, the process proceeds to step S17.

  When the process proceeds to step S17, the control unit 10 controls the reference clock unit 18 so as to advance the reference time to return to the original reference time when the reference time has been changed to be delayed. Then, when the reference time reaches the original reference time, the control unit 10 returns the process to step S13.

  If it is determined in step S14 that the network communication status is bad and the process proceeds to step S16, the control unit 10 controls the reference clock unit 18 to delay the reference time, and then proceeds to step S13. return.

  The control unit 10 executes the processing of the flowchart of FIG. 6 to select and set the index value according to the content type as described above, and the communication status of the network determined using the index value is good or bad. Adjustment of the reference time in accordance with is realized.

[Summary]
As described above, according to the embodiment of the present invention, it is possible to effectively cope with the delay generated on the transmission path due to the deterioration of the network condition or the like in the streaming delivery requiring high real-time property. Yes.

  In the embodiment of the present invention, as described above, by operating the reference clock in the streaming receiver 50, the quality of the reproduced data can be optimized, and streaming that requires high real-time performance is required. It is also possible to respond to. Here, the concept of the reference clock is a concept used in the reproduction of almost all media including so-called MPEG, and typical examples of its use include storage media such as DVDs and the ground. Broadcast media such as digital wave broadcasting, streaming distribution, etc. can be mentioned. However, there is no idea of changing the reference clock in them, and this is because changing the reference clock directly brings a result of skipping images and sounds. However, in recent years, there has been a demand for less delay than video and audio skipping, such as applications that remotely control video games and audio from electronic game devices and home appliances. Applications are emerging. Therefore, in the embodiment of the present invention, comprehensive and effective use of an application for controlling the delay by intentionally shifting the reference clock as described above, thereby remotely operating the electronic game device or the home appliance. Is possible. In addition, as in the present embodiment, when remote control of an electronic game device, a home appliance, or the like is realized by a mobile phone terminal, the mobile phone terminal is used even when outdoors or on the move. It is assumed that the communication quality of the network tends to deteriorate. In the embodiment of the present invention, it is possible to achieve both low latency and good reproduction quality even in such usage.

  In addition, the above description of the embodiment of the present invention is an example of the present invention. For this reason, the present invention is not limited to the above-described embodiment, and it is needless to say that various modifications can be made according to the design or the like as long as the technical idea according to the present invention is not deviated.

It is a block diagram which shows schematic structure of the whole streaming delivery reproduction | regeneration system of this invention embodiment. It is a block circuit diagram which shows the detailed structural example of the streaming receiver of embodiment of this invention. FIG. 3 is a schematic data structure diagram of video data and audio to be distributed in a streaming manner. It is a flowchart which shows the schematic flow of a process when a control part controls a reference | standard timepiece part by execution of the streaming reproduction | regeneration program concerning this invention. An example of content types, an example of a control policy for adjusting the latency by controlling the reference time for each content type, and an example of an index value selected and set according to the content type FIG. 12 is a flowchart showing a schematic flow of a reference time adjustment process when an index value is selected and set according to the type of content as another embodiment of the process when the control unit controls the reference clock unit.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 Control part, 11 Streaming signal receiving part, 12 PTSV extraction part, 13 Video buffer, 14 Audio buffer, 15 PTSA extraction part, 16 Video decoder, 17 Audio decoder, 18 Reference | standard clock part, 19, 51 Image display part, 20, 52 Audio output unit, 21 Key signal transmission unit, 22, 53 Key input unit 30 Television device, 31 Video / audio data output device, 32 Streaming transmitter, 33 Remote control signal transmission unit, 40 Network, 50 Streaming reception Machine

Claims (14)

A streaming signal receiving unit that receives a streaming signal having at least content data and reproduction start instruction time information of the content data through a transmission path in which communication quality is not guaranteed;
A buffer unit for temporarily storing the streaming signal received by the streaming signal receiving unit;
A signal reproduction processing unit that reads the streaming signal temporarily stored in the buffer unit and performs reproduction processing of the streaming signal;
A reference clock unit for generating a reference time when the signal reproduction processing unit reproduces the streaming signal;
A control unit for obtaining a predetermined index value representing a communication status of the transmission path connected to the streaming signal receiving unit, and adjusting a reference time of the reference clock unit according to the index value;
A streaming playback device.   The streaming reproduction apparatus according to claim 1, wherein the control unit adjusts the reference time of the reference clock unit to be delayed when a predetermined index value indicating that the communication state of the transmission path has deteriorated is obtained.   3. The streaming playback apparatus according to claim 2, wherein the control unit adjusts the reference time of the reference clock unit to be restored when the communication state of the transmission path is recovered after delaying the reference time of the reference clock unit. .   4. The streaming playback apparatus according to claim 1, wherein the control unit uses the data accumulation amount of the buffer unit as the predetermined index value.   The streaming playback device according to claim 1, wherein the control unit uses the number of missing signals of the streaming signal received by the streaming signal receiving unit as the predetermined index value.   4. The streaming playback apparatus according to claim 1, wherein the control unit uses the predetermined number of changes in the order of the streaming signals received by the streaming signal receiving unit.   4. The streaming playback apparatus according to claim 1, wherein the control unit uses a fluctuation amount of a signal reception interval of the streaming signal received by the streaming signal receiving unit as the predetermined index value.   4. The streaming playback apparatus according to claim 1, wherein the control unit selects the predetermined index value from a plurality of index values according to the type of content data of the streaming signal.   4. The streaming playback apparatus according to claim 1, wherein the control unit sets the predetermined index value according to a type of content data of the streaming signal. A streaming transmission device that distributes a streaming signal having at least content data and reproduction start instruction time information of the content data through a transmission path whose communication quality is not guaranteed in response to a remote operation signal from the streaming reception device;
Transmitting a remote operation signal to the streaming transmission device, receiving a streaming signal distributed through the transmission path in response to the transmission of the remote operation signal, playing back the received streaming signal based on a reference time, A streaming reception device that obtains a predetermined index value representing the communication status of the transmission path and adjusts the reference time according to the index value;
Streaming delivery playback system consisting of. The streaming transmission device distributes information representing the type of content data included in the streaming signal,
The streaming delivery / playback system according to claim 10, wherein the streaming receiving apparatus selects the predetermined index value from a plurality of index values based on information representing the type of content data. The streaming transmission device distributes information representing the type of content data included in the streaming signal,
The streaming delivery / playback system according to claim 10, wherein the streaming receiving apparatus sets the predetermined index value based on information representing a type of the content data. A streaming signal receiving unit receiving at least a streaming signal having content data and reproduction start instruction time information of the content data through a transmission path whose communication quality is not guaranteed;
Temporarily storing the streaming signal received by the streaming signal receiving unit in the buffer unit;
A step in which a signal reproduction processing unit reads out the streaming signal temporarily stored in the buffer unit and performs reproduction processing of the streaming signal;
A step in which a reference clock unit generates a reference time when the signal reproduction processing unit reproduces the streaming signal;
The control unit obtains a predetermined index value representing the communication status of the transmission path connected to the streaming signal receiving unit, and adjusts the reference time of the reference clock unit according to the index value;
A streaming playback method. A streaming signal having at least content data and reproduction start instruction time information of the content data is received through a transmission path whose communication quality is not guaranteed, and the received streaming signal is reproduced by a signal reproduction processing circuit based on a reference time. When
As a control unit that obtains a predetermined index value representing the communication status of the transmission path and adjusts the reference time according to the index value,
A streaming playback program that allows a computer to function.

Images