JP2007311902A - Stream receiving and reproducing device and streaming network system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a stream receiving and reproducing device and a streaming network system capable of reducing a frequency of occurrence of an underflow of a stream data receiving buffer even in the system wherein it is impossible to control a rate of a stream transmitted by a transmitter side of stream data. <P>SOLUTION: The stream receiving and reproducing device includes: a terminal communication section 110 for receiving stream data and control data distributed via a network 103; a buffer 111 for temporarily storing received stream data; a decoder 112 for decoding the stored stream data; and a terminal control section 113. The terminal control section 113 receives control information associated with the stream data from the terminal communication section 110, temporarily stops decoding the stream data by controlling the decoder 112 so as to increase a data amount of the stream data stored in the buffer 111 when a state of the stream data indicates that the stream data can temporarily be stopped according to the received control information and the terminal control section 113 discriminates that it is required to increase the storage amount of the stream data. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a stream reception / playback apparatus and a streaming network system that use streaming technology for playing back data while receiving data such as audio and moving images via a network.

  The streaming technology is a technology for delivering data that requires real-time performance such as voice and moving images via a network and reproducing the data while delivering the data.

Stream receiving / playback devices using streaming technology can play all data without downloading it,
(1) A large-capacity storage device or the like is not required on the receiving side.

(2) Compared with the method of downloading and reproducing all data, it is possible to shorten the time required from the start of data delivery to the start of data reproduction and increase the data reproduction speed.
There is a feature.

  However, in a stream reception / playback apparatus using streaming technology, a stream data reception buffer may be used due to a communication error on the network or a deterioration in radio wave conditions when data is distributed via a wireless network. There is a problem that underflow occurs.

  When such an underflow of the stream data reception buffer occurs, problems such as audible noise occur.

  As a technique for solving this problem, for example, there is a “transmission system” shown in Patent Document 1. FIG. 5 is a block diagram showing the configuration of the transmission system described in Patent Document 1. As shown in FIG.

  In FIG. 5, the server 30 is connected to the client terminal 20 via the network 10.

  The server 30 stores a plurality of multiplexed files (stream data) having different multiplexing rates obtained by multiplexing images and sound.

  The multiplexed data sent from the server 30 is first buffered by the input buffer 21 of the client terminal 20 and passed to the demultiplexing unit 22.

  The demultiplexing unit 22 separates the data stream into image data and audio data, and sends the data to the image decoding unit 23 and the audio decoding unit 24.

  The image decoding unit 23 decodes the image data and sends the decoded data to the image buffer 25.

  The audio decoding unit 24 decodes the audio data and sends the decoded data to the audio buffer 26.

  The image buffer 25 and the audio buffer 26 play a role of absorbing the decoding time difference between the image decoding unit 23 and the audio decoding unit 24 and synchronizing the image and the audio.

  The synchronized image data and sound data are output as an image and sound from the display 27 and the speaker 28, respectively.

  The rate calculation unit 29 always counts the amount of data within a unit time transmitted from the server 30 to the input buffer 21 via the network 10, for example, calculates the actual transmission rate, and sends the transmission rate information to the server 30. Notify Such notification is performed, for example, every predetermined time.

  The server 30 includes multiplexed files (stream data) 31 to 33 with different encoding rates obtained by multiplexing images and audio, and a multiplexed file changeover switch 34 that switches files according to transmission rate information transmitted from the client terminal 20 side. And comprising.

  The multiplexed files 31 to 33 store the same content with different encoding rates, that is, the same data with different encoding rates. For example, the encoding rate is set to 60 kbps, 45 kbps, or 30 kbps.

  When streaming playback is started in this transmission system, the server 30 starts transmitting moving images to the client terminal 20.

  The client terminal 20 starts the calculation of the actual transmission rate (bps) in the rate calculation unit 29 while starting the buffering of the multiplexed data in the input buffer 21. This substantial transmission rate tends to decrease when the load on the server 30 increases or the network 10 becomes congested. This transmission rate information is notified to the server 30 at regular time intervals, for example.

  The server 30 selects multiplexed files 31 to 33 that are close to or less than the rate notified by the multiplexed file changeover switch 34 based on the transmission rate information notified from the client terminal 20 at regular time intervals. Switch.

  As described above, in the “transmission system” of Patent Document 1, when the actual reception rate at the client terminal 20 changes due to congestion of the network 10 or an increase in the load on the server 30, the client is periodically transmitted. Based on the transmission rate information sent from the terminal 20 to the server 30, the file is switched to the closest file below the transmission rate.

As a result, in this “transmission system”, even when the actual transmission rate fluctuates on the client terminal 20 side, seamless playback can be performed without interruption in streaming playback, and the difficulty in viewing moving images is eliminated. Is done.
JP-A-11-205390

  However, the conventional configuration has a problem that the stream data transmission side (server) can be applied only to a system capable of transmitting multiplexed files (stream data) having different encoding rates.

  In the conventional configuration, the stream data reception side (client terminal) notifies the stream data transmission side (server) of the rate information, which is the actual reception rate of the stream data, and the stream data transmission side (server) ) Has a problem that it can be applied only to a system capable of controlling the rate of stream distribution based on the rate information.

  The present invention has been made in view of such a point, and the stream data transmission side cannot transmit stream data having different encoding rates, or the stream data based on the rate information notified by the stream data reception side. To provide a stream reception / playback apparatus and a streaming network system capable of reducing the frequency of occurrence of underflow in a stream data reception buffer even in a system in which the transmission side cannot control the stream delivery rate. Objective.

  In order to solve this problem, the stream reception / playback apparatus of the present invention includes a communication unit that receives distributed stream data and control data, a storage unit that temporarily stores the stream data received by the communication unit, The state of the stream data can be paused by the decoding means for decoding the stream data stored by the storage means, and the control information related to the stream data received by the communication means, and the amount of the stream data stored Control means for controlling the decoding means to temporarily stop the decoding of the stream data and increasing the data amount of the stream data accumulated in the accumulating means when it is determined that there is a need to increase The structure to comprise is taken.

  According to the stream reception / playback apparatus and the streaming network system of the present invention, the stream data transmission side cannot transmit stream data having different encoding rates, or stream data based on the rate information notified by the stream data reception side. Even in a system in which it is impossible for the transmission side to control the stream delivery rate, the frequency of occurrence of underflow in the stream data reception buffer can be reduced.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram showing a configuration of a streaming network system according to an embodiment of the present invention.

  As shown in FIG. 1, the streaming network system 100 of this example includes a stream reception / playback apparatus (client) 101, a server 102, and a network 103. In FIG. 1, the network 103 exemplifies a wireless network in this example, but is not limited thereto, and may be a wired network.

  The stream reception / playback apparatus 101 includes a terminal communication unit 110, a buffer 111, a decoder 112, and a terminal control unit 113.

  The server 102 includes a server control unit 120, a stream data transmission unit 121, a server communication unit 122, and a stream data storage unit 123.

  First, the configuration of the stream reception / playback apparatus 101 as a client will be described.

  In FIG. 1, the terminal communication unit 110 receives stream data via the network 103 and outputs the received stream data to the buffer 111.

  A buffer 111 serving as a storage unit on the client side temporarily stores stream data input from the terminal communication unit 110. Then, the buffer 111 outputs the accumulated stream data to the decoder 112.

  The decoder 112 decodes the stream data input from the buffer 111 and outputs it to the speaker 114.

  The terminal control unit 113 controls the decoder 112 based on the control information regarding the stream data received from the terminal communication unit 110 and the state of the buffer 111. Specifically, when the terminal control unit 113 determines that the stream data state can be temporarily stopped and the accumulation amount of the stream data needs to be increased, the terminal control unit 113 controls the decoder 112 to decode the stream data. Is temporarily stopped, and the amount of stream data accumulated in the buffer 111 is controlled to be increased.

  The speaker 114 outputs music or the like represented by the stream data input from the decoder 112 to the outside.

  Next, the configuration of the server 102 will be described.

  The stream data storage unit 123 stores stream data to be distributed and control information related to the stream data, for example, song information such as a song name, an album name, an artist name, and a song length, and sends the stream data to the stream data. Output to the unit 121.

  The stream data sending unit 121 outputs the stream data to the server communication unit 122 based on the control of the server control unit 120.

  The server control unit 120 controls the stream data transmission unit 121. In addition, the server control unit 120 reads control information from the stream data storage unit 123 and outputs the read control information to the server communication unit 122.

  The server communication unit 122 transmits the stream data input from the stream data transmission unit 121 and the control information input from the server control unit 120 to the stream reception / playback apparatus 101 via the network 103.

  Next, the operation of the streaming network system 100 configured as described above will be described with reference to FIG. FIG. 2 is a flowchart showing the operation of the streaming network system according to the embodiment of the present invention.

  In the basic operation of normal streaming playback, the stream reception / playback apparatus 101 starts reception of stream data via the network 103 and stores the received stream data in the buffer 111 within a certain time (for example, Tini) from the start of reception.

  Note that the stream reception / playback apparatus 101 is not limited to the case where the stream data received within a certain time is stored in the buffer 111, but the stream reception / playback apparatus 101 may be configured to store a certain amount of stream data in the buffer 111. good.

  The terminal control unit 113 instructs the decoder 112 to start decoding the stream data when the fixed time Tini elapses or when a certain amount of stream data is accumulated in the buffer 111.

  In response to an instruction from the terminal control unit 113, the decoder 112 starts decoding the stream data stored in the buffer 111, and provides the decoded stream data to the speaker 114.

  In this way, the stream reception / playback apparatus 101 plays back the stream data, and the music represented by the received stream data starts to sound from the speaker 114.

  In the subsequent stream data playback state, stream data is received (step ST201), and the received stream data is stored in the buffer 111 (step ST202).

  Terminal control section 113 determines whether or not control information has been received during reception of stream data (step ST203).

  And in step ST203, when it determines with the terminal control part 113 not receiving control information, the process of step ST201-step ST203 is repeated.

  On the other hand, if it is determined in step ST203 that the terminal control unit 113 has received control information, it is determined whether or not the state of the stream data can be paused (step ST204).

  If it is determined in step ST204 that the state of the stream data cannot be paused, the processes in steps ST201 to ST204 are repeated.

  On the other hand, if it is determined in step ST204 that the state of the stream data can be paused, the stream data is streamed based on the amount of stream data accumulated in the buffer 111 (hereinafter referred to as “remaining buffer amount”). It is determined whether or not there is a need to increase the amount of accumulated data (step ST205).

  If it is determined in step ST205 that there is no need to increase the amount of accumulated stream data, the processes in steps ST201 to ST205 are repeated.

  On the other hand, if it is determined in step ST205 that there is a need to increase the amount of accumulated stream data, terminal control section 113 controls decoder 112 to suspend decoding of stream data (step ST206).

  Next, when stream data is received in a state where decoding of the stream data is paused (step ST207), the stream data is accumulated in the buffer 111 (step ST208), and it is necessary to increase the accumulation amount of the stream data again. It is determined whether or not (step ST209).

  If it is determined in step ST209 that there is a need to increase the accumulated amount of stream data, the processes in steps ST207 to ST209 are repeated.

  On the other hand, if it is determined in step ST209 that there is no need to increase the amount of accumulated stream data, the decoding of the stream data is resumed (step ST210), the process returns to step ST201, and the processes from step ST201 to step ST209 are repeated. .

  Next, a method for determining whether or not the state of the stream data can be paused in step ST204 will be described with a specific example.

  Here, as a specific example, the control information that the terminal control unit 113 determines whether or not there is reception in step ST203 includes time information such as a song name, album name, artist name, and song length, and a song segment indicating a song segment. A case where song information such as information, track number information of songs, and song status information indicating a status between songs is included will be described.

  As described above, when the control information includes song information, for example, song segment information, terminal control is performed so as to determine that the state of the stream data can be paused when the stream data is at the song segment. By configuring the unit 113, it is possible to determine whether or not the state of the stream data can be paused.

  Also, if the control information includes information notifying the change of the track number of the song, it is determined that the stream data state can be paused when the stream data is in a state between the songs. By configuring the terminal control unit 113 to do so, it is possible to determine whether or not the state of the stream data can be paused.

  Similarly, if the control information includes information indicating the state between songs, it is determined that the stream data state can be paused when the stream data is in a state between songs. By configuring the terminal control unit 113, it is possible to determine whether or not the state of the stream data can be paused.

  Next, a method for determining whether or not there is a need to increase the accumulated amount of stream data in step ST205 will be described with a specific example.

  As a method for determining whether or not the stream data accumulation amount needs to be increased, for example, a certain threshold value (for example, Bth_Pause) is provided, and if the remaining buffer amount is equal to or less than the threshold value Bth_Pause, the stream data accumulation amount is increased. What is necessary is just to determine with necessity.

  As another determination method, the remaining buffer amount may be observed periodically, and if the remaining buffer amount tends to decrease, it may be determined that there is a need to increase the accumulated amount of stream data.

  As a method for determining whether or not there is a need to increase the accumulation amount of stream data in step ST209, a determination method based on the same determination criterion as in step ST205 may be used, or a different determination method may be used. Also good.

  As a determination method different from step ST205, for example, if the threshold used for determination in step ST209 is Bth_Restart and the remaining buffer amount is equal to or less than Bth_Restart, it may be determined that there is a need to increase the accumulation amount of stream data.

  That is, in this determination method, in other words, when the remaining buffer amount exceeds Bth_Restart, the decoding of the stream data is resumed.

  Note that these determination methods are based on the same determination criteria as in step ST205 if Bth_Pause = Bth_Restart, and are determined based on different determination criteria if Bth_Pause ≠ Bth_Restart.

  Next, as a specific example, a case where a communication error occurs during reproduction of the second song of stream data and the buffer amount is recovered between the second and third songs will be described with reference to FIG.

  FIG. 3 is a diagram showing the time transition of the remaining buffer amount and the stream data state according to the embodiment of the present invention. Here, a case will be described as an example where the control information includes song separation information.

  In FIG. 3, a block 301 represents a time transition of the content of the stream data received by the terminal communication unit 110 of the stream reception / playback apparatus 101. A block 302 represents a time transition of the contents of the stream data decoded by the decoder 112 of the stream reception / playback apparatus 101. A block 303 represents a time transition of the remaining buffer amount of the stream data accumulated in the buffer 111 of the stream reception / playback apparatus 101.

  FIG. 3 shows a state in which a communication error has occurred between time Td and time Te, and the remaining buffer amount of stream data stored in the buffer 111 of the stream reception / playback apparatus 101 is decreasing.

  As shown in FIG. 3, the stream reception / playback apparatus 101 as a client starts receiving stream data at a time Ta, and the decoder 112 starts decoding the stream data from a time Tb after a predetermined time (Tini) has elapsed. .

  Thus, in the stream reception / playback apparatus 101 of this example, in order to secure a certain amount of remaining stream data in the buffer 111, the stream data is accumulated in the buffer 111 from time Ta to time Tb. However, the decoding of the stream data is not started.

  Then, while the first piece of stream data after time Tb is distributed, the accumulation and decoding of the stream data is repeated.

  After that, when receiving the control information including the song segment information indicating the end of the first song, the terminal control unit 113 determines whether or not there is a need to increase the accumulation amount of the stream data based on the remaining buffer amount. To do.

  In the example shown in FIG. 3, since the remaining buffer amount at the end of the first song is B1, it is determined whether or not the remaining buffer amount B1 is less than or equal to the threshold value Bth_Pause. In this example, since B1> Bth_Pause, it is determined that there is no need to increase the accumulation amount of stream data, and the decoding process for the second song is continued.

  After that, when receiving the control information including the song segment information indicating the end of the second song, the terminal control unit 113 determines whether there is a need to increase the accumulation amount of the stream data based on the remaining buffer amount. To do.

  In the example shown in FIG. 3, since the remaining buffer amount at the end of the second song is B2, it is determined whether or not the remaining buffer amount B2 is less than or equal to the threshold value Bth_Pause. In this example, since B2 <Bth_Pause, it is determined that there is a need to increase the amount of accumulated stream data, and the terminal control unit 113 controls the decoder 112 to temporarily stop decoding of the stream data.

  Next, when stream data is received while decoding is paused, stream data is received and stream data is stored in the buffer 111, and it is determined whether there is a need to increase the amount of stream data stored. .

  In the example shown in FIG. 3, since the remaining buffer amount is equal to or less than Bth_Restart from time Tf to time Tg, it is determined that there is a need to increase the accumulated amount of stream data, and decoding of stream data is suspended. Continue (decoding stop section 3021).

  In the example shown in FIG. 3, since the remaining buffer amount exceeds Bth_Restart when the time Tg is exceeded, after the time Tg, decoding of the stream data is resumed and the third music is decoded.

  As described above, in the stream reception / playback apparatus 101 of this example, it is possible to determine whether or not the state of the stream data can be paused based on the control information regarding the received stream data.

  Also, in the stream reception / playback apparatus 101 of this example, it is possible to determine from the remaining buffer amount of the stream data stored in the buffer 111 whether or not there is a need to increase the storage amount of the stream data.

  Then, in the stream reception / playback apparatus 101 of this example, when it is determined that the state of the stream data can be temporarily stopped and the accumulation amount of the stream data needs to be increased, the terminal control unit 113 performs the decoder 112 is controlled to temporarily stop decoding of the stream data, and control to increase the remaining buffer amount of the stream data accumulated in the buffer 111.

  Therefore, in the streaming network system using the stream reception / playback apparatus 101 of the present example, communication errors on the network 103 and deterioration of the radio wave condition are controlled only by the stream reception / playback apparatus 101 on the stream data reception side. It is possible to suppress the frequency of occurrence of underflow of the buffer 111 due to the cause.

  Here, as shown in FIG. 3, the case where the control information includes “song break information” indicating the end of a song has been described as an example. As shown in FIG. 4, even when the control information includes information for notifying the change of the track number of music or information indicating the state between songs, the same control can be performed.

  In the case of the example shown in FIG. 4, information indicating “state between songs” is included in the control information instead of “music break information” indicating the end of a song.

  In the example shown in FIG. 4, for example, at the time Tc, the state of the stream data can be paused by using the control information indicating the “state between songs” instead of the “music break information” indicating the end of the first song. It is different from the example shown in FIG. 3 in that it is determined whether or not there is a need to increase the accumulation amount of stream data.

  Further, in the example shown in FIG. 4, at the time Tf, the state of the stream data can be paused by using the control information indicating the “state between songs” instead of the “music break information” indicating the end of the second song. Is different from the example shown in FIG. 3 in that it is determined whether or not there is a need to increase the accumulation amount of stream data.

  In the above description, the wireless communication method is not particularly limited. For example, Bluetooth or wireless LAN of short-range wireless communication technology may be used.

  Here, when Bluetooth is used for the wireless communication method, for example, the stream data may be transmitted by A2DP (Advanced Audio Distribution Profile).

  Also, the control information related to the stream data may be transmitted using, for example, a Vendor Dependent Command in AVRCP (Audio / Video Remote Control Profile).

  Further, the control information related to the stream data may be transmitted using a Pass Through Command having a Vendor Unique Operation_Id in AVRCP.

  Here, A2DP is a technical specification for transmitting and receiving stream data such as music data in Bluetooth.

  AVRCP is a technical specification for transmitting and receiving stream control commands (Play, Stop, etc.) in Bluetooth.

  Furthermore, Vendor Dependent Command and Vendor Unique Operation_Id are commands that each vendor can independently extend in AVRCP.

  In the above description, although there are portions described with specific values raised, it goes without saying that the present invention is not limited thereto.

  The streaming network system and the stream reception / playback apparatus according to the present invention provide a stream data transmission system based on the system in which the stream data transmission side cannot transmit stream data having different encoding rates or the rate information notified by the stream data reception side. Even in systems where the transmission side cannot control the stream delivery rate, the frequency of underflow in the stream data reception buffer can be reduced, so data such as audio and video can be received via the network. Therefore, the present invention is useful as a streaming network system and a stream reception / playback apparatus using a streaming technology for playback.

The block diagram which shows the structure of the streaming network system which concerns on one embodiment of this invention The flowchart which shows operation | movement of the streaming network system which concerns on one embodiment of this invention. The figure which shows the time transition of the remaining buffer amount which concerns on one embodiment of this invention, and the state of stream data The figure which shows the time transition of the remaining buffer amount which concerns on one embodiment of this invention, and the other state of stream data The block diagram which shows the structure of the transmission system of patent document 1

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Streaming network system 101 Stream receiving / reproducing apparatus 102 Server 103 Network 110 Terminal communication part 111 Buffer 112 Decoder 113 Terminal control part 114 Speaker 120 Server control part 121 Stream data transmission part 122 Server communication part 123 Stream data storage part

Claims (6)

A communication means for receiving the distributed stream data and control data;
Storage means for temporarily storing the stream data received by the communication means;
Decoding means for decoding the stream data stored by the storage means;
The decoding means is controlled when it is determined that the stream data state can be temporarily stopped and the accumulation amount of the stream data needs to be increased based on the control information regarding the stream data received by the communication means. Control means for temporarily stopping the decoding of the stream data and increasing the data amount of the stream data stored in the storage means;
A stream reception / playback apparatus comprising:   2. The stream according to claim 1, wherein the control unit determines that there is a need to increase the accumulation amount of the stream data when the accumulation amount of the stream data accumulated in the accumulation unit is equal to or less than a preset threshold value. Reception playback device.   2. The control information according to claim 1, wherein the control information includes song information of the stream data, and the control means determines that decoding of the stream data can be temporarily stopped if the stream data is in a state between songs. Stream reception / playback device.   The control information includes song information of the stream data, and the control means determines that decoding of the stream data can be paused when detecting a change of a track of the song in the stream data. 1. A stream reception / playback apparatus according to 1. A streaming network system configured by a network including a server that distributes stream data, and a stream reception and reproduction device that receives and reproduces the stream data distributed from the server,
The stream reception / playback apparatus includes a communication unit that receives distributed stream data and control data, a storage unit that temporarily stores the stream data received by the communication unit, and the stream that is stored by the storage unit. Based on the decoding means for decoding data and the control information on the stream data received by the communication means, it is determined that the state of the stream data can be paused and the amount of accumulation of the stream data needs to be increased Control means for controlling the decoding means to temporarily stop the decoding of the stream data and increasing the data amount of the stream data accumulated in the accumulating means,
The server includes: stream data storage means for storing the stream data; stream data transmission means for transmitting the stream data stored by the stream data storage means; control means for transmitting control information relating to the stream data; A streaming network system comprising: stream data distribution means for distributing the stream data to the stream reception / playback apparatus. A communication step for receiving the distributed stream data and control data;
An accumulation step of temporarily accumulating the stream data received in the communication step;
A decoding step for decoding the stream data accumulated in the accumulation step;
When it is determined by the control information related to the stream data received in the communication step that the state of the stream data can be temporarily stopped and the accumulation amount of the stream data needs to be increased, the decoding of the stream data is performed. A control step of temporarily stopping and increasing the data amount of the stream data accumulated in the accumulation step;
A stream receiving method comprising:

Images