JP2004328613A - Video distribution system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video distribution system with which possibility for a receiving buffer to underflow is forecasted and measures to cope therewith can be effectively taken. <P>SOLUTION: A controller of a receiving terminal measures a transmission throughput of a network while a video signal is received (steps S1, S2). When it is possible for the receiving buffer to underflow, it is then decided by using the transmission throughput whether or not reception to the next synchronizing point is quickly completed by switching to a low bit rate (step S3). When the result of said judgement is positive, reception is stopped in the middle of fragment reception, and a fragment of a low bit rate is requested to a server. When data of a low bit rate are sent from the server in response to the request, the receiving terminal receives and reproduces the data. Thus, the receiving terminal requests bit rate without awaiting the complete reception of the fragment, thereby effectively preventing the receiving buffer from underflowing. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a video distribution apparatus, and more particularly to a video distribution system in which media data is transmitted from a server, and the client receives and reproduces the media data via a network. The present invention also relates to a video distribution system suitable for a case where the transmission speed in a network fluctuates.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, there has been known a technique in which the same content created at a plurality of bit rates is stored in a server in advance, and the server switches the bit rate according to an instruction from a receiving terminal, for example, a mobile phone, and distributes the content. In this prior art, the receiving terminal was able to instruct the bit rate from the beginning of the reception and also to change the bit rate during the reception. The receiving terminal is provided with, for example, a button for instructing the server on the bit rate to be received, and the user can instruct the server on the bit rate to be received by operating the button. When the user wants to receive a high-quality image for a part of the received image, for example, the user can operate the button to give an instruction to increase the bit rate. Examples of known documents related to the above-described prior art include, for example, 11-6 “A Study on Design of a Mobile Phone Video Streaming System” of the 2002 Annual Conference of the Institute of Image Information and Television, August 2002, and 11-11. 7 “Study of Dynamic Coding Rate Control in TCP Video Streaming”.
[0003]
However, in the above-mentioned prior art, there was a problem that the receiving terminal could only select the bit rate manually, so that it was not possible to select an optimal bit rate suitable for the circumstances of the line communication. For this reason, the present inventors have invented a moving picture coding bit rate selection method capable of automatically controlling to select an optimum bit rate, and applied for a patent (Japanese Patent Application No. 2002-335035).
[Problems to be solved by the invention]
[0004]
In the above patent application, the client selects the bit rate of the media data to be received next in consideration of the situation of the network throughput and the accumulation status of its own buffer, and selects the media data (fragment) divided in time units. When the reception is completed, the server requests the server for the bit rate fragment.
[0005]
However, in the above patent application, the timing at which the bit rate of the media data can be changed is only when the reception of the fragment is completed. There is no problem if the throughput of the network is as expected, but when the throughput drops rapidly, for example, when entering a tunnel in wireless communication, the reception buffer of the client underflows. When the throughput in the middle of receiving a fragment decreases, the possibility of underflow occurring can be foreseen, but the only way to change the bit rate of media data is to wait for the completion of fragment reception. In addition, there is also a problem that it takes time to recover normal reception even after an underflow occurs in the reception buffer.
[0006]
SUMMARY OF THE INVENTION It is an object of the present invention to solve the above-mentioned problems of the prior art, and to provide a video distribution system capable of predicting a possibility that a receiving buffer will underflow and taking effective measures against it.
[0007]
[Means for Solving the Problems]
In order to achieve the above object, according to the present invention, reception from a moving image data group having a synchronization point so as to be switchable between continuous media data of a plurality of types of encoding bit rates to the synchronization point is completed. Means for selecting a single encoding bit rate each time, and transmitting and receiving moving image data, a means for measuring a transmission throughput of a network for transmitting the moving image data, and receiving the moving image data. Means for determining, using the transmission throughput, whether the reception buffer may underflow, and whether reception to the next synchronization point ends earlier if switching to a lower bit rate is performed, If the determination is affirmative, reception is stopped during reception of data (hereinafter, referred to as a fragment) between the synchronization points, and a low bit rate fragment is requested. And the step, in that and means for receiving a fragment of the low bit rate is characterized.
[0008]
According to this feature, when underflow is expected to occur in the reception buffer, transmission of low bit rate data can be requested without waiting for reception of the fragment being received, and the transmission side responds to this request. Since the transmission of the requested low bit rate data is performed immediately, it is possible to prevent the reception buffer from underflowing as much as possible. Also, even if an underflow occurs, the recovery can be performed at an early stage.
[0009]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, the present invention will be described in detail with reference to the drawings. FIG. 1 shows a schematic configuration diagram of a system to which the present invention is applied.
[0010]
As shown in the figure, the system includes a server 1 that is a transmitting side of moving image content, a network (network) 2 such as a mobile phone network, and a receiving terminal (client) 3 such as a mobile phone and a PDA. ing. The server 1 is connected to a wired information network (not shown) as is well known.
[0011]
The image data storage device 11 of the server 1 stores a plurality of files in which image data or media data having the same contents are created at a plurality of bit rates. As shown in FIG. 2, the media data has the same contents at a plurality of discrete bit rates c [n] (n = 0, 1, 2,...), For example, bit rates c [0]. The bit stream is formed by a bit stream A and a bit stream B at a bit rate c [1], and each bit stream periodically has synchronization points a1, a2, a3,... B1, b2, b3,. Is set. The bit rate c [n] is notified to the receiving terminal 3 in advance.
[0012]
The synchronization points are set at T intervals, for example, at 10 second intervals. At this point, the data can be switched between different bit rates at this point. Hereinafter, a data unit between the synchronization points is referred to as a “fragment”.
[0013]
The image data storage device 11 is accessed by a common gateway interface (CGI) program 12, which is a distribution program, and provides a file at a bit rate requested by the CGI program 12.
[0014]
On the other hand, the receiving terminal 3 includes a receiving buffer 31 for temporarily storing received data, a controller 32 for determining an optimum bit rate and notifying the server 1 of the bit rate via the network 2, A decoder 33 is provided for reading data from the reception buffer 31 and starting video decoding when the accumulated amount becomes equal to or more than a predetermined amount.
[0015]
Next, an embodiment of the present invention will be described. 3A, 3B, and 3C are timing charts of media data for explaining the operation of the present embodiment, and the horizontal axis represents time (t).
[0016]
FIG. 3A shows the relationship between the received data and the reproduced data in this embodiment. In this embodiment, the receiving terminal 3 receives the media data 52 at the media bit rate c, and the decoder 33 reproduces the data 51 of the fragment immediately before the fragment being received, that is, the media data 52 of the n-th fragment. During the reception, the media data 51 of the (n-1) th fragment is reproduced.
[0017]
Next, the operation of the controller 32 of the present embodiment will be described with reference to the flowchart of FIG. In step S1, the media data of the n-th fragment is received. In step S2, the throughput (transmission throughput) v of the network 2 is estimated. In step S3, using the throughput v, it is determined whether or not (1) the reception buffer 31 may underflow, and (2) if the media bit rate is switched to the lowest rate c0, the reception ends earlier. Make a decision. This determination is specifically made by, for example, the following equations (1) and (2).
2 × T-tp <c × (T-tr) / v (1)
c × (T−tr) / v> c0 × T / v + TCP reconnection time (2)
Here, T is the fragment duration time, tp is the point (time) during reproduction of the (n-1) th fragment, tr is the point (time) during reception of the nth fragment (see FIG. 3A), c Is the bit rate of the media data 52 during reception of the n-th fragment.
[0018]
The expression (1) is an expression for determining the above (1), and the expression (2) is an expression for determining the above (2). If the expressions (1) and (2) are satisfied, the judgment in the step S3 becomes affirmative, and if either of the expressions is not satisfied, the judgment becomes negative.
[0019]
If the determination in step S3 is negative, step S6 is followed by step S1 to continue receiving the media data of the n-th fragment. On the other hand, if the determination in step S3 is affirmative, the process proceeds to step S4, where the TCP connection is immediately disconnected. Alternatively, a stop command is sent to the server. Thus, the reception of the media data of the n-th fragment is stopped during the reception.
[0020]
In step S5, as shown in FIG. 3B, the server 1 is requested to transmit the data 53 of the n-th fragment at the lowest bit rate c0. Upon receiving this request, the server 1 starts transmitting the requested fragment of the new bit rate even if the receiving terminal 3 does not recognize that the reception has been stopped. Next, returning to step S1, the data of the n-th fragment is received from the beginning at the bit rate c0. Hereinafter, the same operation as described above is performed.
[0021]
As a result of the above operation, if the determination in step S6 is affirmative, the process proceeds to step S7 to determine whether or not the last fragment has been received. If this determination is negative, n is incremented by 1 and the process returns to step S1, where the (n + 1) th fragment is received. If the determination in step S7 is affirmative, the series of operations described above ends.
[0022]
The reproduction operation of the decoder 33 of the receiving terminal 3 is performed as described above even if the data of the n-th fragment is interrupted during reception and the data of the bit rate c0 of the n-th fragment is re-received from the beginning. n-1) Continue playing the fragment. The data of the bit rate c0 of the n-th fragment is reproduced in the background during the reproduction of the (n-1) -th fragment, and when the reproduction of the (n-1) -th fragment is completed, the newly arrived n-th fragment The data is switched to the data 53 of the fragment bit rate c0 (see FIG. 3B). This makes it possible to continuously reproduce the media data and to prevent the receiving buffer from underflowing as much as possible.
[0023]
As a modified example of the reproducing operation, the data 52 of the bit rate c is reproduced up to the position where it has been received, that is, up to tr of the n-th fragment, and in parallel with this, the media data of the bit rate c0 is decoded in the background. Alternatively, the reproduction may be switched at the point tr.
[0024]
As described above, according to the present embodiment, when there is a possibility that an underflow occurs in the reception buffer during the reception of the n-th fragment, the possibility occurs without waiting for the end of the reception of the n-th fragment. Since the reception can be interrupted at the point in time and the server can be requested to transmit at the lowest bit rate c0, it is possible to take a countermeasure at an early stage and to prevent the occurrence of the underflow as much as possible. Also, the reproduction of the received data can be performed without delay.
[0025]
Another operation example of the controller 32 will be described with reference to FIG. In FIG. 5, the same operations as those in FIG. 4 are denoted by the same step numbers, and description thereof will be omitted. If the determination in step S3 is affirmative, the flow advances to step S4 to immediately send a TCP connection or cancel command, and then to step S11, as shown in FIG. 3 (c). , The (n + 1) th fragment data 54 at the lowest bit rate c0. Upon receiving this request, the server 1 starts transmitting the requested fragment of the new bit rate even if the receiving terminal 3 does not recognize that the reception has been stopped. In step S12, the data 54 of the (n + 1) th fragment is received at the bit rate c0. In step S13, it is determined whether the reception of the (n + 1) -th fragment has been completed. If the determination is negative, the process returns to step S1 to continue the reception at the bit rate c0. On the other hand, if the determination in step S3 is negative, the process proceeds to step S14 to continue receiving the n-th fragment.
[0026]
As described above, in this operation example, when the determination in step S3 is affirmative, the reception of the n-th fragment is immediately interrupted, and the reception is resumed from the (n + 1) -th fragment at the bit rate c0.
[0027]
In this case, the reproducing operation of the decoder 33 is to reproduce the trth of the nth fragment, freeze the image, buffer the data of the (n + 1) th fragment to a predetermined amount, and then reproduce the (n + 1) th fragment. (See FIG. 3C).
[0028]
According to this embodiment, when there is a possibility that the reception buffer underflows, the image display is forcibly frozen, and thereafter, the reception buffer of the bit rate c0 of the (n + 1) th fragment reaches a predetermined amount. Since the decoding / reproduction is resumed after the data is stored in the buffer, the control of the decoding / reproduction is simplified, and the recovery after the receiving buffer underflows is quickened.
[0029]
When the transmission throughput v of the network for transmitting moving image data is significantly lower (ie, a predetermined value or more) than the minimum bit rate c0 of the fragment data, the determination in step S3 is not performed. It is preferable to stop reception and request transmission of the lowest bit rate c0.
[0030]
Next, a second embodiment of the present invention will be described. 6A, 6B, and 6C are timing charts showing the operation of the present embodiment. As is clear from FIG. 6A, this embodiment is an example of the case where the data 61 is reproduced while following the media data 61 of the n-th fragment being received (reproduction data 62).
[0031]
The operation of the controller 32 in the second embodiment is the same as that in FIG. 4 or FIG. 5 except for the determination formula in step S3. That is, (a) similarly to FIG. 4, when the TCP connection disconnection or stop command is sent in step S4, the n-th fragment is requested at the bit rate c0. Alternatively, (a) similarly to FIG. 5, when the TCP connection disconnection or stop command is sent in step S4, the (n + 1) th fragment is requested at the bit rate c0.
[0032]
The decision formulas that replace the formulas (1) and (2) are the following decision formulas (formulas (3) and (4)).
T-tp <c × (T-tr) / v (3)
c × (T−tr) / v> c0 × T / v + TCP reconnection time (4)
When both the expressions (3) and (4) are satisfied, the determination in step S3 is affirmative, and when either one of the expressions is not satisfied, the determination is negative.
[0033]
Next, the reproducing operation of the decoder 33 will be described. In the case of (a), the decoder 33 continues to reproduce the n-th fragment, and reproduces it up to the already received tr. At the same time, the decoding of the media data of the bit rate c0 is performed, and the media data of the bit rate c is switched to the media data of the c0 at tr (see FIG. 6B).
[0034]
In the case of (a), reproduction of the n-th fragment is performed until tr, the image is frozen, the data of the (n + 1) -th fragment is buffered to a predetermined amount, and then reproduction of the (n + 1) -th fragment is started. (See FIG. 6C).
[0035]
As described above, according to the present embodiment, even when the data of the n-th fragment is received and reproduced while following the same fragment, there is a possibility that underflow occurs in the reception buffer during the reception of the n-th fragment. Appears, the reception can be immediately interrupted and the server can be requested to transmit at the lowest bit rate c0, so that the occurrence of the underflow can be prevented as much as possible. Also, the reproduction of the received data can be performed without delay.
[0036]
In the case of (a), when the possibility of the receiving buffer underflowing occurs, the image is forcibly frozen. However, the reception of the data at the bit rate c is immediately stopped, and Since data of bit rate c0 is received and decoding / reproduction is resumed when the reception data has been accumulated in the reception buffer up to a predetermined amount, the recovery of reproduction is quick.
[0037]
Next, a third embodiment of the present invention will be described. This embodiment is an example of a case where the server 1 can read out data at a different bit rate at any time, that is, even in the middle of a fragment.
[0038]
The main part of the operation of the controller 32 in this embodiment will be described with reference to the flowchart in FIG. 7, the same operations as those in FIG. 4 are denoted by the same step numbers, and description thereof is omitted. After the process of sending the command to disconnect or abort the TCP connection in step S4 is completed, in step S21, a transmission request is made at the minimum bit rate c0 by specifying an intermediate position (tr in FIG. 3 or FIG. 6) of the n-th fragment. . In step S22, the receiving terminal 3 receives the data at the bit rate c0 from the halfway point tr.
[0039]
In the reproducing operation of this embodiment, data received at the bit rate c is reproduced up to the midpoint tr of the n-th fragment, and data received at the lowest bit rate c0 is reproduced from the tr.
[0040]
According to this embodiment, when there is a possibility that an underflow may occur in the reception buffer during the reception of the n-th fragment, the reception can be interrupted, and the server can be requested to transmit from the interruption point at the lowest bit rate c0. Therefore, no waste is generated in the received data, and occurrence of underflow can be prevented as much as possible. Further, the reproduction of the received data can be performed continuously without interruption or freeze.
[0041]
In each of the above embodiments, when there is a possibility that the receiving buffer underflows, a request for the lowest bit rate c0 is made. However, the present invention is not limited to this and does not cause underflow. Any bit rate may be requested as long as it is a bit rate. In this case, the bit rate c ′ in the first embodiment is replaced by the following equation (5) instead of the equation (1), and the bit rate c ′ is replaced by the equation (3) in the second embodiment. What satisfies the following equation (6) may be selected.
2 × T−tp <c ′ × T / v (5)
T-tp <c ′ × T / v (6)
[0042]
According to this modification, underflow of the reception buffer can be prevented, and media data having better image quality than the data having the lowest bit rate c0 can be received.
[0043]
The present invention can be applied to video distribution for mobile phones and PDAs, Internet video distribution for PCs, and the like.
[0044]
【The invention's effect】
As apparent from the above description, according to the first and second aspects of the present invention, when it is expected that an underflow occurs in the reception buffer, the data of the low bit rate can be transmitted without waiting for the end of reception of the fragment being received. Can be requested, so that underflow of the receiving buffer can be prevented as much as possible. Also, even if an underflow occurs, the recovery can be performed at an early stage.
[0045]
According to the third aspect of the present invention, it is possible to shift to a low bit rate reproduced image without causing a video freeze. According to the fourth aspect of the present invention, since a video freeze is forcibly caused and received from the beginning of the next received fragment, it is possible to shift to a low bit rate reproduced image with a simple control. Also, even if an underflow occurs, the recovery can be performed at an early stage.
[0046]
Further, according to the invention of claim 5, the bit rate can be switched to a low bit rate even in the middle of a fragment. According to the invention of claim 6, it becomes possible to switch to reception at a low bit rate and reproduce the data in the middle of the fragment.
[0047]
Further, according to the invention of claim 7, underflow of the reception buffer can be prevented as much as possible. According to the eighth aspect of the present invention, underflow of the receiving buffer can be prevented as much as possible while maintaining good image quality as much as possible. According to the ninth aspect of the present invention, when the transmission throughput of the network is significantly reduced, a countermeasure can be taken at an early stage, and an underflow of the reception buffer can be prevented, or when the underflow occurs, an early recovery can be achieved. become able to. Further, according to the tenth aspect, the transmitting side can transmit the low bit rate data requested earlier.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a schematic configuration of a transmission / reception system suitable for a video distribution system according to the present invention.
FIG. 2 is a diagram showing a configuration of a bit stream.
FIG. 3 is a timing chart of the first embodiment showing reception and reproduction of a fragment.
FIG. 4 is a flowchart showing an operation of the first embodiment.
FIG. 5 is a flowchart illustrating another operation example of the first embodiment.
FIG. 6 is a timing chart of a second embodiment showing reception and reproduction of a fragment.
FIG. 7 is a flowchart showing the operation of the third embodiment.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Server, 2 ... Network, 3 ... Receiving terminal, 11 ... Image data storage device, 12 ... CGI program, 31 ... Receiving buffer, 32 ... Controller, 33 ··decoder.

Claims (10)

From a moving image data group having a synchronization point so that it can be switched between continuous media data of a plurality of types of encoding bit rates, a single encoding bit rate is selected each time reception to the synchronization point is completed, A video distribution system for transmitting and receiving moving image data,
Means for measuring the transmission throughput of the network transmitting the moving image data,
During the reception of the moving image data, there is a possibility that the reception buffer underflows, and whether switching to a low bit rate terminates reception until the next synchronization point early, using the transmission throughput Means to determine,
If the determination is affirmative, means for stopping reception during the reception of data (hereinafter, referred to as a fragment) between the synchronization points and requesting a low bit rate fragment;
Means for receiving the low bit rate fragment. The video distribution system according to claim 1,
The means for requesting a low bit rate fragment requests a low bit rate of the same fragment as the fragment being received or the next fragment. The video distribution system according to claim 2,
If the means for requesting the low bit rate fragment requests the same fragment as the fragment currently being received, the reproduction of the received fragment or the data being received is performed at the same time as the newly received low bit rate. That the data is reproduced in the background, and at the time when the reproduction of the received fragment or the received data up to the reception stop time is finished, switching to the reproduction of the newly received low bit rate data is performed. Characteristic video distribution system. The video distribution system according to claim 2,
When the means for requesting the low bit rate fragment requests the low bit rate of the fragment next to the fragment being received, the reproduction is continued up to the point where the data has been received, and the data is frozen where the data is lost. A video distribution system in which reproduction is started from the next fragment data. Video distribution that selects a single encoding bit rate from time to time and selects a single encoding bit rate from a video data group that has synchronization points so that it can be switched between continuous media data with multiple encoding bit rates. Method,
Means for measuring the transmission throughput of the network transmitting the moving image data,
During the reception of the moving image data, there is a possibility that the reception buffer underflows, and whether switching to a low bit rate terminates reception until the next synchronization point early, using the transmission throughput Means to determine,
If the determination is affirmative, means for stopping reception during the reception of data between the synchronization points and requesting low bit rate data following the stopped data;
Means for receiving the low bit rate data. The video distribution system according to claim 5,
A video distribution method, wherein the data being received is played back halfway through the reception, and subsequently, data at a low bit rate subsequent to the received stopped data is played back. The video distribution system according to any one of claims 1 to 6,
A video distribution system, wherein the low bit rate is a minimum bit rate. The video distribution system according to any one of claims 1 to 6,
The video distribution method, wherein the low bit rate is a bit rate at which the reception buffer is not expected to cause underflow. The video distribution system according to any one of claims 1 to 8,
The video distribution system, wherein the transmitting side of the moving image starts transmitting data of the requested new bit rate even when the transmitting side does not recognize the reception stop. From a moving image data group having a synchronization point so that it can be switched between continuous media data of a plurality of types of encoding bit rates, a single encoding bit rate is selected each time reception to the synchronization point is completed, A video distribution system for transmitting and receiving moving image data,
Means for measuring the transmission throughput of the network transmitting the moving image data,
Means for determining whether the transmission throughput of the network is lower than the minimum bit rate of the video group by a predetermined value or more,
If the determination is affirmative, means for stopping reception during the reception of data between the synchronization points (hereinafter, referred to as fragments) and requesting a fragment of the lowest bit rate;
Means for receiving the fragment of the lowest bit rate.

Images