KR100782343B1 - Method of streaming image - Google Patents

Abstract

The present invention relates to a video streaming method over a network, and more particularly, to a Transmission Control Protocol (TCP) friendly streaming method using a decoder buffer controller.

A TCP friendly streaming method using a decoder buffer controller according to the present invention includes: (a) generating an image frame in a video server; (b) transmitting the generated image frame; (c) monitoring the transmission; (d) decoding the received image frame; (e) combining steps (a), (b) and (c) to prevent the playback of the video from being interrupted in the decoder buffer controller and to ensure continuous playback. It is characterized in that the continuous reproduction is ensured by appropriately adjusting the image quality of the image so that there is always an image that can be reproduced for a certain time.

Streaming, Decoder, TCP Friendly Rate Control (TRFC), Fine Granular Scalability (GFS)

Description

Video streaming method {METHOD OF STREAMING IMAGE}

1 is a schematic diagram of a streaming system according to the present invention.

2 is a structural diagram of a decoder buffer according to the present invention.

The present invention relates to a video streaming method over a network, and more particularly, to a TCP friendly streaming method using a decoder buffer controller.

The existing streaming method always transmits images having a constant image quality regardless of network conditions. If the current Internet can adequately guarantee the transmission capacity required for streaming, the existing streaming method is not a problem, but the current Internet is the best-effort network that cannot guarantee the transmission capacity.

In other words, if the network does not support the minimum transmission capacity required for streaming while video is being streamed at a certain quality, the playback client does not arrive and the playback stops for a certain time until the required video arrives. I have to wait.

Also, even if the network has enough capacity to transmit the current streamed video, it cannot use all the extra capacity.

In order to overcome the above problems, methods such as Korean Patent Publication No. 2003-0035525 and Korean Patent Publication No. 2003-0079337 are disclosed.

In the case of Korea Patent Publication 2003-0035525, it predicts the network situation through the interval between packets and adjusts the transmission rate of the image packet according to the predicted result, and adapts the image quality to transmit the image having the best image quality according to the transmission capacity of the given network There is no concept of quality adaptation.

In the case of Korea Patent Publication No. 2003-0079337, the concept of image quality adaptation was introduced, but the latest image encoding technology and transmission method developed for streaming as well as adjusting the image quality based on the storage amount rather than the playback time through the storage capacity of the decoder buffer. It is not adopting.

An object of the present invention to solve the above problems is to introduce a Moving Picture Experts Group (MPEG) -4 Fine Granular Scalability (FGS) image encoding technology that can have various bit rates within a specific range without additional encoding / decoding process There is this.

Another object of the present invention is to adopt a TCP Friendly Rate Control (TFRC) algorithm that predicts the transmission capacity of a real-time network from a TCP-friendly point of view and enables transmission suitable for streaming.

Another object of the present invention is to ensure continuous playback by controlling that the time that can be reproduced with the storage amount of the decoder buffer is maintained constant through a mathematically proven control formula.

A TCP-friendly streaming method using a decoder buffer controller according to the present invention includes: (a) generating a video frame in a video server; (b) transmitting the generated image frame; (c) monitoring the transmission; (d) decoding the received image frame; (e) combining the steps (a), (b) and (c) to prevent the playback of the video from being interrupted in the decoder buffer controller and to ensure continuous playback. It is a TCP-friendly streaming method using a decoder buffer controller that ensures continuous playback by appropriately adjusting the image quality so that there is always a video that can be played for a certain time in the buffer.

By the configuration according to the present invention, it is possible to ensure continuous playback without interruption when streaming video over a network.

Hereinafter, a TCP friendly streaming method using a decoder buffer controller according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a schematic diagram of a streaming server 100 and a playback client 200 used in a TCP friendly streaming method using a decoder buffer controller according to the present invention.

As shown in FIG. 1, the streaming server 100 includes an FGS server 103 and an image frame generated from the FGS server 103 for generating an image frame having an appropriate image quality from encoded FGS (Fine Granular Scalability) images. Streaming buffer 104 is stored and packetized and then transmitted to the network according to the estimated transmission capacity, bandwidth estimator 102 for predicting the transmission capacity of the current network 300, decoder buffer 202 And a decoder buffer controller 101 which calculates an image quality of an image for controlling the decoder buffer 202 with reference to the state information of the network 300, and the playback client 200 is received through the network 300. The video packet is composed of a decoder buffer 202, which is stored and then restored to an image frame for decoding, and an FGS decoder 203 that decodes the restored video frame into an original image.

Here, when streaming starts, the FGS server 103 has a size of f [k] / frame rate with reference to the bitrate f [k] value calculated by the decoder buffer controller 101 at every 1 / frame rate time. Create an image frame.

The video frame generated through the FGS server 103 is stored in the streaming buffer 104 and the transmission capacity C _S [of the network 300 predicted by the transmission capacity measurer 102 using the TCP Friendly Rate Control (TFRC) algorithm. k] is packetized and then transmitted to the network 300.

In addition, the decoder buffer controller 101 transmits the state information e [k] and the capacity measuring instrument 102 of the decoder buffer 202 provided by the state information 301 whenever the state information 301 arrives from the playback client 200. Bit rate f [k] for the image for controlling the decoder buffer 202 is calculated by substituting the transmission capacity C _S [k] of the network 300 predicted by the control equation.

In this case, when the first video packet for streaming arrives, the playback client 200 does not start playback immediately, but starts playback after having an initial playback delay of τ _d .

Then, when playback starts, the FGS decoder 203 observes the decoder buffer 202 every 1 / frame rate second to restore and decode the received video packets into a video frame to be played back.

In addition, the total storage amount q _d [k] of the image stored in the decoder buffer 202 and reception at the periodic time of T to form the state information 301 to be provided to the decoder buffer controller 101 of the streaming server 100 is received. Based on the transmission rate s [k] of the video packet, the state information e [k] 301 is generated through Equation 4 and packetized and transmitted to the streaming server 100.

The value of T is any value that can optimize the streaming system, but is generally preferred to be 1 / frame rate.

In addition, s [k] is a value representing a transmission rate of an image packet transmitted through the actual network 300, and is different from the predicted transmission capacity C _S [k].

2 is a schematic diagram of the state information 301 used in the TCP friendly streaming method using the decoder buffer controller 101 according to the present invention.

As shown in FIG. 2, the present invention is not intended to control the storage amount of the decoder buffer 202 itself, but to control the time that can be reproduced through the storage amount of the decoder buffer 202.

To this end, it should be possible to predict the time that can be reproduced through the storage amount of the decoder buffer 202, which can be calculated as follows.

delete

Here, as illustrated in FIG. 2, an image present in the decoder buffer 202 may store a plurality of images q _d1 , q _d2 , ... q having different bit rates f ₁ , f ₂ , ... f _{N. dN} ).

Further, if there are N kinds of such images in the decoder buffer 202 and the respective bit rate f _n and the storage amount q _dn can be known, the reproduction amount through the storage amount q _d [k] of the image existing in the decoder buffer 202 is reproduced. The time p _d [k] can be calculated as in Equation 1.

In practice, however, it is not easy to know each bit rate and amount of storage, so appropriate approximation is required.

The mathematical analysis shows that the control equation converges the bit rate f [k] and the rate s [k] of the image to the network's transmission capacity C _S [k] near the expected equilibrium point.

Therefore, the rate s [k] can be used as an approximation of the bit rate f [k] for the image, and furthermore, since it can be expected that the transmission rate in the steady state will not change significantly, Equation 1 is expressed as Equation 2 below. .

delete

Further, in order to keep the predicted reproduction time p _d [k] at the initial reproduction delay τ _d , and to make the target value of the decoder buffer 202 storage smaller than the buffer size B, the target value q _T [k] is It can be expressed as Equation 3.

delete

In addition, the state information 301 is expressed by the following equation (4).

delete

As shown in Equation 4, the state information e [k] 301 is a difference between a total storage amount of an image stored in the decoder buffer 202 and a target value of the total storage amount of the decoder buffer 202.

Here, the control equation is as follows.
f [k] = f [k-1] + C _S [k] -C _S [k-1] + A (e [k] -e [k-1]) + B · T · e [k]

delete

In Equation 5, A and B are the controller gains to be obtained through the analysis of convergence intervals, and show the maximum performance when the following values are obtained.

delete

Here, τ _max means the maximum value that a value obtained by adding a round trip time (RTT) of the network 300 and a delay time in the streaming buffer 104 and the decoder buffer 202 may have.

Here, T means an interval in which the state information 301 is periodically updated as described above.

The right side of equation 5 can be divided into two parts.

C _S [k] -C _S [k-1] is a portion that causes the bit rate f [k] for the image to follow the network throughput C _S [k] predicted by the capacity measuring unit 102.

A (e [k] -e [k-1]) + B · T · e [k] is a portion that causes the value of e [k] to converge to 0, resulting in the storage amount q _d [of the decoder buffer 202. k] converges to the target value q _T [k].

The present invention introduces a Moving Picture Experts Group (MPEG) -4 Fine Granular Scalability (GFS) image encoding technology to have various bit rates within a specific range without additional encoding / decoding process.

In addition, the present invention employs a TCP Friendly Rate Control (TFRC) algorithm to predict the transmission capacity of a real-time network in a friendly manner with TCP and to enable transmission suitable for streaming.

In addition, the present invention has the effect of controlling to maintain a constant time that can be reproduced with the storage amount of the decoder buffer through a control formula whose performance has been mathematically verified to ensure continuous playback.

Claims (15)

delete delete delete delete delete In the video streaming method over a network, (a) generating an image frame; (b) transmitting the image frame; And (c) monitoring the transmission; Includes, and in response to the monitoring, convert the bit rate of the video frame, The monitoring step of (c), (c-1) measuring a transmission capacity of the video frame by using a TCP Friendly Rate Control (TFRC) algorithm; (c-2) transmitting the image frame to a decoder buffer; And (c-3) accumulating the video frame in the decoder buffer; Including, video streaming method. delete The method of claim 6, The video streaming method, (e-1) forming state information by predicting a playable time based on a storage amount of the decoder buffer; (e-2) feeding back the status information to a decoder buffer controller; And (e-3) calculating a bit rate of an image frame in the decoder buffer controller using the state information; Including, video streaming method. The method of claim 8, In order to predict the playable time in the step (e-1), the following equation is satisfied,

Where p _d [k] is the total playback time predicted at the k th, q _d [k] is the total amount of video stored in the decoder buffer at kth, s [k] is the transmission rate of the video packet received in the decoder buffer at the kth, q _dn is the storage amount of the n th image stored in the decoder buffer, f _n is the bit rate for the nth image stored in the decoder buffer, N is the total number of images stored in the decoder buffer. The method of claim 8, The step (e-1) uses the difference between the storage amount of the current decoder buffer and the target value of the decoder buffer to form state information. The method of claim 10, And a target value of the decoder buffer is set such that a predicted playback time is maintained as an initial playback delay time. The method of claim 11, The target value setting of the decoder buffer is limited to the size of the decoder buffer when the set target value exceeds the size of the decoder buffer. The method of claim 8, The decoder buffer controller satisfies the following equation, f [k] = f [k-1] + C _S [k] -C _S [k-1] + A (e [k] -e [k-1]) + B · T · e [k] Here, f [k] is a bit rate for the image for controlling the decoder buffer at the k th, C _S [k] is the network throughput predicted by the capacity meter in kth, e [k] is the status information of the decoder buffer at the k th, A and B are decoder buffer controller gains, T is a period of status information, video streaming method. The method of claim 13, Gains A and B of the decoder buffer controller satisfy the following equation,

Here, τ _max is a maximum value that can be obtained by adding a round trip time (RTT) of the network and a delay time in the streaming buffer and the decoder buffer. The method of claim 8, Step (e-1) satisfies the following control formula to form state information.

Here, e [k] is status information of the decoder buffer at the k th, q _d [k] is the total amount of video stored in the decoder buffer at kth, q _T [k] is the target value of the total storage amount of the decoder buffer at the k th, s [k] is the transmission rate of the video packet received in the decoder buffer at the kth, τ _d is the initial regeneration delay time. B is the size of the decoder buffer.

Images