RU2310989C2 - System and method for controlling speed of transmission when providing stream transmission service in real time - Google Patents

Abstract

FIELD: communications engineering, possible use for providing stream transmission service in real time between stream server and stream client, connected through a predetermined network.

SUBSTANCE: in accordance to the invention, during transmission of multimedia content to stream client, stream server controls transmission speed by compensating for a value of accumulated average by means of addition of average value of difference between the level of buffer, messaged by stream client, and the required level of buffer. To prevent the initial reproduction delay, stream server performs control, setting the maximal speed of transmission allowed by network.

EFFECT: prevented overloading or underloading of buffer which occur when a stream transmission service is provided over extensive time span, minimized time delay from the moment of beginning of stream transmission to the moment of reproduction of actual image or sound.

2 cl, 6 dwg

Description

FIELD OF THE INVENTION

This document relates to a system and method for controlling the transmission rate of a real-time streaming service, providing the ability to control the transmission rate of data transmitted from the streaming server to the client using the real-time transport protocol and real-time transmission control protocol, which allows maintaining the buffer level streaming client at a constant level.

Description of the Related Art

Typically, all multimedia data, such as video data, computer animation data, or any combination thereof, is downloaded to a hard disk drive of a personal computer (PC), for example, via Internet access to reproduce this data. However, in a streaming environment, it is not possible to download all the data at once, but it is possible to reproduce the data in real time during buffering without first loading the entire file. In other words, streaming allows you to see data without first downloading the entire file. Generally speaking, streaming is a multimedia data transmission method that enables real-time reading, transmission and playback of data on the Internet.

When the streaming server and the streaming client are connected via a wired or wireless network, and the streaming server transmits multimedia content (information-relevant content), the streaming client that has received the transmission of multimedia content buffers the received content and plays this content in real time.

As a rule, the transmission speed itself provided by the streaming server satisfies the requirements of the transmitted data. In streaming audio and / or video packets transmitted by compression according to the standards of the Expert Group on Cinematography (MPEG), time information for controlling the bit rate provided by the streaming server is recorded in the header.

However, even if the transmission rate is maintained for a long time while providing streaming services, a problem arises. It is impossible to calculate the exact transmission rate even if the mentioned time information has exceptional accuracy, so that even a small difference in the transmission rates can cause a buffer overflow after the mentioned long time.

There is another problem, namely, that the streaming server and the streaming client are independent objects, have different reference clock pulses, and small differences of these clock pulses accumulate during the operation of a cheap crystal oscillator, preventing stable operation.

1 is a first graphical view illustrating a change in a buffer level in accordance with the prior art.

As shown in FIG. 1, the buffer level increases if the transmission rate provided by the streaming server is greater than the speed of data reproduced by the actual streaming client, which leads to the case when the maximum buffer level determined by the streaming client is exceeded.

Another problem is that the method of determining the transmission speed only from the time information contained in the multimedia data causes a time delay from the start of streaming to the start of actual playback.

2 is a second graphical representation illustrating a change in the buffer level in accordance with the prior art, which shows the change in the buffer level after the start of streaming. To capture and reproduce the context of multimedia content transmitted through the MPEG decoder of the streaming client, a certain minimum amount of data is required. The reason is that the streaming client is somewhat capable of allowing delay variation. Therefore, as shown in FIG. 2, the “required buffer level”, to which - as expected - the buffer is filled without reproducing the transmitted data, is determined to a certain level before starting playback.

At the same time, control by simply comparing the actual buffer level and the required buffer level reported by the streaming client can negatively affect the stability of the system, since even a very small difference in the transmission rates between the streaming server and the streaming client can lead to buffer overflow or underload.

The streaming server is required to continuously control the transmission rate in order to maintain the buffer level reported to the streaming clients. In addition, there is a need to fill the buffer at the highest possible speed, so that when streaming starts, fast playback is possible.

Summary of the invention

One object of the present invention is to provide a system and method for controlling the transmission rate of a real-time streaming service, enabling the buffer level to be stably maintained by controlling the transmission rate of multimedia content transmitted between the streaming server and the streaming client.

Another objective of the present invention is to develop a system and method for controlling the transmission rate when providing real-time streaming services in order to minimize the initial playback delay by maximizing the initial transmission rate from the streaming server to the streaming client.

In accordance with one aspect of the present invention, a method for controlling a transmission rate for a real-time streaming service includes real-time streaming of multimedia content to a client connected to the network, the client having a buffer. In this embodiment, multimedia content is shared and transmitted in accordance with a transmission rate. The method also includes sending to the server a buffer level that indicates the amount of multimedia content transmitted to the client stored in the buffer. The method also includes changing the transmission rate to complete the transmission of multimedia content. In this case, the server changes the transmission rate by compensating for the accumulated average over a period of time associated with the difference between the buffer level and the predetermined desired buffer level.

The server changes the predefined transmission rate to complete the transmission of multimedia content by compensating for the accumulated average value for a predetermined period of time associated with the difference between the received buffer level and the predetermined required buffer level.

The network is either a wired network or a wireless network, and in particular is a wireless local area network (LAN), a Bluetooth network, a Zigbee network, or an ultra-wideband (UWB) network.

The client transmits the buffer level through the application message (APP) corresponding to the Real-Time Transmission Control Protocol (RTCP).

The buffer level transmitted by the APP message may include at least one of a user datagram protocol socket socket level (UDP socket), a stream application buffer level that is used by the client, and an MPEG decoder buffer level (MPEG -decoder) belonging to the client, as well as the level of the buffer transmitted by the APP message and containing all of the above categories.

The transmission step in accordance with a predetermined transmission rate further includes transmitting multimedia content with a maximum transmission rate allowed by the network when streaming begins.

The server that started transmission at the maximum transmission rate maintains the maximum transmission rate until the buffer level received by the client at a predetermined interval reaches the required buffer level for the first time.

After that, the server changes the level of the buffer in accordance with a predefined transmission rate and transmits it to play multimedia content if the received buffer level reaches the desired buffer level.

The accumulated average value is calculated, and the difference between the buffer level adopted for the mentioned time period and the required buffer level is multiplied by a weight coefficient, summed and averaged, and the difference associated with the most recently received buffer level is multiplied by a larger weight coefficient. The accumulated average value can be calculated using the following equation:

where Δ _t denotes the accumulated average value, N denotes the value of the buffer level received over a predefined period of time, l _tk denotes the buffer level received at the (tk) -th interval from the client, θ denotes the required buffer level, and ω ^k denotes the weight coefficient having a value from 0 to 1.

The server compensates for the accumulated average value by changing a predefined transmission rate in accordance with the following equation:

where Δ _t denotes the accumulated average value, R _t-1 denotes the transmission rate before the change, and R _t denotes the already changed transmission rate.

A streaming system according to another embodiment of the present invention comprises a server and a client. The server streams multimedia content in real time by splitting and transmitting multimedia content at a certain transmission rate. The client includes a buffer and at a certain interval transmits a buffer level to the server - the amount of multimedia content that has been transferred and stored in the buffer. The server changes the transmission speed so that the difference between the transmitted buffer level and the desired buffer level is compensated using the accumulated average over a period of time. In this embodiment, the transfer of multimedia content ends with the client.

List of drawings

These and / or other aspects and advantages of the present invention will become apparent and easier to read after studying the following detailed description given in connection with the accompanying drawings, while

figure 1 is a graphical representation illustrating a change in the level of the buffer in accordance with the prior art;

figure 2 is a graphical representation illustrating another change in the level of the buffer in accordance with the prior art;

3 is a block diagram illustrating a network including a streaming system in accordance with the present invention;

4 is a state diagram illustrating a message used by a client to transmit a buffer level and corresponding to the present invention;

5 is a flowchart illustrating a method for controlling a transmission rate when providing a real-time streaming service in accordance with the present invention; and

6 is a graphical representation illustrating a change in the level of a client buffer in accordance with the present invention.

Description of Preferred Embodiments

Now with reference to the accompanying drawings will be given a detailed description of embodiments of the present invention.

The system (indicated by 300 and hereinafter referred to as the "streaming system") used in the implementation of the method of controlling the transmission rate for the provision of streaming services in accordance with the present invention comprises a server (310) and a client (330). The server (310) and the client (330) are connected through a network (350), and the server (310) and the client (330) support the Internet Protocol (IP).

The network (350) includes a wired network and a wireless network, and the term “wireless network” may refer to a wireless local area network (LAN), a Bluetooth network, a ZigBee network, an ultra-wideband (UWB) network, and the like.

The server (310) transmits multimedia content including image and sound data compressed in a predetermined manner. Said compression method includes a method in accordance with the standard of the Expert Group on Cinematography (MPEG), which is an international standard. In addition, the server (310) can itself — by using the included MPEG encoder (not shown) —compress multimedia content.

The server (310) controls the streaming packet through a buffer level reported by the client (330) to ensure that the client buffer level (330) is maintained at a predetermined level (hereinafter referred to as the "required buffer level").

Below will be a detailed description of the transmission speed control in relation to the client (330) server (310). Further, it should be noted that the server (310) transmits packets with the maximum transmission rate allowed by the network (350) at the initial stage of the transmission of multimedia content in order to thereby fill the buffer (331) of the client (330) to the desired buffer level in a short period of time.

The client (330) includes a buffer (331) of predetermined capacity and decodes the stream packet transmitted from the server (310) to receive image and / or sound data and reproduces this image and / or sound data through a user interface (not shown ) using a speaker or display device.

The buffer (331) includes a UDP socket buffer, a stream application program buffer, and an MPEG decoder buffer.

The client (330) stores the received stream packet in the buffer (331), decodes and reproduces this stream packet. The client (330) sends - regularly or irregularly - to the server (330) a buffer level indicating the amount of information stored at the current time.

Now, with reference to FIG. 4, an explanation will be given of how the client (310) of the streaming system (300) of the present invention transmits a message to the server (310).

The client (330) uses the application message (APP message) corresponding to the real-time transmission control protocol (RTCP) to notify the server (310) of the buffer level.

Client (310) transmits three types of buffer levels, including a UDP socket buffer level, a streaming application program buffer level, and an MPEG decoder buffer level.

The UDP socket buffer level determines the buffer level allocated to a packet received in accordance with the UDP protocol. The program buffer level of the streaming application refers to the buffer level allocated to the program that plays or processes the streaming data received to support the user interface for multimedia content. In addition, the buffer level of an MPEG decoder is a buffer level characteristic of a buffer allocated to a decoder (not shown) that reproduces multimedia content by decoding it.

The client (310) transmits three types of buffer levels in the byte block by filling the buffers to levels corresponding to the message format shown in FIG. 4. Server (310) uses the sum of the buffer levels of all three types.

Stream data transmitted from the server (310) is first stored in the buffer (331). Stream data stored in the buffer (331) is not transmitted to the MPEG decoder (not shown) until the buffer level that is the required buffer level is reached. If an MPEG decoder (not shown) decodes the data stored in the buffer, this occurs when the buffer level reaches the desired buffer level. In fact, an MPEG decoder (not shown) reads the streaming data to capture the context of the data transmitted and stored in the buffer.

Next, with reference to FIG. 5, a method for controlling a transmission rate when providing a real-time streaming service will be described.

When the transmission of the streaming packet corresponding to the new multimedia content begins, the server (310) initializes the initial transmission rate (R ₀ ), setting (S501) its equal to the maximum transmission speed (R _max ) allowed by the network (350).

The server (310) confirms (S503) the level (l _t ) of the buffer transmitted from the client (330). The server (310) sets (S505) whether the transmitted buffer level (l _t ) exceeds the buffer level (θ) required by the client (330).

If, as a result of step (S505), it is established that the transmitted buffer level (l _t ) exceeds the required buffer level (θ), then the server (310) changes the transmission speed from the maximum transmission rate (R _max ) to the normal transmission rate (R _DATA ) .

The normal bit rate (R _DATA ) of streaming data is the bit rate at which the client (310) performs normal playback. However, the initial received transmission rate is the transmission rate (R _DATA ) specified for the packet to be streamed. In other words, the transmission speed (R _DATA ) is the speed at which the multimedia content transmitted from the server (310) is played (S507).

If, as a result of step (S505), it is established that the transmitted buffer level (l _t ) is less than the required buffer level (θ), then the server (310) maintains the maximum transmission speed (R _max ) in order to minimize the initial delay time until the transmitted buffer level (l _t ) does not reach the desired buffer level (θ).

After that, the server (310) changes (S509 - S515) the transmission rate (R _t ) to compensate for the difference between the level (l _t ) of the buffer newly transmitted from the client (330) and the level (θ) of the buffer required by the client (330 ) (or MPEG decoder of the streaming client) until the streaming of the multimedia content to be streamed is completed.

Next, a detailed explanation of the transmission rate control method in steps (S509 to S515) will be provided.

First of all, the server (310) receives from the client (330) a continuously changing current level (l _t ) of the buffer.

The server (310) searches for the accumulated average value over a predetermined period of time associated with the difference between the required level (θ) of the buffer and the reported level (l _t ) of the buffer, and compensates for the value.

The accumulated average value can be calculated (S511) using the following equation 1.

[Equation 1]

wherein the accumulated average value (Δ _t ) is calculated by the server (310) using a predetermined value (N) of the level (l _t ) of the buffer reported by the most recent in time from the client (330); ω ^k is a real number having a value between 0 and 1 and is a weighting factor used to reduce the influence of the level (l _t ) received earlier from the client (330); θ is the required buffer level.

Server (310) calculates the new transmission rate (R _t ) by adding the accumulated average value (Δ _t ) obtained using equation 1 and multiplied by the previous transmission speed (R _t-1 ), again to the previous speed (R _t-1 ) transmission. The new transmission rate (R _t ) can be obtained (S513) using the following equation 2.

[Equation 2]

Having received a new value of the transmission rate, the server (310) transmits the multimedia content to the client (330) with the transmission rate (R _t ) adjusted by Equation 2 and determines (S515) whether the transmission is completed with said transmission rate.

If, as a result of step (S514), it is established that the transmission of multimedia content is not completed, then the server (310) repeats the operations of steps (S509) - (S515) to continuously change the transmission speed (R _t ) in accordance with changes in the buffer level.

By the aforementioned method, the streaming system (300) according to the present invention can prevent the transmission rate instability in accordance with the conventional method in which the transmitted buffer level is simply compared with the requested buffer level in order to adjust the transmission rate up or down.

FIG. 6 is a graphical representation illustrating a change in a client buffer level in the case of using the transmission rate control method described in connection with FIG. 5 in real-time streaming service; in this view, you can see that the buffer level (331) converges to the requested buffer level.

The transmission rate control in accordance with the present invention is implemented by accessing the buffer level during the last time period [t- (N-1), t], which helps to prevent a sudden change in the buffer level. In addition, information on the most recent buffer level is assigned a greater weight to ensure fast and stable convergence of the data rate to the playback speed provided by the client (330).

From the foregoing, it is obvious that the advantage of the proposed system and method for controlling the transmission rate for the provision of real-time streaming services described above in accordance with the present invention is that the buffer overflow or underload that may occur in the case of the provision of streaming services in for a long time, it can be prevented, and this guarantees the possibility of providing streaming services while maintaining an appropriate buffer level at any time.

The described control can be implemented in a relatively simple way, because it is based on the client buffer level.

In addition, the streaming system of the present invention utilizes the maximum transmission rate allowed by the network during initial transmission in order to guarantee the maximum reduction in time from the start of streaming to the actual image or sound.

The present invention can be embodied in a method, device and system. In addition, if you intend to translate the present invention into computer software, then the constituent elements of the present invention can be replaced by code segments necessary for operations. Programs or code segments can be stored on a medium accessed by a microprocessor and transmitted as computer data associated with carrier signals through transmission media or communication networks. The media accessed by the microprocessor includes any media that can transmit and store information, such as electronic circuits, semiconductor memory devices, random access memory (RAM), flash memory blocks, an electrically erasable programmable read-only memory device (EEPROM), floppy disks, optical disks, hard disks, optical fibers and radio networks. Computer data includes data that can be transmitted through the channels of electrical networks, optical fibers, magnetic fields, radio networks.

The present invention has been described above in various details with reference to specific embodiments and drawings. However, these details should not be construed as imposing limitations on the scope of the claims of the present invention, but merely illustrating some of the currently preferred embodiments. It should also be understood that those skilled in the art will be able to make other modifications or replacements to the described embodiments within the broadly interpreted scope of the claims of the invention. The above embodiments and drawings should not be interpreted in isolation from the ideas and aspects of the present invention.

Claims (18)

1. A method for controlling the transmission rate in real-time streaming, which comprises real-time streaming of multimedia content to a client connected to the server via a network, the client having a buffer, wherein the multimedia content is shared and transmitted in accordance with the transmission rate, send to the server a buffer level indicating the amount of multimedia content transmitted to the client stored in the buffer, change the transmission speed to complete the transmission of multimedia ontenta, wherein the server changes the transmission rate by compensating for the accumulated average value for a period of time related to the difference between said buffer layer and specify required buffer level. 2. The method of claim 1, wherein the network is a wireless network, and the wireless network is a wireless local area network (LAN), a Bluetooth network, a Zigbee network, or an ultra-wideband (UWB) network. 3. The method according to claim 1 or 2, in which the buffer level received by the server is included in the application message transmitted from the client (APP message) corresponding to the real-time transmission control protocol (RTCP). 4. The method according to claim 3, in which the buffer level transmitted by the APP message includes at least one of a buffer level of a port number corresponding to a user datagram protocol (UDP), a buffer level of a streaming application, and a buffer level of a decoder corresponding to the standard of the Expert Group on Cinematography (MPEG decoder). 5. The method according to claim 1, in which the transmission of multimedia content with a certain transmission rate is that transmit multimedia content with the maximum transfer rate allowed by the network, accept the buffer level passed from the client, maintaining a maximum transmission rate until the received buffer level reaches said predefined desired buffer level, change to a predetermined transmission rate for reproducing multimedia content if the received buffer level has reached said predetermined desired buffer level. 6. The method according to claim 1, in which the accumulated average value is calculated, wherein the difference between the buffer level adopted for the said time period and the desired buffer level is multiplied by a weight coefficient, summed and averaged, and the difference associated with the most recent in time the accepted buffer level is multiplied by a larger weight coefficient. 7. The method according to claim 6, in which the accumulated average value is calculated using the following equation:

where Δt is the accumulated average value, N is the value of the buffer level received during a predefined period of time, l _tk is the buffer level received at the (tk) interval from the client, θ is the required buffer level, and ω ^k is the weight coefficient having a value from 0 to 1. 8. The method according to claim 7, in which the completion of the transmission provides for the compensation of the accumulated average value by changing said predefined transmission rate in accordance with the following equation:

where Δt is the accumulated average value, R _t-1 is the transmission rate before the change, and R _t is the already changed transmission rate. 9. The method of claim 1, wherein the network is a wired network. 10. Stream system containing a server connected to the network, and this server streams multimedia content in real time by splitting and transmitting multimedia content at a certain transmission rate, and a client including a buffer, and the client at a certain interval transmits to the server a buffer level indicating the amount of multimedia content that has been transferred and stored in the buffer, and the server changes the transmission speed so that the difference between the transmitted buffer level and the desired buffer level is compensated using the accumulated average over a period of time, and the transfer of multimedia content ends the client. 11. The system of claim 10, wherein the network is a wireless network, and the wireless network is a wireless local area network (LAN), a Bluetooth network, a Zigbee network, or an ultra-wideband (UWB) network. 12. The system of claim 10 or 11, in which the buffer level received by the server is included in the application message sent from the client (APP message) corresponding to the real-time transmission control protocol (RTCP). 13. The system of claim 12, wherein the buffer level transmitted by the APP message includes at least one of a buffer level of a port number corresponding to a protocol of user datagrams of a UDP socket, a buffer level of a streaming application, and an MPEG buffer level decoder. 14. The system of claim 10, in which the server starts transmitting multimedia content with the maximum transfer rate allowed by the network, maintains the maximum transfer rate until the buffer level received from the client at a certain interval reaches the desired buffer level for the first time , and changes the level of the buffer to achieve a predetermined transmission rate and transmits multimedia content for playback if the received buffer level reaches the desired buffer level. 15. The system of claim 10, in which the accumulated average value is calculated in such a way that the difference between the level of the buffer adopted for the mentioned period of time and the desired level of the buffer is multiplied by a weight coefficient, summed and averaged, and the difference associated with the most recent the time taken by the buffer level is multiplied by a larger weight coefficient. 16. The system of clause 15, in which the accumulated average value is calculated using the following equation:

where Δt is the accumulated average value, N is the value of the buffer level received over a predefined period of time, l _tk is the buffer level received at the (tk) interval from the client, θ is the required buffer level, and ω ^t is the weight coefficient having a value from 0 to 1. 17. The system according to clause 16, in which the completion of the transmission provides for the compensation of the accumulated average value by changing the aforementioned predetermined transmission speed in accordance with the following equation:

where Δ _t is the accumulated average value, R _t-1 is the transmission rate before the change, and R _t is the already changed transmission rate. 18. The system of claim 10, in which the network is a wired network.

Images