WO2012053834A2

WO2012053834A2 - Streaming service transmitting/receiving device and method

Info

Publication number: WO2012053834A2
Application number: PCT/KR2011/007827
Authority: WO
Inventors: 정순흥; 서광덕; 이진영; 배성준; 심상우; 탕쯔엉공; 박상택; 류원
Original assignee: 한국전자통신연구원; 연세대학교 산학협력단
Priority date: 2010-10-20
Filing date: 2011-10-20
Publication date: 2012-04-26
Also published as: WO2012053834A3

Abstract

A streaming service transmitting and receiving device and method are provided. An arrival interval jitter is an arrival delay time variation between TCP packets, which arrive at a device receiving a streaming service. The device receiving a streaming service determines a de-jitter buffering time to absorb a network jitter before TS packets in a TCP packet are inputted into a T-STD buffer model. An initial playback delay time of the device receiving a streaming service is determined by the de-jitter buffering time.

Description

Streaming service transmission and reception device and method

The present invention relates to a streaming service transmission / reception system, and more particularly, to an apparatus and method for progressive streaming service transmission / reception using de-jitter buffering time estimation.

As a method of implementing a video on demand (VoD) service through an Internet Protocol (IP) network such as the Internet, there are largely a streaming method and a downloading method. .

In a streaming service, a multimedia payload is packetized into a Real-time Transport Protocol (RTP), and then an RTP packet is loaded on a user datagram protocol (UDP) and transmitted. .

UDP, however, is suitable for real-time multimedia transmission but does not provide the ability to improve the quality of service (QoS) by retransmission of packets that are lost during transmission.

Digital video, such as H.264, is compressed by using prediction based coding between frames. Thus, if one of the packets of digital video is lost, the error due to the lost packet is followed until the blocked video is blocked by a subsequent I-frame received after the lost packet. Propagated through the frames.

In other words, RTP / UDP-based real-time multimedia streaming is suitable for a real-time service requiring low delay, but has a disadvantage in terms of guaranteeing QoS.

If the download method is used, the client system downloads the entire video file and then starts playback. Therefore, the user of the client system can enjoy high quality service. However, the download method has the same problems as 1) and 2) below.

1) The user can watch the video after a long download time.

2) The client side must prepare a storage space to download and save the entire video file in advance.

Progressive streaming services are similar to downloads in that they use the transmission control protocol (TCP) as the transport protocol. In addition, the progressive streaming service is similar to the real-time streaming in that it plays the media data already received in real time while the file including the media data is downloaded. In conclusion, progressive streaming service can be regarded as a technology that utilizes only the advantages of both download and streaming technologies.

MPEG transport stream (TS) is a transmission protocol standard standard for multiplexing compressed data of audio and video and delivering multiplexed compressed data through a broadcasting network such as DVB, ISDB, ATSC, DMB, and the like.

Based on progressive streaming technology, a streaming service for transmitting MPEG TS through a wired or wireless IP network by using HTTP and TCP protocols may be provided.

According to one embodiment of the present invention, a streaming service transmitting / receiving apparatus and method for transmitting a TCP packet including information necessary for calculating arrival interval jitter may be provided.

One embodiment of the present invention can provide a streaming service transmission and reception apparatus and method for calculating the arrival interval jitter based on the arrival time and the expected arrival time of the TCP packet.

According to one aspect of the invention, the operation of loading one or more TS packets in the payload of the TCP packet, setting the first PCR information in the one or more TS packets in the payload to the PCR of the TCP packet, Calculating a timestamp of the TCP packet based on a PCR, recording the timestamp in the TCP packet, and transmitting the TCP packet to a terminal, wherein the timestamp is sent to the terminal of the TCP packet. A streaming service transmission method is provided, which is the expected arrival time of.

The streaming service transmission method may further include checking whether there is a TS packet including the PCR information in a TS header among the one or more TS packets loaded in the payload.

The timestamp may represent a time at which the first byte of the payload arrives at the terminal as a system time clock count value using the system clock frequency of the streaming server.

The time stamp may include a base clock and an extended clock, and the base clock may use a smaller frequency unit than the extended clock.

The length of the bits of the timestamp may be determined based on the precision of the arrival time delay variation required by the terminal.

Writing the timestamp in the TCP packet may store the timestamp in an options field in the header of the TCP packet.

According to another aspect of the invention, the TS packet load unit for loading one or more TS packets in the payload of the TCP packet, setting the first PCR information in the one or more TS packets in the payload to the PCR of the TCP packet A PCR setting unit, a timestamp calculating unit for calculating a timestamp of the TCP packet based on the PCR, a timestamp recording unit for recording the timestamp in the TCP packet, and a transmission unit for transmitting the TCP packet to the terminal; A time stamp is provided with a streaming service transmitting apparatus, wherein the TCP packet is an expected arrival time of the TCP packet.

According to yet another aspect of the present invention, an operation of receiving a first TCP packet, measuring a first clock count of a system time clock at the moment when the first TCP packet is received, and receiving the first TCP packet from the first TCP packet Extracting a first timestamp of the TCP packet, calculating a de-jitter buffering time based on the first clock count, the first timestamp, the second clock count, and the second timestamp; The first timestamp is the reception time of the first TCP packet expected by the server, and the second clock count is a system time clock at the moment a second TCP packet received before the TCP packet is received, and the second time The stamp is provided with a streaming service receiving method, wherein the server is the time of receipt of the second TCP packet expected by the server extracted from the second TCP packet.

The operation of calculating a de-jitter buffering time based on the first clock count, the first timestamp, the second clock count, and the second timestamp may include: generating a first jitter based on the first clock count and the first timestamp; Calculating a first network jitter experienced by a TCP packet, a first arrival between the second TCP packet and the first TCP packet based on the first network jitter and the second network jitter experienced by the second TCP packet Calculating a time interval and calculating the de-jitter buffering time based on the first network jitter and the first arrival time interval.

The calculating of the de-jitter buffering time based on the first network jitter and the first arrival time interval may include: determining the de-jitter buffering time based on the second arrival interval jitter and the first arrival time interval of the second TCP packet. Calculating a first arrival interval jitter of one TCP packet, wherein the arrival interval jitter is an arrival interval delay variation between successive arriving packets, a second arrival interval variance of the second TCP packet, the first arrival time Calculating a first arrival interval variance of the first TCP packet based on an interval and the first arrival time interval, wherein the arrival time variance is a variance for the arrival time interval and a first first arrival interval jitter and the Calculating the de-jitter buffering time based on a first arrival interval variance.

The first timestamp may be in an options field in the header of the first TCP packet.

The streaming service receiving method may further include determining a final de-jitter buffering time among the de-jitter buffering time and the analyzing time, wherein the analyzing time is a time sufficient to receive and analyze the received TCP packets. Can be.

The streaming service receiving method may further include setting the de-jitter buffering time to a buffering time of a transport stream system target decoder of the streaming client.

According to yet another aspect of the present invention, a receiver for receiving a first TCP packet, a clock count measuring unit for measuring a first clock count of the system time clock at the moment the first TCP packet is received, from the first TCP packet A timestamp extractor extracting a first timestamp of the first TCP packet and a de-jitter buffering time calculated based on the first clock count, first timestamp, second clock count, and second timestamp A jitter buffering time calculation unit, wherein the first timestamp is a reception time of the first TCP packet expected by the server, and the second clock count is a value of receiving a second TCP packet received before the TCP packet; And a second system time clock, wherein the second time stamp is a reception time of the second TCP packet expected by the server extracted from the second TCP packet. It is provided.

Provided are a streaming service transmitting / receiving apparatus and method for transmitting a TCP packet including information necessary to calculate arrival interval jitter.

Provided are a streaming service transmitting / receiving apparatus and method for calculating arrival interval jitter based on arrival time and expected arrival time of a TCP packet.

1 illustrates an overview of an HTTP-based progressive streaming service according to an example of the present invention.

2 shows an overview of an MPEG TS based progressive streaming service according to an embodiment of the present invention.

3 illustrates a structure of an MPEG TS packet according to an embodiment of the present invention.

4 illustrates a standard T-STD buffer model according to an example of the present invention.

5 illustrates a change in packet arrival delay time in an IP network according to an embodiment of the present invention.

6 illustrates a TCP payload format composed of n MPEG TS packets according to an embodiment of the present invention.

7 illustrates a structure of a TCP packet including TS packets including PCR information in a header according to an embodiment of the present invention in a TCP payload.

8 illustrates a timing model for estimating the time at which a TCP packet arrives at a receiver according to an example of the present invention.

9 is a flowchart illustrating a method of operating a transmitting side according to an embodiment of the present invention.

10 illustrates a basic structure of a header 310 of a TCP packet and an extension field that may be selectively used according to an embodiment of the present invention.

11 is a structural diagram of a transmitting side 1100 according to an embodiment of the present invention.

12 is a flowchart illustrating a method of operating a receiving side according to an embodiment of the present invention.

13 illustrates a T-STD buffer model including de-jitter buffering according to an embodiment of the present invention.

14 is a structural diagram of a receiving side 1400 according to an example of the present invention.

15 illustrates a delay of media playback time due to de-jitter buffering configured according to an embodiment of the present invention.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited or limited by the embodiments. Like reference numerals in the drawings denote like elements.

The terminal (ie, client) 110 transmits a "GET Request" 130 of the HTTP protocol to inform the server 120 of the service request for the desired content file.

The server 120 transmits the content file to the terminal 110. The server 120 uses TCP as a transport protocol for transmitting the content file to guarantee QoS.

The content file arrives at the terminal 110 by TCP.

The terminal 110 buffers some data of an initially arrived content file.

The terminal 110 displays the data in the TCP packet arriving in real time from when the TCP packet arriving at the terminal 110 fills the buffer by a predetermined amount in order to reduce the service delay time, which is a problem of the conventional download scheme. . That is, the terminal 110 parses some data of the buffered content file to play out video and audio.

Therefore, the HTTP-based progressive streaming service is differentiated from the download method in which the content file is played only after all data of the content file arrives at the terminal 110. In addition, the progressive streaming service based on HTTP can reduce service delay time, which is a problem of the conventional download method, and can guarantee QoS.

Content files transmitted from existing progressive streaming services are QuickTime (.mov), MPEG-4 (.mp4) and Windows Media Video (Windows Media Video) to store compressed video and audio as a single file. (.wmv), Flash Video (.flv), and 3GPP (.3gp) file formats.

These file format specifications are extended based on the ISO based media file format (ISO / IEC 14496-12 (MPEG-4 Part 12)) developed by ISO / IEC MPEG. Multimedia content files produced based on these various file format standards are widely produced, distributed, and distributed through wired and wireless IP networks by progressive streaming services.

Meanwhile, due to convergence of broadcasting and communication, Internet TV and IPTV services, which provide broadcasting services through an IP network, have been expanded.

Instead of the existing mov, mp4, wmv, flv, and 3gp file formats, progressive streaming using broadcast contents produced in the MPEG TS standard may be provided.

2 shows an overview of a service for providing multimedia content produced by the MPEG TS standard by HTTP progressive streaming.

The HTTP progressive streaming server can produce (i.e. encode) video / audio compressed data in MPEG TS standards compressed with various multimedia encoders for use as MPEG TS broadcast content. The produced MPEG TS content may be serviced through HTTP progressive streaming.

In addition, the HTTP progressive streaming server may utilize the MPEG TS content obtained by converting a content file produced by an existing file format standard into MPEG TS for the HTTP progressive streaming service.

A client using HTTP progressive streaming simply requests the desired MPEG TS content from the server through an HTTP GET request.

In embodiments of the present invention to be described below, when MPEG TS content is transmitted to a client through an IP network (eg, a wired or wireless Internet) by a progressive streaming service, a program clock reference included in a header of an MPEG TS packet. A method and apparatus for effectively estimating network jitter by utilizing PCR clock information are disclosed. Network jitter is the arrival time delay variation between TCP packets arriving at the receiving end.

Embodiments of the present invention may estimate inter-arrival jitter, which is a delay variation in arrival time between packets based on network jitter.

De-jitter required to absorb network jitter in designing the Transport Stream System Target Decoder (T-STD) buffer model at the receiving end based on the estimated intra-arrival jitter of the packet. Appropriate size of buffering can be determined.

The de-jitter buffering of the terminal absorbs the difference in arrival time with the packets arriving later by not allowing the receiving side to start the playback process immediately when the first packet arrives at the receiving side. Thus, the de-jitter buffering allows the receiving side to start playing after collecting the packets to some extent so that the original time interval between the packets can be maintained.

Therefore, the length of de-jitter buffering is that after the first packet arrives at the receiver, before the receiver performs the full video and audio playback process, the first packet must wait in the file buffer. Decide when to do it.

If the size of the de-jitter buffering time is set too small, sufficient time margin required to absorb network jitters cannot be provided. Therefore, a problem of buffer underflow is likely to occur within an early time after the reproduction on the receiving side. This causes frequent rebuffering at the receiver during streaming between the sender and receiver. Such refurbishing causes interruption of playback, which severely degrades the quality of service.

Conversely, if the size of the de-jitter buffering time is set too large, the initial playout process immediately following the de-jitter buffering is delayed in proportion to the size of the buffering. Thus, the latency for services experienced by the user increases.

Embodiments of the present invention are directed to the size of the appropriate de-jitter buffering time and the size of the de-jitter buffering required to absorb arrival interval jitter through estimation of arrival interval jitter of TCP packets in an MPEG TS based progressive streaming service. A method and apparatus for determining a corresponding initial playout delay is disclosed.

The initial play delay time corresponds to the time after the first packet arrives at the receiver, and the first packet stays for de-jitter buffering in the TCP file buffer before the packet is delivered to the T-STD buffer model. Thus, the initial playback delay time determines the initial playback time point for the received media (or content).

The MPEG TS packet 300 is a fixed length packet of size 188 bytes.

MPEG TS is a continuous stream of MPEG TS packets 300.

In FIG. 3, the structure of the header 310 and payload 320 of the MPEG TS packet 300 is shown. In addition, the fields in the header 310 are shown hierarchically. The number below the field indicates the size (in bits) of the field.

The header 310 is basically 4 bytes in size (sync byte field to continuity counter field). The size of the header 310 can be increased according to the presence or absence of an adaptation field and an optional field existing to deliver various extended information.

MPEG TS is a standard for effectively multiplexing and delivering video and audio compressed data transmitted from a digital broadcasting service.

The elementary stream of compressed video / audio is packetized elementary stream (PES) packetized. The PES packet is finely divided into fixed length TS packets of 188 bytes in size, and the divided TS packets are transmitted by properly multiplexing video and audio.

MPEG TS is a standard developed for the purpose of digital broadcasting service. Therefore, TS packets are delivered to the receiver through a broadcasting network which is a circuit switched network in which channel quality is relatively stable. Therefore, the packet delay time experienced by MPEG TS packets in the transport channel is relatively short and constant. In addition, a timing buffer model for sequentially processing TS packets arriving at the receiver may also be effectively applied.

In the MPEG system standard (ISO / IEC 13818-1 (Part 1)), the timing buffer model at the receiving side for processing MPEG TS is called a Transport Stream System Target Decoder (T-STD) buffer model. .

In FIG. 4, a structure of a standard T-STD buffer model is disclosed in which multiplexed TS packets related to video, audio, and system information received by multiplexing MPEG MPEG are demultiplexed, and the demultiplexed streams are sequentially buffered.

MPEG TS aims to deliver multimedia data over circuit switched networks with relatively high bandwidth quality and low propagation delay, such as broadcast networks or Asynchronous Transfer Mode (ATM) networks. Developed.

Therefore, in order to transmit MPEG TS over the Internet, which is a packet switched network such as an IP network, the delay variation in arrival time between packets, which is a chronic problem of the Internet, may be estimated accurately. Required. In addition, the standard T-STD model of FIG. 4 designed for MPEG TS transmission through a broadcasting network is also required to be modified based on this estimation.

When packets are transmitted through the wired / wireless Internet based on the IP network, the routing paths through which each of the packets may be different from each other, unlike when the packets are transmitted through the existing broadcasting network. In addition, since the amount of traffic to be processed and the processing time for each router through each of the packets are different from each other, the destination arrival times of each of the packets may be different from each other.

An embodiment of the present invention to be described below discloses a method for accurately estimating arrival delay time variation between TCP packets occurring when MPEG TS is delivered to a client via a progressive streaming service in a wired or wireless Internet environment. In addition, embodiments of the present invention to be described below disclose a modified T-STD model based on the estimated arrival delay variation.

FIG. 5 shows an example of a dynamic variation in the arrival delay time that packets undergo when they are delivered over a general IP network.

In FIG. 5, packets of size h bytes are transmitted through the IP network.

The packets undergo a transmission delay of x1, x2 and x3 through the IP network and arrive at the receiver at t1, t2 and t3 time instants, respectively.

By the way, while time x1 and time x2 are equal to each other, time x3 is larger than x1 and x2. Thus, a variation of arrival delay time by b seconds occurs between t2 and t3. Although the playout gap, such as this variation, should be eliminated, the decoder based on the T-STD buffer model operates smoothly without experiencing buffer overflow and buffer underflow.

Therefore, to absorb this temporal gap before arriving packets are delivered to the decoder, a playout delay through a separate buffering process such as de-jitter buffering is required.

In order to determine an appropriate size required for such a separate buffering process, a method for accurately estimating the arrival time delay variation between packets is required.

If the size of the de-jitter buffering is too small, the risk of buffer underflow increases, resulting in frequent refurbishing. Conversely, if the size of the de-jitter buffering is too large, the buffering time for the arrived data becomes long, resulting in latency of service resulting in poor service quality.

In order to solve this problem, in an embodiment of the present invention to be described later, a method for effectively estimating inter-arrival jitter by utilizing a program clock reference (PCR) value at the receiver side. And a device. Inter-arrival jitter is a variation in arrival delay time between packets.

PCR may be present in an MPEG TS packet header carried in a TCP packet. The computational and implementation complexity of the method and apparatus for estimating inter-arrival jitter by utilizing the PCR information recorded in the TS packet is low.

MPEG TS packets have a fixed size of 188 bytes. Therefore, several TS packets may be carried in the payload of one TCP packet. Considering that the size of the Ethernet maximum transfer unit (MTU) is 1500 bytes, theoretically, up to seven TS packets may be carried in the payload of TCP.

In FIG. 6, a TCP payload format is shown that maps n TS packets into the payload of one TCP packet.

In order to transfer time information used for synchronization between System Time Clocks (STCs) used at the transmitter and the receiver, PCR is present in the header of the TS packet.

The PCR value is a value obtained by sampling the time at the system encoder at a system clock frequency (SCF) of 27 MHz.

The PCR clock can consist of a total of 42 bits. The PCR clock may consist of a 33-bit PCR_base expressed in units of 90 KHz (ie, SCF / 300) and 9 bits of PCR_ext expressed in units of 27 MHz (ie, SCF).

PCR is typically transmitted periodically within 100 ms to maintain STC synchronization between the transmitter and receiver.

Accordingly, there may be a TS packet including PCR information in a header among TS packets constituting the payload format of the TCP packet.

In FIG. 7, payload formats of TCP (n) 710 and TCP (n + 1) 750 are shown, respectively.

TCP (n) 710 is an n th TCP packet containing TS packets including PCR information in a header in a TCP payload.

TCP (n + 1) 750 is the n + 1 th TCP packet which contains TS packets including PCR information in the header in the TCP payload.

That is, TCP (n) and TCP (n + 1) are two TCP packets whose transmission order sequentially neighbors, while including at least one TS packet including PCR information in a header in the TCP payload.

In an embodiment of the present invention to be described below, a method of using PCR information included in the TCP payload to estimate the arrival time delay variation of the TCP packet arriving at the receiving side.

TCP (n), which is the nth TCP packet, includes a plurality of consecutive TS packets in the payload, in a format as described above with reference to FIG. 7. N is an integer of 0 or more.

The horizontal axis of FIG. 8 represents a theoretical time at which a TCP packet arrives at a receiving side and a corresponding TCP timestamp value. The vertical axis represents the index value of each successive data byte sent to the receiver.

TCP (n) utilizes the PCR value of the first TS packet including PCR information as PCR (n), which is a PCR value representative of TCP (n).

n is the index of TCP. That is, the value of n in TCP (n).

PCR (n) is the index of the first PCR in TCP (n).

i _n is the byte index of PCR (n), and i _{n + 1} is the byte index of PCR (n + 1).

The time t (i) at which the i-th data byte arrives at the receiver may be estimated according to Equation 1 below.

Where R (n) is the data rate between the PCR (n) and PCR (n + 1) clocks (ie, the transport rate of the transport stream). R (n) may be calculated according to Equation 2 below.

The time t (n + 1) at which the first byte of the payload of TCP (n + 1) arrives at the receiver may be calculated according to Equation 3 below.

Here, I represents the size of all the data transmitted in bytes by being located ahead of the byte including the last bit indicating the information of PCR (n + 1) among all data included in TCP (n + 1).

That is, t (n + 1) is calculated based on PCR (n + 1), I and R (n).

The transmitting side is a streaming service transmitting apparatus, and the receiving side is a streaming service receiving apparatus.

Operations 910 to 970 show a streaming service transmission method performed by a transmitting side.

At operation 910, one or more TS packets are mapped in TCP (n + 1). That is, one or more TS packets are loaded into the payload of TCP (n + 1).

In operation 920, it is checked whether a TS packet containing PCR information in the TS header exists in the payload of TCP (n + 1). That is, it is checked whether there is a TS packet including PCR information in the TS header among one or more TS packets loaded in the payload of TCP (n + 1).

If a TS packet including PCR information in the TS header exists in the payload of TCP (n + 1), operation 930 is performed, otherwise operation 960 is performed.

That is, when PCR information is not recorded in the headers of one or more TS packets in the payload of TCP (n + 1), TCP time stamps to be described later are not calculated and recorded for TCP (n + 1).

In operation 930, PCR (n + 1) is set.

Initial PCR information in one or more TS packets in the payload of TCP (n + 1) is set to PCR (n + 1) of TCP (n + 1).

In operation 940, a timestamp TCPt (n + 1) of TCP (n + 1) is calculated.

The timestamp TCPt (n + 1) is the expected arrival time to the receiver side of the theoretically expected TCP (n + 1).

The timestamp of TCP may represent a time reference value for streaming data in the TCP packet stream. The visual reference value may be calculated based on the STC from which the PCR is derived.

TCPt (n + 1) is a time corresponding to t (n + 1) of FIG. 8 (that is, the time at which the first byte of the payload of TCP (n + 1) is estimated to arrive at the receiving side). It may be expressed as a clock count value of the STC using.

TCPt (n + 1), like PCR (n + 1), can be divided into a base clock TCPt_base (n + 1) and an extension clock TCPt_ext (n + 1). That is, TCPt (n + 1) may include a base clock and an extended clock.

The base clock can use smaller frequency units than the extended clock.

The base clock may be expressed in units of 90 KHz (ie SCF / 300). The extended clock may be expressed in units of 27 MHz (ie SCF).

The timestamp TCPt (n + 1), the base clock TCPt_base (n + 1) and the extended clock TCPt_ext (n + 1) can be calculated according to Equation 4 below.

That is, TCPt (n + 1), TCPt_base (n + 1) and TCPt_ext (n + 1) may be calculated based on the system clock frequency and t (n + 1). TCPt (n + 1) has a precision of 27 MHz.

By Equation 4, TCPt_base (n + 1) may be represented by a total of 33 bits in length, TCPt_ext (n + 1) may be represented by a total of 9 bits in length, and TCPt_base (n + 1) and TCPt_ext TCPt (n + 1), which is the sum of (n + 1), can be represented by a total length of 42 bits as in PCR.

A modulo operation in which TCPt (n + 1) is reset to zero again when TCPt (n + 1) has reached its maximum value can be used for the calculation of TCPt (n + 1).

The maximum error that occurs when the time is expressed in a clock of 90 KHz is

to be.

Therefore, depending on the requirements of the application, only TCPt_base (n + 1) may be used when the accuracy of the arrival time delay variation may exceed 11 μs. The maximum error of 11 μs that can occur due to the use of only TCPt_base (n + 1) is included in the arrival time delay variation itself.

On the other hand, in an application where an error of 11 mu s cannot be tolerated in the precision of the arrival time delay variation, a full length of 42 bits TCPt (n + 1) may be used.

That is, the length of the bits of TCPt (n + 1) may be determined based on the precision of the arrival time delay variation required by the receiver.

In operation 950, TCPt (n + 1) is written into TCP (n + 1).

TCPt (n + 1) may be stored in the header of TCP (n + 1) and may be stored in the options field in the header.

A specific method of storing TCPt (n + 1) in the header of TCP (n + 1) is described in detail below with reference to FIG.

In operation 960, TCPt (n + 1) is sent to the receiving side.

In operation 970, n is incremented by 1 for the next TCP packet to be processed, and operation 910 is performed again.

In the present embodiment, the transmitting side for generating and transmitting the TCP packet may be a streaming server and may be an HTTP streaming server. In addition, the receiving side receiving the TCP packet may be a client or a terminal. It is possible to receive a TCP packet through HTTP streaming on the receiving side.

Technical contents according to an embodiment of the present invention described above with reference to FIGS. 1 to 8 may be applied to the present embodiment as it is. Therefore, more detailed description will be omitted below.

By default, the header 310 of the TCP packet is 20 bytes in size. In addition, the TCP packet may add header information up to 32 bytes by utilizing the Options field 1010 to include additional information.

Whether the options field 1010 is added or not may be specified through a data offset 1020 (indicating the length of the header).

In the embodiment described above with reference to FIG. 9, the calculated TCP timestamp value may be added to the options field 1010.

The options field can be largely composed of three fields, such as Kind, Length, and Data.

The 1-byte type field indicates the purpose.

The 1-byte length field indicates the total size of the options field corresponding to the current type field.

The data field contains the actual data needed for the purpose indicated by the kind field.

The transmitting side 110 is a streaming service transmitting apparatus (eg, a streaming server).

The transmitting side 1100 includes a TS packet load unit 1110, a PCR setting unit 1120, a time stamp calculation unit 1130, a time stamp recording unit 1140, and a transmission unit 1150.

The TS packet load unit 1110 performs

operations

910, 920, and 970. For example, the TS packet load unit 1110 loads one or more TS packets in the payload of the TCP packet.

The PCR setting unit 1120 performs operation 930. For example, the PCR setting unit 1120 sets first PCR information in one or more TS packets in a payload to PCR of a TCP packet.

The time stamp calculator 1130 performs operation 940. For example, the time stamp calculator 1130 calculates a time stamp of the TCP packet based on the PCR.

The time stamp recording unit 1140 performs operation 950. For example, the time stamp recording unit 1140 records the calculated time stamp in the TCP packet.

The transmitter 1150 performs operation 960. For example, the transmitter 1150 transmits a TCP packet to the terminal.

The operations 1210 to 1295 illustrate a streaming service receiving method performed by the receiving side.

The receiving side receives the TCP packet transmitted by the transmitting side described with reference to FIG. The receiving side utilizes the value of TCP timestamp TCPt (n + 1), which is carried in a TCP packet.

That is, through the difference between the theoretical value of TCPt (n + 1), which is the arrival time of the receiver, and the count value of the STC clock at the actual arrival of the receiver, the receiver receives the TCP packet through the IP network. Estimates of the delay variation of the arrival time experienced.

In operation 1210, receive a TCP packet. The received TCP packet is the n + 1 th packet. Name the n + 1th TCP packet as the first TCP packet. In addition, the n th packet received before the n + 1 th packet is referred to as a second TCP packet. In addition, the clock count of the first TCP packet described below may be referred to as a first clock count.

In operation 1220, the clock count TCPa (n + 1) is measured.

TCPa (n + 1) is a clock count value of the receiving STC at the moment when the n + 1th TCP packet is received. In other words, TCPa (n + 1) is the clock count of the first TCP packet. STC may be based on 27Mhz.

In operation 1230, it is checked whether a timestamp TCPt (n + 1), which is a TCP timestamp, is present in the received TCP packet.

TCPt (n + 1) is the time when the receiving side expected by the transmitting side of the TCP packet receives the TCP packet.

The TCP timestamp may be in the options field in the header of the TCP packet. Thus, it can be checked whether a TCP timestamp exists in the options field in the header of the received TCP packet.

If TCPt (n + 1) is present in the received TCP packet, operation 1240 is performed, otherwise operation 1260 is performed.

In operation 1240, TCPt (n + 1) is extracted from the TCP packet.

TCPt (n + 1) may be extracted from the options field in the header of the TCP packet.

In operation 1245, network jitter N (n + 1) experienced by TCP (n + 1) is calculated. The network jitter N (n + 1) may be calculated according to Equation 5 below.

Network jitter N _base (n when TCPt_base (n + 1) of length 33 bits with a 90 KHz precision is recorded in the TCP packet TCP (n + 1) (or optional fields of the header of the TCP packet) as a timestamp. +1) may be calculated according to Equation 6 below.

TCPa_base (n + 1) is a base clock of TCPa (n + 1) and has a unit of 90 KHz.

The network jitter N _base (n + 1) according to Equation 6 has an error in the range as shown in Equation 7 below, compared to the network jitter N (n + 1) according to Equation 5.

Thus, in an application performed at the receiving side, only a base clock with a precision of 90 KHz can be used to calculate network jitter, unless an error of about 11 s is largely a problem.

TCP (n) is a packet that arrives before TCP (n + 1) arrives at the receiving end. In other words, TCP (n) and TCP (n + 1) are neighboring packets. TCPt (n) is the timestamp of TCP (n). TCPa (n) is the clock count value of the receiving STC at the moment when TCP (n) is received. N (n) is the network jitter experienced by TCP (n).

In operation 1250, an inter packet spacing D (n + 1) between TCP (n) and TCP (n + 1) is calculated.

Arrival time interval D (n +) between TCP (n) and TCP (n + 1) when TCPt (n + 1) and TCPt (n) are used, TCP time stamps of 42 bits in length with a precision of 27 MHz 1) may be calculated according to Equation 8 below.

In addition, when TCPt_base (n + 1) and TCPt_base (n), which are 33-bit long TCP timestamps with a precision of 90 KHz, are used, the arrival time interval D _base between TCP (n) and TCP (n + 1) (n + 1) may be calculated according to Equation 9 below.

An arrival time interval between TCP (n) and TCP (n + 1) obtained based on Equations 8 and 9 may be represented by D (n + 1).

In operation 1255, arrival interval jitter and variance and de-jitter buffering times of the arrival interval jitter are calculated.

Arrival interval jitter is the arrival interval delay variation between successive arriving packets.

The arrival interval jitter J (n + 1) of TCP (n + 1) may be calculated according to Equation 10 below by a moving average.

Where J (n) is the arrival interval jitter of TCP (n).

α (0 ≦ α ≦ 1) is the first smoothing factor. α can adjust the reflectance ratio of recently updated network jitter.

ν (n + 1) is the variance for J (n + 1). In other words, v (n + 1) is the arrival interval variance of J (n + 1). v (n) is the variance over J (n).

ν (n + 1) may be calculated by the moving average according to Equation 11 below.

Where β (0 ≦ β ≦ 1) is the second smoothing factor. β can control the rate of response to variations between recently updated network jitter and existing average jitter.

B (n + 1) is the de-jitter buffering time for effectively absorbing J (n + 1).

Based on Equations 10 and 11, B (n + 1) may be calculated according to Equation 12 below.

Here, K is a parameter for reflecting the fluctuation range of the variance υ (n + 1), which is a deviation from the average arrival interval jitter value in determining the de-jitter buffering time.

By operation 1255, B (n + 1) is calculated based on N (n + 1) and D (n + 1).

By operations 1240 through 1255, B (n + 1) is calculated based on TCPt (n + 1), TCPt (n), TCPa (n + 1) and TCPa (n).

The larger the value of K, the greater the probability that an unusually large fluctuation that can occur in a particular situation in the network can be absorbed by de-jitter buffering.

By B (n + 1), among the packets for a particular file (or content), the initial playout delay time for the first packet to arrive at the receiver is experienced through de-jitter buffering. Can be determined. In addition, based on the determined initial reproduction delay time, an initial reproduction time of the progressive streaming service may be determined in consideration of absorption of network jitter occurring during transmission of a packet through the IP network.

In operation 1260, it is checked whether the analysis time for de-jitter buffering time determination has elapsed.

In order to estimate the appropriate de-jitter buffering time by utilizing the timestamp of the received TCP packet (ie TCP (n + 1)), TCP packets must be received and analyzed for some sufficient time. The time required for such reception and analysis is called the analysis time T.

Sufficient time for the transmission of a TCP packet to a stable packet transmission and reception state may be sufficient as analysis time. Thus, a time of about 3 to 4 seconds is sufficient as the analysis time.

Operation 1280 is performed when analysis time T for determining the de-jitter buffering time has elapsed, otherwise operation 1270 is performed.

At operation 1270, n is incremented by 1 to process the next packet, and operation 1210 is performed.

In operation 1280, it is checked whether the calculated de-jitter buffering time B (n + 1) is greater than the analysis time T. Operation 1290 is performed if the calculated de-jitter buffering time B (n + 1) is greater than analysis time T; otherwise operation 1295 is performed.

In operation 1290, the calculated de-jitter buffering time B (n + 1) is determined as the final de-jitter buffering time t _dj .

In operation 1295, analysis time T is determined as the final de-jitter buffering time t _dj .

That is, in

operations

1280, 1290 and 1295, the larger of B (n + 1) and T is determined as the final de-jitter buffering time t _dj, as shown in Equation 13 below.

Operation 1290 or operation 1295 is performed, followed by operation 1270.

The T-STD buffer model provides de-jitter buffering to absorb jitter that occurs when receiving progressive streaming service over an IP network, based on the arrival interval jitter according to the embodiment of the present invention described above.

That is, the de-jitter buffering time B (n + 1) or t _dj calculated in FIG. 12 may be set as the buffering time of the T-STD of the receiving side.

IP network de-jitter buffering to absorb network jitter takes place inside the file buffer at the receiving end.

After de-jitter buffering, TS packets delivered to the existing T-STD buffer model may be processed according to the operating principle of the existing T-STD buffer model.

The receiving side 1410 is a streaming service receiving apparatus (eg, a terminal).

The receiving side 1400 includes a receiving unit 1410, a clock count measuring unit 1420, a timestamp extracting unit 1430, and a de-jitter buffering time calculating unit 1440.

The receiver 1410 may include a T-STD buffer 1450.

The receiver 1410 performs operation 1210. For example, the receiver 1410 receives TCP (n + 1) and TCP (n).

The clock count measurer 1420 performs operation 1220. For example, the clock count measuring unit 1420 measures the clock count TCPa (n + 1) at the moment when TCP (n + 1) is received, and measures the clock count TCPa (n) at the moment when TCP (n) is received. do.

The time stamp extractor 1430 performs

operations

1230 and 1240. For example, the time stamp extractor 1430 checks whether TCPt (n + 1) exists in TCP (n + 1), and extracts TCPt (n + 1) from TCP (n + 1).

The de-jitter buffering time calculator 1440 performs operations 1244 to 1260, 1280, 1290, and 1295. For example, the de-jitter buffering time calculator 1440 may calculate the de-jitter buffering time B (n + 1) based on TCPt (n + 1), TCPt (n), TCPa (n + 1), and TCPa (n). Or t _dj .

In addition, the de-jitter buffering time calculator 1440 may include N (n + 1), N (n), N _base (n + 1), J (n + 1), J (n), and D (n + 1). ), υ (n + 1), υ (n) and the like.

Technical contents according to an embodiment of the present invention described above with reference to FIGS. 1 to 13 may be applied to the present embodiment as it is. Therefore, more detailed description will be omitted below.

The temporal modeling shown in FIG. 15 is as follows 1) to 3).

1) The first TCP packet arriving at the file buffer stays in the file buffer for the de-jitter buffering time determined by the embodiment of the present invention described above.

2) Since the first TCP packet stays in the file buffer, the playback time of the content represented by the TCP packets is delayed.

3) Immediately after the set de-jitter buffering time, TS packets included in the first packet are transferred to the existing T-STD buffer model.

In Fig. 15, the x axis represents the time at the transmitting side and the receiving side, and the y axis represents the index of the transmitted or received TCP packet.

In FIG. 15, the time 1510 at which the sender transmitted TCP packets, the time 1520 at which the receiver received TCP packets, and the delivery deadline 1530 in the conventional T-STD buffer model are shown. Also shown is a time point 1540 where TS packets in the first TCP packet are delivered to a conventional T-STD buffer for playback.

The end-to-end transmission delay is the difference between the time at which the TCP packets were sent 1510 and the time at which the TCP packets were received 1520, and the de-jitter buffering time (ie, the playback delay time) is the TCP packet. Difference between the reception time 1520 and the propagation deadline 1530 in a typical T-STD buffer model.

The de-jitter buffering time according to the embodiment of the present invention described above should be considered integrally with the T-STD buffer model of the receiver in order to provide a high quality progressive streaming service.

Therefore, in the new T-STD buffer model including the de-jitter buffering process, except for the newly added de-jitter buffering process, the operation principle of the existing T-STD buffer model can be followed as it is. The T-STD buffer model developed as a standard can be utilized without modification in the method and apparatus according to embodiments of the present invention.

Method according to an embodiment of the present invention is implemented in the form of program instructions that can be executed by various computer means may be recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains various modifications and variations from such descriptions. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the claims below but also by the equivalents of the claims.

Claims

Loading one or more TS packets into the payload of the TCP packet;

Setting initial PCR information in the one or more TS packets in the payload to PCR of the TCP packet;

Calculating a timestamp of the TCP packet based on the PCR;

Recording the timestamp in the TCP packet; And

Transmitting the TCP packet to a terminal

And the timestamp is an expected arrival time of the TCP packet to the terminal.
The method of claim 1,

Checking whether there is a TS packet including the PCR information in a TS header among the one or more TS packets loaded in the payload

Further comprising, the streaming service transmission method.
The method of claim 1,

And the timestamp expresses the time at which the first byte of the payload arrives at the terminal as a system time clock count value using the system clock frequency of the streaming server.
The method of claim 3,

The estimated time is calculated based on Equation 1 below.

Where t (n + 1) is the estimated time, PCR (n + 1) is the PCR, and R (n) is the data between the PCR of the previous TCP packet received before the TCP packet and the PCR It is a transmission rate, and I represents the size of all data transmitted in bytes in advance of the byte including the last bit indicating PCR information among all data included in the TCP packet.
The method of claim 4, wherein

R (n) is calculated based on Equation 2 below.

Where i n is the byte index of PCR (n) and in + 1 is the byte index of PCR (n + 1).
The method of claim 1,

The timestamp includes a base clock and an extended clock, and the base clock uses a frequency unit smaller than the extended clock.
The method of claim 6,

The timestamp is calculated based on Equation 3 below, streaming service transmission method.

Where TCPt_base (n + 1) is the base clock, TCPt_ext (n + 1) is the extended clock, TCPt (n + 1) is the timestamp, system_clock_freq is the system clock frequency of the streaming server, T (n + 1) is a time when the first byte of the payload is estimated to arrive at the terminal.
The method of claim 1,

The length of the bits of the timestamp is determined based on the precision of the arrival time delay variation required by the terminal.
The method of claim 1,

Recording the timestamp in the TCP packet stores the timestamp in an options field in a header of the TCP packet.
A TS packet load unit that loads one or more TS packets in a payload of the TCP packet;

A PCR setting unit for setting first PCR information in the one or more TS packets in the payload to PCR of the TCP packet;

A timestamp calculator for calculating a timestamp of the TCP packet based on the PCR;

A timestamp recorder for recording the timestamp in the TCP packet; And

Transmitter for transmitting the TCP packet to the terminal

And the timestamp is an expected arrival time of the TCP packet to the terminal.
Receiving a first TCP packet;

Measuring a first clock count of a system time clock at the moment the first TCP packet is received;

Extracting a first timestamp of the first TCP packet from the first TCP packet;

Calculate a de-jitter buffering time based on the first clock count, a first timestamp, a second clock count, and a second timestamp

Wherein the first timestamp is a reception time of the first TCP packet expected by the server, and the second clock count is a system time clock at the moment a second TCP packet received before the TCP packet is received. And the second timestamp is a reception time of the second TCP packet expected by the server extracted from the second TCP packet.
The method of claim 11,

Computing a de-jitter buffering time based on the first clock count, the first timestamp, the second clock count, and the second timestamp,

Calculating first network jitter experienced by a first TCP packet based on the first clock count and the first timestamp;

Calculating a first arrival time interval between the second TCP packet and the first TCP packet based on the second network jitter experienced by the first network jitter and the second TCP packet; And

Calculate the de-jitter buffering time based on the first network jitter and the first arrival time interval.

Receiving, streaming service receiving method.
The method of claim 12,

Computing the de-jitter buffering time based on the first network jitter and the first arrival time interval,

Calculating a first arrival interval jitter of the first TCP packet based on a second arrival interval jitter of the second TCP packet and the first arrival time interval, wherein the arrival interval jitter is arrival between successive arriving packets. Interval delay variation-;

Calculate a first arrival interval variance of the first TCP packet based on a second arrival interval variance of the second TCP packet, the first arrival time interval and the first arrival time interval, wherein the arrival time variance is Variance over arrival time intervals; And

Calculate the de-jitter buffering time based on a first arrival interval jitter and the first arrival interval variance

Receiving, streaming service receiving method.
The method of claim 13,

The first arrival interval jitter is calculated based on Equation 4 below.

Where J (n + 1) is the first arrival interval jitter, J (n) is the second arrival interval jitter, D (n + 1) is the first arrival time interval, and α is 0 or more 1 It is the following smoothing factor.
The method of claim 13,

The first arrival interval variance is calculated based on Equation 5 below.

Where υ (n + 1) is the first arrival interval variance, υ (n) is the second arrival interval variance, D (n + 1) is the first arrival time interval, and J (n) is The second arrival interval jitter, and β is a smoothing factor of 0 or more and 1 or less.
The method of claim 13,

The de-jitter buffering time is calculated based on Equation 6 below.

Where B (n + 1) is the de-jitter buffering time, J (n + 1) is the first arrival interval jitter, υ (n + 1) is the first arrival interval variance, and k is A parameter for reflecting a fluctuation range of the first arrival interval variance.
The method of claim 11,

And the first timestamp is in an options field in a header of the first TCP packet.
The method of claim 11,

Determining a final de-jitter buffering time among the de-jitter buffering time and the analysis time

Further comprising the analysis time being a time sufficient for received TCP packets to be received and analyzed.
The method of claim 11,

Setting the de-jitter buffering time to the buffering time of the transport stream system target decoder of the streaming client.

Further comprising, the streaming service receiving method.
A receiving unit for receiving a first TCP packet;

A clock count measuring unit measuring a first clock count of a system time clock at the moment the first TCP packet is received;

A timestamp extracting unit for extracting a first timestamp of the first TCP packet from the first TCP packet; And

A de-jitter buffering time calculator configured to calculate a de-jitter buffering time based on the first clock count, a first timestamp, a second clock count, and a second timestamp;

Wherein the first timestamp is a reception time of the first TCP packet expected by the server, and the second clock count is a system time clock at the moment a second TCP packet received before the TCP packet is received. And the second timestamp is a reception time of the second TCP packet expected by the server extracted from the second TCP packet.