KR20210042051A - Multipath-based block transmission system and streaming method for adaptive streaming service - Google Patents

Abstract

Disclosed are a multipath-based block transmission system and a streaming method for an adaptive streaming service. The system and method provides the multipath-based block transmission system and the streaming method for the adaptive streaming service, which improve quality of a user quality of experience (QoE) by using a system and a streaming method, wherein when the adaptive streaming service transmits media segments using multipath, the media server transmits the segments in block units according to a request of a block unit media segment of which block length and quality are determined by measuring an aggregate bandwidth of each path from a client.

Description

Multipath-based block transmission system and streaming method for adaptive streaming service

The present invention relates to an adaptive streaming service, and more particularly, when transmitting a media segment using multiple paths of a network in an adaptive streaming service, a client measures the aggregate bandwidth of each path and blocks from the client. An adaptive streaming service is provided to improve the quality of experience (QoE) by using a system and streaming method in which the media server transmits the segment in block units according to the request of the block unit media segment whose length and quality are determined. It relates to a multi-path based block transmission system and a streaming method for the purpose.

In the video streaming system, when overload traffic occurs beyond the network capacity, the quality of service (QoS) of the video decreases due to packet delay, packet loss, and jitter, so that the playback device can watch it. The quality of experience (QoE) deteriorates.

Recently, as the number of users for video streaming service increases, adaptive video streaming technology using network perception technology has emerged to provide seamless video in heterogeneous networks with limited bandwidth and to increase user experience quality (QoE).

The adaptive video streaming server divides and stores video data of various bit rates into segments, and defines URL and video information of each segment as metadata. The adaptive video streaming playback apparatus receives metadata in an initial streaming session, analyzes it, and requests a segment appropriate to the network situation.

However, since the existing adaptive video streaming service considers only a bandwidth measurement value and a buffer of a playback device when selecting a quality, a bandwidth wasted or a video dropout phenomenon may occur due to not taking into account the content characteristics and the playback device characteristics. The problem of wasting bandwidth occurs when a playback device requests a video that exceeds a displayable pixel and bit rate.

In the case of a media content with a large number of changing frames, a video interruption phenomenon occurs due to a high delay when receiving a corresponding segment due to a high instantaneous bit rate. In order to improve the quality of experience (QoE) of users using video streaming services, a technology that provides seamless streaming services using content, playback devices, and network features is required.

As a prior art 1 related to this, patent registration number 10-1982290 discloses "a streaming system and method based on content characteristics for improving the quality of experience of an adaptive streaming service", and the quality of experience (QoE) of an adaptive streaming service is disclosed. Experience) Content for improving the quality of experience of an adaptive streaming service that minimizes playback interruption and unnecessary quality changes by adjusting the video bitrate using the requested segment size, network bandwidth, and buffer occupancy as a content characteristic for enhancement. It provides a feature-based streaming system and method.

1 is a diagram showing a structure of a client/server system for improving haptic quality based on content characteristics of a conventional adaptive streaming service.

A streaming system based on content characteristics for improving the quality of experience of an adaptive streaming service,

Encoding contents at various bit rates, dividing contents for each bitrate into byte-unit segments, storing manifest files expressing the information of contents, storing media contents, and content-recognition using a rate adaptive method A server that provides an adaptive streaming service of an approach method; And

In order to improve the quality of experience of the adaptive streaming service, when the client requests a manifest file from the server and receives the manifest file from the server in response to the request, the location of the content for each quality, the actual size of the segment, and the playback length It recognizes the information of, and measures the network bandwidth using the size of the segment every time segments are downloaded from the server, and determines the bitrate of the next requested segment based on the information provided by the bandwidth measurement module and the buffer. , Including a client (client) requesting a byte corresponding to the segment of the quality (bitrate) selected in the quality adaptation module,

The quality is the bitrate of the A/V media,

In order to improve the quality of experience (QoE) in the adaptive streaming service, video quality in consideration of the size of a segment to be requested to the server, network bandwidth, and buffer occupancy when streaming based on content characteristics Adjust (bitrate).

The manifest file contains the location and byte range of the content for each bitrate. When a manifest file is requested from the client to the server and the manifest file is received from the server in response to the request, each Recognizes the location of the content by quality, the actual size of the segment, and information on the playback length.

The client (client) is a bandwidth measurement module for measuring a network bandwidth using the size of the segment every time the segment is downloaded from the server; A quality adaptation module that determines a quality of a segment to be requested next based on information provided by the bandwidth measurement module and the buffer; A segment request module for requesting a byte corresponding to a segment of a bitrate selected by the quality adaptation module; And each time each segment is downloaded from the server, the buffer occupied amount is increased by the reproduction length of the segment, and the buffer occupancy rate and the buffer filling rate are calculated.

Content characteristics include the size of the segment to be requested, network bandwidth, and buffer occupancy.

The media server storing the media content encodes the content at various bit rates, divides the content for each quality into byte-unit segments, and stores a manifest file that expresses the information of the content. The inside of the manifest file contains the location and byte range of the content for each bitrate, and when the media server receives the manifest file, the media server recognizes information such as the location of the content for each quality, the actual size of the segment, and the playback length. can do.

The client includes a bandwidth measurement module, a quality adaptation module, a segment request module, and a buffer.

The client's bandwidth estimation module measures the network bandwidth using the size of the segment every time a segment is downloaded from the media server.

Whenever a segment is downloaded from the server, the buffer increases the occupied amount of the buffer by the playback length of the segment, and calculates the occupancy of the buffer and the filling speed of the buffer.

The quality adaptation module determines the quality of the segment to be requested next based on the information provided by the bandwidth measurement module and the buffer.

The segment requesting module requests a byte corresponding to a segment of a bitrate selected by the quality adaptation module.

The content characteristic-based streaming method for improving the haptic quality of an adaptive streaming service includes (a) a client requests a manifest file from a media server that provides an adaptive streaming service of a content-aware approach, and the client Downloading the manifest file to; (b) Before the client starts streaming, the media server requests bytes according to the order of the corresponding segment of the content for each bitrate, and the first after the client downloads the requested manifest file from the media server. When a segment is requested and downloaded, the metadata box of the content is analyzed, and the client uses a rate adaptation scheme to use the actual segment information from the manifest file to provide segment throughput. Measuring and, at the end of the download of the video segment, i) estimation, ii) expectation, and iii) adaptation of the network bandwidth; And (c) when the media server providing the adaptive streaming service of the content-aware approach streams VBR video content, the client is buffered while downloading the segment containing the image data from the server, and the network bandwidth and the buffer Measuring the charging rate and adjusting the video bitrate based on the buffer occupancy and the network bandwidth.

In step (b), since the metadata box includes all the byte positions and ranges of each segment, and the playback length, the actual byte size of each segment is determined without separately analyzing the last byte position of each segment in the client. Able to know.

The first procedure is a procedure in which the client requests a manifest file from the media server and downloads the manifest file from the media server to the client.

The second procedure is a procedure for the client to request bytes according to the order of the corresponding segment of the content for each bitrate before starting streaming. After the client downloads the manifest file from the media server, when the first segment is requested and downloaded, the metadata box of the content is analyzed. Since the metadata box contains all of the byte position and range of each segment, and the playback length, the actual byte size of each segment can be known without separately analyzing the position of the last byte of each segment in the client.

The third procedure starts video streaming by downloading the segment containing the video data. From the third procedure, the client downloads a segment from the media server, measures the network bandwidth and the rate at which the buffer is charged, and adjusts the video bitrate based on the buffer occupancy and network conditions (network bandwidth).

I. Related research

In HTTP-based adaptive video streaming, rate adaptation schemes are classified. Existing rate adaptation schemes can be classified into two main categories: bandwidth-based, and buffer-based rate adaptation schemes.

The bandwidth-based rate adaptation scheme switches the video bit rate up or down based on the estimated network bandwidth. Most of the rate adaptation schemes adopted by commercial video providers fall into this category. However, the inherent time-varying bandwidth results in short-term rate oscillations and reduces the user experience of the streaming service. [1] S. Akhshabi, A. C. Begen, and C. Dovrolis, "An experimental evaluation of rate-adaptation algorithms in adaptive streaming over HTTP," in Proc. ACM MMSys, Feb. 2011, pp. At 157-168, the authors compare rate adaptation schemes of three popular HAS clients: Netflix, Microsoft Smooth Streaming, and Adobe Open Source Media Framework (OSMF) clients. They concluded that none of these were enough, either because they were too aggressive or too conservative. Some clients even jump between the highest video bitrate and the lowest video bitrate. In addition, it has a relatively long response time to a network congestion level change. Due to the complex network conditions, it is still a challenge to accurately predict the network bandwidth.

Buffer-based rate adaptation schemes provide continuous video playback by preventing buffer underflow (rebuffering) and overflow. Choose a rate. Several research works have dealt with a buffer-based approach. [2] J. He, Z. Xue, D. Wu, D. O. Wu,and Y. Wen, "CBM: Online strategieon cost-aware buffer management for mobile video streaming," IEEE Trans. Multimedia, vol. 16, no. 1, pp. 242-252, Jan. In 2014, a cost-aware buffer management strategy is proposed that minimizes the cost caused by unconsumed video segments. [3] Y. Xu,E. Altman, R. El-Azouzi, M. Haddad, S. Elayoubi, and T. Jimenez, "Analysis of bufferstarvation with application to objective Qooptimization of streaming services," IEEE Trans. Multimedia, vol. 16, no. 3, pp. 813-827, Apr. In 2014, the authors characterize buffer starvation by modeling the playback buffer as an M/M//1 Queue. Akamai adopts a buffer-based rate adaptation technology, but adjusts the video bitrate on the server side [4]. The server-side approach will dramatically increase the burden on the web server or cache, thus limiting the support of large-scale multimedia delivery. Preventing buffer underflow can ensure video playback continuity, but can cause frequent video bitrate changes, resulting in poor quality experience for users. .

With the widespread use of smart devices, the demand for video streaming services in various network environments is rapidly increasing. In 2013, the volume of video traffic in the United States was 37.05% of all downstream traffic from Netflix, a leading video streaming service provider, and 17.85% of YouTube, which uses more than 1 billion users and accounts for a third of all Internet users. Occupied the traffic. The market size of video streaming through the network is expected to increase continuously, and video traffic is expected to increase rapidly as UHD content is recently provided. lately. When LTE-A services that support high transmission rates of 150Mbps and 5G services that support at least 10 times to 1000 times faster speeds appear in the near future, the video streaming service provision environment is expected to improve. However, since the server of a company providing a video service uses limited network resources, when users watch a high-quality video, the quality of service may decrease. Accordingly, there is a need for a network optimization technology for handling exploding video traffic and a video streaming service that adaptively adjusts the quality of content according to network conditions (bandwidth).

Patent Registration No. 10-1982290 (Registration Date May 20, 2019), "Streaming System and Method Based on Content Characteristics for Improving the Experienced Quality of Adaptive Streaming Service", Kwangsoo Jeong, Jiwoo Park, Minsu Kim, Applicant: Kwangwoon University

S. Akhshabi, A. C. Begen, and C. Dovrolis, "An experimental evaluation of rate-adaptation algorithms in adaptive streaming over HTTP," in Proc. ACM MMSys, Feb. 2011, pp. 157-168. J. He, Z. Xue, D. Wu, D. O. Wu,and Y. Wen, "CBM: Online strategieon cost-aware buffer management for mobile video streaming," IEEE Trans. Multimedia, vol. 16, no. 1, pp. 242-252, Jan. 2014. Y. Xu, E. Altman, R. El-Azouzi, M. Haddad, S. Elayoubi, and T. Jimenez, "Analysis of bufferstarvation with application to objective Qooptimization of streaming services," IEEE Trans. Multimedia, vol. 16, no. 3, pp. 813-827, Apr. 2014. L. D. Cicco, S. Mascolo, and V. Palmisano, "Feedback control foadaptive live video streaming," in Proc. ACM MMSys, Feb. 2011, pp.145-156.

It is an object of the present invention to solve the above problems in a network optimization technology that processes video traffic and an adaptive streaming service that adaptively adjusts a content bitrate according to a network state, and a media segment is divided using multiple paths. When transmitting, the client measures the aggregate bandwidth of each path, and the media server transmits the segment in block units according to the request of the block unit media segment whose block length and quality are determined from the client. , Quality of Experience), and provides a multi-path-based block transmission system for adaptive streaming service.

Another object of the present invention is to provide a multi-path based block transport streaming method for an adaptive streaming service.

In order to achieve the object of the present invention, a multi-path-based block transmission system for an adaptive streaming service is at least one in order to seamlessly reproduce a high-quality video and improve a user's quality of experience (QoE) in a plurality of network environments. A client requesting a media segment in block units by determining a block length and a bitrate through the above multipaths; And storing media segments of different qualities (bitrate), receiving a request for a media segment in block units from the client, and dividing the media segment into a plurality of block unit partial segments to provide an adaptive streaming service. Including a plurality of media servers for transmitting the segment,

The client is connected to the plurality of media servers through multiple paths of two or more networks, and the client determines the length and bitrate of the block to be requested in consideration of content characteristics, playback device characteristics, and network characteristics. Requests a media segment in block units through a path,

The content characteristic is the size and length of the segment, the playback device characteristic is the amount of segments stored in the buffer, the speed at which the segment is stored in the buffer, and the network characteristic is the bandwidth of each path of two or more networks,

The manifest file includes the segment file name, media segment bitrate, segment duration, segment size, and

The client parses and analyzes the information of the manifest file in a parser to recognize the quality, length, and byte range information of the media segment, and the bandwidth measurement module, block length adaptation module, block quality adaptation module, block scheduling module, and block It includes a unit segment request module, a buffer that calculates the buffer occupancy rate and the buffer charging rate,

After the bandwidth measurement module measures the integrated bandwidth, the block length adaptation module and the block quality adaptation module provide content for seamless video playback in the network environment that changes according to the connection of the user terminal. The length (N) of the block taking into account the characteristics (segment size, length, playback time), playback device characteristics (the amount of segments stored in the buffer, the speed at which segments are stored in the buffer), and network characteristics (the total bandwidth of each path). ) And block quality (bitrate),

In the multipath-based block transmission streaming service, the block length (N) affects the adaptation speed and quality change frequency of the streaming network.

A multipath-based block transmission system including a media server and a client that provides an adaptive streaming service uses the playback device characteristic information (the amount of segments stored in the buffer, the speed at which the segments are stored in the buffer) to determine the length of the block (N ), and the multipath-based block transmission system determines the quality (V) and length (N) of the block to be requested, and then uses a number of HTTP Range Requests to transmit 2N Partial Segments to each media through each path. Make a request to the server.

In order to achieve another object of the present invention, a multi-path-based block transmission streaming method for an adaptive streaming service includes (a) seamlessly playing high-quality video and improving user experience quality (QoE) in multiple network environments. Requesting a media segment in units of blocks by determining a block length and a bitrate through at least one multipath of a network in order to achieve this; And (b) a plurality of media servers store media segments of different quality (bitrate), receive a request for a media segment in block units from the client, and divide the media segment into a plurality of block unit partial segments for adaptive streaming. Including the step of transmitting the media segment to the client in block units to provide a service,

The client is connected to the plurality of media servers through multiple paths of two or more networks, and the client determines the length and bitrate of the block to be requested in consideration of content characteristics, playback device characteristics, and network characteristics. Requests a media segment in block units through a path,

The content characteristic is the size and length of the segment, the playback device characteristic is the amount of segments stored in the buffer, the speed at which the segment is stored in the buffer, and the network characteristic is the bandwidth of each path of two or more networks,

The manifest file includes the segment file name, media segment bitrate, segment duration, segment size, and

The client parses and analyzes the information of the manifest file in a parser to recognize the quality, length, and byte range information of the media segment, and the bandwidth measurement module, block length adaptation module, block quality adaptation module, block scheduling module, and block It includes a unit segment request module, a buffer that calculates the buffer occupancy rate and the buffer charging rate,

After the bandwidth measurement module measures the integrated bandwidth, the block length adaptation module and the block quality adaptation module provide content for seamless video playback in the network environment that changes according to the connection of the user terminal. The length (N) of the block taking into account the characteristics (segment size, length, playback time), playback device characteristics (the amount of segments stored in the buffer, the speed at which segments are stored in the buffer), and network characteristics (the total bandwidth of each path). ) And block quality (bitrate),

In the multipath-based block transmission streaming service, the block length (N) affects the adaptation speed and quality change frequency of the streaming network.

A multipath-based block transmission system including a media server and a client that provides an adaptive streaming service uses the playback device characteristic information (the amount of segments stored in the buffer, the speed at which the segments are stored in the buffer) to determine the length of the block (N ), and the multipath-based block transmission system determines the quality (V) and length (N) of the block to be requested, and then uses a number of HTTP Range Requests to transmit 2N Partial Segments to each media through each path. Make a request to the server.

The multi-path-based block transmission system and streaming method for an adaptive streaming service of the present invention uses a system for transmitting segments in block units and a streaming method when media segments are transmitted using multiple paths in an adaptive streaming service. It has the effect of improving the quality (QoE, Quality of Experience).

By providing a multi-path based block transmission system in an adaptive streaming service, high-quality media can be provided without interruption in playback, and high quality of experience (QoE) can be provided to users by reducing the frequency of change in video playback quality.

The adaptive streaming service is applicable to VoD service to which adaptive streaming technology is applied, IPTV service to which adaptive streaming technology is applied, adaptive streaming service based on Web, adaptive streaming service based on mobile terminal, and ultra-high quality media streaming service.

1 is a diagram showing a structure of a client/server system for improving haptic quality based on content characteristics of a conventional adaptive streaming service.
2 is a diagram showing a scenario of a multipath-based block transmission system.
3 is a diagram illustrating an operation of a multipath-based block transmission system.
4 is a block diagram of a multipath-based block transmission system.
5 is a diagram illustrating a block transmission method in a multipath-based block transmission system.
6 is a flowchart illustrating an operation of a multipath-based block transmission system.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The multi-path-based block transmission system and streaming method for an adaptive streaming service of the present invention determine the quality of content (video segment size, segment playback time) according to network optimization technology and network conditions (bandwidth) for processing video traffic. In an adaptive streaming service that adjusts adaptively, a media segment is transferred to a client (media player) by using at least two multi-paths of the network through each network interface card (NIC) card through the paths of network 1 and network 2. When transmitting, the client measures the bandwidth and the media server transmits the segment in block units according to the request of the block unit media segment whose block length and quality are determined from the client. of experience).

Multi-path based block transmission system and streaming method for adaptive streaming service segment through multi-path to improve user's quality of experience (QoE) by seamlessly playing high-quality video in a multi-path environment and minimizing unnecessary quality changes. Is requested in block units. The multipath-based block transmission system for an adaptive streaming service proposed in the present invention includes content characteristics (segment size, length), playback device characteristics (amount of segments stored in a buffer, speed at which segments are stored in a buffer), and a network. After determining the length and bitrate of the block to be requested in consideration of the characteristics (bandwidth of each path), the media segment is requested in block units through multipath.

For example, a media segment is a plurality of media segments (two-minute data) in which the original video data of the HAS (60-minute data) is divided and split.

A block is a set of multiple media segments (eg, 12 minutes/10 minutes/8 minutes/6 minutes...), and is a unit that requests a media segment in block units to the media server.

2 shows a scenario of a multipath-based block transmission system.

The multipath-based block transmission system seamlessly reproduces high-quality video in at least two or more multipath environments in a network environment and minimizes unnecessary quality changes to improve user experience quality (QoE) through at least one NIC card. Through the above multi-paths, the content characteristics (segment size, length, playback time), playback device characteristics (amount of segments stored in the buffer, speed at which segments are stored in the buffer), and network characteristics (bandwidth of each path) are considered. A client that requests a media segment in block units through multipath after determining the length and bitrate of the block to be requested; And

It stores media segments of different quality (bitrate), receives a media segment request in block units from the client, and divides the media segment into a plurality of block unit partial segments to provide an adaptive streaming service. Including a plurality of media servers (Media Server) to transmit,

The client is connected to the plurality of media servers through multiple paths of two or more networks, and the client determines the length and bitrate of the block to be requested in consideration of content characteristics, playback device characteristics, and network characteristics. Requests a media segment in block units through a path,

The client parses and analyzes the information of the manifest file in the parser to recognize information on the quality, length, and byte range of the media segment, and the client is a bandwidth measurement module, a block length adaptation module, a block quality adaptation module, and a block scheduling module. , A block-by-block segment request module, and a buffer that calculates the buffer occupancy rate and the buffer charging rate,

After the bandwidth measurement module measures the integrated bandwidth, the block length adaptation module and the block quality adaptation module provide content for seamless video playback in the network environment that changes according to the connection of the user terminal. The length (N) of the block taking into account the characteristics (segment size, length, playback time), playback device characteristics (the amount of segments stored in the buffer, the speed at which segments are stored in the buffer), and network characteristics (the total bandwidth of each path). ) And the block quality (bitrate).

In the multipath-based block transmission streaming service, the block length (N) affects the adaptation speed and quality change frequency of the streaming network.

A multipath-based block transmission system including a media server and a client that provides an adaptive streaming service uses the characteristic information of the playback device (the amount of segments stored in the buffer, the speed at which the segments are stored in the buffer), and uses the appropriate block length ( N) must be determined, and the multipath-based block transmission system determines the quality (V) and length (N) of the block to be requested, and then sends 2N partial segments through each path using multiple HTTP Range Requests. Make a request to the media server.

The client is connected to the plurality of media servers through multiple paths of two or more networks, and the client determines the length and bitrate of the block to be requested in consideration of content characteristics, playback device characteristics, and network characteristics. Requests a media segment in block units through a path,

The content characteristics are segment size and length,

The playback device characteristics are the amount of segments stored in the buffer, the speed at which segments are stored in the buffer,

The network characteristic is the bandwidth of each path of two or more networks.

)을 모두 더한 통합 대역폭(

)을 측정한다. 클라이언트는 통합 대역폭(BW1 + BW2)을 기반으로 요청할 블록의 길이와 품질(bitrate)을 결정한 후, 블록(block)을 다수의 부분 세그먼트들로 분할하여 다수의 네트워크(NET1, NET2)의 각 경로를 통해 요청한다. 네트워크들의 다중경로 환경에서 클라이언트는 블록의 길이와 품질(bitrate)이 결정된 블록 단위로 미디어 세그먼트를 미디어 서버로 요청함으로써, 고품질의 비디오 콘텐츠를 끊김 없이 재생하고 품질 변경을 최소화해 사용자의 체감 품질(QoE)을 향상시킨다. In order to utilize all the bandwidth of the network in a multipath environment, the client, for example, the bandwidth of each path of network 1 and network 2 (

) Plus the aggregated bandwidth (

) Is measured. The client determines the length and bitrate of the block to be requested based on the aggregated bandwidth (BW1 + BW2), and then divides the block into a number of partial segments to divide each path of a number of networks (NET1, NET2). Request through. In the multipath environment of networks, the client requests media segments in blocks with the block length and bitrate determined to the media server, thereby seamlessly playing high-quality video content and minimizing quality changes to the user's perceived quality (QoE). ) To improve.

3 is a diagram illustrating an operation of a multipath-based block transmission system.

The client of the multipath-based block transmission system operates in 5 steps.

The client includes a bandwidth measurement module, a block length adaptation module, a block quality adaptation module, a block scheduling module, a block unit segment request module, and a buffer.

The client's bandwidth estimation module measures the aggregated bandwidth of the network (integrated bandwidth of each path) in the Aggregated Bandwidth Estimation step whenever a block-by-block segment is downloaded from the media server.

After the Bandwidth Estimation module measures the integrated bandwidth, the Block Length Adaptation module and the Block Quality Adaptation module seamlessly play video in the network environment that changes according to the connection of the user terminal. For this, the content characteristics (segment size, length, playback time), playback device characteristics (amount of segments stored in the buffer, speed at which segments are stored in the buffer), and network characteristics (total bandwidth of each path) are considered. Determine the length and bitrate of the block.

The Block Length Adaptation module determines the length of the next block to be requested using the aggregate bandwidth of the network, the amount of segments stored in the buffer, and the rate at which segments are stored in the buffer.

The Block Quality Adaptation module determines the quality of the block to be requested using information on the aggregate bandwidth of the network, the length of the block to be requested, and the size of the segment.

The Block Scheduling module determines the byte range of a partial segment to be requested through each path of the network in order to complete the download at the same time so as not to cause a problem of rearrangement of a plurality of segments constituting a block.

In the Block Request stage, the Block Request module determines the byte range of the segment to be requested based on the partial segment size determined by the Block Scheduling module through multiple HTTP Range Requests. Each media server requests a number of Partial Segments that compose a block with a block length and bitrate.

Whenever a segment is downloaded from the media server, the buffer increases the occupied amount of the buffer by the playback length of the segment, and calculates the occupancy of the buffer and the filling speed of the buffer.

4 shows a block diagram of a multipath-based block transmission system.

The Manifest File, which contains information on video content, includes the segment file name, media segment bitrate, segment playback time, and segment size.

When a manifest file containing information about video content is received from a media server to a client (media player), the parser of the client parses and analyzes the information of the manifest file to determine the quality of the media segment ( bitrate), length, and byte range information. In addition, the bandwidth estimation module requests a block from the media server and measures the bandwidth by using the size of the block and the time to receive the block each time it is received.

The Buffer Monitor measures the amount of segments currently stored in the buffer of the playback device and the speed at which the segments are stored in the buffer in order to obtain information on the characteristics of the playback device.

The Block Length Adaptation module determines the length of the next block to be requested using the aggregate bandwidth of the network, the amount of segments stored in the buffer, and the rate at which segments are stored in the buffer.

The Block Quality Adaptation module determines the quality of the block to be requested using information on the aggregate bandwidth of the network, the length of the block to be requested, and the size of the segment.

The block scheduling module determines the byte range of a partial segment to be requested through each path of the network.

The Block Request module transmits the HTTP Range Request message for Partial Segment to the media server through each path through Block Scheduling. Partial segments received from the media server to the client through the request to the media server are stored in the buffer of the playback device, then decoded and played in the media player.

5 shows a block transmission method in a multipath-based block transmission system.

In a multipath-based block transmission streaming service, the block length (N) affects the speed of adaptation to the network change of streaming and the frequency of quality change. If the value of N is large, unnecessary quality changes can be reduced, but it is not possible to quickly adapt to changes in the network, which increases the likelihood of playback interruption. If the value of N is relatively small, it adapts quickly to network changes, but the possibility of unnecessary quality changes increases.

Therefore, a multipath-based block transmission system including a media server and a client that provides an adaptive streaming service uses the playback device characteristic information (the amount of segments stored in the buffer, the speed at which the segments are stored in the buffer) to determine the appropriate N. Decide. The multipath-based block transmission system determines the quality (V) and length (N) of the block to be requested, and then requests 2N Partial Segments to each media server through each path using multiple HTTP Range Requests.

In addition, the multi-path-based block transmission method for the adaptive streaming service of the present invention is (a) in a number of network environments, in order to seamlessly play a high-quality video and improve the quality of experience (QoE) of the user, the client Determining a length and a bitrate of a block through at least one multipath and requesting a media segment for each block; And

(b) Multiple media servers store media segments of different quality (bitrate), receive a request for a media segment in block units from the client, and divide the media segment into a plurality of block unit partial segments to provide an adaptive streaming service. Transmitting the media segment to the client in block units to provide a,

The client is connected to the plurality of media servers through multiple paths of two or more networks, and the client determines the length and bitrate of the block to be requested in consideration of content characteristics, playback device characteristics, and network characteristics. Requests a media segment in block units through a path,

The content characteristic is the size and length of the segment, the playback device characteristic is the amount of segments stored in the buffer, the speed at which the segment is stored in the buffer, and the network characteristic is the bandwidth of each path of two or more networks,

The manifest file includes the segment file name, media segment bitrate, segment duration, segment size, and

The client parses and analyzes the information of the manifest file in a parser to recognize the quality, length, and byte range information of the media segment, and the client is a bandwidth measurement module, a block length adaptation module, a block quality adaptation module, and block scheduling. It includes a module, a segment request module for each block, a buffer that calculates the buffer occupancy rate and the buffer charging rate,

After the bandwidth measurement module measures the integrated bandwidth, the block length adaptation module and the block quality adaptation module provide content for seamless video playback in the network environment that changes according to the connection of the user terminal. The length of the block and the length of the block considering the characteristics (segment size, length, playback time), playback device characteristics (the amount of segments stored in the buffer, the speed at which segments are stored in the buffer), and network characteristics (the total bandwidth of each path). Determine the bitrate of the block.

The client is connected to the plurality of media servers through multiple paths of two or more networks, and the client determines the length and bitrate of the block to be requested in consideration of content characteristics, playback device characteristics, and network characteristics. Requests a media segment in block units through a path,

The content characteristics are segment size and length,

The playback device characteristics are the amount of segments stored in the buffer, the speed at which segments are stored in the buffer,

The network characteristic is the bandwidth of each path of two or more networks,

The manifest file includes the segment file name, media segment bitrate, segment playback time, and segment size.

The client includes a bandwidth measurement module, a block length adaptation module, a block quality adaptation module, a block scheduling module, a block unit segment request module, and a buffer,

When a manifest file containing information about video content is received from the media server to a client (media player), the parser of the client parses and analyzes the information of the manifest file to determine the quality (bitrate), length, and length of the media segment. Recognizing information in a byte range;

Measuring an aggregated bandwidth of a network (an aggregated bandwidth of each path) in an aggregated bandwidth estimation step by the bandwidth measurement module;

Determining, by the block length adaptation module, the length of the next block to be requested using the aggregate bandwidth of the network, the amount of segments stored in the buffer, and the rate at which the segments are stored in the buffer;

Determining, by the block quality adaptation module, a quality of a block to be requested using information on an aggregated bandwidth of the network, a length of a block to be requested, and a size of a segment;

Determining a byte range of a partial segment to be requested through each path of a network in order to simultaneously complete downloading of a plurality of segments constituting a block by the block scheduling module so that a rearrangement problem does not occur;

In the Block Request step by the block unit segment request module, each media through multiple HTTP Range Requests by determining the byte range of the segment to be requested based on the partial segment size determined by the Block Scheduling module. Requesting a plurality of Partial Segments constituting a block having a block length and a bitrate to a server; And

The buffer includes the step of increasing the occupied amount of the buffer by the playing length of the segment each time a segment is downloaded from the media server, and calculating the occupied buffer occupancy rate and the buffer filling rate,

The client is connected to the plurality of media servers through multiple paths of two or more networks, and the client determines the length and bitrate of the block to be requested in consideration of content characteristics, playback device characteristics, and network characteristics. Requests a media segment in block units through a path,

The content characteristic is the size and length of the segment, the playback device characteristic is the amount of segments stored in the buffer, the speed at which the segment is stored in the buffer, and the network characteristic is the bandwidth of each path of two or more networks,

The manifest file includes the segment file name, media segment bitrate, segment duration, segment size, and

The client parses and analyzes the information of the manifest file in a parser to recognize the quality, length, and byte range information of the media segment, and the bandwidth measurement module, block length adaptation module, block quality adaptation module, block scheduling module, and block It includes a unit segment request module, a buffer that calculates the buffer occupancy rate and the buffer charging rate,

After the bandwidth measurement module measures the integrated bandwidth, the block length adaptation module and the block quality adaptation module provide content for seamless video playback in the network environment that changes according to the connection of the user terminal. The length (N) of the block taking into account the characteristics (segment size, length, playback time), playback device characteristics (the amount of segments stored in the buffer, the speed at which segments are stored in the buffer), and network characteristics (the total bandwidth of each path). ) And block quality (bitrate),

In the multipath-based block transmission streaming service, the block length (N) affects the adaptation speed and quality change frequency of the streaming network.

A multipath-based block transmission system including a media server and a client that provides an adaptive streaming service uses the characteristic information of the playback device (the amount of segments stored in the buffer, the speed at which the segments are stored in the buffer), and uses the appropriate block length ( N) must be determined, and the multipath-based block transmission system determines the quality (V) and length (N) of the block to be requested, and then sends 2N partial segments through each path using multiple HTTP Range Requests. Make a request to the media server.

After the bandwidth measurement module measures the integrated bandwidth, the block length adaptation module and the block quality adaptation module provide content for seamless video playback in the network environment that changes according to the connection of the user terminal. The length of the block and the length of the block considering the characteristics (segment size, length, playback time), playback device characteristics (the amount of segments stored in the buffer, the speed at which segments are stored in the buffer), and network characteristics (the total bandwidth of each path). Determine the bitrate of the block.

)을 모두 더한 통합 대역폭(

)을 측정하며, 상기 클라이언트는 통합 대역폭(BW1 + BW2)을 기반으로 요청할 블록의 길이와 품질(bitrate)을 결정한 후, 블록(block)을 다수의 부분 세그먼트들로 분할하여 다수의 네트워크(NET1, NET2)의 각 경로를 통해 블록의 길이와 품질(bitrate)가 결정된 블록 단위로 미디어 세그먼트를 상기 미디어 서버로 요청함으로써, 고품질의 비디오 콘텐츠를 끊김 없이 재생하고 사용자의 체감 품질(QoE)을 향상시킨다. In order to utilize all the bandwidth of the network in a multipath environment, the client, for example, the bandwidth of each path of network 1 and network 2 (

) Plus the aggregated bandwidth (

) Is measured, and the client determines the length and bitrate of the block to be requested based on the aggregated bandwidth (BW1 + BW2), and then divides the block into a plurality of partial segments, Through each path of NET2), a media segment is requested to the media server in block units whose block length and bitrate are determined, thereby seamlessly playing high-quality video content and improving the user's quality of experience (QoE).

The manifest file includes a segment file name, a media segment's bitrate, a segment playback time, and a segment size.

In the above method, in a multipath-based block transmission streaming service, the block length (N) affects the adaptation speed and quality change frequency of the streaming network,

A multipath-based block transmission system including a media server and a client that provides an adaptive streaming service must determine an appropriate N using the playback device characteristic information (the amount of segments stored in the buffer, and the speed at which the segments are stored in the buffer). The multipath-based block transmission system determines the quality (V) and length (N) of the block to be requested, and then requests 2N partial segments to each media server through each path using a number of HTTP Range Requests. .

6 is a flowchart illustrating an operation of a multipath-based block transmission system.

When streaming starts, a manifest file containing information on a media segment is received from a media server that provides an applicable streaming service. The manifest file received from the media server to the client is analyzed by the parser of the client (media player) to recognize the quality (bitrate) and segment characteristics (segment size, length) of the content.

The client requests a block to the media server according to the perceived content characteristics, and measures the bandwidth of the network using the size of the block and the time it takes to download the block whenever it receives a block from the media server in response. do.

The client determines the length and quality of the block to be requested next by using context information such as the measured bandwidth or characteristic information of the playback device (the amount of segments stored in the buffer, the speed at which the segments are stored in the buffer).

The client recognizes the byte range and length of the next media segment to be requested according to the determined bitrate of the media segment, and divides the block to be requested for each path into a number of Partial Segments in consideration of network characteristics (bandwidth of each path).

After dividing a block into a number of Partial Segments, a request message is transmitted to the media server through each path using the byte range of the Partial Segment, and in response to this, it receives a block-by-block media segment from the media server to the client, and plays the media. Until the end of the block is repeatedly requested and received, adaptive streaming service is provided.

The multi-path-based block transmission system and streaming method for an adaptive streaming service of the present invention uses a system for transmitting segments in block units and a streaming method when media segments are transmitted using multiple paths in an adaptive streaming service. It will improve the quality of experience (QoE).

By providing a multi-path based block transmission system in an adaptive streaming service, high-quality media can be provided without interruption in playback, and high quality of experience (QoE) can be provided to users by reducing the frequency of change in video playback quality.

The adaptive streaming service is applicable to VoD service to which adaptive streaming technology is applied, IPTV service to which adaptive streaming technology is applied, adaptive streaming service based on Web, adaptive streaming service based on mobile terminal, and ultra-high quality media streaming service.

Embodiments according to the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, and data structures alone or in combination. Computer-readable recording media include storage, magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions in a magneto-optical media, and a storage medium such as ROM, RAM, flash memory, etc. may be included. Examples of program instructions may include those produced by a compiler, and high-level language codes that can be executed by a computer using an interpreter as well as machine language codes. The hardware device may be configured to operate as one or more software modules to perform the operation of the present invention.

As described above, the method of the present invention is implemented as a program and can be read using software of a computer, such as a recording medium (CD-ROM, RAM, ROM, memory card, hard disk, magneto-optical disk, storage device, etc.). ) Can be stored.

Although it has been described with reference to the preferred embodiments of the present invention, various modifications or variations of the present invention within the scope not departing from the spirit and scope of the present invention described in the following claims by those of ordinary skill in the relevant technical field It will be understood that this can be done.

100: client
201, 202: media server

Claims (6)

In multiple network environments, in order to seamlessly play high-quality video and improve the quality of experience (QoE) of users, the length and bitrate of blocks are determined through at least one multipath and a media segment is requested in units of blocks. A client to do; And
Media segments in blocks to provide adaptive streaming services by storing media segments of different qualities (bitrate), receiving a request for a media segment in block units from the client, and dividing the media segment into a plurality of block unit partial segments to provide an adaptive streaming service. Including a plurality of media servers for transmitting,
The client is connected to the plurality of media servers through multiple paths of two or more networks, and the client determines the length and bitrate of the block to be requested in consideration of content characteristics, playback device characteristics, and network characteristics. Requests a media segment in block units through a path,
The content characteristic is the size and length of the segment, the playback device characteristic is the amount of segments stored in the buffer, the speed at which the segment is stored in the buffer, and the network characteristic is the bandwidth of each path of two or more networks,
The manifest file includes the segment file name, media segment bitrate, segment duration, segment size, and
The client parses and analyzes the information of the manifest file in a parser to recognize the quality, length, and byte range information of the media segment, and the bandwidth measurement module, block length adaptation module, block quality adaptation module, block scheduling module, and block It includes a unit segment request module, a buffer that calculates the buffer occupancy rate and the buffer charging rate,
After the bandwidth measurement module measures the integrated bandwidth, the block length adaptation module and the block quality adaptation module provide content for seamless video playback in the network environment that changes according to the connection of the user terminal. Determine the block length (N) and block quality (bitrate) in consideration of characteristics, playback device characteristics, and network characteristics,
In the multipath-based block transmission streaming service, the block length (N) affects the adaptation speed and quality change frequency of the streaming network.
A multipath-based block transmission system including a media server and a client that provides an adaptive streaming service uses the playback device characteristic information (the amount of segments stored in the buffer, the speed at which the segments are stored in the buffer) to determine the length of the block (N ), and the multipath-based block transmission system determines the quality (V) and length (N) of the block to be requested, and then uses a number of HTTP Range Requests to transmit 2N Partial Segments to each media through each path. A block transmission system based on multi-path for adaptive streaming service requested by a server. The method of claim 1,
The client
When a manifest file containing information about video content is received from the media server to a client (media player), the parser of the client parses and analyzes the information of the manifest file to determine the quality (bitrate), length, and length of the media segment. Recognizes information in a range of bytes,
A bandwidth measurement module for measuring an aggregated bandwidth of a network (an aggregated bandwidth of each path) in an aggregated bandwidth estimation step;
A block length adaptation module that determines the length of a block to be requested next using the aggregate bandwidth of the network, the amount of segments stored in the buffer, and the rate at which the segments are stored in the buffer;
A block quality adaptation module that determines a quality of a block to be requested using information on the aggregate bandwidth of the network, the length of the block to be requested, and the size of the segment;
A block scheduling module that determines a byte range of a partial segment to be requested through each path of the network in order to simultaneously download a plurality of segments constituting a block so as not to cause a rearrangement problem;
In the Block Request stage, the byte range of the segment to be requested is determined based on the partial segment size determined by the Block Scheduling module, and the block length and bitrate to each media server through multiple HTTP Range Requests. A block unit segment request module that requests a plurality of Partial Segments constituting a block having ); And
A buffer for increasing the occupied amount of the buffer by the playback length of the segment each time a segment is downloaded from the media server, and calculating a buffer occupancy rate and a buffer charging speed;
Including a multi-path based block transmission system for an adaptive streaming service. The method of claim 2,
In order to utilize all the bandwidth of the network in a multipath environment, the client, for example, the bandwidth of each path of network 1 and network 2 (

) Plus the aggregated bandwidth (

) Is measured, and the client determines the length and bitrate of the block to be requested based on the aggregated bandwidth (BW1 + BW2), and then divides the block into a plurality of partial segments, NET2) by requesting a media segment in block units in which the block length and bitrate are determined through each path of NET2), seamlessly playing high-quality video content and improving the quality of experience (QoE) of the user, Multipath-based block transmission system for adaptive streaming service. (a) In multiple network environments, in order to play high-quality video seamlessly and improve the quality of user experience (QoE), the client determines the block length and bitrate through at least one multipath of the network. Requesting a media segment in block units; And
(b) Multiple media servers store media segments of different quality (bitrate), receive a request for a media segment in block units from the client, and divide the media segment into a plurality of block unit partial segments to provide an adaptive streaming service. Transmitting the media segment to the client in block units to provide a,
The client is connected to the plurality of media servers through multiple paths of two or more networks, and the client determines the length and bitrate of the block to be requested in consideration of content characteristics, playback device characteristics, and network characteristics. Requests a media segment in block units through a path,
The content characteristic is the size and length of the segment, the playback device characteristic is the amount of segments stored in the buffer, the speed at which the segment is stored in the buffer, and the network characteristic is the bandwidth of each path of two or more networks,
The manifest file includes the segment file name, media segment bitrate, segment duration, segment size, and
The client parses and analyzes the information of the manifest file in a parser to recognize the quality, length, and byte range information of the media segment, and the bandwidth measurement module, block length adaptation module, block quality adaptation module, block scheduling module, and block It includes a unit segment request module, a buffer that calculates the buffer occupancy rate and the buffer charging rate,
After the bandwidth measurement module measures the integrated bandwidth, the block length adaptation module and the block quality adaptation module provide content for seamless video playback in the network environment that changes according to the connection of the user terminal. The length (N) of the block taking into account the characteristics (segment size, length, playback time), playback device characteristics (the amount of segments stored in the buffer, the speed at which segments are stored in the buffer), and network characteristics (the total bandwidth of each path). ) And block quality (bitrate),
In the multipath-based block transmission streaming service, the block length (N) affects the adaptation speed and quality change frequency of the streaming network.
A multipath-based block transmission system including a media server and a client that provides an adaptive streaming service uses the playback device characteristic information (the amount of segments stored in the buffer, the speed at which the segments are stored in the buffer) to determine the length of the block (N ), and the multipath-based block transmission system determines the quality (V) and length (N) of the block to be requested, and then uses a number of HTTP Range Requests to transmit 2N Partial Segments to each media through each path. A multi-path-based block transmission method for an adaptive streaming service requested by a server. The method of claim 4,
When the client receives a manifest file containing information about video content from the media server to a client (media player), the parser of the client parses and analyzes the information of the manifest file to determine the quality of the media segment (bitrate). Recognizing information of a length, a range of bytes, and a byte range;
Measuring an aggregated bandwidth of a network (an aggregated bandwidth of each path) in an aggregated bandwidth estimation step by the bandwidth measurement module;
Determining, by the block length adaptation module, the length of the next block to be requested using the aggregate bandwidth of the network, the amount of segments stored in the buffer, and the rate at which the segments are stored in the buffer;
Determining, by the block quality adaptation module, a quality of a block to be requested using information on an aggregated bandwidth of the network, a length of a block to be requested, and a size of a segment;
Determining a byte range of a partial segment to be requested through each path of a network in order to simultaneously complete downloading of a plurality of segments constituting a block by the block scheduling module so that a rearrangement problem does not occur;
In the Block Request step by the block unit segment request module, each media through multiple HTTP Range Requests by determining the byte range of the segment to be requested based on the partial segment size determined by the Block Scheduling module. Requesting a plurality of Partial Segments constituting a block having a block length and a bitrate to a server; And
Each time the buffer downloads a segment from the media server, increasing the occupied amount of the buffer by the playback length of the segment, and calculating the occupied buffer occupancy rate and the buffer filling rate;
Including, multi-path-based block transmission method for an adaptive streaming service. The method of claim 5,
In order to utilize all the bandwidth of the network in a multipath environment, the client, for example, the bandwidth of each path of network 1 and network 2 (

) Plus the aggregated bandwidth (

) Is measured, and the client determines the length and bitrate of the block to be requested based on the aggregated bandwidth (BW1 + BW2), and then divides the block into a plurality of partial segments, and the network (NET1, NET2) By requesting a media segment in block units for which the block length and bitrate are determined through each path, high-quality video content is seamlessly played and the user's quality of experience (QoE) is improved. Multipath-based block transmission method for streaming service.

Images