KR20210131587A - Adaptive bitrate streaming system and method for efficient transmission of contents - Google Patents

Abstract

In the present disclosure, provided are a system and a method, wherein an optimal bitrate is calculated in a logic server according to a user's network bandwidth in real time to determine the resolution of an ROI area and a scaling area, and images of both eyes are separately and respectively transmitted to reduce a bitrate and a delay, which are the levels of quality experienced by customers. A real-time variable bitrate transmission system comprises: a logic server which calculates an optimal bitrate on the basis of usage statistics, quality statistics, and resolution information; a bitrate adapter which determines a resolution applied to each of the ROI region and the scaling region according to the optimal bitrate calculated by the logic server for original content divided into an ROI region and a scaling region; an encoder for encoding each of a left-eye image and a right-eye image of the original content according to the resolution determined by the bitrate adapter; and a transmission unit for transmitting each of the encoded left-eye image and the right-eye image to a user terminal at the optimal bitrate.

Description

ADAPTIVE BITRATE STREAMING SYSTEM AND METHOD FOR EFFICIENT TRANSMISSION OF CONTENTS

The present disclosure relates to a system and method for efficient streaming data transmission, and more particularly, a system for determining the resolution of an ROI region and a scaling region using an adaptive bitrate streaming technology and transmitting left and right binocular images, respectively, and the same it's about how

Recently, a huge amount of video content is being consumed around the world. Unlike the download method, which cannot be played while downloading, the streaming method divides a large amount of audio or video into several pieces using compression technology and transmits it, so it is possible to download and play it at the same time without storing it on the hard disk. . Video technology is gradually advanced, and various methods are being sought to provide a service without buffering by streaming the optimal video quality according to the network environment.

As a streaming method generally used, there is a progressive download method. In this way, once a media source is selected, the corresponding content is downloaded and played to the end. If the network condition is not good, there is a problem of continuously experiencing buffering delay during content playback.

To solve this problem, if adaptive bitrate streaming is used, the video content is divided into several pieces and stored, and the resolution is determined according to the user's network condition and streamed piece by piece to optimize the user condition. content streaming can be provided.

Patent Publication No. 10-2019-0042568 discloses a distribution device, a distribution method, a reception device, a reception method, a program, and a content distribution system that enable signaling of an ROI identifier of an image distributed by broadcasting or the Internet, DASH (Dynamic Adaptive Streaming over HTTP), which is one of the variable bitrate transmission methods in Internet streaming, is used.

In the present disclosure, the user network bandwidth is fed back in real time, and the logic server calculates the optimal bit rate accordingly to determine the resolution of the ROI area and the scaling area, and separates and transmits the left and right eye images. A system and method for reducing delay are provided.

A real-time variable bitrate transmission system according to an embodiment includes a logic server for calculating an optimal bitrate based on at least one of usage statistics, quality statistics, and resolution information; A bitrate adapter that determines a resolution applied to each of the ROI region and the scaling region according to the optimal bitrate calculated by the logic server for the original content divided into an ROI region and a scaling region adapter); an encoder for encoding according to the resolution determined by the bitrate adapter; and a transmitter for transmitting the encoded content to the user terminal at the optimal bit rate.

A real-time variable bitrate transmission method according to an embodiment comprises: calculating, in a logic server, an optimal bitrate based on at least one of usage statistics, quality statistics, and resolution information; In the bitrate adapter, for original content divided into an ROI region and a scaling region, the resolution applied to each of the ROI region and the scaling region is determined according to the optimal bitrate calculated by the logic server step; encoding in the encoder according to the resolution determined by the bitrate adapter; and transmitting, in the transmission unit, the encoded content to the user terminal at the optimal bit rate.

A user terminal according to an embodiment includes a receiving unit for receiving content transmitted from a real-time variable bit rate transmission system; a decoder for decoding the content; and a transmission unit for transmitting feedback information to the real-time variable bit rate transmission system, wherein the content is an optimal bit rate based on usage statistics, quality statistics, and resolution information in a logic server of the real-time variable bit rate transmission system. (bitrate), and in the bitrate adapter of the real-time variable bitrate transmission system, for the original content divided into an ROI region and a scaling region, according to the optimal bitrate calculated by the logic server, the A resolution applied to each of the ROI region and the scaling region is determined, and the encoder of the real-time variable bitrate transmission system encodes according to the resolution determined by the bitrate adapter, and a transmission unit of the real-time variable bitrate transmission system , the encoded content is transmitted to the user terminal at the optimal bit rate.

The encoder according to an embodiment encodes each of a left-eye image and a right-eye image of the original content, and the transmitter transmits each of the left-eye image and the right-eye image to a user terminal.

The logic server according to an embodiment may be configured to configure the optimal bitrate value based on an end-to-end (E2E) latency value based on at least one of the usage statistics, the quality statistics, and the resolution information. to calculate

The usage statistics according to an embodiment include at least one of usage time, usage time, streaming location information, content type, and device type information, and the quality statistics include network throughput and packet loss. loss rate) and at least one of information on average round trip delay (Average DL RTT).

The logic server according to an embodiment receives a feedback of downlink bandwidth information from the user terminal.

The logic server according to an embodiment uses the downlink bandwidth information to calculate the optimal bit rate.

Through this disclosure, the service quality can be improved by applying the bit rate based on the bit rate and delay, which are indicators of customer experience that are one step higher than the simple buffering time or network download speed, and variable bit rates for each content based on customer usage statistics. This will allow the efficient use of the network.

1 is a conceptual diagram of a real-time variable bit rate transmission system according to an embodiment.
2 is a conceptual diagram illustrating a process of calculating an optimal bitrate value based on at least one of usage statistics, quality statistics, and resolution information in a logic server according to an exemplary embodiment.
3 is a conceptual diagram of determining a resolution applied to each of the ROI region and the scaling region according to the optimal bit rate calculated by a logic server for original content divided into an ROI region and a scaling region according to an embodiment. .
4 illustrates an example for explaining a reduced ratio compared to the original when encoding media original content by dividing it into an ROI region and a scaling region according to an embodiment.
5 is a diagram illustrating a reduced delay when transmitting left and right eyes separately according to an exemplary embodiment.
6 is a flowchart illustrating a real-time variable bit rate transmission method according to an embodiment.
7 illustrates a relationship between a configuration of a user terminal and a real-time variable bitrate system according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, since various changes may be made to the embodiments, the scope of the patent application is not limited or limited by these embodiments. It should be understood that all modifications, equivalents and substitutes for the embodiments are included in the scope of the rights.

The terms used in the examples are used for description purposes only, and should not be construed as limiting. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present specification, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiment belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

In addition, in the description with reference to the accompanying drawings, the same components are given the same reference numerals regardless of the reference numerals, and the overlapping description thereof will be omitted. In the description of the embodiment, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the embodiment, the detailed description thereof will be omitted.

In addition, in describing the components of the embodiment, terms such as first, second, A, B, (a), (b), etc. may be used. These terms are only for distinguishing the components from other components, and the essence, order, or order of the components are not limited by the terms. When it is described that a component is “connected”, “coupled” or “connected” to another component, the component may be directly connected or connected to the other component, but another component is between each component. It will be understood that may also be "connected", "coupled" or "connected".

Components included in one embodiment and components having a common function will be described using the same names in other embodiments. Unless otherwise stated, a description described in one embodiment may be applied to another embodiment, and a detailed description in the overlapping range will be omitted.

<system>

1 is a conceptual diagram of a real-time variable bit rate transmission system according to an embodiment.

Referring to FIG. 1 , the components and flow of the real-time variable bit rate transmission system 100 are illustrated. The real-time variable bit rate transmission system 100 includes a logic server 120 , a bit rate adapter 130 , an encoder 140 , and a transmission unit 150 , and the media source 110 includes a logic server 120 and a bit rate. A rate adapter 130 is provided. The logic server 120 may receive information on usage statistics and quality statistics from the statistics server 122 , and may also receive resolution information 124 . The logic server 120 calculates an optimal bitrate based on at least one of this information. In the bitrate adapter 130 , the original content divided into a region of interest (ROI) maintaining a high resolution and a scaling region in which the resolution is changed is determined according to the calculated optimal bitrate. A resolution applied to each of the ROI region and the scaling region is determined. The encoder 140 performs encoding according to the resolution determined by the bitrate adapter 130 . The transmitter 150 transmits the content encoded by the encoder 140 to the user terminal 160 at the optimal bit rate.

When encoding is performed, the encoder 140 may encode each of the left-eye image and the right-eye image of the original content, and the transmitter 150 transmits each of the left-eye image and the right-eye image of the encoded content to the user terminal with the optimal bit. rate can be transmitted. This will be described in detail with reference to FIG. 5, which will be described later.

The user terminal 160 feeds back downlink bandwidth information to the logic server 120 , and the logic server 120 uses this information to calculate the optimal bit rate. When a change is detected by checking downlink bandwidth information according to a specific period in the user terminal 160, this information is fed back to the logic server 120, and the logic server 120 adjusts the variable resolution for the original media accordingly. apply

2 is a conceptual diagram illustrating a process of calculating an optimal bitrate value based on at least one of usage statistics, quality statistics, and resolution information in a logic server according to an embodiment.

In the logic server, the end-to-end latency (E2E latency) value (Y) of each item (X) of the usage statistics and the quality statistics is stored, and the content bit rate value of each item (X) ( Store contents bitrate value)(Z). And by calculating the correlation index (correlation) of each item (X) and the E2E latency value according to each item, an item with a high degree of relevance (eg, an item greater than 0.7) is designated as a core item, and the If there is a value different from the average value among the core items, the predicted contents bitrate value (W) is calculated by multiplying the weights (A, B, C) for the corresponding bit rates. In the case of resolution information, the E2E latency value (Y) of the variable resolution index is stored, the content bitrate value (Z) at that time is stored, and the weight is calculated and multiplied to calculate the expected content bitrate value (W) do.

The usage statistics include at least one of usage time, usage time, streaming location information, content type, and device type information, and the quality statistics include network throughput, packet loss rate, and average At least one of information on round-trip delay (Average DL RTT) is included.

3 is a conceptual diagram of determining a resolution applied to each of the ROI region and the scaling region according to the optimal bit rate calculated by a logic server for original content divided into an ROI region and a scaling region according to an embodiment. .

3 illustrates a change in resolution when the variable resolution is applied by dividing the ROI region and the scaling region. In the case of the existing adaptive bitrate media streaming, when the network bandwidth is small due to the user's poor network usage, a low resolution is applied to the entire frame and adaptively transmitted. However, this causes inconvenience to the user due to the low resolution. In the present disclosure, the ROI region of the media is maintained at a relatively high resolution, and the compression ratio is adjusted for the remaining scaling region so that the quality can be considered.

In the case of a good network situation, all regions of the image 310 may be encoded with high resolution and transmitted to the user terminal. As described above, when the downlink bandwidth information is fed back from the user terminal and the bandwidth is reduced, the image 330 has a high resolution of the ROI region 331 in the center, which is mainly observed by the user, and the resolution of the remaining scaling region 333 . is low-encoded and transmitted to the user terminal. At this time, the resolution of each region is determined according to the optimal bit rate calculated by the logic server 120 of FIG. 1 in the bit rate adapter 130 of FIG. 1 according to the method of FIG. 2 . The division of the left eye and the right eye is described together with FIG. 6 in relation to binocular separation transmission.

4 illustrates an example for explaining a reduced ratio compared to the original when encoding media original content by dividing it into an ROI region and a scaling region according to an embodiment.

3 shows an embodiment in which the resolutions of the ROI region and the scaling region are determined differently. In this regard, the media source 410 and the encoding resolution 430 of FIG. ) is shown. Assuming that the field of view (FoV) of the user terminal is 100°, the reduced ratio after encoding compared to the original media is obtained as follows. Since the same applies to the left and right eyes, calculations were made based only on the left eye.

The resolution of the existing media source 410 was 1600 * 1440 = 2304000 (pixel), and the resolution of the ROI region 412, which is a fixed resolution, was (60/100) * (60/100) * 1600 * 1440 = 829440 (pixel), the resolution of the scaling area 414 is 2304000-829440 = 1474560 (pixel). Looking at the resolution 430 after encoding, the encoding resolution is 1200 * 1080 = 1296000 (pixel), and since the resolution of the ROI area 432 is maintained, it is still 829440 (pixel) the same as the resolution of the ROI area 412 of the media source. . Accordingly, the resolution of the scaling region 434 after encoding becomes 1296000-829440 = 466560 (pixel). Here, the number of pixels (1200*1080) of the encoding resolution 430 may be flexible according to the bandwidth fed back from the aforementioned user terminal 160 .

If the reduced ratio of the resolution of the scaling region in which the resolution is changed after encoding is calculated, it can be seen that 466560/1474560 = 0.316, that is, it is reduced to about 32%. Accordingly, when the content bitrate is calculated after encoding, if the existing bitrate is 50Mbps, after encoding, 50M*{(100-32)/100}=34 Mbps.

Through this, it can be confirmed that the bitrate is reduced when the bitrate adapter determines a different resolution for each of the ROI region and the scaling region in the real-time bitrate transmission system.

5 is a diagram illustrating a decrease in delay when transmitting left and right eyes separately according to an exemplary embodiment.

The existing method 510 for bundling both left and right eyes and transmitting them as a container file, and a method 530 for transmitting each of the left and right eyes separately are disclosed. Both the left-eye image and the right-eye image are encoded (512, 532) in the encoder of the real-time variable bitrate transmission system, transmitted (514, 534) in the transmitter, and decoded (516, 536) in the decoder of the user terminal, When the left and right eyes are separated and transmitted compared to the case of transmission, the packet size is reduced, and each process is processed in parallel to finally reduce the delay 540 .

<Method>

6 is a flowchart illustrating a real-time variable bitrate transmission method 600 according to an embodiment.

The logic server calculates an optimal bitrate based on usage statistics, quality statistics and resolution information (602), and for the original content divided into an ROI region and a scaling region in the bitrate adapter, the logic A resolution applied to each of the ROI region and the scaling region is determined (604) according to the optimal bitrate calculated by the server, and the encoder encodes (606) according to the resolution determined by the bitrate adapter, and a transmission unit transmits the encoded content to the user terminal at the optimal bit rate (608). The downlink bandwidth information is periodically fed back from the user terminal ( 610 ), and the logic server calculates an optimal bit rate again using this information ( 602 ).

The step 602 of the logic server calculating the optimal bitrate may include an End to End (E2E) delay based on at least one of usage statistics, quality statistics, and resolution information in the same manner as described above with respect to FIG. 2 . It is calculated based on a (latency) value, and the usage statistics include at least one of usage time, usage time, streaming location information, content type, and device type information, and the quality statistics include network throughput, It includes at least one of information about a packet loss rate and an average round trip delay (Average DL RTT).

In Fig. 6, the step 604 of the bitrate adapter determining the resolution applied to each region according to the optimal bitrate calculated in the logic server for the original content divided into the ROI region and the scaling region is the same as in Figs. It proceeds in the same way as described above.

In FIG. 6, the encoder encodes according to the resolution determined by the bitrate adapter (606), and the transmitter transmits the encoded content to the user terminal at the optimal bit rate (608) in relation to binocular separation transmission. 5 may proceed as described above. When the encoder performs encoding, the encoding may be performed into each of the left-eye image and the right-eye image of the original content, and the transmitter may transmit each of the left-eye image and the right-eye image of the encoded content to the user terminal at the optimal bit rate.

In FIG. 6 , the step 610 of receiving downlink bandwidth information from the user terminal proceeds as described above in FIG. 1 .

<User terminal>

7 illustrates a relationship between a configuration of a user terminal and a real-time variable bitrate system according to an embodiment.

The user terminal 160 of FIG. 7 includes a receiver 710 that receives the content transmitted 760 from the transmitter 150 of the real-time variable bitrate transmission system 100, and a decoder 730 that decodes the received content, respectively. ), and a transmission unit 750 for periodically transmitting 770 feedback information to the logic server 120 of the real-time variable bit rate transmission system 100 .

The content is end-to-end based on at least one of usage statistics, quality statistics, and resolution information in the logic server 120 of the variable bit rate transmission system 100, as described above with reference to FIGS. 2 to 5 . to End; E2E) Calculates an optimal bitrate based on a latency value, and divides it into an ROI region and a scaling region in the bitrate adapter 130 of the real-time variable bitrate transmission system For the original content, a resolution applied to each of the ROI region and the scaling region is determined according to the optimal bitrate calculated by the logic server, and in the encoder 140 of the real-time variable bitrate transmission system, the The encoded content is encoded according to the resolution determined by the bitrate adapter, and the encoded content is transmitted (760) to the user terminal 160 at the optimal bitrate in the transmission unit 150 of the real-time variable bitrate transmission system.

The logic server 120 calculates the optimal bitrate, and the bitrate adapter 130 applies it to each region according to the optimal bitrate calculated by the logic server 120 for the original content divided into the ROI region and the scaling region. The resolution is determined, the encoder 140 encodes according to the resolution determined by the bitrate adapter 130, and the transmission unit 150 transmits the encoded content to the user terminal 160 at the optimal bit rate. The process proceeds in the same manner as described above with reference to FIGS. 2 to 5 .

The transmitting unit 150 transmits the encoded content to the user terminal 160 at the optimal bit rate is specifically transmitting (760) to the receiving unit 710 in the user terminal 160 .

The decoder 720 decodes the left-eye image and the right-eye image received by the receiver 710. At this time, the binocular separation technique related to FIG. 5 may be applied (536).

The transmission unit 750 of the user terminal 160 periodically feedbacks 770 the downlink bandwidth information to the logic server 120 of the real-time variable bit rate transmission system 100 . The logic server 120 uses this to calculate the optimal bitrate.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

The software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

As described above, although the embodiments have been described with reference to the limited drawings, those skilled in the art may apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (15)

a logic server for calculating an optimal bitrate based on at least one of usage statistics, quality statistics, and resolution information;
A bitrate adapter that determines a resolution applied to each of the ROI region and the scaling region according to the optimal bitrate calculated by the logic server for the original content divided into an ROI region and a scaling region adapter);
an encoder for encoding according to the resolution determined by the bitrate adapter; and
Transmission unit for transmitting the encoded content to the user terminal at the optimal bit rate
Including, real-time variable bitrate transmission system.
According to claim 1,
The encoder encodes each of a left-eye image and a right-eye image of the original content,
The transmitter transmits each of the left-eye image and the right-eye image to a user terminal,
Real-time variable bitrate transmission system.
According to claim 1,
The logic server calculates the optimal bitrate value based on an end-to-end (E2E) latency value based on at least one of the usage statistics, the quality statistics, and the resolution information.
Real-time variable bitrate transmission system.
4. The method of claim 3,
The usage statistics include at least one of usage time, usage time, streaming location information, content type and device type information,
The quality statistics include at least one of information about network throughput, packet loss rate, and average round-trip delay (Average DL RTT),
Real-time variable bitrate transmission system.
According to claim 1,
The logic server receives a feedback of downlink bandwidth information from the user terminal,
Real-time variable bitrate transmission system.
6. The method of claim 5,
The logic server uses the downlink bandwidth information to calculate the optimal bitrate,
Real-time variable bitrate transmission system.
calculating, in the logic server, an optimal bitrate based on at least one of usage statistics, quality statistics, and resolution information;
In the bitrate adapter, for the original content divided into an ROI region and a scaling region, the resolution applied to each of the ROI region and the scaling region is determined according to the optimal bitrate calculated by the logic server step;
encoding in the encoder according to the resolution determined by the bitrate adapter; and
Transmitting, in the transmission unit, the encoded content to the user terminal at the optimal bit rate
Including, real-time variable bitrate transmission method.
8. The method of claim 7,
The encoder encodes each of a left-eye image and a right-eye image of the original content,
The transmitter transmits each of the left-eye image and the right-eye image to a user terminal,
Real-time variable bitrate transmission method.
8. The method of claim 7,
The logic server calculates the optimal bitrate value based on an end-to-end (E2E) latency value based on at least one of the usage statistics, the quality statistics, and the resolution information.
Real-time variable bitrate transmission method.
10. The method of claim 9,
The usage statistics include at least one of usage time, usage time, streaming location information, content type and device type information,
The quality statistics include at least one of information about network throughput, packet loss rate, and average round-trip delay (Average DL RTT),
Real-time variable bitrate transmission method.
8. The method of claim 7,
The logic server receives a feedback of downlink bandwidth information from the user terminal,
Real-time variable bitrate transmission method.
12. The method of claim 11,
The logic server uses the downlink bandwidth information to calculate the optimal bitrate,
Real-time variable bitrate transmission method.
A computer-readable recording medium containing a program for performing the method of any one of claims 7 to 12.
in a user terminal.
a receiver for receiving content transmitted from a real-time variable bit rate transmission system;
a decoder for decoding the content; and
Transmission unit for transmitting feedback information to the real-time variable bit rate transmission system
including,
The content is
In the logic server of the real-time variable bitrate transmission system, calculating an optimal bitrate based on at least one of usage statistics, quality statistics, and resolution information,
In the bitrate adapter of the real-time variable bitrate transmission system, for the original content divided into an ROI region and a scaling region, each of the ROI region and the scaling region according to the optimal bitrate calculated by the logic server determines the resolution applied for
In the encoder of the real-time variable bitrate transmission system, encoding according to the resolution determined by the bitrate adapter,
In the transmitting unit of the real-time variable bit rate transmission system, the encoded content is transmitted to the user terminal at the optimal bit rate,
user terminal.
15. The method of claim 14,
The content is
The encoder encodes each of a left-eye image and a right-eye image of the original content,
Each of the left-eye image and the right-eye image is transmitted to the user terminal by the transmitter,
user terminal.

Images