KR20210077841A - Method and apparatus for providing high quality real time media streaming service with low latency - Google Patents

Abstract

Provided are a method of providing a high-quality/low-latency real-time media streaming service and an apparatus thereof. The method, which is a method of operating an apparatus for managing the quality of experience (QoE), includes the following steps of: collecting transmission information of real-time media stream packets and condition information about each of a plurality of wireless networks from a multichannel client dividing and transmitting the packets through the plurality of wireless networks, respectively; calculating buffering time and transmission time intervals for a packet having a changed jitter, based on the transmission information of the packets and the condition information; and transmitting the calculated buffering time and transmission time intervals to a multichannel gateway merging the media stream packets received from the multichannel client through the plurality of wireless networks to transmit the same to a terminal. The buffering time and transmission time intervals are used to restore the packet having the changed jitter as an original packet transmitted from the multichannel client.

Description

Method and device for providing high-quality, low-latency real-time media streaming service {METHOD AND APPARATUS FOR PROVIDING HIGH QUALITY REAL TIME MEDIA STREAMING SERVICE WITH LOW LATENCY}

The present invention relates to a method and apparatus for providing a high-quality, low-latency real-time media streaming service.

The high-quality, low-latency real-time media streaming service provides low-latency transmission of high-quality multimedia content such as Ultra High Definition (UHD) content.

In general, a high-quality, low-latency real-time media streaming service operates based on User Datagram Protocol (UDP) in a dedicated line-based wired network. The reason for using UDP is that it enables high-speed processing in terms of video encoding/decoding and transmission at regular intervals, so that the Constant Bit Rate (CBR) required for real-time media streaming can be used. However, since UDP is not responsible for data reception, it does not guarantee stability. Therefore, UDP is used in a fast and reliable dedicated line-based wired network.

Recently, with the generalization of smart devices, the demand for a media streaming service in a wireless network is rapidly increasing. In addition, 5G services that support speeds that are at least 10 times and up to 1000 times faster than the existing LTE (Long-Term Evolution) are being provided. Therefore, the demand of users who want to provide a high-quality, low-latency real-time media streaming service in a wireless network environment is expected to increase.

In the wireless network, the packet loss rate and transmission bandwidth change over time, and accordingly, the phenomenon of disconnection occurs frequently in the transmission of high-capacity multimedia data. Therefore, it is difficult to use the media streaming service using UDP provided in the existing dedicated line-based wired network in an unstable wireless network environment.

The problem to be solved by the present invention is to provide a media streaming service using UDP (User Datagram Protocol) to a wireless network through a virtual private network (VPN)-based multi-net using Jitter Stable technology. It is to provide a method and an apparatus for stably providing the same.

According to an embodiment of the present invention, as a method of operating an apparatus for managing quality of experience (QoE), the packet is transmitted from a multi-path client that divides a real-time media stream packet and transmits the packet through a plurality of wireless networks. Collecting transmission information and status information for each of the plurality of wireless networks, using the transmission information of the packet and the status information, calculating a buffering time and transmission time interval for a packet with a changed jitter and transmitting the calculated buffering time and transmission time interval to a multi-path gateway that merges the media stream packets received from the multi-path client through the plurality of wireless networks and transmits them to the terminal. The time and the transmission time interval are used to restore the jitter-changed packet to the original packet transmitted from the multi-path client.

After the step of transmitting, the buffering time and the transmission time interval are recalculated using the queue length change information of the jitter buffer received from the multi-path gateway, and the recalculated buffering time and the transmission time interval are set to the multi-path gateway. The method may further include transmitting the jitter buffer to the jitter buffer, and after buffering the jitter-changed packet for the buffering time, output the jitter buffer in units of the transmission time interval.

The packet may be a User Datagram Protocol (UDP) packet using a constant bit rate (CBR).

The packet transmission information may include the number of packets per unit time of the packet transmitted in a predetermined time unit from the multi-path client and an average size of the packets.

The status information may include information measured by the multi-path client for each of the plurality of wireless networks, and the measured information may include a Round Trip Time (RTT) and a bandwidth (BW).

The transmission time interval is calculated using a packet transmission interval included in the packet transmission information and a maximum value of the number of packets per second that can be transmitted in each of the plurality of wireless networks, and the maximum value of the number of packets per second is, The calculation may be performed using an average value of Round Trip Time (RTT) measured in each of a plurality of wireless networks, an average size of a transport packet included in the transmission information of the packet, and a bandwidth of each of the plurality of wireless networks.

The buffering time may be calculated using a maximum buffering time, the packet transmission interval, and the transmission time interval, and the maximum buffering time may be a preset value.

After the collecting step, by calculating the standard deviation of the RTT (Round Trip Time) values measured in each of the plurality of wireless networks, the priority is determined in the order of the least change in the wireless network state, and the prioritization information is divided into the multiple The method further includes transmitting the packet to a path client, wherein the prioritization information may be used when the multi-path client transmits the packet through a plurality of networks.

According to another embodiment of the present invention, as a method of operating a multi-path gateway, a packet whose jitter is changed from a device for managing Quality of Experience (QoE) is converted to an original packet transmitted from the multi-path client. Receiving a buffering time and a transmission time interval for recovery, receiving the segmented real-time media stream packets from a multi-path client through a plurality of wireless networks and merging them into one packet, and merging the merged packets in the jitter buffer After buffering for a buffering time, transmitting to the terminal at the transmission time interval.

After the transmitting step, transmitting the queue length change information of the jitter buffer to the apparatus for managing the quality of experience, the buffering time and transmission time interval recalculated by reflecting the queue length change information to manage the quality of experience The method may further include the steps of: receiving the merged packet in the jitter buffer by the recalculated buffering time, and then transmitting the merged packet to the terminal at the recalculated transmission time interval.

The merging may include merging the divided real-time media stream packets into one Transmission Control Protocol (TCP) packet, and between the merging and transmitting, multiple merge tunneling headers and The method further comprising the step of removing a VPN (Virtual Private Network) tunneling header to restore a User Datagram Protocol (UDP) packet, wherein the transmitting includes buffering the UDP packet by the buffering time in the jitter buffer and then It can be transmitted to the terminal at the transmission time interval.

The UDP packet is transmitted in a predetermined time unit using a constant bit rate (CBR), and the transmission time interval is the number of packets per unit time of the UDP packet measured by the multi-path client and the plurality of packets. It is calculated using the maximum number of packets per unit time that can be transmitted in each wireless network, and the buffering time can be calculated using the number of packets per unit time of the UDP packet and the transmission time interval.

According to another feature of the present invention, the quality of experience management apparatus includes a communication interface respectively connected to a multi-path client and a multi-path gateway, a memory storing a program, and at least one processor executing the program, The path client divides the real-time media stream packet using UDP (User Datagram Protocol) and transmits it through a plurality of wireless networks, respectively, and the multi-path gateway receives the divided real-time media stream packet and converts it into one packet. collecting the transmission information of the real-time media stream packet and the status information for each of the plurality of wireless networks from the multi-path client, the processor collects the transmission information of the packet and the status information calculating a jitter stabilization value using the method, and transmitting the jitter stabilization value to the multi-path gateway, wherein the jitter stabilization value is received by the multi-path gateway through the plurality of wireless networks It is used to restore the jitter-changed packet to the original packet transmitted by the multi-path client.

The jitter stabilization value includes a buffering time and a transmission time interval, and the buffering time and the transmission time interval are, after buffering the one packet in the jitter buffer of the multi-path gateway, to the terminal at a constant time unit. can be used to transmit.

After the transmitting step, receiving the queue length change information of the jitter buffer from the multi-path gateway, and calculating the buffering time and the transmission time interval by reflecting the queue length change information to the multi-path gateway It may further include the step of transmitting.

According to the embodiment, by using the existing encoder and decoder for broadcasting as it is, it is possible to provide a high-quality, low-latency real-time media streaming service anywhere. Therefore, infrastructure can be more easily provided to customers who need high-quality real-time broadcasting, and cost savings can be obtained by using a low-cost wireless network compared to a high-cost leased-line wired network.

1 is a block diagram of a multi-wireless network system that provides a high-quality, low-latency real-time media streaming service according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating a detailed configuration and connection relationship of each of the multi-path client and the multi-path gateway of FIG. 1 .
Figure 3 is a flow chart showing a high-quality, low-delay real-time media streaming service method according to an embodiment of the present invention.
Figure 4 is a flow chart showing a high-quality low-delay real-time media streaming service method according to another embodiment of the present invention.
5 is a flowchart showing a high-quality, low-latency real-time media streaming service method according to another embodiment of the present invention.
6 is a hardware configuration diagram according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those of ordinary skill in the art to which the present invention pertains can easily implement them. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part "includes" a certain element, it means that other elements may be further included, rather than excluding other elements, unless otherwise stated.

In addition, terms such as “…unit”, “…group”, and “module” described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware or software or a combination of hardware and software. have.

The devices described in the present invention are composed of hardware including at least one processor, a memory device, a communication device, and the like, and a program executed in combination with the hardware is stored in a designated place. The hardware has the configuration and capability to implement the method of the present invention. The program includes instructions for implementing the method of operation of the present invention described with reference to the drawings, and is combined with hardware such as a processor and a memory device to execute the present invention.

As used herein, "transmission or provision" may include not only direct transmission or provision, but also transmission or provision indirectly through another device or using a detour path.

In the present specification, expressions described in the singular may be construed in the singular or plural unless an explicit expression such as “a” or “single” is used.

In this specification, the same reference numbers refer to the same components regardless of the drawings, and "and/or" includes each and every combination of one or more of the referenced components.

In this specification, terms including an ordinal number such as first, second, etc. may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present disclosure, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

In the flowchart described with reference to the drawings in this specification, the order of operations may be changed, several operations may be merged, some operations may be divided, and specific operations may not be performed.

In the present specification, a terminal is a generic concept meaning a user terminal in communication, and is a user equipment (UE), a mobile station (MS), a mobile terminal (MT), a subscriber station (SS), a portable subscriber station (PSS), and an AT ( Access Terminal), mobile station, mobile terminal, subscriber station, portable subscriber station, user equipment, access terminal, wireless device, etc., may be called terms such as UE, MS, MT, SS, PSS, AT, mobile station, mobile terminal, subscription It may include all or part of the functions of a home country, a portable subscriber station, a user equipment, an access terminal, a wireless device, and the like.

A terminal is a base station (BS), an access point (Access Point, AP), a radio access station (RAS), a Node B (Node B), an advanced Node B (evolved NodeB, eNodeB), a transmission and reception base station ( It can be connected to a remote server by accessing a network device such as a Base Transceiver Station (BTS) or Mobile Multihop Relay (MMR)-BS.

Base Station (BS) is an Access Point (AP), Radio Access Station (RAS), Node B (Node B), Base Transceiver Station (BTS), MMR (Mobile Multihop Relay) )-BS, etc. may refer to, and may include all or some functions of an access point, a radio access station, a Node B, a transceiver base station, an MMR-BS, and the like.

The terminal of the present specification is a communication terminal of various types, such as a mobile terminal such as a smart phone, a tablet terminal such as a smart pad and a tablet PC, a computer, and a television, and may include a plurality of communication interfaces.

The communication interface may vary. For example, the communication interface is a short-range wireless network interface such as Wi-Fi/Wireless Local Area Network (WLAN)/Bluetooth, and 3G/LTE (Long Term Evolution)/LTE-A (Long Term) It may include a mobile communication network interface such as Evolution-Advanced)/5G, and a terminal manufacturer may add various communication interfaces.

In this specification, a WiFi interface, an LTE interface, and a 5G interface will be described as examples, but the communication interface is not limited thereto.

In the present specification, multi-net aggregation (hereinafter, collectively referred to as 'MA') is a technology for transmitting traffic using a plurality of wireless networks at the same time. treated with Through the multi-network merging technology, a terminal can be connected to a plurality of wireless networks at a time, and one service/application communicates by merging a plurality of networks as one network, regardless of the type of network or the number of networks. Accordingly, the multi-network aggregation technology can rapidly transmit and receive a large amount of traffic by using a plurality of available network resources.

Multi-network merge (MA) transmission is a technology that transmits data by merging a plurality of communication networks. Data transmitted through one path is divided into multiple homogeneous networks or multiple heterogeneous networks for transmission, or data transmitted through multiple paths. The transmitted data can be bundled into one path and transmitted. Multi-network merge transmission may be referred to as multi-path transmission in the sense of simultaneously transmitting data through a plurality of paths.

The merging technology of a plurality of wireless networks enables various merge transmissions depending on the merge transport layer such as L2/link layer merging, L3/network layer merging, L4/transport layer merging, and L7/application layer merging. As a representative technology of the multi-network merging (MA) technology, there is a multi-path TCP (hereinafter, 'MPTCP').

Embodiments of the present invention are 3GPP (3rd Generation Partnership Project) Long Term Evolution (LTE), LTE-A, 3GPP2, IEEE (Institute of Electrical and Electronics Engineers) system, 3GPP such as FS_NextGen (Study on Architecture for Next Generation System). It may be supported by standard documents related to at least one of the 5G systems. That is, steps or parts not described in order to clearly reveal the technical spirit of the present invention among the embodiments of the present invention may be supported by the document. In addition, all terms disclosed in this document may be explained by the standard document.

Now, a high-quality, low-latency real-time media streaming service method and a multi-wireless network system providing the same will be described with reference to the drawings.

1 is a block diagram of a multi-wireless network system that provides a high-quality, low-latency real-time media streaming service according to an embodiment of the present invention.

Referring to FIG. 1 , a multi-wireless network system can be defined as a system for transmitting a high-quality, low-latency real-time media stream using a multi-network merging (MA) technology. This multi-wireless network system includes a camera 100 , a server 200 , a multi-path gateway (Multi Path-Gateway, MP-GW) 300 , a terminal 400 , and a Quality Of Experience (QoE) manager 500 ) is included.

The camera 100 is a device for capturing and acquiring an image to be transmitted to the terminal 400 .

The server 200 includes an encoder 210 and a multi-path client (Multi Path-Client, MP-C) 230 .

The encoder 210 encodes an image output by the camera 100 into a real-time media stream format and outputs it to the multi-path client 230 . According to one embodiment, the encoder 210 receives the video and audio captured by the camera 100, encodes it, and then converts it according to the broadcast stream of the ATSC 3.0 format (Advanced Television Systems Committee standard 3.0 version) and outputs it. can

The multi-path client 230 transmits/receives data to and from the multi-path gateway 300 based on MPTCP Virtual Private Network (VPN) tunneling. The multi-path client 230 transmits a real-time media stream to the terminal 400 through the multi-path gateway 300 .

The multi-path client 230 includes a plurality of communication interfaces, and may be connected to a plurality of networks at one time through the plurality of communication interfaces. Here, the plurality of communication interfaces may include a 5G interface, an LTE interface, a WiFi interface, and the like.

The multi-path client 230 creates an MPTCP session with the multi-path gateway 300 in units of subflows, and transmits and receives data through a plurality of communication interfaces.

The multi-path client 230 may be an application mounted on the server 200 . The multi-path client 230 performs authentication, state management, and traffic processing for multi-network merging, and divides the single-path flow output by the encoder 210 into multi-path subflows and transmits the split.

The multi-path client 230 creates a VPN with the multi-path gateway 300 by requesting the creation of a VPN to the multi-path gateway 300 through a plurality of wireless networks (5G, LTE, WiFi, etc.).

When the VPN is created, the multi-path client 230 performs an MPTCP service authentication procedure with the multi-path gateway 300 by requesting authentication to the multi-path gateway 300 through the VPN.

If the MPTCP service authentication procedure is successful, the multi-path client 230 divides and transmits the real-time media stream to the multi-path gateway 300 through a heterogeneous network (eg, 5G network, LTE network, Wi-Fi network) according to the distribution policy. do.

The multi-path gateway 300 merges data received through a plurality of communication interfaces and transmits the merged data to the terminal 400 . The multi-path gateway 300 merges each real-time media stream received from the multi-path client 230 through a plurality of wireless networks and outputs the merged real-time media streams to the terminal 400 in real time.

The terminal 400 includes a decoder 410 and a display device 430 .

The decoder 410 decodes the media stream received from the multi-path gateway 300 and transmits it to the display device 430 . The display device 430 plays and outputs the decoded media stream.

In this way, in multi-network aggregation (MA) transmission such as MPTCP, a plurality of physically separated networks can be logically merged as one network so that a plurality of networks can be used at the same time. In particular, MPTCP can divide a session created for one service into a plurality of subflows, and through this, an access network can be dynamically added or released.

Therefore, MPTCP provides stable service regardless of changes in the physical environment by merging multiple networks to increase the transmission speed when merged transmission is possible, while transmitting traffic to another wireless network when the condition of one wireless network becomes poor. can do.

However, in general, a media streaming service provided based on User Datagram Protocol (UDP) is provided over a dedicated line based wired network. The state of the wireless network changes in real time irregularly.

By merging UDP packets based on TCP through MPTCP VPN, packets can be reliably delivered to their destination even in multiple wireless network environments. However, due to TCP characteristics (eg, congestion control, etc.), a problem occurs in the fixed bit rate (CBR) of UDP packets. That is, the traffic interval varies non-uniformly. This problem becomes more inconsistent because of multi-network merging. This is because multi-network merging causes delays because wireless networks with different network environments (bandwidth, latency, etc.) are merged based on TCP and reordered.

Therefore, in order to stably provide a real-time media streaming service using a multi-network merge (MA) transmission technology, an operation adaptive to real-time changes in the state of each wireless network is required.

In the embodiment of the present invention, a jitter stabilization function (hereinafter, collectively referred to as 'JSF') is used to solve the delay problem. JSF can be divided into two main functions.

JSF includes a function of buffering UDP packets whose jitter is changed while passing through MPTCP and MPTCP VPN. The JSF includes a function of transmitting the buffered UDP packets to the decoder at the interval transmitted by the encoder 210 .

Jitter represents the variation in inter-arrival time between packets.

The UDP buffering time (UDP_BUF_TIME) and UDP transmission time interval (UDP_SEND_TIME_INTERVAL) that can stabilize jitter are very important to provide high-quality, low-latency real-time media streaming service. For example, if the UDP buffering time (UDP_BUF_TIME) is long, it is possible to provide a seamless streaming service by stably transmitting an image, but the delay increases and real-time performance is deteriorated. On the other hand, if the UDP transmission time interval (UDP_SEND_INTERVAL_TIME) is too short, low delay can be achieved, but it cannot respond to the change of the multi-wireless network environment, and there may be a disconnection.

Therefore, it is necessary to derive the optimal UDP buffering time (UDP_BUF_TIME) and UDP transmission time interval (UDP_SEND_INTERVAL_TIME) that can minimize jitter by reflecting the multi-wireless network environment.

In the embodiment of the present invention, the optimal UDP buffering time (UDP_BUF_TIME) and UDP transmission time interval (UDP_SEND_INTERVAL_TIME) are determined through the quality of experience (QoE) manager 500 connected to the multi-path client 230 and the multi-path gateway 300. Calculate.

The quality of experience (QoE) manager 500 monitors the status of a plurality of wireless networks and the transmission status of media streams through these two devices 230 and 300, and based on this, the optimal UDP buffering time (UDP_BUF_TIME) and UDP transmission time By deriving the interval (UDP_SEND_INTERVAL_TIME), the quality of experience (QoE) is controlled.

As described above, since the UDP buffering time (UDP_BUF_TIME) and the UDP transmission time interval (UDP_SEND_INTERVAL_TIME) are used as jitter stabilizing values, in this specification, they are collectively referred to as a 'jitter stabilizer value'.

FIG. 2 is a diagram illustrating a detailed configuration and connection relationship of each of the multi-path client and the multi-path gateway of FIG. 1 .

Referring to FIG. 2 , a multi-path client (MP-C) 230 includes a UDP routing unit 231 , an MPTCP-VPN processing unit 232 , an MPTCP processing unit 233 , and a plurality of communication interfaces 234 , 235 , 236 . and a status monitor 237 .

The UDP routing unit 231 receives a UDP packet from the encoder 210 . The UDP routing unit 231 transmits the UDP packet to the MPTCP-VPN processing unit 232 based on L3 routing. Since the UDP routing unit 231 operates based on CBR, it routes UDP packets at regular intervals (UDP_PPS, UDP_Packet per Second).

When the multi-path client 230 boots, the MPTCP-VPN processing unit 232 connects to the quality of experience (QoE) manager 500 using the preset IP address of the quality of experience (QoE) manager 500 .

The MPTCP-VPN processing unit 232 obtains the IP address of the multi-path gateway 300 from the quality of experience (QoE) manager 500 . The MPTCP-VPN processing unit 232 requests generation of the MPTCP VPN from the multi-path gateway 300 using the obtained IP address. The MPTCP-VPN processing unit 232 creates the multi-path gateway 300 and the MPTCP VPN.

When the MPTCP VPN is created, the MPTCP-VPN processing unit 232 adds a VPN header and an MPTCP header to the UDP packet received from the UDP routing unit 231 , and delivers them to the MPTCP processing unit 233 .

The MPTCP processing unit 233 transmits the VPN header and the UDP packet to which the MPTCP header is added to the multi-path gateway 300 through the plurality of air interfaces 234 , 235 , and 236 .

At this time, the MPTCP processing unit 233 divides the UDP packet received from the MPTCP-VPN processing unit 232 . The MPTCP processing unit 233 transmits a partial packet of the divided UDP packet through a subflow of the air interface 234 of the first network (eg, 5G). The MPTCP processing unit 233 transmits another partial packet of the divided UDP packet through a subflow of the air interface 235 of the second network (eg, LTE network). The MPTCP processing unit 233 may transmit the remaining data of the divided UDP packet through a subflow of the air interface 236 of a third network (eg, WiFi network).

The status monitor 237 measures the UDP packet interval (UDP_PPS) output by the UDP routing unit 231 . The status monitor 237 measures a Round Trip Time (RTT) and a bandwidth (BANDWITH) of each of a plurality of wireless networks. The status monitor 237 transmits the measured UDP packet interval (UDP_PPS) information and a plurality of wireless network status information to the quality of experience (QoE) manager 500 periodically or at a preset specific point in time.

The multi-path gateway 300 includes a plurality of air interfaces 301, 302, 303, an MPTCP processing unit 304, an MPTCP-VPN processing unit 305, a UDP redirect processing unit 306, a jitter stabilization unit 307 and a UDP proxy ( 308).

The MPTCP processing unit 304 reorders TCP packets through the plurality of air interfaces 301 , 302 , and 303 to form one TCP flow, and outputs it to the MPTCP-VPN processing unit 305 .

The MPTCP-VPN processing unit 305 creates the multi-path client 230 and the MPTCP VPN according to the request of the MPTCP-VPN processing unit 232 . The MPTCP-VPN processing unit 305 restores the TCP packet received from the MPTCP processing unit 304 to UDP. The MPTCP-VPN processing unit 305 removes the MPTCP header and the VPN header from the TCP packet received from the MPTCP processing unit 304 and restores it to a UDP packet. The MPTCP-VPN processing unit 305 outputs the restored UDP packet to the UDP redirect processing unit 306 .

The UDP redirect processing unit 306 forcibly outputs the UDP packet output from the MPTCP-VPN processing unit 305 to the jitter stabilization unit 307 .

The jitter stabilization unit 307 performs the JSF function described above. The jitter stabilization unit 307 buffers UDP packets whose traffic jitter has changed a lot while passing through TCP for a certain period of time (UDP_BUF_TIME) and restores them to an interval similar to the interval (UDP_SEND_INTERVAL_TIME) transmitted from the encoder 210 again. carry out

Here, the UDP buffering time (UDP_BUF_TIME) and the UDP transmission time interval (UDP_SEND_INTERVAL_TIME) are calculated as jitter stabilization values, and these jitter stabilization values are provided from the quality of experience (QoE) manager 500 .

The UDP proxy 308 transmits the UDP packet adjusted by the jitter stabilization unit 307 to the terminal 400 .

The quality of experience (QoE) manager 500 calculates a jitter stabilization value based on the status report received from the status monitor 237 . Then, the jitter stabilization value is provided to the jitter stabilization unit 307 and applied to the UDP packet to be transmitted to the decoder 410 of the terminal 400 .

Additionally, the terminal 400 is highly likely to receive the streaming packet while moving. Accordingly, the quality of experience (QoE) manager 500 can efficiently cope with the movement of the terminal 400 by using the location information of the terminal 400 and information on the base station to which the terminal 400 is connected.

A method of controlling a high-quality, low-latency real-time media streaming service using the jitter stabilization value will be described.

Figure 3 is a flow chart showing a high-quality, low-delay real-time media streaming service method according to an embodiment of the present invention.

Referring to FIG. 3 , the status monitor 237 of the multipath client 230 measures the interval (PPS) of UDP packets received from the encoder 210 and the average size of the UDP packets ( S101 ). The status monitor 237 measures the RTT and bandwidth of each of a plurality of wireless networks (S103).

The state monitor 237 transmits a measurement report including the information measured in steps S101 and S103 to the quality of experience (QoE) manager 500 (S105). The status monitor 237 may transmit a measurement report at a preset point in time, periodically or at the request of the quality of experience (QoE) manager 500 .

The quality of experience (QoE) manager 500 calculates a UDP buffering time (UDP_BUF_TIME) and a UDP transmission time interval (UDP_SEND_INTERVAL_TIME) based on the measurement report received in step S105 ( S107 ).

The quality of experience (QoE) manager 500 may calculate the UDP buffering time UDP_BUF_TIME as shown in Equation 1 below.

Here, MAX_BUF_TIME is the maximum buffering time and is a value that an administrator can set arbitrarily. For example, it may be set to 5 seconds. R_USIT is the UDP packet interval measured in step S101 and received through step S105. USIT is the UDP transmission time interval (UDP_SEND_INTERVAL_TIME).

The UDP transmission time interval (UDP_SEND_INTERVAL_TIME) may be calculated through Equation (2).

Here, R_UDP_PPS is an interval (Packets per second) of UDP packets received by the multi-path client 230 from the encoder 210 . MAX_PPS _i is the number of packets per second that each wireless network (i∈5G, LTE, WiFi) can send. MAX_PPS _i may be calculated through Equation 3 below.

Here, avgRTT _i is the average RTT value of each wireless network (i∈5G, LTE, WiFi). UDP_PACKET_SIZE is the average UDP packet size measured in step S101 and received in step S105. BW _i is the bandwidth of each wireless network (i∈5G, LTE, WiFi) measured in step S103 and received in step S105.

The quality of experience (QoE) manager 500 generates a jitter stabilization value notification including the UDP buffering time (UDP_BUF_TIME) and the UDP transmission time interval (UDP_SEND_INTERVAL_TIME) calculated in step S107 to generate a jitter stabilization unit ( 307) (S109).

The jitter stabilization unit 307 of the multi-path gateway 300 performs JSF based on the jitter stabilization value received in step S109 (S111). That is, after buffering the UDP packet received from the UDP redirect processing unit 306 of FIG. 2 by the UDP buffering time (UDP_BUF_TIME), it is output to the decoder 410 of the terminal 400 at the UDP transmission time interval (UDP_SEND_INTERVAL_TIME).

Figure 4 is a flow chart showing a high-quality low-delay real-time media streaming service method according to another embodiment of the present invention. In particular, as a process of updating the jitter stabilization value, it may be performed after step S111 of FIG. 3 .

Referring to FIG. 4 , the jitter stabilization unit 307 of the multi-path gateway 300 transmits a report on the result of performing the JSF to the quality of experience (QoE) manager 500 ( S201 ). In this case, the JSF result report includes the amount of jitter queue length change. The jitter stabilization unit 307 may transmit the JSF result report periodically or at a specific time (S201).

The status monitor 237 of the multipath client 230 transmits a measurement report including the information measured in steps S101 and S103 of FIG. 3 to the quality of experience (QoE) manager 500 (S203). The measurement report may be transmitted to the quality of experience (QoE) manager 500 periodically or at a specific time.

The quality of experience (QoE) manager 500 recalculates the UDP buffering time (UDP_BUF_TIME) and the UDP transmission time interval (UDP_SEND_INTERVAL_TIME) based on the JSF result information received in step S201 and the measurement report received in step S203 ( S205 ). ).

The quality of experience (QoE) manager 500 may recalculate the UDP transmission time interval (UDP_SEND_INTERVAL_TIME) (NEW_USIT) using Equation (4).

Here, UDP_SEND_PPS refers to the PPS of the UDP packet output by the multi-path gateway 300 to the decoder 410 using the USIT calculated through Equation (2). Q_PPS means a value expressed in terms of the number of packets per second of the change in the length of the JSF queue.

Quality of experience (QoE) manager 500 is calculated through Equation 4 The UDP buffering time (UDP_BUF_TIME) is recalculated by applying the UDP transmission time interval (UDP_SEND_INTERVAL_TIME) (NEW_USIT) to Equation (1).

The quality of experience (QoE) manager 500 generates a jitter stabilization value update notification including the UDP buffering time (UDP_BUF_TIME) and the UDP transmission time interval (UDP_SEND_INTERVAL_TIME) recalculated in step S205 to stabilize jitter of the multi-path gateway 300 . It is transmitted to the unit 307 (S207).

Then, the jitter stabilization unit 307 of the multi-path gateway 300 performs JSF based on the jitter stabilization value updated in step S207 (S209). That is, after buffering the UDP packet received from the UDP redirect processing unit 306 of FIG. 2 by the UDP buffering time (UDP_BUF_TIME) updated in step S207, the terminal 400 uses the updated UDP transmission time interval (UDP_SEND_INTERVAL_TIME) in step S207. output to the decoder 410 of Through this process, the jitter stabilization unit 307 may restore the TCP packet similarly to the packet output by the encoder 210 and output it to the decoder 410 .

As such, the quality of experience (QoE) manager 500 continuously updates the UDP buffering time (UDP_BUF_TIME) and the UDP transmission time interval (UDP_SEND_INTERVAL_TIME) by using the information received from the state monitor 237 and the jitter stabilization unit 307 . do. Accordingly, the jitter stabilization unit 307 may perform JSF by continuously reflecting the real-time UDP packet transmission state and the wireless network environment state of the server 200 .

In addition, Figure 5 is a flow chart showing a high-quality low-latency real-time media service method according to another embodiment of the present invention. In this case, FIG. 5 is performed separately from the JSF operation of FIGS. 3 and 4 .

Referring to FIG. 5 , the status monitor 237 repeatedly measures the RTT value of each wireless network a plurality of times (eg, 30 times) for a predetermined interval (eg, 5 seconds) (S301). The state monitor 237 transmits a measurement report including the RTT value of each wireless network measured in step S301 to the quality of experience (QoE) manager 500 (S303).

The quality of experience (QoE) manager 500 calculates the standard deviation of the RTT values of each wireless network, and compares the degree to which the state change of each network is flat based on the standard deviation (S305).

The quality of experience (QoE) manager 500 gives a high priority to a network having a constant (flat) state change per second (S307). For example, if the width of the state change per second of the WiFi network is the smallest and the width of the state change per second of the 5G network is the largest, the quality of experience (QoE) manager 500 sets the network priority to WiFi network > LTE network > 5G determined by the network.

The quality of experience (QoE) manager 500 transmits the determined network prioritization information to the MPTCP processing unit 233 (S309).

The MPTCP processing unit 233 determines the amount of data to be allocated for each transmission network based on the prioritization information (S311). For example, if the divided media stream packets are ①, ②, ③, ④, ⑤, ⑥, ⑦, assign ①, ②, ③, ④ to the WiFi network, assign ⑤, ⑥ to the LTE network, and 5G ⑦ can be assigned to the network.

Meanwhile, FIG. 6 is a hardware configuration diagram of a computing device according to an embodiment of the present invention.

Referring to FIG. 6 , the server 200 , the multipath gateway 300 , the terminal 400 , and the quality of experience (QoE) manager 500 described in FIGS. 1 to 5 are computing devices operated by at least one processor. At 600 , a program including instructions described to perform the operations of the present invention may be executed.

Hardware of the computing device 600 may include at least one processor 601 , a memory 603 , a storage 605 , and a communication interface 607 , and may be connected through a bus 609 . In addition, hardware such as an input device and an output device may be included.

The computing device 600 may be loaded with various software including an operating system capable of driving a program.

The processor 601 is a device for controlling the operation of the computing device 600 , and may be various types of processors that process instructions included in a program, for example, a central processing unit (CPU), a micro processor unit (MPU) ), microcontroller unit (MCU), graphic processing unit (GPU), and the like. The memory 603 may load a corresponding program so that instructions described to execute the operations of the present invention are processed by the processor 601 . The memory 603 may be, for example, read only memory (ROM), random access memory (RAM), or the like. The storage 605 may store various data, programs, etc. required for executing the operation of the present invention. The communication interface 607 may be a wired/wireless communication module.

The embodiment of the present invention described above is not implemented only through the apparatus and method, and may be implemented through a program for realizing a function corresponding to the configuration of the embodiment of the present invention or a recording medium in which the program is recorded.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improved forms of the present invention are also provided by those skilled in the art using the basic concept of the present invention as defined in the following claims. is within the scope of the right.

Claims (15)

A method of operating a device for managing quality of experience (QoE), comprising:
Collecting transmission information of the packet and status information for each of the plurality of wireless networks from a multi-path client that divides real-time media stream packets and transmits them through a plurality of wireless networks,
Calculating a buffering time and a transmission time interval for a packet in which jitter is changed using the transmission information and the state information of the packet, and
and transmitting the calculated buffering time and transmission time interval to a multi-path gateway that merges media stream packets received from the multi-path client through the plurality of wireless networks and transmits them to a terminal,
The buffering time and the transmission time interval are
used to restore the jitter-changed packet to the original packet transmitted from the multi-path client. In claim 1,
After the step of transmitting,
Recalculating the buffering time and the transmission time interval using the queue length change information of the jitter buffer received from the multi-path gateway, and transmitting the recalculated buffering time and transmission time interval to the multi-path gateway. including,
The jitter buffer is
After buffering the jitter-changed packet for the buffering time, outputting the jitter-changed packet in units of the transmission time interval. In claim 2,
The packet is
A User Datagram Protocol (UDP) packet using a Constant Bit Rate (CBR), a method of operation. In claim 2,
The packet transmission information is
An operating method comprising the number of packets per unit time of the packets transmitted in a predetermined time unit from the multi-path client and an average size of the packets. In claim 2,
The status information is
Including information measured by the multi-path client for each of the plurality of wireless networks,
The measured information is
A method of operation, including Round Trip Time (RTT) and Bandwidth (BW). In claim 2,
The transmission time interval is
It is calculated using the packet transmission interval included in the packet transmission information and the maximum value of the number of packets per second that can be transmitted in each of the plurality of wireless networks,
The maximum value of the number of packets per second is,
The method is calculated by using an average value of Round Trip Time (RTT) measured in each of the plurality of wireless networks, an average size of a transport packet included in the packet transmission information, and a bandwidth of each of the plurality of wireless networks. In claim 6,
The buffering time is
calculated using the maximum buffering time, the packet transmission interval and the transmission time interval,
The maximum buffering time is,
A preset value, the method of operation. In claim 1,
After the collecting step,
calculating the standard deviation of RTT (Round Trip Time) values measured in each of the plurality of wireless networks, determining priorities in the order in which the wireless network state changes are least, and transmitting the prioritization information to the multi-path client further comprising,
The prioritization information is
used when the multi-path client transmits the packet through a plurality of networks. A method of operating a multi-path gateway, comprising:
Receiving a buffering time and a transmission time interval for restoring the jitter-changed packet to the original packet transmitted from the multi-path client from the device managing Quality of Experience (QoE);
receiving the segmented real-time media stream packets from the multi-path client through a plurality of wireless networks and merging them into one packet; and
Buffering the merged packets as much as the buffering time in the jitter buffer and transmitting the merged packets to the terminal at the transmission time interval
comprising, a method of operation. In claim 9,
After the step of transmitting,
Transmitting the queue length change information of the jitter buffer to the apparatus for managing the quality of experience;
Receiving the buffering time and transmission time interval recalculated by reflecting the queue length change information from the apparatus for managing the quality of experience, and
Buffering the merged packets by the recalculated buffering time in the jitter buffer, and then transmitting the merged packets to the terminal at the recalculated transmission time interval
Further comprising, the method of operation. In claim 9,
The merging step is
Merging the divided real-time media stream packets into one Transmission Control Protocol (TCP) packet,
Between the step of merging and the step of transmitting,
The method further comprises the step of removing a multiple merge tunneling header and a VPN (Virtual Private Network) tunneling header from the one TCP packet and restoring it to a User Datagram Protocol (UDP) packet,
The transmitting step is
After buffering the UDP packet by the buffering time in the jitter buffer, transmitting the UDP packet to the terminal at the transmission time interval. In claim 11,
The UDP packet is
It is transmitted in a predetermined time unit using a constant bit rate (CBR),
The transmission time interval is
It is calculated using the number of packets per unit time of the UDP packets measured by the multi-path client and the maximum number of packets per unit time that can be transmitted in each of the plurality of wireless networks,
The buffering time is
It is calculated using the number of packets per unit time of the UDP packet and the transmission time interval. communication interfaces respectively connected with multi-path clients and multi-path gateways;
memory to store the program, and
at least one processor executing the program;
The multi-path client,
Real-time media stream packets using UDP (User Datagram Protocol) are divided and transmitted through a plurality of wireless networks, respectively.
The multi-path gateway,
Receives the divided real-time media stream packets, merges them into one packet, and transmits them to the terminal,
The processor is
collecting transmission information of the real-time media stream packet and status information for each of the plurality of wireless networks from the multi-path client;
calculating a jitter stabilization value using the packet transmission information and the state information; and
transmitting the jitter stabilization value to the multi-path gateway;
The jitter stabilization value is
The multi-path gateway is used to restore jitter-changed packets received through the plurality of wireless networks into original packets transmitted by the multi-path clients. In claim 13,
The jitter stabilization value is
buffering time and transmission time interval;
The buffering time and the transmission time interval are
After buffering the one packet in the jitter buffer of the multi-path gateway, it is used to transmit to the terminal in a predetermined time unit, the apparatus for managing quality of experience. 15. In claim 14,
After the step of transmitting,
Receiving the queue length change information of the jitter buffer from the multi-path gateway, and
Calculating the buffering time and the transmission time interval by reflecting the queue length change information, and transmitting the buffering time and the transmission time interval to the multi-path gateway
Further comprising, a quality of experience management device.

Images