KR102491247B1 - Method and Apparatus for Controlling User's QoE in Real-time Multimedia Service - Google Patents

Abstract

The present invention relates to a method for controlling real-time multimedia service quality in the middle of a network and an apparatus for the same, wherein a terminal apparatus receives and reproduces packets of contents suitable for a current service environment in a streaming manner from a contents providing apparatus, and the computing capability of the terminal apparatus By measuring the network transmission capability between the terminal device and the content providing device, requesting various operations such as generating auxiliary information, generating FEC packets, filtering, etc. to the content providing device according to the measurement result, and performing various operations according to the request, Content can be provided to the user without a temporary interruption caused by a change in the service environment.

Description

Method and Apparatus for Controlling User's QoE in Real-time Multimedia Service on Network

The present invention relates to a method for controlling quality of service on a network and an apparatus therefor, and more particularly, by performing various operations such as creating auxiliary information based on a real-time streaming service in the middle of a network, generating an FEC packet, or filtering, It relates to a method and apparatus capable of efficiently changing the resolution and quality of content and providing content seamlessly.

The contents described in this part merely provide background information on the present embodiment and do not constitute prior art.

A real-time multimedia service is one of multimedia data transmission methods, and refers to a method of reproducing or consuming audio and video content. These real-time multimedia services can be divided into interactive services, interactive services, and streaming services according to the type of service. Among them, the streaming service does not download all of the large-capacity multimedia data, and when only some of the data to be played is received, real-time can be played with

Due to such a streaming service, real-time performance can be further guaranteed in a live broadcasting service such as live broadcasting or live broadcasting, compared to a conventional download type.

However, consumers who consume content want to receive high-definition content stably and without interruption, as in the case of downloading content from a streaming service and playing it in the form of VoD. The available bit rate of the network for transmitting must be sufficient, and if the available bit rate is not sufficient, there is a limit to providing high-definition content without interruption.

This results in loss of transport packets due to insufficient available bit rate or congestion, resulting in deterioration of service contents.

In addition, when the available bit rate of the network is low enough to transmit high-quality content but high enough to transmit low-quality content, or when the available bit rate fluctuates greatly depending on the network situation, the content providing device provides low-quality content to provide stable service. It is transmitted to the user, which causes a waste of bandwidth because the available bit rate cannot be fully used.

In addition, when the communication network that connects consumers consuming content and the content providing device is disconnected and the service cannot be stably provided, or when the traffic for a specific service increases rapidly, the load rate of the content providing device increases and the load rate increases. There is a case where the speed decreases due to the increase in .

Therefore, there is a demand for methods capable of improving the user's QoE by adaptively responding to these various problems in the middle of the network.

Korean Patent Publication No. 10-2016-0005663 (Name: MMT device and method for low-latency service, publication date: 2016.01.15)

The present invention has been proposed to satisfy the above-mentioned needs, and uses the maximum available bit rate of the network by using selective additional transmission of auxiliary information and basic packet control according to priority, or multipath when connection fails during service. When the server load rate and load rate increase due to a rapid increase in traffic of a specific service and the server speed decreases, various methods such as a function to increase efficiency through server distributed processing, that is, load balancing, are provided. It is intended to provide a user QoE increasing method and apparatus capable of improving service quality by using the.

In particular, the terminal device requests transmission of auxiliary information for increasing the resolution of the content according to the terminal device's computing capability and the network transmission capability between the terminal device and the content providing device, and uses the auxiliary information provided according to the request to provide current service. An object of the present invention is to provide a method and apparatus for increasing the resolution of content that can provide high-resolution content to a user while improving the resolution of the content being played and without interruption in receiving the content.

In addition, a content providing device named in the present invention may be an entity existing in the middle of a network as well as a main server initially holding content composed of basic packets. This entity could be an MMT MANE, it could be a cloud, a CDN, etc.

To achieve the above object, a method for controlling the quality of a real-time multimedia service on a network according to an embodiment of the present invention is a method for controlling the quality of a multimedia service in a terminal device that receives and plays content in a streaming manner from a content providing device. The method includes: receiving a basic packet of the content from the content providing device in a streaming manner; measuring packet processing capability in a streaming service environment in which content is received and reproduced from the content providing device; and the measured packet processing capability If the predetermined condition is satisfied, transmitting a feedback message including detailed request information for controlling service quality to the content providing device, receiving a changed basic packet from the content providing device according to the detailed request information, or The method may include receiving auxiliary information together with the basic packet and reproducing the content based on the received changed basic packet or basic packet and auxiliary information.

At this time, the detailed request information includes a request to transmit auxiliary information for increasing the resolution of the content or increasing the viewpoint of the content, a request for non-transmission of certain data requesting transmission except for a certain data area of the multimedia content, and content Among one or more frame types constituting , a filtering request requesting filtering and transmission of at least one frame type, a function stop request requesting stopping a specific function already being performed, and an error detection packet transmission request for correcting a bit error It may include at least one, and the reproducing may, when auxiliary information is received according to the requested detailed information, combine the auxiliary information with the basic packet to increase the resolution of the content or increase the time of the content. The transmitting may request transmission of at least one of the first auxiliary information for increasing the spatial resolution and the second auxiliary information for increasing the temporal resolution in consideration of the measured packet processing capability. A feedback message containing request details may be sent.

In addition, the reproducing step may include, when the first auxiliary information is received, reproducing by increasing the spatial resolution of the content through interpolation using the first auxiliary information, and when receiving the second auxiliary information, and reproducing a temporal resolution of the content by temporally combining a basic packet of the content and second auxiliary information, and if the measured packet processing capability does not satisfy a preset condition, the reception The method may further include reproducing content based on one basic packet, and the feedback message may further include at least one of a packet ID, request execution information, and request execution start time information.

In addition, the packet is composed of at least one Media Fragment Unit (MFU), which is the minimum decoding unit included in the payload of the packet, or at least one Media Processing Unit (MPU) including the at least one MFU. Media Transport) packet, and if the measured packet processing capability does not satisfy a preset condition after the reproducing step, the step of requesting to stop transmission of the packet according to the detailed request information may be further included, The packet processing capability may include at least one of an arithmetic capability of the terminal device and a network transmission capability between the terminal device and the content providing device.

In addition, the requesting step may include a first condition in which a difference between the measured computing power and the computing power required to process the basic packet is greater than the computing power required to process the auxiliary information and the measured network transmission capability and basic content When at least one of the second conditions, which is a case in which a difference in network transmission capability required to transmit is greater than a network transmission capability required to transmit the auxiliary information, is satisfied, the transmission of the auxiliary information may be requested.

On the other hand, the above-described real-time multimedia service quality control method may be provided as a computer-readable recording medium on which a program for executing the above-described method is recorded, and is implemented to execute the above-described method and stored in a computer-readable recording medium. It may also be provided as a stored computer program.

A terminal device according to the present invention for achieving the above object is a terminal device that controls the service quality of the content while receiving and playing content through a streaming service. By measuring packet processing capability in a streaming service environment in which a basic packet of the content is received through the communication module and the content is received and reproduced from the content providing device in a streaming manner from the communication module and the content providing device, When the packet processing capability satisfies a preset condition, a feedback message including detailed request information for controlling service quality is transmitted to the content providing device, and when a packet of content according to the requested detailed information is received, the request A control module for controlling reproduction of the content based on detailed information may be included.

According to the present invention, while a terminal device receives and reproduces packets of content corresponding to the current service environment in a streaming manner from a content providing device, measuring the computing capability of the terminal device and the network transmission capability between the terminal device and the content providing device, According to the measurement result, the content providing device requests various operations such as auxiliary information generation, FEC packet generation, and filtering, and performs various operations according to the request, thereby providing the user with content without temporary interruption due to changes in the service environment. can provide

In addition, various effects other than the above effects may be disclosed directly or implicitly in detailed descriptions according to embodiments of the present invention to be described later.

1 is a diagram for explaining the configuration of a system implementing a real-time multimedia service control method according to an embodiment of the present invention.
2 is a diagram for explaining an MMT packet structure according to an embodiment of the present invention.
3 is a block diagram showing the main configuration of a terminal device according to the present invention.
4 to 6 are flowcharts for explaining the operation process of a terminal device according to an embodiment of the present invention.
7 is a block diagram showing the main configuration of the content providing device according to the present invention.
8 is a flowchart illustrating an operation process of a content providing device according to an embodiment of the present invention.

In order to clarify the characteristics and advantages of the problem solving means of the present invention, the present invention will be described in more detail with reference to specific embodiments of the present invention shown in the accompanying drawings.

However, detailed descriptions of well-known functions or configurations that may obscure the gist of the present invention will be omitted in the following description and accompanying drawings. In addition, it should be noted that the same components are indicated by the same reference numerals throughout the drawings as much as possible.

The terms or words used in the following description and drawings should not be construed as being limited to a common or dictionary meaning, and the inventor may appropriately define the concept of terms for explaining his/her invention in the best way. It should be interpreted as a meaning and concept consistent with the technical idea of the present invention based on the principle that there is. Therefore, the embodiments described in this specification and the configurations shown in the drawings are only one of the most preferred embodiments of the present invention, and do not represent all of the technical ideas of the present invention. It should be understood that there may be equivalents and variations.

In addition, terms including ordinal numbers, such as first and second, are used to describe various components, and are used only for the purpose of distinguishing one component from other components, and to limit the components. Not used. For example, a second element may be termed a first element, and similarly, a first element may be termed a second element, without departing from the scope of the present invention.

In addition, terms used in this specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In addition, terms such as "include" or "having" described in this specification are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or the other It should be understood that the above does not preclude the possibility of the presence or addition of other features, numbers, steps, operations, components, parts, or combinations thereof.

In addition, terms such as “unit”, “unit”, and “module” described in the specification refer to a unit that processes at least one function or operation, and may be implemented as hardware or software or a combination of hardware and software. Also, "a or an", "one", "the" and similar related words in the context of describing the invention (particularly in the context of the claims below) Unless indicated or otherwise clearly contradicted by context, both the singular and the plural can be used.

In addition to the above-mentioned terms, specific terms used in the following description are provided to aid understanding of the present invention, and the use of these specific terms may be changed in other forms without departing from the technical spirit of the present invention.

In addition, embodiments within the scope of the present invention include computer-readable media having or conveying computer-executable instructions or data structures stored thereon. Such computer readable media can be any available media that can be accessed by a general purpose or special purpose computer system. By way of example, such computer readable media may be in the form of RAM, ROM, EPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage, or computer executable instructions, computer readable instructions or data structures. physical storage media such as, but not limited to, any other medium that can be used to store or convey any program code means in a computer system and which can be accessed by a general purpose or special purpose computer system. .

In the following description and claims, the technical background for implementing the packet transmission method according to the present invention will be described on the assumption that MMT (MPEG Media Transport) is used.

That is, the content providing device 300 described in the embodiment of the present invention may be a MANE (MMT Aware Network Entity) of MMTP (MPEG Media Transport Protocol), and the packet described in the embodiment of the present invention is an MMT packet. It can be.

Meanwhile, a detailed description of the structure of such an MMT packet will be described later.

However, this is only an example for convenience to help understanding of the present invention, and the present invention is not necessarily limited to the MMT technology.

Then, a real-time multimedia service quality control method according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram showing the configuration of a system for controlling service quality in a streaming service, that is, a real-time multimedia service, according to an embodiment of the present invention.

Referring to FIG. 1 , a system for controlling service quality in a real-time multimedia service according to an embodiment of the present invention may include a terminal device 200 and a content providing device 300, and the terminal device 200 ) and the content providing device 300 are linked through the communication network 100.

Then, with reference to FIG. 1, each component will be schematically described.

First, the communication network 100 serves to transmit data for data transmission and reception between the terminal device 200 and the content providing server 300, and depending on the system implementation method, Ethernet, xDSL (ADSL, VDSL), HFC (Hybrid Fiber Coaxial Cable), FTTC (Fiber to The Curb), FTTH (Fiber To The Home), etc. Wireless communication methods such as Wimax, High Speed Downlink Packet Access (HSDPA), Long Term Evolution (LTE), and Long Term Evolution Advanced (LTE-A) may be used. In addition to the above communication methods, other widely known or future It can include all types of communication methods to be developed.

The terminal device 200 refers to a user's device capable of transmitting and receiving various types of data with the content providing device 300 through the communication network 100 according to a user's manipulation.

The terminal device 200 can perform voice or data communication through the communication network 100, and includes a memory for storing programs and protocols for transmitting and receiving and processing data, and a microprocessor for calculating and controlling various programs by executing them. etc. can be provided. In addition, the terminal device 200 of the present invention may be implemented in various forms. For example, the terminal device 200 described in this specification is a mobile terminal such as a smart phone, a tablet PC, a personal digital assistant (PDA), a portable multimedia player (PMP), and an MP3 player. Of course, a fixed terminal such as a smart TV, a laptop computer, or a desktop computer may be used, and any device capable of transmitting and receiving data according to the present invention may be used in an embodiment of the present invention. It can be used as a terminal device 200 according to the

Meanwhile, the terminal device 200 according to the present invention may request content transmission to the content providing device 300 according to a user's input and receive the content from the content providing device 300 in a streaming manner.

In this case, the content may be received in units of packets, and the packets may be MMT packets.

Meanwhile, the terminal device 200 may receive a packet of content data requested by the user from the content providing device 300 according to the user's input.

At this time, the service experienced by the user in a streaming service environment in which content is received and played from the content providing device 300, such as the computing capability of the terminal device 200 or the network transmission capability between the terminal device 200 and the content providing device 300. In order to increase quality, that is, QoE, there is a need to adjust packets received from the content providing device 300 .

As one representative example, when packet processing capability is difficult to receive high-resolution content as it is, that is, when a number of temporary interruptions occur to receive high-resolution content, low-resolution content is received from the content providing device 300 , To provide content to the user without interruption, in the case of transmitting high-resolution content because network transmission capability or packet processing capability such as computing capability of the terminal device 200 improves again while transmitting low-resolution content, temporary A disconnection phenomenon occurs, and when low-resolution content is continuously transmitted in order not to cause a disconnection phenomenon, a waste of network transmission capacity occurs and the user cannot receive high-resolution content, and the user's QoE (Quality of Experience) is declining.

To explain this in more detail, in an environment where it is difficult to transmit content with HD resolution, the content providing device 300 transmits content with SD resolution to the terminal device 200. In the case of improvement, when HD-level content is transmitted, a temporary disconnection occurs, and when content with SD-level resolution is continuously transmitted, network transmission capacity is wasted and the user cannot receive HD-level content. is to do

In order to solve this problem, the terminal device 200 of the present invention measures the packet processing capability including at least one of the computing capability and the network transmission capability of the terminal device 200, and the measured result satisfies a preset condition. If satisfied, a feedback message including detailed information requested for controlling the QoS is transmitted to the content providing device 300 to control the QoS.

In the above example, when a content providing device 300 is requested to transmit auxiliary information and the auxiliary information according to the request is received, the content resolution can be increased using the auxiliary information.

That is, when the terminal device 200 receiving SD-level content and providing it to the user recognizes that the transmission environment has improved, it requests and receives auxiliary information transmission from the content providing device 300, and uses the auxiliary information to transmit the SD-level content. It is to increase the resolution of content to HD level content.

Through this method, the terminal device 200 can provide HD-level content to the user while solving the disconnection phenomenon that occurs when the previously received content of SD-level resolution is continuously received and replaced with HD-level content. By doing so, it is possible to increase the user's QoE for the corresponding content.

If the measured result does not satisfy the preset condition, the request is stopped through the information in the feedback message including detailed request information such as a request for transmission of auxiliary information, and the low-resolution content (SD-class resolution in the above example) content) to the user.

A detailed method for increasing the content resolution of the terminal device 200 will be described later.

Meanwhile, the packet processing capability including at least one of the computing capability of the terminal device 200 and the network transmission capability between the terminal device 200 and the content providing device 300 receives and reproduces content from the content providing device 300. It may refer to packet processing capability in a streaming service environment.

That is, it means the packet processing capability in the entire process from the terminal device 200 requesting the content, receiving the content in packet units, and providing it to the user. ) of computing power and network transmission capability between the terminal device 200 and the content providing device 300 may be included.

Finally, the content providing device 300 is used to transmit content data such as videos stored in the content providing device 300 or received from an external device (not shown) to the terminal device 200 through the communication network 100. It is a device.

In the present invention, the content providing device 300, when the terminal device 200 transmits a feedback message including detailed request information such as a request for transmission of auxiliary information, a packet of content according to the requested detailed information (for example, detailed request information) Auxiliary information according to) is generated and transmitted to the terminal device 200.

A detailed description thereof will also be described later.

Meanwhile, the content providing device 300 transmits content data in packet units to the terminal device 200. In an embodiment of the present invention, the packet may be an MMT packet as described above.

Therefore, prior to a detailed description of an embodiment of the present invention, the structure of an MMT packet transmitted from the content providing device 300 to the terminal device 200 will be described with reference to FIG. 2 .

Referring to FIG. 2 , the MMT package 10 is the largest logically structured MMT entity among the data models defined in the MMT functional domain, and means one independent content such as a movie. The content providing device 300 that provides a streaming content providing service such as a video may generate such an MMT package and provide it to the terminal device 200. The MMT package 10 is logically structured consisting of a collection of various entities. can be a collection of Such an MMT package 10 may include one or more assets 20, presentation information 11, transport characteristics 13, and descriptive information, for example, package information.

Among them, the presentation information (Presentation Information, 11) includes information on the relationship between the plurality of assets 20, that is, spatial and temporal relationships, and one content includes a plurality of MMT packages. When made of , information for indicating a relationship between a plurality of MMT packages may be further included. The presentation information 11 may be a combination of hypertext markup language (HTML) and composition information documents as an example of the presentation information 11 .

Transport characteristics (ADC; Asset Delivery Characteristics, 13) represent QoS (Quality of Service) requirements and statistics for delivery of the asset 20. A plurality of assets 20 may be related to one transport characteristic 13 . At this time, the transport characteristics (transport characteristics) 13 may be used to set parameters of the MMT payload and MMT protocol by an entity that encapsulates the package for effective delivery of the asset 20 . In addition, the transport characteristics 13 may include delivery characteristic information required to determine delivery conditions of the MMT asset 20 or the MMT packet, for example, traffic description parameters and QoS. A descriptor (QoS descriptor) may be included. This transfer characteristic information can be expressed in the form of parameters irrelevant to the communication network environment.

The asset 20 refers to an element of a package that encapsulates encoded media data such as audio, video, or web page data having timed or non-timed properties, and refers to the largest logical data unit. These assets 20 refer to data entities that include data with the same transfer characteristics. For example, if the MMT package 10 is the largest entity among the data models defined in the MMT functional area and means one independent content such as a movie, the asset 20 is audio data, video data, and script data. It means a data unit constituting the MMT package 10 such as These assets 20 logically group MPUs sharing the same asset ID.

In addition, the MMT packet of the present invention may actually include an MMT packet header and an MMT payload.

Then, the method for controlling the real-time multimedia service quality according to the present invention and the configuration of the apparatus according to the method will be reviewed through FIGS. 3 to 8.

Hereinafter, the configuration and operation process of the terminal device 200 according to an embodiment of the present invention will be described.

3 to 6 are diagrams for explaining the configuration and operation process of the terminal device 200 according to the present invention, specifically, FIG. 3 is a block diagram showing the configuration of the terminal device 200, and FIG. 4 is the terminal device 200 is a flowchart for explaining a method for controlling the quality of service of a real-time multimedia service, and FIG. 5 describes an embodiment of the auxiliary information transmission request message syntax transmitted from the terminal device 200 to the content providing device 300. It is a drawing for

6 is a flowchart for explaining an embodiment in which the terminal device 200 requests generation of auxiliary information for increasing the resolution of content.

Referring to FIG. 3 , a terminal device 200 may include a communication module 210 , a storage module 230 and a control module 250 .

The communication module 210 is for transmitting and receiving data to and from the content providing device 300 through the communication network 100, requests the provision of content according to the user's input through the communication module 210, and provides the content according to the request. In addition, it is possible to transmit an auxiliary information transmission request message and receive auxiliary information accordingly.

The storage module 230 may store content or auxiliary information received through the communication module 210 .

Meanwhile, although not shown in detail in FIG. 3 , the storage module 230 may include a buffer (not shown) for temporarily storing packets such as the received content or auxiliary information for a certain period of time.

The control module 250 is a device that controls the overall operation process of the terminal device 200 according to the present invention.

Looking at the overall operation process of the terminal device 200 controlled by the control module 250 through FIG. 4, the terminal device 200 is a packet (hereinafter referred to as .

That is, the basic packet may be a packet with a fixed resolution or a packet with a resolution adaptively set before starting a service.

And it is determined whether the measured packet processing capability satisfies a preset condition (S3), and if the set condition is satisfied, a feedback message including request detailed information is transmitted to the content providing device 300 (S5), Accordingly, the changed basic packet may be received, or additional information may be received from the content providing device 300 together with the basic packet (S7), and content may be reproduced based on the received packets based on the requested detailed information. (S9).

On the other hand, the predetermined condition is to determine whether the packet processing capability in the service environment is above a certain level, and may be the computing capability or network transmission capability (eg, network resource) of the terminal device 200, etc. , As an example of the above condition, the difference between the current computing capability of the terminal device 200 and the computing capability required for the terminal device 200 to receive and process the basic packet is changed to process the basic packet or the basic packet and auxiliary information together. It is possible to determine whether a predetermined condition is satisfied by determining whether the computing power is greater than the required computing power, and in another embodiment, a basic packet in which the difference between the current network transmission capacity and the network transmission capacity required to transmit the basic packet is changed is transmitted, or a basic packet and a basic packet are transmitted. It is possible to determine whether a predetermined condition is satisfied by determining whether the transmission capability of the network is greater than the network transmission capability required to process the auxiliary information together.

Of course, it may be determined that a preset condition is satisfied only when both of the above two embodiments, that is, an embodiment relating to the computing capability and the network transmission capability of the terminal device 200 are satisfied.

When the reception of the content requested by the terminal device 200 is completed (S11), the corresponding operation is terminated.

On the other hand, if the packet processing capability measured in step S1 does not satisfy the preset condition, the feedback message is not transmitted, and the content packet that can be provided to the user in the current service environment, that is, the basic packet, is received and reproduced as it is. will do

Meanwhile, a feedback message transmitted from the terminal device 200 to the content providing device 300 to control the quality of service may include detailed request information, and the detailed request information may include the terminal device 200 for controlling the quality of service. ) is included, which will be described in detail with reference to FIG. 5 .

Referring to FIG. 5, the above feedback message may be named VNF_Request_Message, and this VNF_Request_Message may be included in an MMT Cross Layer Interface (CLI).

Meanwhile, the syntax of this feedback message may include message_id information consisting of 16 bits and version information consisting of 8 bits. The message_id and version information may be the same as the NAM Feedback message included in the MMT CLI.

Packet ID (packet_id) information is an identifier for distinguishing a packet to be transmitted, and can be expressed using a 16-bit packet_id or a 7-bit flow_label in MMTP (MMT Protocol).

The requested detailed information (VNF_number) is a field including detailed information about content requested by the terminal device 200 to the content providing device 300, and may consist of 7 bits.

As for the detailed information that the requested detailed information may include, for example, when the value is '001', it may be to request transmission of auxiliary information for increasing spatial resolution (hereinafter referred to as 'first auxiliary information'). .

At this time, the content providing device 300 that has received the request message containing the detailed request information in which the value of '001' is recorded includes information on the boundary of video content in accordance with the first auxiliary information transmission request. After generating 1 auxiliary information and receiving the first auxiliary information, the terminal device 200 may provide HD video to the user by interpolating the original SD video content.

In addition, when the requested detailed information value is '010', it may be a value requesting transmission of auxiliary information (second auxiliary information) for temporal resolution enhancement.

The content providing device 300 receiving the request message including the detailed request information of the '010' value includes information on the motion vector of video content and the size and ratio of blocks in accordance with the second auxiliary information transmission request. After generating the second auxiliary information, the terminal device 200 that has received the second auxiliary information may combine the second auxiliary information and original video content packets to provide content having an increased resolution to the user.

If the requested detailed information value is '011', transmission can be requested except for a certain data area of the corresponding miltermedia service.

For example, video content for providing a video multimedia service may be divided into a video data part, an audio data part, a subtitle part, etc., and request transmission except for the subtitle part or transmission except for the audio data part may be requested. There is.

On the other hand, if the value is '100', it is possible to request transmission of I frames and P frames by filtering B frames among packets of content, and if it is '101', filtering P frames among packets of contents to request transmission of I frames. and B frames may be requested to be transmitted.

That is, the frames constituting the multimedia content packet may be divided into one or more frame types, filtering a specific frame type among them and requesting transmission of only the remaining frame types.

However, the value of the above-described detailed request information and request information according to each value may be set differently depending on the case.

Specifically, detailed information requesting to stop a specific function already being performed may be included, and an error detection packet transmission request for a Forward Error Correction (FEC) request for correcting a bit error may be included.

For example, in general, an FEC packet for bit error detection is inserted between content packets to detect a bit error through the FEC packet. If the packet processing capability is less than a certain value, the FEC packet is not inserted, that is, By receiving only pure content packets excluding FEC packets and providing content, it is possible to reduce the load of the terminal device.

At this time, if the packet processing capability is continuously measured and the measured result satisfies a preset condition, that is, if the packet processing capability is improved, it is possible to request transmission of the FEC packet again.

As another example, a value requesting transmission of auxiliary information (hereinafter referred to as 'third auxiliary information') for increasing a viewpoint may be additionally set.

That is, it is possible to request transmission of auxiliary information for providing content provided on a 2D screen on a 3D screen.

The request performance information (VNF_mode) is a field value indicating whether the terminal device 200 performs a request according to the detailed request information or suspends the execution of the request, and may consist of 1 bit.

That is, if the above value is 1, a request according to request detailed information may be performed, and if the above value is 0, a request to stop performing the request may be made.

On the other hand, if it is possible to know whether the request has been performed or stopped based on the set value of the detailed request information, the request performance information field will not be necessary.

The request execution start time information (VNF_start_time) may indicate the start time of the request action included in the request message, and an absolute time unit may be used in MMT.

) 및 단말장치(200)와 컨텐츠 제공 장치(300) 간의 네트워크 전송 능력(

)을 포함하는 패킷 처리 능력을 측정한다(S101).In order to help a detailed understanding of the multimedia service quality control method by the terminal device 200 according to the present invention, with reference to FIG. 6, an implementation of adjusting the resolution of content by requesting transmission of auxiliary information from the content providing device 300. Looking at an example, while the terminal device 200 continuously receives packets for low-resolution content (hereinafter, referred to as 'basic packets') from the content providing device 300, the terminal device 200 has an arithmetic capability (

) and network transmission capability between the terminal device 200 and the content providing device 300 (

) is measured (S101).

)의 차이가 컨텐츠 제공 장치(300)로부터 수신할 보조 정보를 처리하는 데 필요한 연산능력(

)보다 큰지, 즉,

-

>

인지를 판단하고(이하, 제1 조건, S103), 측정된 네트워크 전송 능력과 기본 패킷으로 이루어진 컨텐츠, 즉, 저해상도의 컨텐츠를 전송하는데 필요한 네트워크 전송 능력(

)의 차이가 컨텐츠 재생 장치(300)가 보조 정보를 전송하는데 필요한 네트워크 전송 능력(

)보다 큰 지, 즉,

-

>

인지를 판단하여(이하, 제2 조건, S105), 상기 설정된 조건을 만족하지 않으면 보조 정보 전송을 요청하기 않고, 저해상도의 컨텐츠만을 지속하여 수신하여 재생하고, 상기 두 조건 중 적어도 하나의 조건을 만족하면, 컨텐츠 제공 장치(300)에 보조 정보 전송을 요청한다(S107).Thereafter, the terminal device 200 determines whether the measured packet processing capability, that is, the measured computing capability and network transmission capability satisfy a preset condition. Specifically, looking at an embodiment of the condition, the measured computing capability And the basic packet currently being received by the terminal device 200, that is, the computing power required to process the low-resolution content (

The difference in ) is the computational power required to process the auxiliary information to be received from the content providing device 300 (

), that is,

-

>

It is determined (hereinafter, the first condition, S103), and the network transmission capability required to transmit the measured network transmission capability and the content consisting of basic packets, that is, the low-resolution content (

) is the network transmission capability required for the content playback device 300 to transmit auxiliary information (

), that is,

-

>

It is determined (hereinafter, the second condition, S105), and if the set condition is not satisfied, auxiliary information transmission is not requested, and only low-resolution content is continuously received and reproduced, and at least one of the two conditions is satisfied. If so, the content providing device 300 is requested to transmit auxiliary information (S107).

At this time, the terminal device 200 requests transmission of at least one of the first auxiliary information for spatial resolution increase and the second auxiliary information for temporal resolution increase in consideration of the measured network transmission capability and computational capability. can

That is, the terminal device 200 considers the measured network transmission capability and packet processing capability, such as computing capability, to temporally increase the resolution of the content (for example, from 30 fps to 60 fps) to increase the motion vector of the content. In order to request the content providing device 300 to transmit auxiliary information including information about, or to spatially increase the resolution of content (eg, SD-level video to HD-level video), the boundary of the video A request may be made to the content providing device 300 to transmit auxiliary information including information about the part.

Also, the requested auxiliary information may be auxiliary information (hereinafter referred to as third auxiliary information) that increases a viewpoint.

That is, if the content currently provided from the content providing device 300 is binocular content, a request to transmit third auxiliary information for providing the content with four more points of view by increasing two more points of view. can request

After that, when auxiliary information is received from the content providing device 300 (S109), using the received auxiliary information, the terminal device 200 increases the resolution of the content currently received and provides it to the user (S111). , If the received auxiliary information is first auxiliary information, the spatial resolution of the content is increased through interpolation using the first auxiliary information, and if the received auxiliary information is second auxiliary information, the packet of the content being received Temporal resolution of the content may be increased by temporally combining the P and the second auxiliary information to generate a new packet.

Thereafter, when reception of the content requested by the user is completed, or when the user inputs a request to stop content transmission, that is, when content playback is stopped, the above process is terminated and content reception is not completed without the user's request to stop content transmission. If not, return to step S101 and repeat the entire process (S113). If the condition of step S103 or step S105 is not satisfied while repeating the above process, the terminal device 200 is instructed to stop transmitting auxiliary information. A request is made to the content providing device 300 (S115).

Meanwhile, although not shown in FIG. 3, the terminal device 200 according to the present invention uses an input module (not shown) for receiving a user's request, content or auxiliary information received from the content transmission device 300, and resolution. It may include an output module (not shown) for reproducing content or the like with increased .

In the above, the configuration and operation process of the terminal device 200 according to the embodiment of the present invention have been reviewed.

Hereinafter, the configuration and operation process of the content providing device 300 according to an embodiment of the present invention will be described.

7 is a block diagram showing the configuration of the content providing device 300, and FIG. 8 is a flowchart showing the operation process of the content providing device 300.

Referring to FIG. 7 , the content providing device 300 may include a communication module 310 , a storage module 330 and a control module 350 .

The communication module 310 is for transmitting and receiving data with the terminal device 200 through the communication network 100, receives an auxiliary information transmission request message and a content transmission request from the terminal device 200, and the terminal device according to the request. Auxiliary information and content may be transmitted to (200).

The storage module 330 may store data of various contents to be transmitted to the terminal device 200 through the communication module 310 .

The control module 350 is a device for controlling the overall operation process of the content providing device 300 according to the present invention, and may include an auxiliary information generating module 351, the auxiliary information generating module 351 is a terminal device When 200 requests generation of auxiliary information through a feedback message, auxiliary information is generated according to the request.

When receiving a feedback message from the terminal device 200 through the communication module 310, the control module 350 checks the detailed request information included in the feedback message, and the content packet or frame according to the detailed request information. It can be controlled to modify the configuration of or create a new packet such as auxiliary information and transmit it to the terminal device 200.

In addition, when the terminal device 200 again requests to stop the request through the request detailed information, the generation of auxiliary information is stopped accordingly or the configuration of the content packet or frame modified according to the requested detailed information is returned to the original, It can be controlled to be transmitted to the device 200.

In order to help understand the operation process of the content providing device 300, an embodiment in which auxiliary information is generated and transmitted to the terminal device 200 to adjust the content resolution will be described in detail with reference to FIG. 7. The providing device 300 receives an auxiliary information transmission request from the terminal device 200 while transmitting a basic packet, that is, content having a low resolution, to the terminal device 200 (S201), and assists in response to the auxiliary information transmission request. Information is generated (S203).

In this case, according to the value of the auxiliary information transmission request message, at least one of first auxiliary information for increasing spatial resolution, second auxiliary information for increasing temporal resolution, and third auxiliary information for increasing the view of content may be generated. and, when generating the first auxiliary information, it may be generated by including information about the boundary part, such as the position of the boundary in the content currently being transmitted, that is, the position of the boundary in the basic content, the nature of the boundary, etc., and generating the second auxiliary information In this case, it can be generated by including motion information including motion vector information and block size and ratio information in the content currently being transmitted, that is, basic content.

The content providing device 300 that has generated auxiliary information according to the request of the terminal device 200 transmits the generated auxiliary information to the terminal device 200 (S205).

Thereafter, when the transmission of the content requested by the user is completed (S207), the above process is terminated, and if the transmission of the content is not completed, the entire process is repeated again in step S201. When a request to stop transmitting information is received (S209), transmission of auxiliary information is stopped (S211), and the content currently being transmitted is transmitted to the terminal device 200. During this process, the transmission of auxiliary information is requested again from the terminal device 200 Upon reception, it returns to step S201 and repeats the above process.

As set forth above, while this specification contains many specific implementation details, they should not be construed as limiting as to the scope of any invention or claimables, but rather as may be specific to a particular embodiment of a particular invention. It should be understood as a description of the features.

Further, while operations are depicted in the drawings in a specific order, it should not be understood that all illustrated operations must be performed or that those operations must be performed in the specific order shown or in sequential order to obtain desired results. In certain cases, multitasking and parallel processing can be advantageous. Further, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and the program components and systems described may generally be integrated together into a single software product or packaged into multiple software products. You have to understand that you can.

The present description presents the best mode of the invention and provides examples to illustrate the invention and to enable those skilled in the art to make and use the invention. The specification thus prepared does not limit the invention to the specific terms presented. Therefore, although the present invention has been described in detail with reference to the above-described examples, a person skilled in the art may make modifications, changes, and modifications to the present examples without departing from the scope of the present invention.

Therefore, the scope of the present invention should not be determined by the described embodiments, but by the claims.

The present invention relates to a real-time multimedia service quality control method and an apparatus therefor, and more particularly, by performing various operations such as creating auxiliary information based on a real-time streaming service, generating an FEC packet, or filtering, the resolution and quality of content It relates to a method and apparatus for efficiently changing and providing content without interruption.

According to the present invention, while a terminal device receives and reproduces packets of content corresponding to the current service environment in a streaming manner from a content providing device, measuring the computing capability of the terminal device and the network transmission capability between the terminal device and the content providing device, According to the measurement result, the content providing device requests various operations such as auxiliary information generation, FEC packet generation, and filtering, and performs various operations according to the request, thereby providing the user with content without temporary interruption due to changes in the service environment. can provide

Therefore, it is possible to contribute to the development of the streaming service industry through the above real-time multimedia service quality control method, and in addition, the present invention has industrial applicability because it is not only sufficiently commercially available or commercially viable, but also realistically and clearly practicable. .

100: communication network 200: terminal device 300: content providing device
210: communication module of terminal device 230: storage module of terminal device
250: control module of terminal device 310: communication module of content providing device
330: storage module of content providing device
350: control module of the content providing device
351: auxiliary information generation module

Claims (14)

A multimedia service quality control method in a terminal device that receives and reproduces content from a content providing device in a streaming manner,
receiving a basic packet of the content in a streaming manner from the content providing device;
Measuring packet processing capability in a streaming service environment in which content is received and reproduced from the content providing device;
If the measured packet processing capability satisfies a preset condition, transmitting a feedback message including detailed information requested for controlling service quality to the content providing device;
further receiving auxiliary information related to QoS control according to the requested detailed information in a process of continuously receiving the basic packets from the content providing device; and
reproducing the content based on the received basic packet and auxiliary information;
Real-time multimedia service quality control method comprising a. The method of claim 1, wherein the requested detailed information
Auxiliary information transmission request for increasing the resolution of the content or increasing the viewpoint of the content, a certain data ratio transmission request requesting transmission except for a certain data area of the multimedia content, and one or more frame types constituting the content and at least one of a filtering request for filtering and transmitting at least one of the frame types, a function stop request for requesting to stop a specific function currently being performed, and an error detection packet transmission request for correcting a bit error. A real-time multimedia service quality control method. The method of claim 2, wherein the reproducing step
When auxiliary information is received according to the requested detailed information, the auxiliary information is combined with the basic packet to increase the resolution of the content or increase the viewing time of the content to reproduce the real-time multimedia service quality control method. The method of claim 3, wherein the transmitting step
Considering the measured packet processing capability, a feedback message including request detailed information for requesting transmission of at least one auxiliary information of first auxiliary information for increasing spatial resolution and second auxiliary information for increasing temporal resolution A real-time multimedia service quality control method characterized by transmitting. 5. The method of claim 4, wherein the reproducing step
If the first auxiliary information is received, reproducing by increasing the spatial resolution of the content through interpolation using the first auxiliary information; and
If the second auxiliary information is received, temporally combining the basic packet of the content and the second auxiliary information, and reproducing the content by increasing the temporal resolution;
Real-time multimedia service quality control method comprising a. According to claim 1,
playing content based on the received basic packet when the measured packet processing capability does not satisfy a preset condition;
Real-time multimedia service quality control method further comprising a. The method of claim 1, wherein the feedback message
A method for controlling real-time multimedia service quality, further comprising at least one of a packet ID, information on whether or not to perform a request, and information on a start time of performing a request. The method of claim 1, wherein the packet
MMT (MPEG Media Transport) packet composed of at least one Media Fragment Unit (MFU), which is the minimum decoding unit included in the payload of the packet, or at least one Media Processing Unit (MPU) including the at least one MFU A real-time multimedia service quality control method, characterized in that. The method of claim 1, wherein after the reproducing step,
requesting interruption of packet transmission according to the requested detailed information when the measured packet processing capability does not satisfy a preset condition;
Real-time multimedia service quality control method further comprising a. 10. The method of claim 9, wherein the packet processing capability
The real-time multimedia service quality control method comprising at least one of an operating capability of the terminal device and a network transmission capability between the terminal device and the content providing device. 11. The method of claim 10, wherein the requesting step
Condition 1 in which the difference between the measured computing power and the computing power required to process the basic packet is greater than the computing power required to process the auxiliary information and the measured network transmission capability and network transmission capability required to transmit the basic content and requesting transmission of the auxiliary information when at least one condition of the second conditions is satisfied, in which the difference in is larger than the network transmission capability required to transmit the auxiliary information. A computer-readable recording medium on which a program for executing the method according to any one of claims 1 to 11 is recorded. A computer program implemented to execute the method according to any one of claims 1 to 11 and stored in a computer readable recording medium. In a terminal device that controls the service quality of the content while receiving and playing content through a streaming service,
A communication module interworking with a communication network to transmit and receive packets with a content providing device; and
By measuring the packet processing capability in a streaming service environment in which a basic packet of the content is received through the communication module in a streaming manner from the content providing device and the content is received and reproduced from the content providing device, the packet processing capability is If a predetermined condition is satisfied, a feedback message including detailed request information for controlling service quality is transmitted to the content providing device, and when a packet of content according to the requested detailed information is received, based on the requested detailed information Including; a control module for controlling to play the content;
The control module
In the process of continuously receiving the basic packets from the content providing device, auxiliary information related to QoS control corresponding to the requested detailed information is further received, and the QoS is adjusted by applying the auxiliary information to the basic packets. A terminal device characterized in that it controls to reproduce content.

Images