KR100918385B1 - Method and Apparatus for informing the information about the reference discardable slice in Scalable Video Coding - Google Patents

Abstract

The present invention is not used for inter-layer reference among lower layer information in scalable video encoding and decoding supporting temporal, spatial and image quality scalability. The present invention relates to a method and apparatus for performing decoding without an error when a part of a lower layer is removed and an information about the same is attempted to switch the layer to the remaining lower layer.

Video, Hierarchical B Picture, Discardable Layer

Description

Apparatus and method for notification slice removal notification in scalable video stream coding {Method and Apparatus for informing the information about the reference discardable slice in Scalable Video Coding}

The present invention is not used for inter-layer reference among lower layer information in scalable video encoding and decoding supporting temporal, spatial and image quality scalability. The present invention relates to a method and apparatus for performing decoding without an error when a part of a lower layer is removed and an information about the same is attempted to switch the layer to the remaining lower layer.

The present invention is derived from the research conducted as part of the IT new growth engine core technology development project of the Ministry of Information and Communication and the Ministry of Information and Communication Research and Development. [Task Management Number: 2005-S-103-02, Title: Ubiquitous in Tongbang Convergence Environment] Content Service Technology Development]

An extractor removes an unwanted layer from an input scalable bitstream and generates a decodeable partial bitstream. It can be located after the encoder, on a server or a node in the network, or before the decoder.

If the layer remover leaves only the information necessary to decode a target layer of one of several layers, and removes information not necessary for decoding of the target layer not only from the upper layer but also from the information of the lower layer, another layer remover or When a terminal having a decoder attempts a layer transformation that changes a lower layer into a target layer than the target layer, a phenomenon in which the image used as the reference image is already removed from the changed target layer may occur.

In order to solve this problem, in the present invention, when the encoding information of an image used as a reference image is removed between images in one layer, the decoder or the next layer remover is notified to the decoder so that the decoder layer is lower than the target layer. The error caused by the absence of the reference image when the conversion is attempted to be decoded is prevented.

As described above, the present invention can eliminate the possibility of an error that the existing method does not think, and thus improve the error-tolerant coding efficiency.

Also, in order to approach the current single-layer coding more or to adapt adaptively to the network situation, even if the information of the corresponding layer is removed by using any means, a subsequent intermediate transmitter uses It is possible to know whether or not it can be decoded, and by using this, it is possible to limit the layer transformation to the corresponding layer, thereby reducing the probability of error occurring during decoding.

In one preferred embodiment of the present invention, a reference slice removal notification device includes a receiving unit for receiving a scalable video stream including reference information which is information associated with a discardable layer; A removal unit for removing a discardable layer based on the reference information when a network condition is worse than a first reference value; And a reference information updater for updating the reference information by adding information on which discardable layer is removed for each of the base layer and at least one higher layer of the scalable video stream to the reference information.

According to another preferred embodiment of the present invention, a scalable encoder includes: a reference information generator for generating reference information which is information on which discardable layer is removed for each of the base layer and at least one or more upper layers; A receiver configured to receive a scalable video stream comprising a; And a layer switching determination unit configured to determine whether to perform layer switching from the target layer to be decoded to the lower layer based on the reference information according to the bandwidth situation.

According to a preferred embodiment of the present invention, a method of notifying reference slice removal may include receiving a scalable video stream including reference information which is information associated with a discardable layer; Removing the discardable layer based on the reference information when the network condition worsens than the first reference value; And a reference information updating step of updating the reference information by adding information on which discardable layer has been removed for each of the base layer and at least one higher layer of the scalable video stream to the reference information.

According to another preferred embodiment of the present invention, a scalable encoding method may include: a reference information generator configured to generate reference information which is information on which discardable layer has been removed for each base layer and at least one or more upper layers; Receiving a scalable video stream comprising; And a layer switching determination step of determining whether to perform layer switching from a target layer to be decoded to a lower layer based on the reference information according to a bandwidth situation.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. It should be noted that the same elements among the drawings are denoted by the same reference numerals and symbols as much as possible even though they are shown in different drawings.

In the following description of the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

In order to transmit video effectively in the internet or wireless communication environment where the transmission bandwidth changes frequently, not only the encoder with high compression characteristics but also the scalability function to adaptively transmit the video data according to the network environment and the performance of the terminal should be provided. do.

Using the scalable coding method, a video can be compressed into a plurality of layers and a high quality video can be received by sending a base layer and an enhancement layer information together in a situation where bandwidth is sufficient. In the situation where the bandwidth is low, only the base layer information is transmitted, so that the video quality may be slightly reduced or the video size is small, thereby providing a seamless service.

Using the scalable coding method, a video is compressed into multiple layers so that a high quality video can be received by sending base layer and enhancement layer information together when bandwidth is sufficient. In the situation where the bandwidth is lowered, only the base layer information is sent, so that the video quality may be slightly reduced or the video size is small, thus providing a seamless service.

However, when the base layer and the upper layer are sent at the same time, the bit amount increases because the lower layer is transmitted even when the upper layer is decoded due to the layer coding. Therefore, the compression efficiency is somewhat lower than the method of coding only the higher layer separately.

In order to solve this disadvantage, in SVC, when only the upper layer is decoded, the layer which transmits only the lower layer referred to by the upper layer and does not transmit other information is defined as a discardable layer.

As a method of removing information of the corresponding layer during transmission in SVC, there is a method of missing and sending a discardable layer when transmitting. However, in order to access the bit amount of single layer coding, more various methods of information removal are required. Therefore, the present invention intends to provide a configuration that secures the concept of the discardable layer and informs of the removed discardable layer in order to correspond to various methods of removing information in the future.

The discardable layer is largely classified into 1) a method of distinguishing an image referenced from an upper layer and an unreferenced image on a macroblock basis, and 2) a method of distinguishing an image referenced from an upper layer and an unreferenced image on a per-picture basis. can do.

* DISCARDABLE  Approach to layers

Macro Block Unit Approach

1 illustrates a concept of a method of accessing a discardable layer on a macroblock basis. The method of classifying the discardable layer in units of macroblocks is to classify using a slice group.

slice group 0 (110) represents a set of macroblocks for inter-layer prediction in a higher layer, and slice group 1 (120) represents a set of macroblocks for inter-layer prediction in a higher layer.

The advantage of the macroblock-based approach is that the referenced macroblocks and the unreferenced macroblocks are precisely separated, leaving only the referenced macroblocks, thereby improving coding efficiency when an arbitrary layer is discarded.

However, since the macroblocks in a slice group are not always continuous, slice must know which macroblock it has and then input additional header information.

2. picture (Picture) Unit approach

2 illustrates a concept of a method of accessing a discardable layer in picture units.

The picture unit approach is a method using a hierarchical structure of the SVC, as shown in FIG.

If the difference between the bit amount of the picture encoded by inter-layer reference and the bit amount of the encoded picture without using the inter-layer reference is not large, the entire encoder is encoded without using the inter-layer reference method.

However, if the bit amount is more than a certain level, the picture can be coded using a large number of inter-layer reference methods, and thus the picture is encoded using the inter-layer reference method.

The advantage of the picture-based approach is that the process is simple and the amount of overhead for using any layer is small. On the other hand, since the picture-based approach uses or does not use the entire picture, since the picture has a referenced portion 210 and an unreferenced portion 220, the portion of the macro block that is not referenced in the case of a picture using an inter-layer reference method is used. In case 220 is a picture that is transmitted without being used and does not use an inter-layer reference method, coding is performed without referring to the block part 210 to which reference is made. Therefore, coding efficiency is reduced in this part.

The discardable layer used in the present invention may be divided into macroblock units or picture division units.

3 illustrates a configuration of a reference slice removal notification device as a preferred embodiment of the present invention.

Leaving only the information necessary to decode a target layer of one of several layers of a bitstream encoded with scalable video coding (SVC) in an extractor located at a transmission origin, a network or a terminal with a decoder, Not only the upper layer but also the information unnecessary for decoding the target layer (that is, the information of the lower layer not used for inter-layer reference) is removed from the information of the lower layer.

In this case, the reference slice removal notification apparatus 300 of the present invention adds information about the removed discardable layer to the scalable video stream to notify the next decoder or the next layer remover of the absence of the reference image, thereby causing the absence of the reference image. Prevent errors due to

According to an embodiment of the present invention, the reference slice removal notification device 300 includes a receiver 310, a remover 320, and a reference information updater 330. The reference slice removal notification device 300 may be implemented in a layer remover located in an encoder, a router, a decoder, or the like. Each component is as follows.

The receiver 310 receives a scalable video stream including reference information that is information related to a discardable layer. The reference information is not used for inter-layer reference or information about the disc double layer, which is a lower layer that is not currently needed for decoding the target layer to be decoded, and any disc for each base layer and at least one higher layer in the removal unit 320. Contains information about whether the cardable layer has been removed.

The remover 320 removes the discardable layer based on the reference information received by the receiver 310 when the network condition deteriorates, for example, when the bandwidth is narrowed or the terminal transmit / receive situation is not good. This will be described in detail with reference to FIG. 4.

The reference information updater 330 updates the reference information by adding, to the reference information, information about which discardable layer has been removed for each base layer or at least one higher layer of the scalable video stream. By updating the reference information, each layer may inform the next decoder or the next layer remover via the SVC bitstream whether the picture used as the reference in the layer has been removed.

The hierarchical switching determination unit 340 deteriorates the network situation than the first reference value set by the user or the system, and discards the discardable layer. The second reference value set by the system) may determine whether to perform hierarchical switching. That is, it is determined whether to change the layer from the target layer to be decoded to the lower layer.

Conventionally, when layer switching is performed, an error occurs because it is not possible to determine whether a discardable layer exists in a lower layer due to deterioration of a network situation.

On the other hand, the layer switching determination unit 340 of the present invention, if there is a discardable layer already removed in the lower layer based on the reference information having information on which discardable layer is discarded in each layer. By not performing switching, the occurrence of errors can be reduced.

4 illustrates an example of selectively performing inter-layer prediction based on a frame in scalable video coding according to a preferred embodiment of the present invention.

As shown in FIG. 4, the scalable video stream is transmitted to the terminal 440 through the server 410, the transmitter 1 420, and the transmitter 2 430. In this case, the transmitters 1 and 2 may be routers.

Scalable video consists of a base layer and three upper layers. When the network bandwidth is narrowed, the transmitter 1 420 discards the discardable slices 421, 422, and 423 in order to perform decoding while maintaining the quality. Even in this case, the target layer can decode layer 3 without error.

However, when the network bandwidth becomes narrower, the transmit end 2 430 attempts layer switching to layer 2 or layer 1. However, in this case, both layers 1 and 2 already include discarded slice slices, and an error occurs when the layer switching is attempted.

In order to solve this problem, in the present invention, as described above, the information on the discardable layer removed by the reference information update unit 330 is added and updated, and the layer switching determination unit 340 uses the reference information. It is determined whether to perform layer switching based on this.

FIG. 5 illustrates a concept used to inform whether or not to remove an image used as an intra-layer reference for each layer as a preferred embodiment of the present invention.

As in the case of the transmitter 1 420 of FIG. 4, as a result of the determination by the layer switching determination unit 340 based on the reference information, there is no problem in the target layer decoding, but there is a problem in the lower layer decoding by deleting the information of the lower layers. In this case, inter_prediction_reference_slice_dropped_layer is set to 0 (510) to limit layer switching to the lower layer.

However, if the inter_prediction_reference_slice_dropped_layer is 1 (520, 530), the lower layer indicates that the block or slice used for the reference in the layer has not been discarded. Therefore, in this case, layer switching to the lower layer is possible.

6 (a) to 6 (c) illustrate a syntax of an SEI message to which the concept of the present invention is added, as a preferred embodiment of the present invention.

The message of FIG. 6 is sent per IDR access unit, and the semantics of this message are valid until after the same type of SEI message.

The inter_prediction_reference_slice_dropped_layer [i] of Fig. 6 is specified as follows. In other words, when [i] is equal to 1, the I-th layer indicates that the block used for reference in the layer has not been discarded. If the value is 0, the layer is already limited to layer switching to that layer by removing any information used for reference.

7 is a flowchart illustrating a reference slice removal notification method according to a preferred embodiment of the present invention.

The reference slice removal notification method is substantially the same as or similar in configuration to the above-described reference slice removal notification device, and thus, details thereof will be referred to.

First, an encoder or a router receives a scalable video stream including reference information which is information related to a discardable layer (S710). When the network situation worsens than the first reference value set as the bandwidth narrows, the discardable layer is removed based on the reference information (S720).

Thereafter, the reference information is updated by adding information on which discardable layer is removed for each of the base layer and at least one higher layer of the scalable video stream to the reference information (S730).

If the network situation is worsened based on the above reference information, it is determined whether to perform layer switching (S740). It should be noted that if the discardable layer has already been removed from the previous encoder or router, steps S720 and S730 have been performed, and thus it may be determined whether layer switching is performed based on reference information immediately after receiving the scalable video stream.

The best embodiments have been disclosed in the drawings and specification above. Although specific terms have been used herein, they are used only for the purpose of describing the present invention and are not used to limit the scope of the present invention as defined in the meaning or claims.

Therefore, those skilled in the art will understand that various modifications and equivalent other embodiments are possible from this. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

1 illustrates a concept of a method of accessing a discardable layer on a macroblock basis.

2 illustrates a concept of a method of accessing a discardable layer in picture units.

3 illustrates a configuration of a reference slice removal notification device as a preferred embodiment of the present invention.

4 illustrates an example of selectively performing inter-layer prediction based on a frame in scalable video coding according to a preferred embodiment of the present invention.

FIG. 5 illustrates a concept used to inform whether or not to remove an image used as an intra-layer reference for each layer as a preferred embodiment of the present invention.

6 (a) to 6 (b) illustrate the syntax of an SEI message to which the concept of the present invention is added, as a preferred embodiment of the present invention.

7 is a flowchart illustrating a reference slice removal notification method according to a preferred embodiment of the present invention.

Claims (17)

A receiver configured to receive a scalable video stream including reference information which is information associated with a discardable layer; A removal unit for removing a discardable layer based on the reference information when a network condition is worse than a first reference value; And And a reference information updater for updating the reference information by adding information on which discardable layer has been removed for each of the base layer and the at least one higher layer of the scalable video stream to the reference information. See slice removal notification device. The method of claim 1, wherein the reference information is Information about the discardable layer, which is a lower layer that is not used for inter-layer reference or that is not currently required for decoding the target layer to be decoded, and which discardable layer has been removed for each layer in the removal unit. Reference slice removal notification device comprising the information on. The method of claim 1, wherein the network situation is worse than the second reference value. A layer switching determining unit configured to determine whether to perform layer switching from a target layer to be currently decoded based on the reference information to a lower layer; In this case, the second reference value indicates that the network situation is worse than the first reference value reference slice removal notification device, characterized in that. The method of claim 2, wherein the hierarchical switching determination unit The reference slice removal notification device, characterized in that the layer switching is not performed if there is already a discardable layer in the lower layer. The method of claim 1, wherein the discardable layer is Reference slice removal notification device, characterized in that divided into macroblock units or picture units. The method of claim 1, wherein the network situation Reference slice removal notification device, characterized in that the bandwidth situation or the terminal situation. The apparatus of claim 1, wherein the reference slice removal notification device is Reference slice removal notification device, characterized in that implemented in the layer extractor (extractor) located in the encoder, decoder, router. A reference information generation unit for generating reference information which is information on which discardable layer has been removed for each of the base layer and at least one higher layer; A receiver configured to receive a scalable video stream comprising a; And And a layer switching determination unit configured to determine whether to perform layer switching from a target layer to be decoded to a lower layer based on the reference information according to a bandwidth situation. The method of claim 8, wherein the hierarchical switching determination unit And a layer switching is not performed when there is a discardable layer already removed in the lower layer based on the reference information. Receiving a scalable video stream comprising reference information that is information associated with the discardable layer; Removing the discardable layer based on the reference information when the network condition worsens than the first reference value; And And a reference information updating step of updating the reference information by adding information on which discardable layer has been removed for each of the base layer and the at least one higher layer of the scalable video stream to the reference information. How to Notify Reference Slice Removal. The method of claim 10, wherein the reference information is Information about the discardable layer, which is a lower layer that is not used for inter-layer reference or is not currently required for decoding the target layer to be decoded, and which discardable layer has been removed for each layer in the removal step. Reference slice removal notification method comprising the information on. 11. The method of claim 10, wherein the network situation is worse than the second reference value. A layer switching determining step of determining whether to perform layer switching from a target layer to be decoded to a lower layer based on the reference information; In this case, the second reference value is a reference slice removal notification method, characterized in that the network conditions worse than the first reference value. The method of claim 11, wherein the determining of the layer switching is performed. And if no discardable layer is already present in the lower layer, switching of the reference slice is not performed. The method of claim 10, wherein the discardable layer is The reference slice removal notification method, characterized in that divided into macroblock units or picture units. 11. The method of claim 10, wherein the network situation is Reference slice removal notification method, characterized in that the bandwidth situation or the terminal situation. A reference information generation unit for generating reference information which is information on which discardable layer has been removed for each of the base layer and at least one higher layer; Receiving a scalable video stream comprising; And And a layer switching decision step of determining whether to perform layer switching from a target layer to be currently decoded to a lower layer based on the reference information according to a bandwidth situation. The method of claim 16, wherein the determining of the layer switching comprises: And a layer switching is not performed when there is a discardable layer already removed in the lower layer based on the reference information.

Images