KR20230156005A - Level setting method and apparatus using the same - Google Patents

Abstract

Disclosed is a level setting method and a video decoding device using the same.
According to an embodiment of the present invention, a method of setting a level for each of one or more areas includes the steps of decoding a definition syntax element related to level definition and a designation syntax element related to target designation from a bitstream; defining one or more levels based on the defining syntax elements; and setting a target level designated by the designated syntax element among the defined levels to a target area designated by the designated syntax element.

Description

Level setting method and video decoding device using the same {LEVEL SETTING METHOD AND APPARATUS USING THE SAME}

The present invention relates to video encoding and decoding, and more specifically, to a level setting method that improves encoding and decoding efficiency by setting different levels or tiers for each region, and to a video decoding device using the same.

Since video data has a larger amount of data than audio data or still image data, it requires a lot of hardware resources, including memory, to store or transmit it without processing for compression.

Therefore, typically, when storing or transmitting video data, an encoder is used to compress the video data and store or transmit it, and a decoder receives the compressed video data, decompresses it, and plays it. These video compression technologies include H.264/AVC and HEVC (High Efficiency Video Coding), which improves coding efficiency by about 40% compared to H.264/AVC.

However, the size, resolution, and frame rate of images are gradually increasing, and the amount of data that needs to be encoded is also increasing accordingly, so a new compression technology with better coding efficiency and higher picture quality improvement effect than existing compression technologies is required.

In order to meet these needs, the present invention aims to provide improved video encoding and decoding technology. In particular, one aspect of the present invention is to predefine one or more levels or tiers, and to define different levels or tiers for each target area. It is related to technology that improves the efficiency of encoding and decoding by setting .

One aspect of the present invention is a method of setting a level for each of one or more areas, comprising the steps of: decoding a definition syntax element related to level definition and a designation syntax element related to target designation from a bitstream; defining one or more levels based on the defining syntax elements; and setting a target level designated by the designated syntax element among the defined levels to a target area designated by the designated syntax element.

Another aspect of the present invention is a video decoding device that sets a level for each of one or more regions, and decodes a definition syntax element related to level definition and a designation syntax element related to target designation from a bitstream. wealth; And defining one or more levels based on the definition syntax element, and comprising control means for setting the target level specified by the specified syntax element among the defined levels to the target area specified by the specified syntax element. Provides a video decoding device that

As described above, according to an embodiment of the present invention, it is possible to implement encoding and decoding optimized for the image quality characteristics of the target area by setting different levels or tiers for each target area.

In addition, according to another embodiment of the present invention, by setting different levels or tiers for each target area, the viewport area corresponding to the actual image display target can be distinguished from other areas, making it possible to implement selective decoding. .

1 is an example block diagram of a video encoding device that can implement the techniques of the present disclosure.
Figure 2 is a diagram to explain a method of dividing a block using the QTBTTT structure.
Figure 3 is a diagram for explaining a plurality of intra prediction modes.
Figure 4 is an example block diagram of a video decoding device that can implement the techniques of the present disclosure.
5 and 6 are flowcharts for explaining an embodiment of the present invention for a level or tier setting method implemented in a video encoding device and a video decoding device.
Figure 7 is a diagram for explaining an embodiment of the present invention that implements selective decoding for each region by setting different levels or tiers for each target region.
Figures 8 and 9 are flowcharts for explaining an embodiment of the present invention for specifying a target level or target tier implemented in a video encoding device and a video decoding device.
10 and 11 are flowcharts for explaining an embodiment of the present invention for defining a level or tier implemented in a video encoding device and a video decoding device.

Hereinafter, some embodiments of the present invention will be described in detail through illustrative drawings. In adding identification codes to components in each drawing, it should be noted that the same components are given the same codes as much as possible even if they are shown in different drawings. Additionally, in describing the present invention, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present invention, the detailed description will be omitted.

1 is an example block diagram of a video encoding device that can implement the techniques of the present disclosure. Hereinafter, the video encoding device and its sub-configurations will be described with reference to FIG. 1.

As shown in FIG. 1, the image encoding device includes a block division unit 110, a prediction unit 120, a subtractor 130, a transform unit 140, a quantization unit 145, an encoding unit 150, and an inverse quantization unit. It may be configured to include a unit 160, an inverse transform unit 165, an adder 170, a filter unit 180, and a memory 190.

Each component of the video encoding device may be implemented as hardware or software, or may be implemented as a combination of hardware and software. Additionally, the function of each component may be implemented as software and a microprocessor may execute each function of each software (component).

One image (video) consists of multiple pictures. Each picture is divided into a plurality of regions and encoding is performed for each region. For example, one picture can be divided into one or more slices and/or tiles, and one or more tiles can be defined as a tile group. Each slice or tile is divided into one or more Coding Tree Units (CTUs). And each CTU is divided into one or more CUs (Coding Units) by a tree structure.

Information applied to each CU is encoded with the syntax of the CU, and information commonly applied to CUs included in one CTU is encoded with the syntax of the CTU. In addition, information commonly applied to all blocks within one tile is encoded with the syntax of the tile or with the syntax of the tile group to which the tile belongs, and information applied to all blocks constituting one picture is encoded as a picture parameter. It is encoded in a set (PPS, Picture Parameter Set) or picture header.

Furthermore, information commonly referenced by a plurality of pictures is encoded into a Sequence Parameter Set (SPS), and information commonly referenced by one or more SPSs is encoded into a Video Parameter Set (VPS).

The block division unit 110 determines the size of the CTU (Coding Tree Unit). Information about the size of the CTU (CTU size) is encoded with SPS or PPS syntax and transmitted to the video decoding device.

The block division unit 110 divides each picture constituting the image into a plurality of CTUs (Coding Tree Units) having a predetermined size, and then repeatedly divides the divided CTUs using a tree structure. Split recursively. A leaf node in the tree structure becomes a coding unit (CU), the basic unit of encoding.

The tree structure includes Quad Tree (QT), in which the parent node is divided into four child nodes (or child nodes) of the same size, and Binary Tree (QT), in which the parent node is divided into two child nodes. Tree, BT), and a Ternary Tree (TT) in which the upper node is divided into three lower nodes in a 1:2:1 ratio.

Additionally, a structure in which two or more of QT structure, BT structure, and TT structure are mixed may be included. For example, a Quad Tree plus Binary Tree (QTBT) structure may be used, and a Quad Tree plus Binary Tree Ternary Tree (QTBTTT) structure may be used.

Figure 2 is a diagram to explain a method of dividing a block using the QTBTTT structure. As shown in Figure 2, the CTU can first be divided into a QT structure. Quadtree splitting can be repeated until the size of the splitting block reaches the minimum block size (MinQTSize) of the leaf node allowed in QT.

If the leaf nodes of QT are not larger than the maximum block size (MaxBTSize) of the root node allowed in BT, they may be further divided into one or more of the BT structure or TT structure. In the BT structure and/or TT structure, there may be multiple division directions.

For example, in some examples, there may be two types: a type that splits the block of the node horizontally (i.e., horizontal splitting) and a type that splits it vertically (i.e., vertical splitting).

As shown in FIG. 2, when BTTT splitting is performed, a flag indicating whether the nodes are split, a flag indicating the splitting direction (vertical or horizontal), and/or a flag indicating the splitting type (Binary or Ternary) are used for video decoding. Can be signaled to the device.

Meanwhile, there may be an additional type that divides the block of the corresponding node into two asymmetric blocks. The asymmetric form may include dividing the block of the corresponding node into two rectangular blocks with a size ratio of 1:3, dividing the block of the corresponding node diagonally, etc.

When QTBT is used as another example of a tree structure, the CTU may first be divided into a QT structure, and then the leaf nodes of the QT may be further divided into a BT structure.

A CU can have various sizes depending on the QTBT or QTBTTT division from the CTU. Hereinafter, the block corresponding to the CU (i.e., leaf node of QTBTTT) to be encoded or decoded is referred to as the 'current block'.

The prediction unit 120 predicts the current block and generates a prediction block. The prediction unit 120 may include an intra prediction unit 122 and an inter prediction unit 124. In general, each current block in a picture can be coded predictively. Prediction of the current block may be performed using intra prediction technology using data from a picture including the current block, or inter prediction technology using data from a picture coded before the picture including the current block.

The intra prediction unit 122 predicts pixels within the current block using pixels (reference pixels) located around the current block within the current picture including the current block. There are multiple intra prediction modes depending on the prediction direction. For example, as shown in FIG. 3, the plurality of intra prediction modes may include non-directional modes including planar mode and DC mode and 65 directional modes. The surrounding pixels and calculation formulas to be used are defined differently for each prediction mode.

The intra prediction unit 122 may determine the intra prediction mode to be used to encode the current block. In some examples, intra prediction unit 122 may encode the current block using multiple intra prediction modes and select an appropriate intra prediction mode to use from the tested modes.

For example, the intra prediction unit 122 calculates rate-distortion values using rate-distortion analysis for several tested intra-prediction modes, and selects the rate-distortion value with the best rate-distortion characteristics among the tested modes. You can also select intra prediction mode.

The intra prediction unit 122 selects one intra prediction mode from a plurality of intra prediction modes and predicts the current block using surrounding pixels (reference pixels) and an operation equation determined according to the selected intra prediction mode. Information about the selected intra prediction mode is encoded by the encoder 150 and transmitted to the video decoding device.

The inter prediction unit 124 searches for a block most similar to the current block within a reference picture that has been encoded and decoded before the current picture through a motion estimation process, and uses the block searched through a motion compensation process to find the current block. Create a prediction block for the block. Inter prediction can generally be divided into uni-directional prediction and bi-directional prediction depending on the prediction direction.

The inter prediction unit 124 generates a motion vector corresponding to the displacement between the current block in the current picture and the prediction block in the reference picture. Typically, motion estimation is performed on the luma component, and motion vectors calculated based on the luma component are used for both the luma component and the chroma component.

Motion information including information about the reference picture and information about the motion vector used to predict the current block is encoded by the encoder 150 and transmitted to the video decoding device.

Various methods can be used to reduce or minimize the amount of bits required to encode motion information. Representative examples of these various methods include Skip mode, Merge mode, and AMVP (Adaptive (Advanced) motion vector predictor) mode.

In Skip mode, a preset number of candidate blocks are selected from neighboring blocks, one to be used as motion information of the current block among the motion information of the selected candidate blocks is selected, and then the index value of the selected motion information is encoded and used by the video decoding device. It is signaled as In the HEVC standard, the index value for Skip mode is expressed through merge_idx syntax.

In Merge mode, first, a preset number of candidate blocks are selected from neighboring blocks. When the candidate blocks are selected, the inter prediction unit 124 constructs a merge list containing the candidate blocks, and selects motion information to be used as motion information for the current block from among the motion information of the candidate blocks included in the list. and generate a merge index value to identify the selected motion information (selected candidate block).

The index value of the selected motion information, that is, the merge index value, is encoded and signaled to the video decoding device. In the HEVC standard, the index value for Merge mode is expressed through merge_idx syntax.

In AMVP mode, first, motion vector predictor (MVP) candidates for the motion vector of the current block are derived using neighboring blocks of the current block. When predicted motion vector candidates are derived, the inter prediction unit 124 determines a predicted motion vector (mvp) for the motion vector of the current block, subtracts the determined predicted motion vector from the motion vector of the current block, and produces a differential motion vector (motion vector). Calculate vector difference (mvd). The calculated differential motion vector is encoded and signaled to the video decoding device.

The process of determining a predicted motion vector from predicted motion vector candidates may be implemented through a predefined function (eg, median value operation, average value operation, etc.). In this case, the video decoding device is set to apply a predefined function.

Since the neighboring blocks used to derive predicted motion vector candidates correspond to blocks for which encoding and decoding have already been completed, the video decoding device also already recognizes the motion vectors for these neighboring blocks. Accordingly, since information for identifying predicted motion vector candidates does not need to be encoded, the video encoding apparatus encodes only information about the differential motion vector and information about the reference picture used to predict the current block.

The process of determining a predicted motion vector from the predicted motion vector candidates may be implemented by selecting one of the predicted motion vector candidates. In this case, information for identifying the determined prediction motion vector is additionally encoded, along with information about the differential motion vector and information about the reference picture used to predict the current block.

The subtractor 130 generates a residual block by subtracting the current block from the prediction block generated by the intra prediction unit 122 or the inter prediction unit 124, and the transform unit 140 generates a residual block having pixel values in the spatial domain. The residual signal within the block is converted into a transform coefficient in the frequency domain.

The conversion unit 140 may convert the residual signals in the residual block using the size of the current block as a conversion unit, divide the residual block into a plurality of smaller sub-blocks, and convert the residual signals into a conversion unit of the sub-block size. You may.

There may be various ways to divide the residual block into smaller sub-blocks. For example, it can be divided into predefined sub-blocks of the same size, or QT (Quad Tree) method partitioning with the residual block as the root node can be used.

The quantization unit 145 quantizes the transform coefficients output from the transform unit 140 and outputs the quantized transform coefficients to the encoder 150.

The encoder 150 generates a bitstream by encoding the quantized transform coefficients using an encoding method such as CABAC. In addition, the encoder 150 encodes and signals information such as CTU size, QT split flag, BTTT split flag, split direction, and split type related to block splitting, so that the video decoding device can split the block in the same way as the video coding device. make it possible

Furthermore, the encoder 150 encodes information about the prediction type indicating whether the current block is encoded by intra prediction or inter prediction, and generates intra prediction information (i.e., intra prediction mode) according to the prediction type. information) or inter prediction information (information about reference pictures and motion vectors) is encoded.

The inverse quantization unit 160 inversely quantizes the quantized transform coefficients output from the quantization unit 145 to generate transform coefficients. The inverse transform unit 165 converts the transform coefficients output from the inverse quantization unit 160 from the frequency domain to the spatial domain to restore the residual block.

The addition unit 170 restores the current block by adding the restored residual block and the prediction block generated by the prediction unit 120. Pixels in the restored current block are used as reference pixels for intra prediction of the next block.

The filter unit 180 applies filtering to the restored pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transformation/quantization. Perform filtering on The filter unit 180 may include a deblocking filter 182 and a SAO filter 184.

The deblocking filter 180 filters the boundaries between restored blocks to remove blocking artifacts caused by block-level encoding/decoding, and the SAO filter 184 performs additional filtering on the deblocking-filtered image. Perform.

The SAO filter 184 corresponds to a filter used to compensate for the difference between the restored pixel and the original pixel caused by lossy coding. The restored block filtered through the deblocking filter 182 and the SAO filter 184 is stored in the memory 190. When all blocks in one picture are reconstructed, the reconstructed picture is later used as a reference picture for inter prediction of blocks in the picture to be encoded.

Although not shown in FIG. 1, the video encoding device may further include control means for setting different or the same level or tier for each region. This control means may be implemented in the same physical configuration (processor, etc.) as the sub-configurations shown in FIG. 1, or may be implemented in a different physical configuration than the sub-configurations shown in FIG. Details about the control means will be described later.

Figure 4 is an example block diagram of a video decoding device that can implement the techniques of the present disclosure. Hereinafter, the video decoding device and its sub-configurations will be described with reference to FIG. 4.

As shown in FIG. 4, the image decoding device includes a decoder 410, an inverse quantization unit 420, an inverse transform unit 430, a prediction unit 440, an adder 450, a filter unit 460, and a memory ( 470).

Like the video encoding device shown in FIG. 1, the video decoding device may have each component implemented as a hardware chip, the function of each component may be implemented as software, and a microprocessor may be configured to execute the functions of each software. there is.

The decoder 410 decodes the bitstream received from the video encoding device, extracts information related to block division (segmentation information of the luma block and/or division information of the chroma block), and uses this to select the current block to be decoded. decision, and extract the prediction information and information about the residual signal necessary to restore the current block.

The decoder 410 extracts information about the CTU size from a Sequence Parameter Set (SPS) or Picture Parameter Set (PPS), determines the size of the CTU, and divides the picture into CTUs of the determined size. Additionally, the decoder 410 determines the CTU as the highest layer of the tree structure, that is, the root node, extracts segmentation information from the bitstream, and then uses this to segment or restore the block.

In addition, the decoder 410 extracts information about whether the block is BT-divided and the division type (division direction) for the node corresponding to the leaf node of the QT division and divides the corresponding leaf node into a BT structure.

As another example, when dividing or restoring a block using the QTBTTT structure, the decoder 410 extracts information (flag) about whether the QT is divided, divides each node into four nodes of the lower layer, and divides the QT. For nodes corresponding to leaf nodes (nodes where QT division no longer occurs), extract information about whether it is further divided into BT or TT, information about the direction of division, and information about the division type that distinguishes whether it is a BT structure or a TT structure. Split recursively into BT or TT structures.

In this way, when the current block to be decoded is determined using the partition information, the decoder 410 extracts information about the prediction type indicating whether the current block was intra-predicted or inter-predicted.

When prediction type information indicates intra prediction, the decoder 410 extracts syntax elements for intra prediction information (intra prediction mode) of the current block. When prediction type information indicates inter prediction, the decoder 410 extracts syntax elements for inter prediction information, that is, information indicating a motion vector and the reference picture referenced by the motion vector (motion information of the current block). .

Meanwhile, the decoder 410 extracts information about the quantized transform coefficients of the current block as information about the residual signal.

The inverse quantization unit 420 inversely quantizes the quantized transform coefficients, and the inverse transformation unit 430 inversely transforms the inverse quantized transform coefficients from the frequency domain to the spatial domain to restore the residual signals, thereby generating a residual block for the current block. .

The prediction unit 440 may include an intra prediction unit 442 and an inter prediction unit 444. The intra prediction unit 342 is activated when the prediction type of the current block is intra prediction, and the intra prediction unit 342 is activated when the prediction type of the current block is intra prediction. Unit 344 is activated when the prediction type of the current block is intra prediction.

The intra prediction unit 442 determines the intra prediction mode of the current block among a plurality of intra prediction modes using syntax elements for the intra prediction mode extracted from the decoder 410, and determines the intra prediction mode around the current block according to the determined intra prediction mode. Predict the current block using reference pixels.

The inter prediction unit 444 determines the motion vector of the current block and the reference picture referenced by this motion vector using the syntax elements for the inter prediction mode extracted from the decoder 410, and determines the motion vector of the current block and the reference picture referred to by the motion vector. Predict the current block.

The adder 450 restores the current block by adding the residual block output from the inverse transform unit 430 and the prediction block output from the inter prediction unit 444 or the intra prediction unit 442. Pixels in the restored current block are used as reference pixels for intra prediction of the block to be decoded later.

The filter unit 460 includes a deblocking filter 462 and a SAO filter 464. The deblocking filter 462 removes blocking artifacts caused by block-level decoding by deblocking and filtering the boundaries between restored blocks.

The SAO filter 464 performs additional filtering on the reconstructed block after deblocking filtering to compensate for the difference between the reconstructed pixel and the original pixel caused by lossy coding.

The restored blocks filtered through the deblocking filter 462 and the SAO filter 464 are stored in the memory 470, and when all blocks in one picture are restored, the restored picture is interconnected with the blocks in the picture to be encoded later. It is used as a reference picture for prediction.

Although not shown in FIG. 4, the video decoding device may further include control means for setting different or the same level or tier for each region. This control means may be implemented in the same physical configuration (processor, etc.) as the sub-configurations shown in FIG. 4, or may be implemented in a different physical configuration than the sub-configurations shown in FIG. 4. Details about the control means will be described later.

Meanwhile, the control means included in the video decoding device corresponds to the above-described control means included in the video encoding device in terms of its function. Hereinafter, in order to easily distinguish between the control means included in the video decoding device and the control means included in the video encoding device, the control means included in the video decoding device is referred to as decoding control means, and the control means included in the video encoding device is referred to as decoding control means. The means shall be referred to as encoding control means.

Before a detailed description of the present invention, terms used in this specification are defined.

'Area' referred to in this specification refers to an object for setting a level or tier, and this area may include tiles, slices, tile groups, slice segments, etc.

The 'area header' referred to in this specification is a concept defined in relation to the 'area'. When the area corresponds to a tile, the area header corresponds to the tile header, and when the area corresponds to a slice, the area header is It corresponds to a slice header, and if the area corresponds to a tile group, the area header corresponds to the tile group header.

Additionally, when the region corresponds to a picture, the region header corresponds to a Picture Parameter Set (PPS) or a picture header, and when the region corresponds to multiple pictures, the region header corresponds to a Sequence Parameter Set (SPS). Parameter Set), and the area header for the area commonly referenced by one or more SPS corresponds to the Video Parameter Set (VPS).

Conventional standards such as HEVC used concepts such as profile, level, and tier in consideration of the application to which the standard technology is applied and the performance of the video decoding device (decoder).

A profile refers to a specification set in advance for various applications to which standard technology is applied. In HEVC, profiles such as 'Main profile', 'Main10 profile', and 'Main still picture profile' are established.

Level is a concept used in consideration of the fact that even if the profile is the same, there may be differences in processing performance depending on the characteristics of the video decoding device. The maximum resolution and frame rate of the video that can be processed can be determined according to the level value.

Tier refers to restrictions on the maximum bit rate, and is a concept used because even if it is the same profile and level, there are cases where the video is compressed to high resolution and high quality depending on the application and there are cases where it is not. In other words, the tier is a regulation related to the memory 470 of the video decoding device.

The conventional method for setting profiles, levels, tiers, etc. sets the levels, tiers, and profiles to a single value for a sequence containing one or more images (pictures). That is, the conventional method applies or sets levels, tiers, and profiles in units of one or more sequences.

Unlike these conventional methods, the present invention corresponds to an invention that further improves the efficiency of encoding and decoding by using an area with a size smaller than a picture as a reference unit for setting levels and/or tiers. In the following, the present invention will be described focusing on an embodiment in which the level among the levels and tiers is set for each area. However, an embodiment for setting the tier for each area and an embodiment for setting both the level and the tier for each area are also proposed in the present invention. It can be implemented through this method.

5 and 6 are flowcharts for explaining an embodiment of the present invention for a level or tier setting method implemented in a video encoding device and a video decoding device. Hereinafter, a basic embodiment of the present invention regarding a level setting method will be described with reference to FIGS. 5 and 6.

As shown in FIG. 5, the control means included in the video encoding device, that is, the encoding control means, defines one or more levels using a definition syntax element, which is a syntax element that defines a level (S510). An example of a definition syntax element is represented in Table 1 below.

pic_parameter_set_rbsp(　) { Descriptor … … num_base_tier_level_set_minus1 for( i = 0; i <= num_base_tier_level_set_minus1; i++ ) { base_tier_level_set_id[i] base_tier_flag[　i　] base_level_idc[　i　] } … }

In Table 1 above, num_base_tier_level_set_minus1 represents the number of levels or tiers to be defined minus 1 (i.e., represents the number of levels to be defined), and base_tier_level_set_id[i] represents the id of the level and tier set. , base_tier_flag[i] represents the value of the ith tier, and base_level_idc[i] represents the value of the ith level.

Table 1 shows only an example of a set of levels or tiers to be defined in the form of an id, but depending on the embodiment, the set of levels or tiers to be defined may be expressed in the form of an index (idx) (base_tier_level_set_idx). The definition syntax element may be defined in the PPS position as shown in Table 1, or may be defined in one or more of PPS, SPS, VPS, and SEI positions.

When one or more levels are defined, the encoding control means uses a designation syntax element to designate a target area corresponding to the target of level setting and a target level, which is the level to be set in this target area (S520).

The specified syntax element may include a syntax element that specifies the target area and a syntax element that specifies the target level. A designated syntax element that specifies a target area may be referred to as an 'area-specified syntax element', and a designated syntax element that specifies a target level may be referred to as a 'level-specified syntax element'.

An example of designation syntax elements (region designation syntax element and level designation syntax element) is represented in Table 2 below.

tile_header(　) { Descriptor tile_idx base_tier_level_set_id … }

In Table 2 above, tile_idx represents an area specification syntax element that specifies the index of the target area implemented as a tile, and base_tier_level_set_id represents the level and tier to be applied or specified to the target area indicated by tile_idx among the predefined levels and tiers. Represents level and tier specification syntax elements.

Table 2 shows area designation syntax elements and level and tier designation syntax elements assuming that the target area corresponds to a tile. However, as described above, the target area may correspond not only to tiles but also to slices, tile groups, slice segments, etc., so area designation syntax elements and level and tier designation syntax elements must be defined or specified in the header of the unit to which the target area corresponds. It can be. For example, if the target area corresponds to a tile group, area designation syntax elements and level and tier designation syntax elements may be defined or specified in this tile group header. The same applies to other tables presented below.

When the designation of the target area and target level is completed, the encoder 150 encodes the definition syntax element and the designation syntax element (S530). The encoded syntax elements are included in the bitstream and signaled to the video decoding device (S540).

As shown in FIG. 6, the decoder 410 first parses the definition syntax element and the designated syntax element in the bitstream signaled from the video encoding device and decodes them (S610). As described above, the designation syntax element may include an area designation syntax element that specifies the target area and a level designation syntax element that specifies the target level.

The control means of the video decoding device, that is, the decoding control means, defines one or more levels based on definition syntax elements (S620). When one or more levels are defined, the decoding control means sets the target level designated by the level designation syntax element among the defined levels to the target area indicated by the area designation syntax element (S630).

When the level setting is completed, the video decoding device decodes the corresponding areas based on the level set in each area (S640).

As such, the present invention is configured to set a level or tier for each region rather than setting a single level or tier for a sequence consisting of one or more pictures, so that encoding and decoding optimized for different picture quality characteristics for each region are performed. It becomes possible to implement.

In addition, the present invention records images in all directions, such as 360 video, and decodes and plays images from a specific location (viewport) and images from other locations with differential quality, or decodes and plays only viewport images. It can be used more usefully when desired. A drawing to explain this use example is shown in FIG. 7.

In FIG. 7, the entire area 700 of the 360 video may be composed of a plurality of areas 710, of which #1, #2, #25, #26, #29, and Areas #30 represent the viewport area.

The encoding or decoding control means sets different levels or tiers in the viewport area (#1, #2, #25, #26, #29, and #30) and other areas to encode and encode high or low image quality relative to the viewport area. By allowing decoding to be applied, encoding and decoding of differentiated quality can be implemented.

In addition, the encoding or decoding control means sets the viewport area (#1, #2, #25, #26, #29, and #30) to be decoded to a separate level or tier, so that the viewport area (# 1, #2, #25, #26, #29 and #30) can be decoded and played back.

For example, if the video decoding device supports MPEG-4 AVC Main profile, Level 3 (L3) and enables decoding and playback of L3 or lower, and the viewport area of the 360 video is at the L3 level and the other areas are at the L5 level, By setting the level for the viewport area to L3, decoding and playback of the viewport area only can be implemented.

Figures 8 and 9 are flowcharts for explaining an embodiment of the present invention for specifying a target level or target tier implemented in a video encoding device and a video decoding device. Hereinafter, an embodiment of the present invention for level designation will be described with reference to FIGS. 8 and 9.

The definition syntax element may include a syntax element defining a default level (default definition syntax element) and a syntax element defining one or more extra levels (extra definition syntax element). Here, the default level corresponds to the basic level among the levels that can be assigned to the target area, and the extra level corresponds to the level excluding the default level.

An example of a default definition syntax element is represented in Table 3 below.

seq_parameter_set_rbsp(　) { Descriptor … … default_tier_flag default_level_idc … }

In Table 3 above, default_tier_flag corresponds to the default tier syntax element indicating the value of the default tier to be defined, and default_level_idc corresponds to the default level syntax element indicating the value of the default level to be defined.

An example of an extra definition syntax element is represented in Table 4 below.

pic_parameter_set_rbsp(　) { Descriptor … … num_extra_tier_level_set for( i = 0; i < num_extra_tier_level_set; i++ ) { extra_tier_level_set_id[i] extra_tier_flag[　i　] extra_level_idc[　i　] } … }

In Table 4 above, num_extra_tier_level_set represents the number of extra levels and extra tiers to be additionally defined, extra_tier_level_set_id[i] represents the id of the extra level and extra tier to be additionally defined, and extra_tier_flag[i] is the extra level to be additionally defined. Indicates the tier value, and extra_level_idc[i] represents the value of the extra level to be additionally defined.

Table 4 shows only an example representing a set of additionally defined levels or tiers in the form of an id. However, depending on the embodiment, the set of levels or tiers to be additionally defined may be expressed in the form of an index (idx) (extra_tier_level_set_idx). there is. This equally applies to other tables presented below.

The encoding control means defines a default level using a default definition syntax element and defines one or more extra levels using an extra definition syntax element (S810).

It must be decided which of the defined default levels and extra levels will be used as the target level, and the encoding control means determines this by including a default use syntax element indicating whether to use the default level in the designated syntax element (S820 ).

When the default use syntax element included in the designated syntax element indicates the use of the default level (S830), the video decoding device can use the default level as the target level according to the instructions of the default use syntax element, so the encoding control means is used as the target level. No separate syntax elements or information are added for level determination.

On the other hand, if the default use syntax element does not indicate the use of the default level (S830), the process of selecting a level to use as the target level among one or more extra levels must be performed in the video decoding device, so for this purpose, the encoding control means is A separate syntax element (level designation syntax element) for determining the target level is added to the designation syntax element (S840).

An example of default usage syntax elements and level specification syntax elements is shown in Table 5.

tile_header(　) { Descriptor tile_idx default_tier_level_flag if ( !default_tier_level_flag ) extra_tier_level_set_id … }

In Table 5 above, default_tier_level_flag represents a default usage syntax element implemented in the form of a flag, and extra_tier_level_set_id represents a level (tier) designation syntax element. The definition syntax elements and designated syntax elements determined through these processes are encoded by the encoder 150 and signaled to the video decoding device (S850).

Table 5 is an example of a tile group, and the header corresponding to each slice, tile group, and slice segment may also indicate default usage syntax elements and/or level (tier) designation syntax elements.

As shown in FIG. 9, the decoder 410 parses the definition syntax elements and designated syntax elements from the bitstream signaled from the video encoding device and decodes them (S910).

The decoding control means defines a default level based on a default definition syntax element included in the definition syntax element, and defines one or more extra levels based on an extra definition syntax element included in the definition syntax element (S920).

The decoding control means determines whether the default use syntax element included in the designated syntax element indicates use of the default level (S930), and determines either the default level or the extra level as the target level based on the determination result. .

Specifically, when the default use syntax element indicates the use of the default level, the decoding control means determines a predefined default level as the target level (S940) and sets the target level (default level) in the target area ( S970).

On the other hand, if the default use syntax element does not indicate the use of the default level, the decoding control means determines the level indicated by the level designation syntax element among predefined extra levels as the target level (S960), and places the level in the target area. Set the target level (indicated level among extra levels) (S970). Here, the level designation syntax element is signaled from the video encoding device only when the default use syntax element does not indicate use of the default level.

10 and 11 are flowcharts for explaining an embodiment of the present invention for defining levels or tiers implemented in a video encoding device and a video decoding device. Hereinafter, an embodiment of the present invention regarding the definition of a level or tier will be described with reference to FIGS. 10 and 11.

While the embodiment described through FIGS. 8 and 9 corresponds to an embodiment of which of the two levels to designate to the target area on the premise that the default level and the extra level are defined separately, through FIGS. 10 and 11 The described embodiments correspond to embodiments that selectively define extra levels.

In this embodiment, the definition syntax element may include a syntax element defining the default level (default definition syntax element) and an additional definition syntax element indicating whether an extra level is defined. An example of default definition syntax elements and additional definition syntax elements is represented in Table 6 below.

seq_parameter_set_rbsp(　) { Descriptor … … default_tier_flag default_level_idc extra_tier_level_flag … }

In Table 6 above, default_tier_flag and default_level_flag each correspond to a syntax element indicating the value of the default tier and a syntax element indicating the value of the default level, as described above. extra_tier_level_flag corresponds to an additional definition syntax element that indicates whether an extra level is defined.

As shown in FIG. 10, the encoding control means defines the default level using the default definition syntax element (S1010), and further includes an additional definition syntax element in the definition syntax element to determine whether to define an extra level (S1010). S1020).

When the additional definition syntax element indicates the definition of an extra level (S1030), the encoding control means defines one or more extra levels by further including the extra definition syntax element in the definition syntax element (S1040).

In contrast, when the additional definition syntax element does not indicate the definition of an extra level (S1030), the encoding control means does not additionally define the extra level. That is, the encoding control means does not include extra definition syntax elements in definition syntax elements.

An example of additional definition syntax elements and extra definition syntax elements is represented in Table 7 below.

pic_parameter_set_rbsp(　) { Descriptor … … if (extra_tier_level_flag) { num_extra_tier_level_set_minus1 for( i = 0; i <= num_extra_tier_level_set_minus1; i++ ) { extra_tier_level_set_id[i] extra_tier_flag[　i　] extra_level_idc[　i　] } } … }

As shown in Table 7, an extra level (extra_level_flag) or an extra tier (extra_tier_flag) is additionally defined only when the additional definition syntax element (extra_tier_level_flag) indicates the definition of an extra level or tier.

Table 7 shows only an example of a set of additionally defined levels or tiers in the form of an id. However, depending on the embodiment, the set of levels or tiers to be additionally defined may be expressed in the form of an index (idx) (extra_tier_level_set_idx). there is.

Table 7 shows an example where the additional definition syntax element is implemented in the form of a flag to dichotomously indicate whether or not an extra level is defined. However, depending on the embodiment, the additional definition syntax element is the number of extra levels to be defined (n, n is 0) It may also be implemented in the form of information indicating an integer or more.

In the case of an embodiment in which the additional definition syntax element indicates the number of extra levels, the encoding control means does not define the extra level for the additional definition syntax element indicating 0 (n = 0), but 1 or more (n ≥ 1) ), the number of extra levels corresponding to the additional definition syntax element indicating (the number indicated by the additional definition syntax element) can be defined.

The definition syntax elements and designated syntax elements determined through the above-described processes are encoded by the encoder 150 and signaled to the video decoding device (S1050).

As shown in FIG. 11, the decoder 410 parses the definition syntax elements and designated syntax elements from the bitstream signaled from the video encoding device and decodes them (S1110).

The decoding control means defines the default level based on the default definition syntax element included in the definition syntax element (S1120), and determines whether the additional definition syntax element included in the definition syntax element indicates the definition of an extra level (S1130). .

When the additional definition syntax element indicates the definition of an extra level, the decoding control means defines one or more extra levels based on the extra definition syntax element (S1140). Here, the extra definition syntax element is signaled from the video encoding device only when the additional definition syntax element indicates the definition of the extra level.

If the additional definition syntax element is implemented in a form that indicates the number of extra levels, the decoding control means defines the extra level in the same number as the number indicated by the additional definition syntax element, and sets the default level according to the instructions of the default usage syntax element. Alternatively, the above-described embodiment may be performed using any one of the extra levels as the target level.

Conversely, if the additional definition syntax element does not indicate the definition of an extra level, the decoding control means terminates the level definition process without defining the extra level (since the extra definition syntax element is not signaled) and determines whether the default level is used. The default level can be set in the target area without the above-described process of determining . Here, the default level corresponds to the target level.

Depending on the embodiment, the above-described process of determining whether to use the default level may be performed even when the extra level is not defined because the additional definition syntax element does not indicate the definition of the extra level. For example, if the default use syntax element does not indicate the use of the default level, the encoding control means signals the level actually applied to the target area by including it in the designated syntax element, and the decoding control means signals this actually applied level (signaled It can be set in the target area using the actual level, applied level) as the target level.

An embodiment of determining the level actually applied to the target area, that is, the application level, as the target level can be implemented without determining additional definition syntax elements. That is, the encoding control means signals only the default definition syntax element without additional definition syntax elements, and the decoding control means defines the default level based on the signaled default definition syntax element and then determines the default use syntax element.

The decoding control means sets a predefined default level in the target area if the default use syntax element indicates use of the default level, and sets the signaled application level to the target area if the default use syntax element does not indicate use of the default level. It can be set to .

An example of syntax elements used in this embodiment is shown in Tables 8 and 9 below.

seq_parameter_set_rbsp(　) { Descriptor … … default_tier_flag default_level_idc … }

As expressed in Table 8, the default level and/or default tier may be defined using default definition syntax elements (default_tier_flag, default_level_idc).

tile_header(　) { Descriptor tile_idx default_tier_level_flag if (!default_tier_level_flag) { tier_flag level_idc } … }

As expressed in Table 9, whether to use the default level and/or tier may be determined by the default use syntax element (default_tier_level_flag). If the default use syntax element indicates the use of a default level and/or tier, a predefined default level and/or default tier may be set in the target area. Alternatively, if the default usage syntax element does not indicate the use of a default level and/or tier, the application level (level_idc) and/or application tier (tier_flag) may be set in the target area for setting the level and/or tier. You can.

The above description is merely an illustrative explanation of the technical idea of the present embodiment, and those skilled in the art will be able to make various modifications and variations without departing from the essential characteristics of the present embodiment. Accordingly, the present embodiments are not intended to limit the technical idea of the present embodiment, but rather to explain it, and the scope of the technical idea of the present embodiment is not limited by these examples. The scope of protection of this embodiment should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of rights of this embodiment.

410: Decryption unit 420: Inverse quantization unit
430: Inverse transformation unit 440: Prediction unit
442: Intra prediction unit 444: Inter prediction unit
450: adder 460: filter

Claims (8)

In an apparatus for decoding level information for each of a plurality of regions divided from each of the pictures included in a video sequence from a supplemental enhancement information (SEI) message of a bitstream,
The level information is information for defining the level of decoding ability of the device, including the maximum resolution that the device can process,
The device includes at least one processor,
The processor,
Decrypt the default level information to define at least one default level,
Decode a first flag indicating whether extra information is defined, wherein the extra information is related to the default level information and is used to set a level corresponding to the area,
Decrypt the additional information according to the first flag,
Characterized in that setting a level corresponding to the area using the default level information and the additional information. According to paragraph 1,
The default level information indicates at least one default level applicable to pictures included in the video sequence,
The additional information includes number information for indicating the number of additional levels applicable to the area, and additional level syntax elements equal to the number of additional levels, wherein the additional level syntax elements each represent an additional level. device to do. According to paragraph 2,
The processor,
Decode one or more specified syntax elements for selecting one of the at least one default level and the additional levels from a header containing syntax elements related to the region,
The device is characterized in that, setting a level corresponding to the region among the at least one default level and the additional levels using the one or more specified syntax elements. According to paragraph 3,
The one or more specified syntax elements include:
a second flag to indicate whether the at least one default level applies to the area, and
An indicator that is decrypted when it is indicated by the second flag that the at least one default level does not apply and indicates one of the additional levels
A device comprising: According to paragraph 4,
The processor,
When the second flag indicates that the at least one default level is not applied, set the additional level indicated by the indicator among the additional levels to the level corresponding to the area,
When the second flag indicates that the at least one default level is applied, the device sets the at least one default level to a level corresponding to the area. According to paragraph 2,
The device is characterized in that the default level information is decoded from the header of the video sequence, and the additional information is decoded from the header of the picture level. In a video encoding device for encoding level information for each of a plurality of regions divided from each of the pictures included in a video sequence into an SEI (supplemental enhancement information) message of a bitstream,
The level information is information for defining the level of decoding ability of the video decoding device, including the maximum resolution that the video decoding device can process,
The video encoding device includes at least one processor,
The processor,
Encode default level information to define at least one default level,
Encoding a first flag indicating whether extra information is defined, wherein the extra information is related to the default level information and is used to set a level corresponding to the area,
According to the first flag, the video decoding device encodes the additional information so that the video decoding device sets a level corresponding to the region using the default level information and the additional information. In a method for storing a bitstream containing level information for each of a plurality of regions divided from each of the pictures included in a video sequence,
The level information is information for defining the level of decoding ability of the video decoding device, including the maximum resolution that the video decoding device can process,
The above method is,
generating the bitstream by encoding the level information into supplemental enhancement information (SEI); and
comprising storing the bitstream in a non-transitory recording medium,
Encoding the level information is,
A first flag indicating whether extra information is defined, and encoding default level information to define at least one default level, the additional information being related to the default level information and Used to set the level corresponding to the area; and
Encoding the additional information so that the video decoding device sets a level corresponding to the region using the default level information and the additional information according to the first flag.
A method comprising:

Images