RU2802368C2 - Syntax elements for video coding or decoding - Google Patents

Abstract

FIELD: video coding.

SUBSTANCE: invention relates to syntax elements for encoding or decoding video. The result is that efficient coding tool signalling is the transfer of information representing the coding tools used for coding, for example, from an encoder device to a receiver device (for example, a decoder or display), so that the appropriate tools are used for the decoding stage. These tools include new segmentation modes, new intra-prediction modes, increased flexibility for sampled adaptive bias, and a new luminance compensation mode for bi-prediction blocks based on a single flag capable of determining whether independent coding trees or general coding trees are used for coding luminance and chrominance blocks, while the single flag mentioned above is inserted into the high-level syntax elements of the frame's encoded data.

EFFECT: increasing the coding efficiency.

14 cl, 12 dwg, 11 tbl

Description

Field of technology to which the invention relates

At least one of the present embodiments generally relates to syntax elements for encoding or decoding video.

State of the art

To achieve high compression efficiency, image and video coding schemes typically use prediction and transform to exploit spatial and temporal redundancy in video content. Typically, intra or inter prediction is used to exploit intra- or inter-frame correlation, then the differences between the original block and the predicted block, often referred to as prediction errors or prediction residuals, are transformed, quantized and entropy encoded . To reconstruct video, the compressed data is decoded through inverse processes corresponding to entropy encoding, quantization, transform, and prediction.

The essence of the invention

According to a first aspect of at least one embodiment, a video signal is presented, wherein the video signal is formatted to include information according to a video coding standard and comprises a bitstream comprising video content and high-level syntax information, wherein said high-level syntax information comprises at least , one parameter of a parameter representing the segmentation type, a parameter representing the intra-prediction mode type, a parameter representing the adaptation of block sizes for the contour filter based on the sampled adaptive offset, and a parameter representing the illumination compensation mode for the bi-prediction blocks.

According to a second aspect of at least one embodiment, a storage medium is provided, wherein the storage medium contains video signal data encoded thereon, wherein the video signal is formatted to include information according to a video encoding standard and contains a bitstream containing video content and high-level syntax information wherein said high-level syntax information comprises at least one parameter of a parameter representing a segmentation type, a parameter representing an intra prediction mode type, a parameter representing an adaptation of block sizes for the loop filter based on the sampled adaptive offset, and a parameter representing a compensation mode illumination for biforecasting blocks.

According to a third aspect of at least one embodiment, equipment is provided, wherein the equipment includes a video encoder for encoding frame data for at least one block in the frame, wherein the encoding is performed using at least one parameter of a parameter, representing a type of segmentation, a parameter representing a type of intra-prediction mode, a parameter representing block size adaptation for the contour filter based on the sampled adaptive offset, and a parameter representing a light compensation mode for the bi-prediction blocks, and wherein said parameters are inserted into high-level syntax elements of the encoded data frames.

According to a fourth aspect of at least one embodiment, a method is provided, the method comprising encoding frame data for at least one block in the frame, wherein the encoding is performed using at least one parameter of a parameter representing a segmentation type, a parameter representing the intra-prediction mode type, a parameter representing the adaptation of block sizes for the contour filter based on the sampled adaptive offset, and a parameter representing the light compensation mode for the bi-prediction blocks, and inserting parameters into high-level syntax elements of the encoded frame data.

According to a fifth aspect of at least one embodiment, equipment is provided, wherein the equipment includes a video decoder for decoding frame data for at least one block in the frame, wherein the decoding is performed using at least one parameter of the parameter, representing the type of segmentation, a parameter representing the type of the intra prediction mode, a parameter representing the adaptation of block sizes for the contour filter based on the sampled adaptive offset, and a parameter representing the illumination compensation mode for the bi-prediction blocks, and wherein said parameters are obtained from high-level syntax elements of the encoded data frames.

According to a sixth aspect of at least one embodiment, a method is provided, the method comprising obtaining parameters from high-level syntax elements of encoded frame data and decoding the frame data for at least one block in the frame, wherein the decoding is performed using, at least one parameter of a parameter representing a segmentation type, a parameter representing an intra-prediction mode type, a parameter representing an adaptation of block sizes for the contour filter based on the sampled adaptive offset, and a parameter representing a light compensation mode for the bi-prediction blocks.

According to a seventh aspect of at least one embodiment, a non-transitory computer-readable medium is provided, wherein the non-transitory computer-readable medium contains data content generated according to the third or fourth aspect.

According to a seventh aspect of at least one embodiment, there is provided a computer program containing program code instructions executable by a processor, wherein the computer program implements steps of a method according to at least the fourth or sixth aspect.

According to an eighth aspect of at least one embodiment, there is provided a computer program product that is stored on a non-transitory computer readable medium and contains program code instructions executable by a processor, wherein the computer program product implements the method steps of at least the fourth or sixth aspect.

Brief description of drawings

Fig. 1 illustrates an example of a video encoder 100, such as a High Efficiency Video Coding (HEVC) encoder.

Fig. 2 illustrates a block diagram of an example video decoder 200, such as a HEVC decoder.

Fig. 3 illustrates an example of a coding tree unit and a compressed region coding tree.

Fig. 4 illustrates an example of dividing a CTU into coding units, prediction units, and transformation units.

Fig. 5 illustrates an example of a quadtree plus binary tree (QTBT) representation of a CTU.

Fig. 6 illustrates an example superset of coding unit segmentation.

Fig. 7 illustrates an example of an intra prediction mode with two reference layers.

Fig. 8 illustrates an exemplary embodiment of a compensation mode for bidirectional illumination compensation.

Fig. 9 illustrates the interpretation of bt-split-flag according to an exemplary embodiment.

Fig. 10 illustrates a block diagram of an example system in which various aspects and embodiments are implemented.

Fig. 11 illustrates a flowchart of an example encoding method according to an embodiment using the new encoding tools.

Fig. 12 illustrates a flowchart of an example portion of a decoding method according to an embodiment using the new encoding tools.

Carrying out the invention

In at least one embodiment, use of the new encoding tools described below results in increased encoding efficiency. In an exemplary embodiment, effective signaling of these encoding tools is the transmission of information representing the encoding tools used for encoding, for example, from an encoder device to a receiving device (for example, a decoder or display), such that the corresponding tools are used for the decoding stage . These tools include new segmentation modes, new intraprediction modes, increased flexibility for sampled adaptive offset, and a new light compensation mode for biprediction blocks. Therefore, the proposed new syntax, which collects multiple encoding tools, provides more efficient video encoding.

Fig. 1 illustrates an example of a video encoder 100, such as a High Efficiency Video Coding (HEVC) encoder. Fig. 1 may also illustrate an encoder that makes improvements to the HEVC standard, or an encoder using technologies similar to HEVC, such as a JEM encoder being developed through JVET. ).

In this application, the terms "recovered" and "decoded" may be used interchangeably, the terms "encoded" or "coded" may be used interchangeably, and the terms "image", "frame" and "cine frame" may be used interchangeably . Typically, but not necessarily, the term "reconstructed" is used on the encoder side, while "decoded" is used on the decoder side.

Before encoding, the video sequence may go through pre-encoding processing (101). For example, it is performed by applying a color transform to the input color frame (eg, converting from RGB 4:4:4 to YCbCr 4:2:0) or performing a re-conversion of components of the input frame to produce signal propagations that are more resistant to compression (eg , using frequency correction of the histogram of one of the color components). Metadata may be associated with preprocessing and attached to the bitstream.

In HEVC, in order to encode a video sequence using one or more frames, the frame is segmented (102) into one or more slices, where each slice may include one or more slice segments. . A segment of serial macroblocks is organized into coding units, prediction units, and transformation units. The HEVC specification distinguishes between "blocks" and "units", with a "block" addressing a specific region in the sample array (eg luminance, Y) and a "unit" including co-located blocks of all encoded color components (Y, Cb , Cr or monochrome), syntactic elements and predictive data that are associated with blocks (for example, motion vectors).

For HEVC encoding, a frame is segmented into square-shaped coding tree blocks (CTBs) with a configurable size, and a sequential set of coding tree blocks is grouped into a series of sequential macroblocks. The Coding Tree Unit (CTU) contains the CTB of coded color components. The CTB represents a segmentation root of a quadtree into coding blocks (CBs), and a coding block may be segmented into one or more prediction blocks (PBs) and forms a segmentation root of a quadtree into transform blocks (TB). According to the encoding unit, the prediction unit and the transform unit, a coding unit (CU) includes prediction units (PUs) and a tree-structured set of transformation units (TUs), the PU includes prediction information for all color components, and the TU includes syntactic structure of residual coding for each color component. The CB, PB and TB size of the luminance signal component is applied to the corresponding CU, PU and TU. In this application, the term "block" may be used to mean, for example, any of CTU, CU, PU, TU, CB, PB and TB. In addition, "block" can also be used to mean macroblock and segment, as specified in H.264/AVC or other video coding standards, and more generally, to mean an array of data of various sizes.

In the example encoder 100, a frame is encoded by encoder elements as described below. The frame to be encoded is processed in CU units. Each CU is encoded using internal or mutual mode. When the CU is encoded in intra mode, it performs intra prediction (160). In reciprocal mode, motion estimation (175) and motion compensation (170) are performed. The encoder determines (105) which of the internal mode or the inter-mode should be used for encoding the CU, and indicates the intra/inter-mode decision via a prediction mode flag. The prediction residuals are calculated by subtracting (110) the predicted block from the original image block.

CUs in internal mode are predicted from reconstructed adjacent samples in an identical series of consecutive macroblocks. A set of 35 intraprediction modes are available in HEVC, including DC, flat and 33 angular prediction modes. The internal prediction reference is reconstructed from the row and column adjacent to the current block. The reference element is twice the size of the block in the horizontal and vertical directions using available samples from previously reconstructed blocks. When the angle prediction mode is used for intra prediction, reference samples can be copied along the direction indicated by the angle prediction mode.

The applicable intra-luminance signal prediction mode for the current block may be encoded using two different options. If an applicable mode is included in a constructed list of three most probable modes (MPMs), the mode is signaled by an index in the MPM list. Otherwise, the mode is signaled by binarizing a fixed length mode index. The three most probable modes are extracted from the internal prediction modes of the top and left neighboring blocks.

For a mutual CU, the corresponding encoding block is further segmented into one or more prediction blocks. Inter-prediction is performed at the PB level, and the corresponding PU contains information regarding how inter-prediction is performed. Motion information (eg, motion vector and reference frame index) may be signaled in two ways, namely, “fusion mode” and “advanced motion vector prediction (AMVP)”.

In merge mode, the video encoder or decoder assembles a candidate list based on the blocks already encoded, and the video encoder signals an index for one of the candidates in the candidate list. At the decoder side, the motion vector (MV) and the reference frame index are reconstructed based on the signaled candidate.

In AMVP, a video encoder or decoder assembles candidate lists based on motion vectors determined from already encoded blocks. The video encoder then signals an index in the candidate list to identify a motion vector predictor (MVP), and signals a motion vector difference (MVD). At the decoder side, the motion vector (MV) is reconstructed as MVP+MVD. The applicable reference frame index is also explicitly encoded in the PU syntax for AMVP.

The prediction residuals are then transformed (125) and quantized (130), which includes at least one embodiment for adapting a chrominance signal quantization parameter described below. Transformations are, in general, based on separable transformations. For example, the DCT transform is first applied in the horizontal direction, then in the vertical direction. In recent codecs such as JEM, the transforms used in both directions (e.g. DCT in one direction, DST in the other) can be different, resulting in a wide variety of 2D transforms, whereas in earlier codecs, a variety of 2D transforms for a given size block is usually limited.

The quantized transform coefficients, as well as motion vectors and other syntactic elements, are entropy encoded (145) to output the bit stream. The encoder can also skip the transform and apply quantization directly to the unconverted residual signal based on the 4x4 TU. The encoder can also bypass both conversion and quantization, i.e. the remainder is encoded directly without applying a conversion or quantization process. With direct PCM encoding, prediction is not applied and samples of coding units are directly encoded into the bitstream.

The encoder decodes the coded block to provide a reference element for additional predictions. The quantized transform coefficients are dequantized (140) and back-transformed (150) to decode the prediction residuals. By combining (155) the decoded prediction residuals and the predicted block, the image block is reconstructed. In-loop filters (165) are applied to the reconstructed frame, for example, to perform deblocking/SAO (sampled adaptive offset) filtering to reduce encoding artifacts. The filtered image is stored in the reference frame buffer (180).

Fig. 2 illustrates a block diagram of an example video decoder 200, such as a HEVC decoder. In the example of decoder 200, the bit stream is decoded by decoder elements, as described below. Video decoder 200 generally performs a decoding pass inverse to the encoding pass as described in FIG. 1, which performs video decoding as part of video data encoding. Fig. 2 may also illustrate a decoder that makes improvements to the HEVC standard, or a decoder using technologies similar to HEVC, such as a JEM decoder.

Specifically, the decoder input includes a video bit stream that may be generated by video encoder 100. The bit stream is first entropy decoded (230) to obtain transform coefficients, motion vectors, frame segmentation information, and other encoded information. Frame segmentation information indicates the size of the CTU and the manner in which the CTU is partitioned into CUs and possibly into PUs, if applicable. The decoder can therefore divide (235) a frame into CTUs, and each CTU into CUs, according to the segmentation information of the decoded frames. The transform coefficients are dequantized (240), which includes at least one embodiment for adapting a chrominance signal quantization parameter described below, and dequantized (250) to decode the prediction residuals.

By combining (255) the decoded prediction residuals and the predicted block, the image block is reconstructed. The predicted block may be obtained (270) from intra-prediction (260) or motion-compensated prediction (ie, inter-prediction) (275). As described above, AMVP and pooling mode-based technologies can be used to extract motion vectors for motion compensation, which can use interpolation filters to compute interpolated values for sub-integer reference block samples. In-loop filters (265) are applied to the reconstructed image. The filtered image is stored in the reference frame buffer (280).

The decoded frame may further undergo post-decoding processing (285), such as inverse color conversion (eg, conversion from YCbCr 4:2:0 to RGB 4:4:4) or inverse reconversion, which performs the inversion of the reconversion process performed at processing before encoding (101). Post-decoding processing may use metadata extracted during pre-encoding processing and signaled in the bitstream.

Fig. 3 illustrates an example of a coding tree unit and a compressed region coding tree. In the HEVC video compression standard, a frame is divided into so-called coding tree units (CTUs), which are typically 64x64, 128x128, or 256x256 pixels in size. Each CTU is represented by an encoding tree in a compressed region. It represents a division into a quadtree of CTUs, in which each leaf is called a "coding unit (CU)".

Fig. 4 illustrates an example of dividing a CTU into coding units, prediction units, and transformation units. For each CU, some internal or inter-prediction parameters (prediction information) are then provided. To do this, it is spatially segmented into one or more prediction units (PUs), with each PU assigned some prediction information. The internal or intercoding mode is assigned at the CU level.

New emerging video compression tools that include a representation of encoding tree units in a compressed domain are proposed to represent frame data in a more flexible manner. The advantage of this more flexible coding tree representation is that it provides improved compression efficiency compared to the CU/PU/TU layout of the HEVC standard.

Fig. 5 illustrates an example of a quadtree plus binary tree (QTBT) representation of a CTU. The quadtree plus binary tree (QTBT) coding tool provides this increased flexibility. QTBT consists of a coding tree in which coding units can be partitioned as a quadtree and as a binary tree. The coding unit partitioning is determined at the encoder side through a rate-based distortion optimization procedure that defines a QTBT representation of the CTU with a minimum rate-based distortion cost function. In QTBT technology, the CU has a square or rectangular shape. The coding unit size is always a power of 2 and typically ranges from 4 to 128. In addition to this variety of rectangular shapes for the coding unit, this CTU representation has the following different characteristics compared to HEVC. QTBT decomposition of a CTU consists of two stages: first, the CTU is decomposed into a quadtree, then each leaf of the quadtree can be further partitioned in binary form. This is illustrated on the right side of the drawing, in which the solid lines represent the lines for the phases of the quadtree decomposition, and the dotted lines represent the binary decomposition that is spatially embedded in the leaves of the quadtree. In the inner series of successive macroblocks, the segmentation structure into blocks of luma signals and chrominance signals is separated and determined independently. CU segmentation into prediction units or conversion units is no longer used. In other words, each coding unit systematically consists of one prediction unit (such as a 2Nx2N segment of prediction units) and one transformation unit (without partitioning into a transformation tree).

Fig. 6 illustrates an example superset of coding unit segmentation. In asymmetric binary tree (ABT) partitioning modes, a rectangular encoding unit with size (width and height), which should be split through one of the asymmetric binary partitioning modes, e.g. HOR_UP (horizontally up), should result in 2 encoding subunits with corresponding rectangular dimensions And In addition, so-called ternary CU tree segmentation can be used, leading to the set of possible segments shown in FIG. 5. Ternary tree consists of splitting a CU into a tree-like sub-CU with size (1/4, 1/2, 1/4) relative to the parent CU in the orientation in question (eg: HOR_TRIPLE for horizontal splitting mode).

Significant improvements in coding efficiency are achieved using one or more embodiments of the new topologies described above.

Fig. 7 illustrates an example of an intra prediction mode with two reference layers. In fact, new internal prediction modes are covered in improved syntax. The first new internal doom mode is called "multiple reference internal prediction (MRIP)." This tool makes it easy to use multiple reference layers for internal block prediction. Typically, 2 reference layers are used for intraprediction, with each reference layer consisting of a left reference array and a top reference array. Each reference layer is constructed by replacing reference samples and then pre-filtered.

Using each reference layer, a prediction is constructed for the block, typically as done in HEVC or JEM. The final prediction is formed as a weighted average of the predictions made from the two reference layers. Predictions from the closest reference layer are weighted higher than predictions from the farthest layer. Typical weighting factors used are 3 and 1.

In the last part of the prediction process, multiple reference layers can be used to smooth the boundary samples for certain prediction modes, using a mode-dependent process.

In an exemplary embodiment, the video compression tool also includes an adaptive block size for sampled adaptive offset (SAO). This tool is a loop filter specified in HEVC. In HEVC, the SAO process classifies the reconstructed samples of one block into several classes, and the samples belonging to some classes are adjusted with biases. SAO parameters are encoded per block or can be inherited only from the left or top adjacent blocks.

Additional improvements are offered by setting the SAO palette mode. An SAO palette retains identical SAO parameters per block, but blocks can inherit these parameters from all other blocks. This provides greater flexibility for SAO by expanding the range of possible SAO parameters for a single block. The SAO palette consists of a set of different SAO parameters. For each block, an index is encoded to indicate which SAO parameter should be used.

Fig. 8 illustrates an exemplary embodiment of a compensation mode for bidirectional illumination compensation. Illumination Compensation (IC) provides the ability to correct block prediction samples (SMC) obtained through motion compensation by possibly considering spatial or temporal variation in local illumination.

S _IC =a _i . S _MC +b _i

In the case of bi-prediction, the IC parameters are estimated using samples from two reference CU samples, as illustrated in FIG. 8. Firstly, the IC parameters (a, b) are estimated between two reference blocks, then the IC parameters (ai, bi) i=0.1 between the reference and current blocks are extracted as follows:

Where:

If the CU dimension is less than or equal to 8 in width or height, the IC parameters are estimated using a unidirectional process. When the CU size is greater than 8, the choice between using IC parameters extracted from a bidirectional or unidirectional process is made by selecting IC parameters that minimize the difference in the average of the IC-compensated reference blocks. Bidirectional Optical Flow (BIO) is enabled using the Bidirectional Light Compensation tool.

The plurality of tools introduced above are selected and used by the video encoder 100 and must be obtained and used by the video decoder 200. To this end, information representing the use of these tools is carried in a coded bit stream generated by the video encoder and obtained by the video decoder 200 at form of high-level syntactic information. To this end, appropriate syntax is specified and described later in this document. This syntax, which brings together many tools, provides increased coding efficiency.

This syntax structure uses the HEVC syntax structure as a basis and includes some additional syntax elements. Syntactic elements reported in italic bold font and highlighted in gray in the following tables correspond to additional syntactic elements according to the exemplary embodiment. It should be noted that syntactic elements may have forms or names that differ from those shown in the syntax tables while simultaneously processing identical functions. Syntax elements may be permanently located at different levels, for example, some syntax elements may be located in a sequence parameter set (SPS), and some syntax elements may be located in a frame parameter set (PPS).

Table 1 below illustrates a sequence parameter set (SPS) and introduces new syntax elements inserted into this parameter set, according to at least one exemplary embodiment, more specifically: multi_type_tree_enabled_primary, log2_min_cu_size_minus2, log2_max_cu_size_minus4, log2_max_tu_size_minus2, sep_tree_mode_intra, multi_type _tree_enabled_secondary, sps_bdip_enabled_flag, sps_mrip_enabled_flag, use_erp_aqp_flag, use_high_perf_chroma_qp_table, abt_one_third_flag, sps_num_intra_mode_ratio.

pic_parameter_set_rbsp() { Descriptor sps_video_parameter_set_id u(4) sps_max_sub_layers_minus1 u(3) sps_temporal_id_nesting_flag u(1) profile_tier_level(1, vps_max_sub_layers_minus1) sps_seq_parameter_set_id ue(v) chroma_format_idc ue(v) if(chroma_format_idc==3) { separate_colour_plane_flag u(1) pic_width_in_luma_samples ue(v) pic_height_in_luma_samples ue(v) conformance_window_flag u(1) if(conformance_window_flag) { conf_win_left_offset ue(v) conf_win_right_offset ue(v) conf_win_top_offset ue(v) conf_win_bottom_offset ue(v) } bit_depth_luma_minus8 ue(v) bit_depth_chroma_minus8 ue(v) log2_max_pic_order_cnt_lsb_minus4 ue(v) sps_sub_layer_ordering_info_present_flag u(1) for (i=(sps_sub_layer_ordering_info_present_flag ? 0 : sps_max_sub_layers_minus1)
- i<=sps_max_sub_layers_minus1; i++){ sps_max_dec_pic_buffering_minus1[i] ue(v) sps_max_num_reorder_pics[i] ue(v) sps_max_latency_increase_plus1[i] ue(v) } multi_type_tree_enabled_primary ue(v) log2_min_cu_size_minus2 ue(v) log2_max_cu_size_minus4 ue(v) log2_max_tu_size_minus2 ue(v) sep_tree_mode_intra u(1) if(sep_tree_mode_intra) multi_type_tree_enabled_secondary ue(v) sample_adaptive_offset_enabled_flag u(1) pcm_enabled_flag u(1) if(pcm_enabled_flag) { pcm_sample_bit_depth_luma_minus1 u(4) pcm_sample_bit_depth_chroma_minus1 u(4) log2_min_pcm_luma_coding_block_size_minus3 ue(v) log2_diff_max_min_pcm_luma_coding_block_size ue(v) pcm_loop_filter_disabled_flag u(1) } num_short_term_ref_pic_sets ue(v) for (i=0; i<num_short_term_ref_pic_sets; i++) st_ref_pic_set(i) long_term_ref_pics_present_flag u(1) if(long_term_ref_pics_present_flag) { num_long_term_ref_pics_sps ue(v) for (i=0; i<num_long_term_pics; i++) { lt_ref_pic_poc_lsb_sps [i] u(v) used_by_curr_pic_lt_sps_flag[i] u(1) } } sps_temporal_mvp_enabled_flag u(1) vui_parameters_present_flag u(1) if(vui_parameters_present_flag) vui_parameters() use_imv u(1) fruc_merge_mode u(1) if(fruc_merge_mode) { idx_num_template ue(v) idx_num_template_ic ue(v) fruc_template_affine u(1) } mode_bilateral_TM ue(v) optical_flow_filtering ue(v) atmvp_flag u(1) sps_lic_enabled_flag u(1) sps_bidir_ic_present_flag u(1) sps_use_intra_emt u(1) sps_use_inter_emt u(1) sps_nsst_enabled_flag u(1) sps_cross_component_prediction_enabled_flag u(1) sps_intra_4tap_filter_enabled_flag u(1) sps_intra_boundary_filter_enabled_flag u(1) sps_obmc_flag u(1) if(sps_obmc_flag) obmc_blk_size ue(v) obmc_for_sub_block u(1) sps_affine_enabled_flag u(1) use_NL_Bil_flag u(1) sps_bdip_enabled_flag u(1) sps_mrip_enabled_flag u(1) num_predicted_coef_signs u(4) mc_frame_pad u(1) use_erp_aqp_flag u(1) use_chroma_qp_table if(use_chroma_qp_table) use_high_perf_chroma_qp_table u(1) log2_intra131modes_min_area_minus6 ue(v) log2_intra65modes_min_area_minus4 ue(v) abt_one_third_flag u(1) sps_num_intra_mode_ratio u(1) if(sps_range_extension_flag) sps_range_extension() if(sps_multilayer_extension_flag) sps_multilayer_extension()/* specified in Appendix F */ if(sps_3d_extension_flag) sps_3d_extension()/* specified in Appendix I */ if(sps_extension_5bits) while(more_rbsp_data()) sps_extension_data_flag u(1) rbsp_trailing_bits() }

Table 1. Modified set of sequence parameters

New syntax elements are specified as follows:

- use_high_perf_chroma_qp_table: This syntax element specifies a chroma QP table used to extract the QP used in decoding the chroma components associated with a run of macroblocks as a function of the underlying chroma QP associated with the chroma components of the run of slices in question.

- multi_type_tree_enabled_primary: This syntax element specifies the type of segmentation enabled in the encoded series of successive macroblocks. In at least one implementation, this syntax element is signaled at the sequence level (in the SPS) and thereby applies to all coded runs of consecutive macroblocks that use this SPS. For example, the first value of this syntax element allows segmentation into quadtree and binary tree (QTBT) (NO-SPLIT, QT-SPLIT, HOR and VER in Fig. 5), the second value allows segmentation into QTBT plus ternary tree (TT) (NO-SPLIT, QT-SPLIT, HOR, VER, HOR_TRIPLE and VER_TRIPLE from Fig. 5), the third value allows segmentation into QTBT plus asymmetric binary tree (ABT) (NO-SPLIT, QT-SPLIT, HOR, VER, HOR -UP, HOR_DOWN, VER_LEFT and VER_RIGHT of Fig. 5), the fourth value allows segmentation into QTBT plus TT plus ABT (all partitioning cases of Fig. 5).

- sep_tree_mode_intra: This syntax element indicates whether or not a separate coding tree is used for luma and chroma blocks. When sep_tree_mode_intra is equal to 1, the luma and chrominance blocks use independent encoding trees, and therefore the segmentation of the luma and chrominance blocks is independent. In at least one implementation, this syntax element is signaled at the sequence level (in the SPS) and thereby applies to all coded runs of consecutive macroblocks that use this SPS.

- multi_type_tree_enabled_secondary: This syntax element indicates that the segmentation type is enabled for chroma blocks in a coded series of consecutive macroblocks. The values that this syntax element can take are typically identical to the values of the multi_type_tree_enabled_primary syntax element. In at least one implementation, this syntax element is signaled at the sequence level (in the SPS) and thereby applies to all coded runs of consecutive macroblocks that use this SPS.

- sps_bdip_enabled_flag: This syntax element indicates that bidirectional intra prediction, as described in JVET-J0022, is enabled in the coded runs of consecutive macroblocks contained in the coded video sequence in question.

- sps_mrip_enabled_flag: This syntax element indicates that the multiple reference intra prediction tool is used when decoding the coded runs of consecutive macroblocks contained in the coded video bitstream in question.

- use_erp_aqp_flag: This syntax element indicates whether or not spatially adaptive quantization is enabled for VR360 ERP content, as described in JVET-J0022, in a coded series of consecutive macroblocks.

- abt_one_third_flag: This syntax element indicates whether or not asymmetric segmentation into two segments with a horizontal or vertical dimension of 1/3 and 2/3 or 2/3 and 1/3, respectively, of the horizontal or vertical dimension of the initial CU is activated, in encoded series of successive macroblocks.

- sps_num_intra_mode_ratio: This syntax element specifies how to extract the internal prediction number used for block sizes equal to a multiple of 3 in width or height.

In addition, the following three syntax elements are used to control the size of coding units (CU) and transformation units (TU). In at least one implementation, these syntax elements are signaled at the sequence level (in the SPS) and thereby apply to all coded runs of consecutive macroblocks that use that SPS.

- log2_min_cu_size_minus2 indicates the minimum CU size.

- log2_max_cu_size_minus4 indicates the maximum CU size.

- log2_max_tu_size_minus2 indicates the maximum TU size.

In an embodiment, the new syntax elements introduced above are not introduced in a sequence parameter set, but in a frame parameter set (PPS).

Table 2 below illustrates a sequence parameter set (SPS) and introduces new syntax elements inserted into this parameter set, according to at least one exemplary embodiment, more specifically:

- slice_sao_size_id: This syntax element specifies the size of blocks for which SAO is applied for a coded series of consecutive macroblocks.

- slice_bidir_ic_enable_flag: Indicates whether or not bidirectional light compensation is enabled for a coded series of consecutive macroblocks.

slice_segment_header() { Descriptor first_slice_segment_in_pic_flag u(1) if(nal_unit_type>=BLA_W_LP && nal_unit_type<=RSV_IRAP_VCL23) no_output_of_prior_pics_flag u(1) slice_pic_parameter_set_id ue(v) if(!first_slice_segment_in_pic_flag){ if(dependent_slice_segments_enabled_flag) dependent_slice_segment_flag u(1) slice_segment_address u(v) } if(!dependent_slice_segment_flag){ for (i=0; i<num_extra_slice_header_bits; i++) slice_reserved_flag[i] u(1) slice_type ue(v) if(output_flag_present_flag) pic_output_flag u(1) if(separate_colour_plane_flag==1) color_plane_id u(2) if(nal_unit_type !=IDR_W_RADL && nal_unit_type !=IDR_N_LP) { slice_pic_order_cnt_lsb u(v) short_term_ref_pic_set_sps_flag u(1) if(!short_term_ref_pic_set_sps_flag) st_ref_pic_set(num_short_term_ref_pic_sets) else if(num_short_term_ref_pic_sets>1) short_term_ref_pic_set_idx u(v) if(long_term_ref_pics_present_flag) { if(num_long_term_ref_pics_sps>0) num_long_term_sps ue(v) num_long_term_pics ue(v) for(i=0; i<num_long_term_sps+num_long_term_pics; i++) { if(i<num_long_term_sps) { if(num_long_term_ref_pics_sps>1) lt_idx_sps[i] u(v) } else { poc_lsb_lt[i] u(v) used_by_curr_pic_lt_flag[i] u(1) } delta_poc_msb_present_flag[i] u(1) if(delta_poc_msb_present_flag[i]) delta_poc_msb_cycle_lt[i] ue(v) } } if(sps_temporal_mvp_enabled_flag) slice_temporal_mvp_enabled_flag u(1) } if(sample_adaptive_offset_enabled_flag) { slice_sao_luma_flag u(1) if(ChromaArrayType != 0) slice_sao_chroma_flag u(1) } if(slice_sao_luma_flag || slice_sao_chroma_flag) { slice_sao_size_id u(3) if(slice_type==P || slice_type==B) { num_ref_idx_active_override_flag u(1) if(num_ref_idx_active_override_flag) { num_ref_idx_l0_active_minus1 ue(v) if(slice_type==B) num_ref_idx_l1_active_minus1 ue(v) } if(lists_modification_present_flag && NumPicTotalCurr>1) ref_pic_lists_modification() if(slice_type==B) mvd_l1_zero_flag u(1) if(cabac_init_present_flag) cabac_init_flag u(1) if(slice_temporal_mvp_enabled_flag) { if(slice_type==B) collocated_from_l0_flag u(1) if((collocated_from_l0_flag &&num_ref_idx_l0_active_minus1>0) | |
(! collocated_from_l0_flag &&num_ref_idx_l1_active_minus1>0)) collocated_ref_idx ue(v) } if((weighted_pred_flag && slice_type==P) | |
(weighted_bipred_flag && slice_type==B)) pred_weight_table() if(sps_ic_present_flag && slice_type !=I) { slice_ic_enable_flag u(1) if(sps_bidir_ic_present_flag && slice_ic_enable_flag) slice_bidir_ic_enable_flag u(1) max_search_depth_region_tree ue(v) for (i=0; i<max_search_depth_region_tree; i++) max_search_depth_prediction_tree [i] ue(v) if(slice_type !=I) for (i=0; i<max_search_depth_region_tree; i++) max_search_depth_prediction_tree_chroma [i] if(slice_type !=I) max_num_merge_cand ue(v) slice_qp_delta se(v) if(pps_slice_chroma_qp_offsets_present_flag) { slice_cb_qp_offset se(v) slice_cr_qp_offset se(v) } if(chroma_qp_offset_list_enabled_flag) cu_chroma_qp_offset_enabled_flag u(1) if(deblocking_filter_override_enabled_flag) deblocking_filter_override_flag u(1) if(deblocking_filter_override_flag) { slice_deblocking_filter_disabled_flag u(1) if(!slice_deblocking_filter_disabled_flag) { slice_beta_offset_div2 se(v) slice_tc_offset_div2 se(v) } } if(pps_loop_filter_across_slices_enabled_flag &&
(slice_sao_luma_flag || slice_sao_chroma_flag | |
! slice_deblocking_filter_disabled_flag)) slice_loop_filter_across_slices_enabled_flag u(1) } if(tiles_enabled_flag || entropy_coding_sync_enabled_flag) { num_entry_point_offsets ue(v) if(num_entry_point_offsets>0) { offset_len_minus1 ue(v) for (i=0; i<num_entry_point_offsets; i++) entry_point_offset_minus1[i] u(v) } } if(pps_clip_adaptive_enable_flag) clip_adaptive_flag u(1) if(clip_adaptive_flag) { clip_adaptive_y_min ue(v) clip_adaptive_y_max ue(v) clip_adaptive_flag_chroma u(1) if(clip_adaptive_flag_chroma) { clip_adaptive_c0_min ue(v) clip_adaptive_c0_max ue(v) clip_adaptive_c1_min ue(v) clip_adaptive_c1_max ue(v) } } } if(sps_affine_enabled_flag && slice_type !=I) { affine_control_flag[0 ] u(1) affine_control_flag[1 ] u(1) } if(slice_segment_header_extension_present_flag) { slice_segment_header_extension_length ue(v) for (i=0; i<slice_segment_header_extension_length; i++) slice_segment_header_extension_data_byte[i] u(8) } byte_alignment() }

Table 2. Header of a series of consecutive macroblocks

Tables 3, 4, 5, 6 below illustrate coding tree syntax modified to support additional segmentation modes according to at least one exemplary embodiment. In particular, the syntax to enable asymmetric segmentation is specified in coding_binary_tree(). New syntax elements are inserted into the coding tree syntax, more precisely:

- asymmetricSplitFlag, which indicates whether asymmetric binary splitting is enabled or not for the current CU, and

- asymmetric_type, which specifies the type of asymmetric binary partition. It can take two values to allow horizontal splitting up or down, or vertical splitting left or right.

Additionally, the btSplitMode parameter is interpreted differently.

coding_tree_unit() { Descriptor xCtb=(% CtbAddrInRs PicWidthInCtbsY)<<CtbLog2SizeY yCtb=(CtbAddrInRs/PicWidthInCtbsY)<<CtbLog2SizeY if(slice_sao_luma_flag || slice_sao_chroma_flag) if(slice_type==I) { coding_tree(xCtb, yCtb, CtbSizeX, CtbSizeY, LUMA_TREE, 0, 0) coding_tree(xCtb, yCtb, CtbSizeX, CtbSizeY, CHROMA_TREE, 0, 0) } else { coding_tree(xCtb, yCtb, CtbSizeX, CtbSizeY, LUMA_CHROMA_TREE, 0, 0) } }

Table 3. coding_tree_unit

coding_tree(x0, y0, log2CbSize, cqtDepth){ Descriptor if(x0+(1<<log2CbSize)<=pic_width_in_luma_samples &&
- y0+(1<<log2CbSize)<=pic_height_in_luma_samples &&
log2CbSize==MinCbLog2SizeY) split_cu_flag[x0][y0] ae(v) if(split_cu_flag[x0][y0]) { x1=x0+(1<<(log2CbSize-1)) y1=y0+(1<<(log2CbSize-1)) coding_tree(x0, y0, log2CbSize-1, cqtDepth+1) if(x1<pic_width_in_luma_samples) coding_tree(x1, y0, log2CbSize-1, cqtDepth+1) if(y1<pic_height_in_luma_samples) coding_tree(x0, y1, log2CbSize-1, cqtDepth+1) if(x1<pic_width_in_luma_samples && y1<pic_height_in_luma_samples) coding_tree(x1, y1, log2CbSize-1, cqtDepth+1) } else coding_binary_tree(x0, y0, 1<<log2CbSize, 1<<log2CbSize, cqtDepth) }

Table 4. coding_tree

coding_binary_tree(x0, y0, width, height, cqtDepth) { Descriptor if(btSplitAllowed(x0,y0,width, height){ bt_split_mode(x0,y0,width, height,cqtDepth) } if(btSplitFlag) { if(btSplitMode==HOR) { x1=x0 y1=y0+(height>>1) sub_width_1=sub_width_0=width; sub_height_1=sub_height_0=(height>>1) } else if(btSplitMode==VER) { x1=x0+(width>>1) y1=y0 sub_width_1=sub_width_0=(width>>1) sub_height_1=sub_height_0=height } else if(btSplitMode==HOR_UP) { x1=x0 y1=y0+(height>>2) sub_width_1=sub_width_0=width sub_height_0=(height>>2) sub_height_1=((height *3)>>2) } else if(btSplitMode==HOR_DOWN) { x1=x0 y1=y0+((height*3)>>2) sub_width_1=sub_width_0=width sub_height_0=((height *3)>>2) sub_height_1=(height>>2) } else if(btSplitMode==VER_LEFT) { x1=x0+(width>>2) y1=y0 sub_width_0=width>>2 sub_width_1=(width *3)>>2 sub_height_1=sub_height_0=height } else if(btSplitMode==VER_RIGHT) { x1=x0+(width*3)>>2 y1=y0 sub_width_0=(width*3)>>2 sub_width_1=width>>2 sub_height_1=sub_height_0=height } coding_binary_tree(x0, y0, sub_width, sub_height, cqtDepth) if(x1<pic_width_in_luma_samples && y1<pic_height_in_luma_samples) coding_binary_tree(x1, y1, sub_width, sub_height, cqtDepth) } } else coding_unit(x0, y0, width, height) }

Table 5. coding_binary_tree

bt_split_mode(x0,y0,width, height,cqtDepth){ Descriptor if(btSplitAllowed(x0,y0,width, height){ btSplitFlag[x0][y0][cbSizeX][cbSizeY] ae(v) } if(btSplitFlag[x0][y0][cbSizeX][cbSizeY]) { if(horizontalSplitAllowed && verticalSplitAllowed){ btSplitOrientation[x0][y0][cbSizeX][cbSizeY ] ae(v) } if(btSplitOrientation==HOR && horizontal_asymmetric_allowed || btSplitOrientation==VER && vertical_asymmetric_allowed){ asymmetricSplitFlag[x0][y0][cbSizeX][cbSizeY ] ae(v) if(asymmetricSplitFlag==true) { asymmetric_type[x0][y0][cbSizeX][cbSizeY ] ae(v) } } } }

Table 6. bt_split_mode

Fig. 9 illustrates the interpretation of bt-split-flag according to at least one exemplary embodiment. The two syntax elements asymmetricSplitFlag and asymmetric_type are used to specify the value of the btSplitMode parameter; btSplitMode specifies the binary split mode that is applied to the current CU; btSplitMode is extracted differently from the prior art. In JEM, it can accept values HOR, VER. In the improved syntax, in addition, it can take the values HOR_UP, HOR_DOWN, VER_LEFT, VER_RIGHT, according to the segmentation, as illustrated in FIG. 6.

Tables 7, 8, 9 and 10 below illustrate the syntax elements of a coding unit and introduce a new syntax element for the bidirectional intra prediction mode.

coding_unit (x0, y0, cuWidth, cuHeight, treeMode){ Descriptor if(transquant_bypass_enabled_flag) cu_transquant_bypass_flag ae(v) if(slice_type !=I) cu_skip_flag[x0][y0] ae(v) if(cu_skip_flag[x0][y0]) cu_data_merge(x0, y0, cuWidth, cuHeight, treeMode) else { if(slice_type !=I) pred_mode_flag ae(v) if(CuPredMode[x0][y0]==MODE_INTRA) cu_data_intra(x0, y0, cuWidth, cuHeight, treeMode) else { merge_flag[x0][y0] ae(v) if(merge_flag[x0][y0]) cu_data_merge(x0, y0, cuWidth, cuHeight, treeMode) else cu_data_inter(x0, y0, cuWidth, cuHeight, treeMode) if(cuPredMode[x0][y0] !=MODE_INTRA && !(merge_flag[x0][y0]) rqt_root_cbf ae(v) if(rqt_root_cbf) cu_residual_data(x0, y0, cuWidth, cuHeight, treeMode) } } }

Table 7. coding_unit

cu_data_merge(x0, y0, cuWidth, cuHeight, treeMode){ Descriptor if(fruc_merge_enabled_flag) fruc_merge_mode[x0][y0] ae(v) if(fruc_merge_mode[x0][y0]) if(lic_enabled_flag && fruc_merge_mode[x0][y0]==1) lic_flag[x0][y0] ae(v) if(fruc_merge_mode[x0][y0]==1 && MaxNumMergeCand>1) merge_idx[x0][y0] ae(v) if(lic_enabled_flag) lic_flag[x0][y0] ae(v) else if(MaxNumMergeCand>1) merge_idx[x0][y0] }

Table 8. cu_data_merge

cu_data_intra(x0, y0, cuWidth, cuHeight, treeMode){ Descriptor if(treeMode and LUMA_TREE) { prev_intra_luma_pred_flag[x0][y0] ae(v) if(prev_intra_luma_pred_flag[x0][y0]) { merge_idx[x0][y0] ae(v) } else { second_mpm_flag[x0][y0] ae(v) if(ciip_luma_mpm_flag[x0][y0]) second_mpm[x0][y0 ] f(5) else rem_intra_luma_pred_mode[x0][y0] ae(v) } if(bdip_enable_flag && ((luma_mode>=2 && luma_mode<18) || (luma_mode>=114 && luma_mode<130) bdip_enable_flag [x0][y0] ae(v) } if(treeMode and CHROMA_TREE) { intra_chroma_pred_mode[x0][y0] ae(v) } }

Table 9. cu_data_intra

cu_data_inter(x0, y0, cuWidth, cuHeight, treeMode){ Descriptor if(slice_type==B) inter_pred_idc[x0][y0 ] ae(v) if(affine_enabled_flag && cuWidth>8 && cuHeight>8) affine_flag[x0][y0] ae(v) if(inter_pred_idc[x0][y0] !=PRED_L1) { if(num_ref_idx_l0_active_minus1>0) ref_idx_l0[x0][y0] ae(v) if(affine_flag[x0][y0]) { mvd_gr0(x0, y0, 0, AFFINE_LEFT) ae(v) mvd_gr0(x0, y0, 0, AFFINE_RIGHT) ae(v) } else mvd_ gr0(x0, y0, 0, AFFINE_OFF) mvp_l0_flag[x0][y0] ae(v) } if(inter_pred_idc[x0][y0] !=PRED_L0) { if(num_ref_idx_l1_active_minus1>0) ref_idx_l1[x0][y0] ae(v) if(mvd_l1_zero_flag && inter_pred_idc[x0][y0]==PRED_BI) { MvdL1[x0][y0][0]=0 MvdL1[x0][y0][1]=0 } else if(affine_flag[x0][y0]) { mvd_gr0(x0, y0, 1, AFFINE_LEFT) ae(v) mvd_gr0(x0, y0, 1, AFFINE_RIGHT) ae(v) } else mvd_gr0(x0, y0, 1, AFFINE_OFF) mvp_l1_flag[x0][y0] ae(v) } if(imv_enabled_flag && CuHasNonZeroMvd) imv_mode[x0][y0 ] ae(v) if(affine[x0][y0] || CuHasNonZeroMvd) { if(inter_pred_idc[x0][y0] !=PRED_L1) { if(affine_flag[x0][y0]) { mvd_remain(x0, y0, 0, AFFINE_LEFT) ae(v) mvd_remain(x0, y0, 0, AFFINE_RIGHT) ae(v) } else mvd_remain(x0, y0, 0, AFFINE_OFF) ae(v) } if(inter_pred_idc[x0][y0] !=PRED_L0) { if(mvd_l1_zero_flag && inter_pred_idc[x0][y0]==PRED_BI) { MvdL1[x0][y0][0]=0 MvdL1[x0][y0][1]=0 } else if(affine_flag[x0][y0]) { mvd_gr0(x0, y0, 1, AFFINE_LEFT) ae(v) mvd_gr0(x0, y0, 1, AFFINE_RIGHT) ae(v) } else mvd_gr0(x0, y0, 1, AFFINE_OFF) ae(v) } } if(obmc_enabled_flag && cuWidth*cuHeight<=16*16) obmc_flag[x0][y0] ae(v) if(lic_enabled_flag && !affine_flag[x0][y0]) lic_flag[x0][y0] ae(v) }

Table 10. cu_data_inter

cu_residual_data(x0, y0, cuWidth, cuHeight, treeMode){ Descriptor if(treeMode and CHROMA_TREE) { cbf_cb[x0][y0 ] ae(v) cbf_cr[x0][y0 ] ae(v) } if(treeMode and LUMA_TREE) { if(CuPredMode[x0][y0] !=MODE_INTRA | |
- cbf_cb[x0][y0] || cbf_cr[x0][y0]) cbf_luma[x0][y0] ae(v) } if(treeMode and CHROMA_TREE) { if(cbf_cb[x0][y0]) residual_coding(x0, y0, cuWidth/2, cuHeight/2, treeMode, 1) if(cbf_cr[x0][y0]) residual_coding(x0, y0, cuWidth/2, cuHeight/2, treeMode, 2) } if(treeMode and LUMA_TREE) { if(emt_enable_flag && cuWidth<=64 && cuHeight<=64) emt_cu_flag[x0][y0] ae(v) residual_coding(x0, y0, cuWidth, cuHeight, treeMode, 0) }

Table 11. cu_residual_data

Several of the syntax elements explained in this document are specified as arrays. For example, btSplitFlag is specified as a 4-dimensional array indexed by the horizontal and vertical positions in the frame and by the horizontal and vertical dimensions of the encoded block. To simplify the notation, indices are not stored in the semantic description of syntactic elements (for example, btSplitFlag[x0][y0][cbSizeX][cbSizeY] is simply marked as btSplitFlag).

Fig. 10 illustrates a block diagram of an example system in which various aspects and embodiments are implemented. The system 1000 may be implemented as a device including various components described below and configured to perform one or more of the aspects described herein. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital media set-top boxes, digital television receivers, personal video recording systems, connected home appliances, encoders, transcoders and servers . The elements of system 1000, alone or in combination, may be implemented in a single integrated circuit, multiple ICs, and/or discrete components. For example, in at least one embodiment, the processing and encoder/decoder elements of system 1000 are distributed across multiple ICs and/or discrete components. In various embodiments, the system 1000 is operatively connected to other similar systems or to other electronic devices, for example, through a communications bus or through dedicated input and/or output ports. In various embodiments, system 1000 is configured to implement one or more aspects described herein.

System 1000 includes at least one processor 1010 configured to execute instructions loaded thereon to implement, for example, various aspects described herein. Processor 1010 may include embedded storage, an input/output interface, and various other circuitry as is known in the art. System 1000 includes at least one storage device 1020 (eg, volatile storage and/or non-volatile storage). System 1000 includes a data storage device 1040, which may include non-volatile memory and/or volatile memory, including, but not limited to, EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash memory, storage magnetic disk and/or optical disk drive. The storage device 1040 may include an internal storage device, an attached storage device, and/or a network accessible storage device, as non-limiting examples.

System 1000 includes a configured encoder/decoder module 1030, for example, configured to process data to provide encoded video or decoded video, and encoder/decoder module 1030 may include its own processor and memory. Encoder/decoder module 1030 represents module(s) that may be included in a device to perform encoding and/or decoding functions. As is known, the device may include one or both of encoding and decoding modules. Additionally, encoder/decoder module 1030 may be implemented as a separate element of system 1000 or may be included in processor 1010 as a combination of hardware and software, as is known to those skilled in the art.

Program code that must be loaded into processor 1010 or encoder/decoder 1030 to perform various aspects described herein may be stored on storage device 1040 and then loaded into storage device 1020 for execution by processor 1010. In accordance with various embodiments implementation, one or more of the processor 1010, memory 1020, data storage device 1040, and encoder/decoder module 1030 may store one or more different elements while performing the processes described herein. Such stored elements may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from processing equations, formulas, operations, and operational logic.

In several embodiments, memory in processor 1010 and/or encoder/decoder module 1030 is used to store instructions and provide random access memory for processing that is required during encoding or decoding. However, in other embodiments, a storage device external to the processing device (eg, the processing device may be either a processor 1010 or an encoder/decoder module 1030) is used for one or more of these functions. The external storage device may be a memory device 1020 and/or a data storage device 1040, such as dynamic volatile memory and/or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store the operating system of the television receiver. In at least one embodiment, a fast external dynamic volatile memory device, such as RAM, is used as random access memory for video encoding and decoding operations, for example, MPEG-2, HEVC or VVC (Video Coding Standard) .

Input to elements of the system 1000 may be provided through various input devices, as indicated at block 1130. Such input devices include, but are not limited to: (i) an RF portion that receives an RF signal transmitted, for example, over an air interface via a broadcast transmitter device, (ii) composite input pin, (iii) USB input pin, and/or (iv) HDMI input pin.

In various embodiments, the input devices of block 1130 have associated corresponding input processing elements, as is known in the art. For example, the RF portion may be associated with elements necessary to (i) select a desired frequency (also called “signal selection” or “band-limiting a signal”), (ii) downconvert the selected signal, (iii) limiting the frequency band again to a narrower frequency band to select (for example) a frequency band of the signal, which may be referred to as a channel in particular embodiments, (iv) demodulating the down-converted signal with a limited frequency band, (v) performing error correction, and (vi) demultiplexing to select the desired stream of data packets. The RF portion of various embodiments includes one or more elements to perform these functions, for example, frequency selection modules, signal selection modules, band limiting modules, channel selection modules, filters, frequency downconverters, demodulators, modules error correction and demultiplexers. The RF portion may include a tuner that performs various of these functions, including, for example, downconverting the received signal to a lower frequency (e.g., to an intermediate frequency or near baseband frequency) or to a baseband frequency. frequency In one embodiment of a set-top box, the RF portion and its associated input processing element receives an RF signal transmitted over a wired (eg, cable) medium and performs frequency selection by filtering, down-converting, and filtering back to the desired frequency band. Various embodiments rearrange the order of the above (and other) elements, remove some of these elements, and/or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, such as, for example, inserting amplifiers and an analog-to-digital converter. In various embodiments, the RF portion includes an antenna.

Additionally, the USB and/or HDMI pins may include associated interface processors for connecting the system 1000 to other electronic devices via USB and/or HDMI connections. It should be understood that various aspects of input processing, such as Reed-Solomon error correction, may be implemented, for example, in a separate input processing IC or processor 1010 as needed. Likewise, aspects of USB or HDMI interface processing may be implemented in separate interface ICs or in processor 1010 as needed. The demodulated and demultiplexed stream, after error correction, is provided to various processing elements, including, for example, a processor 1010 and an encoder/decoder 1030, operating in combination with memory and storage elements to process the data stream as needed for presentation on an output device.

Various elements of the system 1000 may be provided in an integrated housing. In an integrated package, various elements can be interconnected and communicate data among themselves using a suitable interconnection arrangement, for example, an internal bus, as is known in the art, including an I2C bus, circuit boards and printed circuit boards.

System 1000 includes a communications interface 1050 that allows communication with other devices via a communications link 1060. The communication interface 1050 may include, but is not limited to, a transceiver device configured to transmit and receive data over the communication channel 1060. Communication interface 1050 may include, but is not limited to, a modem or network card, and communication channel 1060 may be implemented, for example, in a wired and/or wireless environment.

The data is streamed to the system 1000, in various embodiments, using a Wi-Fi network such as IEEE 802.11. The Wi-Fi signal of these embodiments is received over a communication channel 1060 and a communication interface 1050 that adapt to Wi-Fi communication. The link 1060 of these embodiments typically connects to an access point or router that provides access to external networks, including the Internet, to enable streaming applications and other communications over the networks. Other embodiments provide streaming data to system 1000 using a set-top box that delivers the data over an HDMI connection of input unit 1130. Still other embodiments provide streaming data to system 1000 using an RF connection of input unit 1130.

System 1000 may provide output to various output devices, including display 1100, speakers 1110, and other peripherals 1120. Other peripherals 1120 include, in various example embodiments, one or more of a standalone DVR, disc player, stereo system , lighting systems, and other devices that provide a function based on the output of the system 1000. In various embodiments, control signals are transmitted between the system 1000 and the display 1100, speakers 1110, or other peripheral devices 1120 using signaling such as AV.Link, CEC, or other communication protocols that provide control between devices with or without user intervention. Output devices may be operatively coupled to system 1000 via dedicated connections via respective interfaces 1070, 1080, and 1090. Alternatively, output devices may be coupled to system 1000 using communications link 1060 via communications interface 1050. The display 1100 and speakers 1110 may be integrated in one module with other components of the system 1000 in an electronic device, such as, for example, a television receiver. In various embodiments, display interface 1070 includes a display driver, such as, for example, a timing controller (T Con) chip.

The display 1100 and speaker 1110 may alternatively be separate from one or more other components, for example, if the RF input portion 1130 is part of a separate set-top box. In various embodiments in which the display 1100 and speakers 1110 are external components, the output signal may be provided through dedicated output connections including, for example, HDMI ports, USB ports, or COMP pins. The implementations described herein may be implemented, for example, in a method or process, hardware, firmware, data stream, or signal. Even if explained only in the context of one form of implementation (eg, explained only as a method), the implementation of the explained features may also be implemented in other forms (eg, as a device or program). The equipment may be implemented, for example, in associated hardware, software and firmware. The methods may be implemented, for example, in equipment such as, for example, a processor, which means processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communications devices, such as, for example, computers, cellular phones, portable/personal digital devices (PDAs), and other devices that facilitate the exchange of information between end users.

Fig. 11 illustrates a flowchart of an example encoding method according to an embodiment using the new encoding tools. This encoding method may be accomplished by the system 1000 described in FIG. 10, and more specifically, may be implemented by processor 1010. In at least one embodiment, at step 1190, processor 1010 selects the encoding tools to be used. The selection can be made using various technologies. The selection may be made by the user through an encoding configuration parameter (typically a flag) that specifies the tools to be used (as well as corresponding parameters if necessary) during the encoding and decoding processes. In an exemplary embodiment, the selection is made by setting a value to a value in a file, wherein the flag is read by an encoder that interprets the value of the flag to select which tool should be used. In an exemplary embodiment, the selection is made through a manual operation of selecting, by the user, encoding configuration parameters using a graphical user interface that is processed by the encoder device. Once this selection is made, encoding is performed at step 1193 using the selected tool from among other encoding tools, and the choice of tool is signaled in high-level syntax elements (eg, the next SPS, PPS, or even a runtime header).

Fig. 12 illustrates a flowchart of an example portion of a decoding method according to an embodiment using the new encoding tools. Such a decoding method may be performed by the system 1000 described in FIG. 10, and more specifically, may be implemented by a processor 1010. In at least one embodiment, at step 1200, the processor 1010 accesses, extracts, and analyzes a signal (eg, received at an input interface or read from a media support tool). high-level syntax elements to specify the tools that are selected at the encoder. At step 1210, these tools are used to perform decoding and generate a decoded image, which, for example, can be provided to or displayed on the device.

Reference to “one embodiment” or “embodiment” or “one implementation” or “implementation”, as well as other variations thereof, means that the particular feature, structure, characteristic, etc. described in connection with the embodiment , is included in at least one embodiment. Thus, occurrences of the phrase "in one embodiment" or "in an embodiment" or "in one implementation" or "in an implementation", and all variations thereof, appearing at various places in the detailed description do not necessarily refer to the same embodiment.

Additionally, this application or its claims may refer to "definition" of various pieces of information. Determining information may include, for example, one or more of evaluating information, calculating information, predicting information, or retrieving information from a storage device.

Additionally, this application or its claims may refer to “accessing” various pieces of information. Accessing information may include, for example, one or more of receiving information, retrieving information (eg, from a storage device), storing information, moving information, copying information, calculating information, predicting information, or evaluating information.

Additionally, this application or its claims may refer to “receiving” various pieces of information. Receiving, like "accessing", should be a general term. Receiving information may include, for example, one or more of accessing or retrieving information (eg, from a memory device or an optical multimedia storage device). Additionally, "receiving" is typically provided in one way or another during operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or evaluating information. .

It should be appreciated that the use of any of the following "/", "and/or" and "at least one of", for example, in the cases of "A/B", "A and/or B" and "at least At least one of "A and B" is intended to cover the choice of only the first listed option (A), or the choice of only the second listed option (B), or the choice of both options (A and B). As a further example, in the cases of "A, B and/or C" and "at least one of A, B and C", such language is intended to cover the selection of only the first listed option (A) or the selection of only the second listed option (B), or selecting only the third listed option (C), or selecting only the first and second listed options (A and B), or selecting only the first and third listed options (A and C), or selecting only the second and third listed options (B and C), or select all three options (A and B, and C). This may extend, as will be apparent to those skilled in the art, to all of the elements listed.

It will be apparent to those skilled in the art that implementations may also generate a plurality of signals formatted to carry information that, for example, may be stored or transmitted. The information may include, for example, instructions for implementing a method or data generated by one of the described implementations. For example, the signal may be formatted to carry the bitstream of the described embodiment. This signal, for example, may be formatted as an electromagnetic wave (eg, using the radio frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding the data stream and modulating a carrier with the encoded data stream. The information that the signal carries, for example, may be analog or digital information. As you know, a signal can be transmitted over many different wired or wireless communication lines. The signal may be stored on a computer readable medium.

In a variation of the first, second, third, fourth, fifth, sixth, seventh and eighth aspect of at least an embodiment, a parameter representing a segmentation type comprises a first value for quadtree and binary tree segmentation, a second value for tree segmentation quadtree and binary tree plus ternary tree segmentations, a third value for quadtree and binary tree segmentations plus asymmetric binary tree segmentations, and a fourth value for quadtree and binary tree segmentations plus ternary tree segmentations plus asymmetric binary tree segmentations.

In a variation of the first, second, third, fourth, fifth, sixth, seventh and eighth aspect of at least one embodiment, said parameters further comprise a block size for which a sampled adaptive offset is applied to the encoded series of successive macroblocks.

In a variation of the first, second, third, fourth, fifth, sixth, seventh and eighth aspect of at least one embodiment, the intra prediction mode type comprises at least one of an intra bi prediction mode and a multiple reference intra prediction mode.

Claims (18)

1. A video encoder for encoding video frame data, wherein the video encoder is configured to encode frame data for at least one block in the frame based on a single flag configured to determine whether independent trees are used for encoding the luma and chrominance signal blocks encodings or general encoding trees, and insert a single flag into high-level syntax elements of the encoded frame data. 2. The device according to claim 1, wherein the only flag is 1, the blocks of luminance signals and chrominance signals use independent coding trees. 3. The apparatus of claim 1 or 2, wherein a single flag is configured to be signaled in a sequence parameter set at the sequence level to apply to all encoded slices using said sequence parameter set. 4. A method for encoding video frame data, wherein the method is carried out by a video encoder and contains the steps of: encoding frame data for at least one block in the frame based on a single flag determining whether the encoding of the luma and chroma blocks uses independent encoding trees or common encoding trees, and insert a single flag into high-level syntax elements of the encoded frame data. 5. The method of claim 4, wherein, when the single flag is 1, the luma and chroma blocks use independent coding trees. 6. The method of claim 4 or 5, wherein a single flag is signaled in the sequence parameter set at the sequence level to apply to all encoded slices using said sequence parameter set. 7. A video decoder for decoding video frame data, wherein the video encoder is configured to obtain parameters from high-level syntax elements of the encoded frame data and decode frame data for at least one block in the frame based on a single flag configured to determine whether coding for luma and chrominance blocks independent coding trees or common coding trees. 8. The device according to claim 7, wherein the only flag is 1, the blocks of luminance signals and chrominance signals use independent coding trees. 9. The apparatus of claim 7 or 8, wherein the single flag is configured to be signaled in the sequence parameter set at the sequence level to apply to all encoded slices using said sequence parameter set. 10. A method for decoding video frame data, wherein the method is carried out by a video encoder and contains the steps of: obtaining parameters from high-level syntax elements of the encoded frame data, and decoding frame data for at least one block in the frame based on a single flag that determines whether independent coding trees or common coding trees are used when encoding the luma and chroma blocks. 11. The method of claim 10, wherein when the single flag is 1, the luma and chroma blocks use independent coding trees. 12. The method of claim 10 or 11, wherein a single flag is signaled in the sequence parameter set at the sequence level to apply to all encoded slices using said sequence parameter set. 13. A non-volatile machine-readable medium on which instructions with program code are stored, executed by the processor to implement the stages of the method according to claim 4. 14. A non-volatile machine-readable medium on which instructions with program code are stored, executed by the processor to implement the stages of the method according to claim 10.

Images