TW202116068A - Video encoding/decoding method and apparatus - Google Patents

Abstract

A method for signaling subpicture structures for coded video is provided. A video decoder receives data from a bitstream to be decoded as a sequence of video pictures. The video decoder receives from the bitstream subpicture specification for one or more subpictures in the sequence of video pictures. The subpicture specification identifies a position and a size for each subpicture by providing an index that identifies a coding tree unit (CTU) for the subpicture. The video decoder reconstructs each subpicture for the sequence of video pictures according to the subpicture specification.

Description

Communication technology of sub-picture structure

The present invention relates to video coding and decoding, and more specifically, the present invention relates to a method of transmitting a sub-image structure.

Unless otherwise stated herein, the methods described in this section are not prior art with respect to the scope of patent applications listed below, and the introduction of this section is not recognized as prior art.

High-efficiency video coding (HEVC) is the latest international video coding and decoding standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). The input video signal is predicted from the reconstructed signal, which is derived from the codec image area. The prediction residual signal is processed by linear transformation. The transform coefficients are quantized and entropy coded together with other auxiliary information in the bit stream. After the inversely quantized transform coefficients are inversely transformed, the reconstruction signal is generated from the prediction signal and the reconstruction residual signal. With loop filtering, the reconstructed signal is further processed to remove codec artifacts. The decoded image is stored in the frame buffer to predict the future image in the input video signal.

In HEVC, a coded image is divided into non-overlapped square block regions represented by an associated coding tree unit (CTU). The coded image is represented by a collection of segments, and each segment includes an integer number of CTUs. Each CTU in the fragment is processed in raster scanning order. A bi-predictive (B for short) segment can use up to two motion vectors and a reference index to be decoded by intra prediction or inter prediction to predict the sample value of each block. A predictive (predictive, P for short) segment can use at most one motion vector and reference index to decode through intra prediction or inter prediction to predict the sample value of each block. Intra (intra, I for short) segments are only decoded using intra prediction.

With the recursive quadtree (QT) structure, the CTU can be divided into multiple non-overlapping coding units (CU) to adapt to various local motion and texture features. The CTU can also be divided into one or more smaller-sized CUs by using a quadtree with nested multi-type trees using binary and ternary divisions. The resulting CU partition can be square or rectangular in shape.

One or more prediction units (prediction, PU for short) are designated for each CU. The prediction unit, together with the associated CU syntax, serves as the basic unit for transmitting predictor information. The specified prediction process is used to predict the value of the pixel sample associated within the PU. The CU may be further divided using a residual quadtree (RQT) structure to represent the associated prediction residual signal. The leaf nodes of the RQT correspond to the transform unit (TU for short). The transform unit includes a transform block (TB) of luminance samples with a size of 8x8, 16x16 or 32x32 or four transform blocks with a luminance sample of 4x4, and the chroma of the image in the 4:2:0 color format. Two corresponding transform blocks of the sample. Integer transform is applied to the transform block, and the level value of the quantized coefficient is entropy coded into a bit stream along with other auxiliary information.

The terms coding tree block (CTB for short), coding block (CB for short), prediction block (PB for short) and transform block (TB for short) are defined as the designation and CTU, CU , 2-D sample array of color components associated with PU and TU. Therefore, CTU is composed of one luminance CTB, two chrominance CTBs and related syntax elements. A similar relationship is valid for CU, PU and TU. Tree segmentation is usually applied to both luminance and chrominance, but it is an exception when a certain minimum size of chrominance is reached.

The following summary of the invention is only illustrative and is not intended to be limited in any way. That is, the following summary is provided to introduce the concepts, highlights, benefits, and advantages of the new and non-obvious technology described herein. Alternative but not all implementations are further described in the detailed description below. Therefore, the following content of the invention is not used to determine the essential features of the requested subject matter, nor is it used to determine the scope of the requested subject matter.

Some embodiments provide a method for transmitting a sub-picture structure of an encoded video. The video decoder receives data from the bit stream that will be decoded into a sequence of video images. The video decoder receives the sub-image specifications of one or more sub-images in the video image sequence from the bit stream. The sub-image specification identifies the position and size of each sub-image by providing an index for each sub-image, where the index identifies the codec tree unit (CTU) in the image in the order of raster scan. The video decoder reconstructs each sub-image of the video image sequence according to the sub-image specification.

In some embodiments, the syntax element in the sequence parameter set (SPS) of the video image sequence indicates that there are one or more sub-images in the video image sequence. The SPS may also include a syntax element that specifies the number of sub-images of the video image sequence and the identifier of each sub-image. The identifier of the sub-image may also be sent in the segment header and/or picture parameter sets (PPS) of the video image in the video image sequence. In some embodiments, a syntax element in the PPS of the video image in the video image sequence indicates that all segments of the video image are rectangular.

In some embodiments, the CTU identified by the raster scan in the image is located at the corner of the sub-image (for example, the upper left or the lower right). In some embodiments, indexes are assigned to raster-scan sub-image grids within the image, and different sub-image grids are assigned different indexes. In some embodiments, the index identifier is defined as a CTU or a sub-image grid corresponding to a CTU, so that the boundary of the sub-image grid is defined along the boundary of the CTU. In some embodiments, the index of the position and size of the sub-image is sent in the SPS of the video image sequence.

In the following detailed description, many specific details are illustrated by examples to provide a thorough understanding of related teachings. Any changes, derivatives and/or extensions based on the teachings described herein are within the protection scope of the present disclosure. In some instances, well-known methods, processes, components, and/or circuits related to one or more example implementations disclosed herein may be described at a relatively high level without details to avoid unnecessarily obscuring the present disclosure Aspects of the teaching. I. Send sub-image structure

A sub-image is a rectangular area of one or more segments in the image, and the segment is composed of one or more tiles/bricks. Each tile/brick is CTU aligned. When sub-images exist in the image, the number of sub-images may be greater than or equal to two. The segments forming the sub-image may be rectangular. In some embodiments, by indicating the grid index of the lower right sub-image in the raster scan order for each sub-image in the image, a grid in units of CTB is used to specify the sub-image structure in the image .

In some embodiments, the video encoder may send sub-picture specifications (and the video decoder may receive sub-picture specifications). The following Table 1A is an example syntax table of the sequence parameter set (SPS) raw byte sequence payload (RBSP) used to send sub-image information: Table 1A: SPS sends sub-image information seq_parameter_set_rbsp() { Descriptor sps_decoding_parameter_set_id u(4) sps_video_parameter_set_id u(4) sps_max_sub_layers_minus1 u(3) sps_reserved_zero_5bits u(5) profile_tier_level( sps_max_sub_layers_minus1) gdr_enabled_flag u(1) sps_seq_parameter_set_id ue(v) chroma_format_idc ue(v) if( chroma_format_idc == 3) separate_colour_plane_flag u(1) pic_width_max_in_luma_samples ue(v) pic_height_max_in_luma_samples ue(v) subpics_present_flag u(1) if( subpics_present_flag) { max_subpics_minus1 u(8) subpic_grid_col_width_minus1 u(v) subpic_grid_row_height_minus1 u(v) for( i = 0; i ＜ NumSubPicGridRows; i++) for( j = 0; j ＜ NumSubPicGridCols; j++) subpic_grid_idx [i ][ j] u(v) for( i = 0; i ＜= NumSubPics; i++) { subpic_treated_as_pic_flag [i] u(1) loop_filter_across_subpic_enabled_flag [i] u(1) } } bit_depth_luma_minus8 ue(v) bit_depth_chroma_minus8 ue(v) … sps_extension_flag u(1) if( sps_extension_flag) while( more_rbsp_data()) sps_extension_data_flag u(1) rbsp_trailing_bits() }

The following Table 1B is another example syntax table of the sequence parameter set (SPS) raw byte sequence payload (RBSP), which provides the specification of the sub-picture. Table 1B : SPS designated sub-images seq_parameter_set_rbsp() { Descriptor sps_decoding_parameter_set_id u(4) sps_video_parameter_set_id u(4) … subpics_present_flag u(1) if( subpics_present_flag) { max_subpics_minus2 u(8) subpic_grid_col_width_minus1 u(v) subpic_grid_row_height_minus1 u(v) for( i = 0; i ＜ NumSubPicGridRows; i++) for( j = 0; j ＜ NumSubPicGridCols; j++) subpic_grid_idx [i ][ j] u(v) for( i = 0; i ＜= NumSubPics; i++) { subpic_treated_as_pic_flag [i] u(1) loop_filter_across_subpic_enabled_flag [i] u(1) } } bit_depth_luma_minus8 ue(v) bit_depth_chroma_minus8 ue(v) … sps_extension_flag u(1) if( sps_extension_flag) while( more_rbsp_data()) sps_extension_data_flag u(1) rbsp_trailing_bits() }

The following table 1C is another example syntax table of the sequence parameter set (SPS) original byte sequence payload (RBSP), which provides the specification of the sub-image: Table 1C : SPS designated sub-image seq_parameter_set_rbsp() { Descriptor sps_decoding_parameter_set_id u(4) sps_video_parameter_set_id u(4) … subpics_present_flag u(1) if( subpics_present_flag) { num_subpics_minus2 u(8) subpic_grid_col_width_minus1 u(v) subpic_grid_row_height_minus1 u(v) for( i = 0; i ＜ NumSubPicGridRows; i++) for( j = 0; j ＜ NumSubPicGridCols; j++) subpic_grid_idx [i ][ j] u(v) for( i = 0; i ＜= NumSubPics; i++) { subpic_treated_as_pic_flag [i] u(1) loop_filter_across_subpic_enabled_flag [i] u(1) } } bit_depth_luma_minus8 ue(v) bit_depth_chroma_minus8 ue(v) … sps_extension_flag u(1) if( sps_extension_flag) while( more_rbsp_data()) sps_extension_data_flag u(1) rbsp_trailing_bits() }

The following table 1D is another example syntax table of the sequence parameter set (SPS) original byte sequence payload (RBSP), which provides the specification of the sub-image: Table 1D : SPS designated sub-image Seq_parameter_set_rbsp() { Descriptor sps_decoding_parameter_set_id u(4) sps_video_parameter_set_id u(4) sps_max_sub_layers_minus1 u(3) sps_reserved_zero_5bits u(5) profile_tier_level( sps_max_sub_layers_minus1) gdr_enabled_flag u(1) sps_seq_parameter_set_id ue(v) chroma_format_idc ue(v) if( chroma_format_idc == 3) separate_colour_plane_flag u(1) pic_width_max_in_luma_samples ue(v) pic_height_max_in_luma_samples ue(v) log2_ctu_size_minus5 u(2) subpics_present_flag u(1) if( subpics_present_flag) { num_subpics_minus1 u(8) signalled_subpic_grid_flag u(1) if( signalled_subpic_grid_flag) { subpic_grid_col_width_minus1 u(v) subpic_grid_row_height_minus1 u(v) } else { subpic_grid_col_width_minus1 = 0 subpic_grid_row_height_minus1 = 0 } bottom_right_subpic_grid_idx_length_minus1 u(v) for( i = 0; i ＜ num_subpics_minus1; i++) { bottom_right_subpic_grid_idx_delta [i] u(v) if( i ＞ 0) subpic_grid_idx_delta_sign_flag [i] u(1) } for( i = 0; i ＜ num_subpics_minus1; i++) { subpic_treated_as_pic_flag [i] u(1) loop_filter_across_subpic_enabled_flag [i] u(1) } } bit_depth_luma_minus8 ue(v) bit_depth_chroma_minus8 ue(v) … sps_extension_flag u(1) if( sps_extension_flag) while( more_rbsp_data()) sps_extension_data_flag u(1) rbsp_trailing_bits() }

The syntax element subpics_present_flag being 1 indicates that the sub-picture parameter currently exists in the SPS RBSP syntax. In some embodiments, when subpics_present_flag is equal to 1, the value of rect_slice_flag is set to 1. The syntax element subpics_present_flag being 0 indicates that there are currently no sub-picture parameters in the SPS RBSP syntax. In some embodiments, the value of subpics_present_flag in the RBSP of the SPS when the bit stream is the result of the sub-bit stream extraction process and only contains a subset of the sub-images of the input bit stream of the sub-bit stream extraction process It is set to 1.

The syntax element max_subpics_minus2 plus 2 specifies the maximum number of sub-pictures that may exist in a coded video sequence (CVS). In some embodiments, max_subpics_minus2 is limited to the range of 0 to 254. The value 255 is reserved for future use.

The syntax element num_subpics_minus1 plus 1 specifies the number of sub-pictures that may exist in the CVS. In some embodiments, the value of num_subpics_minus1 is limited to a range of 0 to 254. The value 255 is reserved for future use. As shown in Table 1D, with the syntax element num_subpics_minus1, the number of sub-pictures existing in the CVS is directly sent in the SPS.

The syntax element num_subpics_minus2 plus 2 specifies the number of sub-pictures that may exist in the CVS. In some embodiments, the value of num_subpics_minus2 is limited to a range of 0 to 254. The value 255 is reserved for future use.

The syntax element subpic_grid_col_width_minus1 plus 1 specifies the width (in CtbSizeY) of each element of the sub-image recognition word grid (excluding the rightmost grid column of the image). The length of the syntax element is Ceil (Log2 (pic_width_max_in_luma_samples/CtbSizeY)) bits. If it does not exist, subpic_grid_row_width_minus1 is inferred to be 0.

The syntax element subpic_grid_row_height_minus1 plus 1 specifies the height (in CtbSizeY) of each element in the sub-image recognition word grid (excluding the bottom grid row of the image). The length of the syntax element is Ceil (Log2 (pic_height_max_in_luma_samples / CtbSizeY)) bits. If it does not exist, subpic_grid_row_height_minus1 is inferred to be 0.

The variable NumSubPicGridRows is exported as follows: NumSubPicGridRows = (Pic_height_max_in_luma_samples + subpic_grid_row_height_minus1 * CtbSizeY + CtbSizeY – 1)/ (Subpic_grid_row_height_minus1 * CtbSizeY + CtbSizeY)

The syntax element bottom_right_subpic_grid_idx_length_minus1 plus 1 specifies the number of bits used to represent the syntax element bottom_right_subpic_grid_idx_delta[i]. The value of bottom_right_subpic_grid_idx_length_minus1 shall be in the range of 0 to Ceil (Log2(NumSubPicGridRows * NumSubPicGridCols)) − 1, including the endpoints.

When i is greater than 0, the variable bottom_right_subpic_grid_idx_delta [i] specifies the difference between the sub-image grid index in the lower right corner of the i-th sub-image and the sub-image grid index in the lower right corner of the i-1th sub-image . The variable bottom_right_subpic_grid_idx_delta[0] specifies the sub-image grid index of the lower right corner of the 0th sub-image.

The syntax element subpic_grid_idx_delta_sign_flag [i] equal to 1 represents the positive sign of bottom_right_subpic_grid_idx_delta [i]. sign_bottom_right_subpic_grid_idx_delta [i] equal to 0 means the negative sign of bottom_right_subpic_grid_idx_delta [i].

Figures 1a-e conceptually show a grid unit based on CTB or CTU, which is used to specify sub-images of a video sequence. Figure 1a shows a coded video sequence (CVS) 100 including several video images. The video image (for example, image 110) in the sequence 100 is divided into CTUs. Figure 1b shows that the images of the sequence are divided into sub-image grids for specifying sub-images. Each sub-image grid 120 corresponds to an integer number of CTUs, so that each sub-image grid is defined based on the boundary of the CTU or CTB. Figure 1c shows an example in which each sub-image grid 120 corresponds to exactly one CTU or CTB. Figure 1d shows an example in which each sub-image grid 120 corresponds to 2×1 CTUs or CTBs. Figure 1e shows an example in which each sub-image grid 120 corresponds to 2×3 CTUs or CTBs. In some embodiments, SPS syntax elements such as Spic_grid_col_width_minus1 and subpic_grid_row_height_minus1 define the sub-image grid according to CTU or CTB. In addition, the size of each CTU or CTB is sent in the SPS with the syntax element log2_ctu_size_minus5.

Figure 2 shows a grid of sub-images based on CTU or CTB, which is indexed for specifying sub-images. Each sub-image grid in the image 110 corresponds to an index that can be used when the size (eg, width, height) and/or position of the sub-image is being sent by the video codec. The specified elements of the sub-image recognition word grid are indexed in raster scan order. In some embodiments, the sub-images are also indexed in raster scan order.

As shown in the figure, the image of the sequence 100 (for example, the image 110) is defined as having four sub-images 210, 220, 230, and 240, and these sub-images are defined using a sub-image grid based on CTU or CTB. The index associated with the sub-image grid is used to specify the size and position of the sub-images 210-240.

In some embodiments, the position of each sub-image is specified based on the index associated with the sub-image grid at the corner of the sub-image (eg, the lower right corner or the upper left corner). In this figure, the position of sub-image 210 is index 27, the position of sub-image 220 is index 36, the position of sub-image 230 is index 79, and the position of sub-image 240 is index 84. By referring to the lower right corner position of another sub-image, SPS syntax elements such as bottom_right_subpic_grid_idx_delta [i] and subpic_grid_idx_delta_sign_flag [i] are used to specify the lower right corner position of the sub-image. Or, in some embodiments, each CTU/CTB-based sub-image grid can be associated with an X index and a Y index, and the position of each sub-image can be determined by the sub-image grid in the upper left corner of the sub-image. The X and Y indexes are specified.

In some embodiments, the maximum number of sub-images (in CVS) in a Video Parameter Set (VPS) is specified. The following Table 3 is an example syntax table of VPS, which specifies the maximum number of sub-pictures in CVS. Table 3: VPS specifies the maximum number of sub-images in CVS Video_parameter_set_rbsp() { Descriptor vps_video_parameter_set_id u(4) vps_max_layers_minus1 u(6) vps_max_subpics_minus2 u(8) if( vps_max_layers_minus1 ＞ 0) vps_all_independent_layers_flag u(1) for( i = 0; i ＜= vps_max_layers_minus1; i++) { vps_layer_id [i] u(6) if( i ＞ 0 && !vps_all_independent_layers_flag) { vps_independent_layer_flag [i] u(1) if( !vps_independent_layer_flag[ i]) for( j = 0; j ＜ i; j++) vps_direct_dependency_flag [i ][ j] u(1) } } if( vps_max_layers_minus1 ＞ 0) { vps_output_layers_mode u(2) if( vps_output_layers_mode = = 2) for( i = 0; i ＜ vps_max_layers_minus1; i++) vps_output_layer_flag [i] u(1) } vps_constraint_info_present_flag u(1) vps_reserved_zero_7bits u(7) if( vps_constraint_info_present_flag) general_constraint_info() vps_extension_flag u(1) if( vps_extension_flag) while( more_rbsp_data()) vps_extension_data_flag u(1) rbsp_trailing_bits() }

The syntax element vps_max_subpics_minus2 plus 2 (or vps_max_subpics_minus1 plus 1) specifies the maximum allowable number of sub-pictures that refer to the VPS in each CVS. In some embodiments, the syntax element vps_max_subpics_minus2 is limited to the range of 0 to 254. The value 255 is reserved for future use.

In some embodiments, the parameters related to the sub-image are sent in a picture parameter set (Picture Paramter Set, PPS for short). The following Table 4 shows an example syntax table of PPS including sub-picture information. Table 4: PPS designated sub-image ID pic_parameter_set_rbsp() { Descriptor pps_pic_parameter_set_id ue(v) pps_seq_parameter_set_id ue(v) pic_width_in_luma_samples ue(v) pic_height_in_luma_samples ue(v) conformance_window_flag u(1) if( conformance_window_flag) { conf_win_left_offset ue(v) conf_win_right_offset ue(v) conf_win_top_offset ue(v) conf_win_bottom_offset ue(v) } output_flag_present_flag u(1) single_ tile_in_pic_flag u(1) if( !single_tile_in_pic_flag) { uniform_tile_spacing_flag u(1) … loop_filter_across_bricks_enabled_flag u(1) if( loop_filter_across_bricks_enabled_flag) loop_filter_across_slices_enabled_flag u(1) } if( rect_slice_flag) { signalled_slice_id_flag u(1) if( signalled_slice_id_flag) { signalled_slice_id_length_minus1 ue(v) for( i = 0; i ＜= num_slices_in_pic_minus1; i++) slice_id [i] u(v) } } if(subpics_present_flag) { signalled_subpic_id_flag u(1) if( signalled_subpic_id_flag) { signalled_subpic_id_length_minus1 ue(v) for( i = 0; i ＜= num_subpics_minus1; i++) subpic_id [i] u(v) } } entropy_coding_sync_enabled_flag u(1) … slice_header_extension_present_flag u(1) pps_extension_flag u(1) if( pps_extension_flag) while( more_rbsp_data()) pps_extension_data_flag u(1) rbsp_trailing_bits() }

In some embodiments, the syntax element rect_slice_flag of 0 in the PPS specifies that the tiles in each segment are in raster scan order and no segment information is sent in the PPS. The syntax element rect_slice_flag is 1 indicating that the tiles in each segment cover the rectangular area of the image, and the segment information is sent in the PPS. When subpics_present_flag is equal to 1, the value of rect_slice_flag should be equal to 1. In some embodiments, when brick_splitting_present_flag is equal to 1, the value of rect_slice_flag is set to 1. If it does not exist, the syntax element rect_slice_flag is inferred to be equal to 1.

The syntax element signalled_subpic_id_flag of 1 specifies that the sub-picture ID of each sub-picture is sent. The syntax element signalled_subpic_id_flag being 0 means that the sub-picture ID is not sent. If it does not exist, the value of signalled_subpic_id_flag is inferred to be equal to zero.

The syntax element signalled_subpic_id_length_minus1 plus 1 specifies the number of bits used to represent the syntax element subpic_id [i] when it exists, and the syntax element subpicture_id in the fragment header. In some embodiments, the value of signalled_subpic_id_length_minus1 is limited to the range of 0 to 7 (including 0 and 7). If it does not exist, the value of signalled_subpic_id_length_minus1 is inferred to be equal to Ceil(Log2(Max(2, num_subpics_minus1 + 1))) − 1.

The syntax element subpic_id [i] specifies the sub-picture ID of the i-th sub-picture. The length of the subpuic_id [i] syntax element is signalled_subpic_id_length_minus1 + 1 bit. If it does not exist, for each i in the range of 0 to num_subpics_minus1 (including 0 and num_subpics_minus1), the value of subpic_id[i] is inferred to be equal to i. Segments with the same sub-image ID collectively form a sub-image area.

The syntax element subpicture_id specifies the subpicture ID to which the current segment belongs. The length of the subpic_id syntax element is Ceil (Log2 (num_subpics_minus1 + 1)) bits. The value of subpicture_id is a mapping to subpic_id[i] specified in PPS. The transmitted sub-picture ID subpic_id [i] of the i-th sub-picture may be added to the PPS, and the transmitted sub-picture ID subpicure_id mapped to subpic_id [i] in the PPS may be added to the segment header. The following Table 5 shows an example syntax table of a segment header including sub-picture information. Table 5: The sub-image ID specified by the fragment header slice_header() { Descriptor slice_pic_parameter_set_id ue(v) if( subpics_present_flag) subpicture_id u(v) if( rect_slice_flag | | NumBricksInPic ＞ 1) slice_address u(v) if( !rect_slice_flag && !single_brick_per_slice_flag) num_bricks_in_slice_minus1 ue(v) non_reference_picture_flag u(1) slice_type ue(v) … if( slice_header_extension_present_flag) { slice_header_extension_length ue(v) for( i = 0; i ＜ slice_header_extension_length; i++) slice_header_extension_data_byte [i] u(8) } byte_alignment() }

In some embodiments, assuming that the sub-picture ID is unchanged for the CVS, the sub-picture ID may be sent in the segment header, PPS and/or SPS of the CVS. For example, according to Table 6 below , the transmitted sub-picture ID subpic_id [i] of the i-th sub-picture can be sent in SPS (rather than PPS): Table 6 : SPS specifies the sub-picture ID Seq_parameter_set_rbsp() { ... for( i = 0; i ＜ num_subpics_minus1; i++) { subpic_id [i] u(v) subpic_treated_as_pic_flag [i] u(1) loop_filter_across_subpic_enabled_flag [i] u(1) } } ... II. Example video encoder

Figure 3 illustrates an example video encoder 300 supporting sub-images. As shown in the figure, the video encoder 300 receives an input video signal from a video source 305 and encodes the signal into a bit stream 395. The video encoder 300 has a number of components or modules, which are used to encode the signal from the video source 305, at least including some selected from the following components: a transformation module 310, a quantization module 311, and an inverse quantization module 314 , Inverse transform module 315, intra-frame estimation module 320, intra-frame prediction module 325, motion compensation module 330, motion estimation module 335, in-loop filter 345, reconstructed image buffer 350, MV buffer 365 and MV prediction module 375, and entropy encoder 390. The motion compensation module 330 and the motion estimation module 335 are part of the inter prediction module 340.

In some embodiments, the modules 310-390 are modules of software instructions executed by one or more processing units (for example, processors) of a computing device or an electronic device. In some embodiments, the modules 310-390 are modules of hardware circuits implemented by one or more integrated circuits (ICs) of the electronic device. Although the modules 310-390 are illustrated as separate modules, some of the modules may be combined into a single module.

The video source 305 provides a raw video signal, which can present the pixel data of each video frame without compression. The subtractor 308 calculates the difference between the original video pixel data of the video source 305 and the predicted pixel data 313 from the motion compensation module 330 or the intra prediction module 325. The transform module 310 converts the difference (or residual pixel data or residual signal 309) into transform coefficients (for example, by performing discrete cosine transform or DCT). The quantization module 311 quantizes the transform coefficients into quantized data (or quantized coefficients) 312, which is encoded into a bit stream 395 by the entropy encoder 390.

The inverse quantization module 314 inversely quantizes the quantized data (or quantized coefficients) 312 to obtain transform coefficients, and the inverse transform module 315 performs inverse transform on the transform coefficients to generate a reconstruction residual 319. The reconstruction residual 319 is added to the predicted pixel data 313 together to generate the reconstructed pixel data 317. In some embodiments, the reconstructed pixel data 317 is temporarily stored in a line buffer (not shown) for intra-picture prediction and spatial MV prediction. The reconstructed pixels are filtered by the in-loop filter 345 and stored in the reconstructed image buffer 350. In some embodiments, the reconstructed image buffer 350 is a memory external to the video encoder 300. In some embodiments, the reconstructed image buffer 350 is the internal memory of the video encoder 300.

The intra-frame estimation module 320 performs intra-frame prediction based on the reconstructed pixel data 317 to generate intra-frame prediction data. The intra prediction data is provided to the entropy encoder 390 to be encoded as a bit stream 395. The intra prediction data is also used by the intra prediction module 325 to generate predicted pixel data 313.

The motion estimation module 335 generates the MV to perform inter-frame prediction on the reference pixel data of the previously decoded frame stored in the reconstructed image buffer 350. These MVs are provided to the motion compensation module 330 to generate predicted pixel data.

Instead of encoding the complete actual MV in the bitstream, the video encoder 300 uses MV prediction to generate a predicted MV, and the difference between the MV used for motion compensation and the predicted MV is encoded as residual motion data and It is stored in the bitstream 395.

The MV prediction module 375 generates a predicted MV based on a reference MV, which is generated to encode a previous video frame, that is, a motion compensation MV for performing motion compensation. The MV prediction module 375 retrieves the reference MV from the previous video frame in the MV buffer 365. The video encoder 300 stores the MV generated for the current video frame in the MV buffer 365 as a reference MV for generating the predicted MV.

The MV prediction module 375 uses the reference MV to create the predicted MV. The predicted MV can be calculated by spatial MV prediction or temporal MV prediction. The difference (residual motion data) between the prediction MV of the current frame and the motion compensation MV (motion compensation motion vector, MC MV for short) may be encoded into the bit stream 395 by the entropy encoder 390.

The entropy encoder 390 uses an entropy coding technique such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding to encode various parameters and data into the bit stream 395. The entropy encoder 390 encodes various header elements, flags, and quantized transform coefficients 312 and residual motion data into the bit stream 395 as syntax elements. The bit stream 395 is stored in a storage device or transmitted to the decoder via a communication medium (such as a network).

The in-loop filter 345 performs a filtering or smoothing operation on the reconstructed pixel data 317 to reduce coding artifacts, especially at the boundary of the block. In some embodiments, the filtering operation performed includes sample adaptive offset (SAO for short). In some embodiments, the filtering operation includes an adaptive loop filter (ALF for short).

FIG. 4 conceptually shows a part of the video encoder 300 that performs sub-image transmission. As shown in the figure, the entropy encoder 390 receives signaling from the video source 305, which specifies the parameters 410 of the sub-image, which will exist in the current encoding sequence 420 of the video image. These parameters 410 may indicate the number of sub-pictures existing in the current coding sequence. The parameter 410 may also indicate the position and geometry (height, width, and size) of each sub-image. The quantized coefficients 312 of different sub-images are provided to the data path of the encoder 300.

Based on the parameters of the sub-image, the entropy encoder 390 generates syntax elements that are used as the sub-image specification in the bit stream 395. These syntax elements may include the recognition word of the sub-image, the number of sub-images, and the position and geometric shape of the sub-image, and the position and geometric shape of the sub-image are specified according to the CTB/CTU-based sub-image grid. These syntax elements can be stored in the SPS of the currently encoded video sequence 420, the PPS of a single image in the video sequence, the segment header of a single segment in the sequence of images, and/or the VPS of the entire video. Examples of these syntax elements are described by referring to Tables 1A-1D and 3-6 above.

Figure 5 conceptually illustrates a process 500 for providing sub-picture specifications at a video encoder. In some embodiments, one or more processing units (eg, processors) of the computing device implement the encoder 300 to execute the process 500 by executing instructions stored in a computer-readable medium. In some embodiments, the electronic device implementing the decoder 300 performs the process 500.

The encoder receives (at block 510) the data in the bit stream to be encoded as a sequence of video images. The encoder sends (at block 520) the sub-image specifications of one or more sub-images in the video image sequence in the video image in the bitstream. In some embodiments, the syntax element in the SPS of the video image sequence indicates that there are one or more sub-images in the video image sequence. The SPS may also include a syntax element that specifies the number of sub-images of the video image sequence and the identification word of each sub-image. The identifier of the sub-image can also be sent in the segment header and/or PPS of the video image in the video image sequence. In some embodiments, the syntax element in the PPS of the video image in the video image sequence indicates that all segments of the video image are rectangular.

By providing an index for identifying the CTU of the sub-image 830, the encoder identifies (at block 530) the location and size of each sub-image.

In some embodiments, the identified CTU is located at a corner of the sub-image (eg, upper left or lower right). In some embodiments, indexes are assigned to sub-image grids, and different sub-image grids are assigned different indexes. In some embodiments, the index identifies a sub-image grid, which is defined as a CTU or corresponds to a CTU, so that the boundary of the sub-image grid is defined along the boundary of the CTU. In some embodiments, in the SPS of the video image sequence, the index of the sub-image position is sent.

According to the sub-picture specification, the encoder encodes each sub-picture of the video image sequence (at block 540). III. Example video decoder

Figure 6 shows an example video decoder 600 that supports sub-images. As shown in the figure, the video decoder 600 is an image decoding or video decoding circuit that receives a bit stream 695 and decodes the content of the bit stream into pixel data of a video frame for display. The video decoder 600 has several elements or modules for decoding the bit stream 695, including some selected from the following elements: inverse quantization module 611, inverse transformation module 610, intra prediction module 625, motion compensation Module 630, in-loop filter 645, decoded image buffer 650, MV buffer 665, MV prediction module 675 and parser 690. The motion compensation module 630 is a part of the inter prediction module 640.

In some embodiments, the modules 610-690 are modules of software instructions executed by one or more processing units (eg, processors) of the computing device. In some embodiments, the modules 610-690 are hardware circuit modules implemented by one or more ICs of the electronic device. Although the modules 610-690 are shown as separate modules, some modules may be combined into a single module.

The parser 690 (or entropy decoder) receives the bit stream 695, and performs initial analysis according to the syntax defined by the video codec or image codec standard. The parsed syntax elements include various header elements, flags, and quantized data (or quantized coefficients) 612. The parser 690 uses entropy coding and decoding techniques such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding to parse various syntax elements.

The inverse quantization module 611 performs inverse quantization on the quantized data (or quantized coefficients) 612 to obtain transform coefficients, and the inverse transform module 610 performs inverse transform on the transform coefficients 616 to generate a reconstructed residual signal 619. The reconstructed residual signal 619 is added to the predicted pixel data 613 from the intra-frame prediction module 625 or the motion compensation module 630 to generate decoded pixel data 617. The decoded pixel data is filtered by the in-loop filter 645 and stored in the decoded image buffer 650. In some embodiments, the decoded image buffer 650 is a memory external to the video decoder 600. In some embodiments, the decoded image buffer 650 is the internal memory of the video decoder 600.

The intra prediction module 625 receives intra prediction data from the bit stream 695 and generates predicted pixel data 613 from the decoded pixel data 617 stored in the decoded image buffer 650 accordingly. In some embodiments, the decoded pixel data 617 is also stored in a line buffer (not shown) used for intra-picture prediction and spatial MV prediction.

In some embodiments, the content of the decoded image buffer 650 is used for display. The display device 655 either retrieves the content of the decoded image buffer 650 for direct display, or retrieves the content of the decoded image buffer back to the display buffer. In some embodiments, the display device receives pixel values from the decoded image buffer 650 through pixel transmission.

The motion compensation module 630 generates predicted pixel data 613 from the decoded pixel data 617 stored in the decoded image buffer 650 according to the motion compensation MV (MC MV). By adding the residual motion data received from the bitstream 695 and the predicted MVs received from the MV prediction module 675, these motion-compensated MVs are decoded.

The MV prediction module 675 generates a predicted MV based on a reference MV, which is generated to decode a previous video frame, for example, a motion compensation MV for performing motion compensation. The MV prediction module 675 retrieves the reference MV of the previous video frame from the MV buffer 665. The video decoder 600 stores the motion compensation MV generated for decoding the current video frame in the MV buffer 665 as a reference MV for generating the predicted MV.

The in-loop filter 645 performs filtering or smoothing operations on the decoded pixel data 617 to reduce decoding artifacts, especially at block boundaries. In some embodiments, the performed filtering operation includes sample adaptive offset (SAO for short). In some embodiments, the filtering operation includes an adaptive loop filter (ALF for short).

FIG. 7 conceptually shows a part of the video decoder 600 that performs sub-picture transmission. As shown in the figure, the entropy decoder 690 provides quantization coefficients 612 to the data path of the video decoder 600, and the video decoder 600 generates pixel data to be displayed on the display device 655 for different sub-images. The display device may display the received pixel data according to the sub-picture parameter 710, where the sub-picture parameter 710 is used for the sub-picture that will appear in the current encoding sequence 720 of the picture. The parameter 710 may also indicate the number of sub-pictures that will appear in the currently encoded sequence. These parameters can also indicate the position and geometry (size, height, width) of each sub-image. The entropy decoder 690 provides sub-picture parameters based on the syntax elements decoded from the bitstream 695.

As shown in the figure, the entropy decoder (parser) 690 receives from the bitstream 695 syntax elements used as sub-picture specifications. These syntax elements may include the recognition words of the sub-image, the number of sub-images, and the location and geometric shape of the sub-image, and the location and geometric shape of the sub-image are specified according to the CTB/CTU-based sub-image grid. These syntax elements can be stored in the SPS of the currently encoded video sequence 720, the PPS of a single image in the video sequence, the segment header of a single segment in the sequence of images, and/or the VPS of the entire video. Examples of these syntax elements are described by referring to Tables 1A-1D and 3-6 above.

Figure 8 conceptually illustrates a process 800 for processing sub-picture specifications at the video decoder. In some embodiments, one or more processing units (eg, processors) of the computing device implement the decoder 600 to execute the process 800 by executing instructions stored in a computer-readable medium. In some embodiments, the electronic device implementing the decoder 600 performs the process 800.

The decoder receives (at block 810) from the bitstream the data to be decoded into a sequence of video images. The decoder receives (at block 820) the sub-image specifications of one or more sub-images in the video image sequence from the bitstream. In some embodiments, the syntax element in the SPS of the video image sequence indicates that there are one or more sub-images in the video image sequence. The SPS may also include a syntax element that specifies the number of sub-images of the video image sequence and the identification word of each sub-image. The identifier of the sub-image can also be sent in the segment header and/or PPS of the video image in the video image sequence. In some embodiments, the syntax element in the PPS of the video image in the video image sequence indicates that all segments of the video image are rectangular.

The decoder identifies (at block 830) the location and size of each sub-image by providing an index that identifies the CTU of the sub-image. In some embodiments, the identified CTU is located at a corner of the sub-image (eg, upper left or lower right). In some embodiments, indexes are assigned to sub-image grids, and different sub-image grids are assigned different indexes. In some embodiments, the index identifies a sub-image grid, which is defined as a CTU or corresponds to a CTU, so that the boundary of the sub-image grid is defined along the boundary of the CTU. In some embodiments, the index of the sub-image position is sent in the SPS of the video image sequence.

According to the sub-image specification, the decoder reconstructs (at block 840) each sub-image of the sequence of video images. IV. Example electronic system

Many of the above features and applications are implemented as software processes, which are designated as a set of instructions recorded on a computer readable storage medium (also known as a computer readable medium). When these instructions are executed by one or more computing or processing units (for example, one or more processors, processor cores, or other processing units), they cause the processing units to perform the actions indicated in the instructions. Examples of computer-readable media include, but are not limited to, compact disk read only memory (CD-ROM), flash drives, random-access memory (RAM) chips, Hard disk drives, erasable programmable read only memory (erasable programmable read only memory, referred to as EPROM), electronically erasable programmable read only memory (electrically erasable programmable read only memory, referred to as EEPROM). The computer-readable medium does not include carrier waves and telecommunications signals transmitted through wireless or wired connections.

In this manual, the term "software" means to include firmware in read-only memory or an application program stored in a magnetic storage device, which can be read into memory for processing by the processor . At the same time, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program, while retaining different software inventions. In some embodiments, multiple software inventions can be implemented as independent programs. Finally, any combination of independent programs that together implement the software invention described herein is within the scope of the present invention. In some embodiments, when a software program is installed to operate on one or more electronic systems, the software program defines one or more specific machine implementations that execute and implement the operations of the software program .

Figure 9 conceptually illustrates an electronic system 900 implementing some embodiments of the present disclosure. The electronic system 1100 may be a computer (for example, a desktop computer, a personal computer, a tablet computer, etc.), a telephone, a personal digital assistant (PDA for short), or any other kind of electronic device. Such electronic systems include various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 900 includes a bus 905, a processing unit 910, a graphics-processing unit (GPU) 915, a system memory 920, a network 925, a read-only memory 930, a permanent storage device 935, an input device 940 and Output device 945.

The bus 905 collectively represents all system buses, peripheral device buses, and chipset buses of the numerous internal devices that are communicatively connected with the electronic system 900. For example, the bus 905 is communicatively connected with the processing unit 910 through the GPU 915, the read-only memory 930, the system memory 920 and the permanent storage device 935.

From these various memory units, the processing unit 910 retrieves the instructions to be executed and the data to be processed in order to perform the processing of the present disclosure. In different embodiments, the processing unit 910 may be a single processor or a multi-core processor. Some instructions are passed to the GPU 915 and executed by the GPU 915. The GPU 915 can offload various calculations or supplement the image processing provided by the processing unit 910.

The read-only memory 930 stores static data and instructions required by the processing unit 910 and other modules of the electronic system. On the other hand, the permanent storage device 935 is a read-write memory device. The device is a non-volatile memory unit that stores instructions and data even when the electronic system 900 is turned off. Some embodiments of the present disclosure use a large-capacity storage device (such as a floppy disk or optical disc and its corresponding drive) as the permanent storage device 935.

Other embodiments use removable storage devices (such as floppy disks, flash memory devices, etc., and their corresponding disk drives) as permanent storage devices. Similar to the permanent storage device 935, the system memory 920 is a read-write memory device. However, unlike the permanent storage device 935, the system memory 920 is a volatile read-write memory, such as a random access memory. The system memory 920 stores some instructions and data required by the processor during operation. In some embodiments, the process according to the present disclosure is stored in the system memory 920, the permanent storage device 935, and/or the read-only memory 930. For example, various memory units include instructions for processing multimedia clips according to them. In some embodiments. From these various memory units, the processing unit 910 retrieves the instructions to be executed and the data to be processed to execute the processes of some embodiments.

The bus bar 905 is also connected to the input device 940 and the output device 945. The input device 940 enables the user to send information and selection commands to the electronic system. The input device 940 includes an alphanumeric keyboard and pointing device (also referred to as a “cursor control device”), a camera (for example, a webcam), a microphone or similar devices for receiving voice commands and the like. The output device 945 displays images generated by the electronic system or otherwise outputs data. The output device 945 includes a printer and a display device, such as a cathode ray tube (CRT) or a liquid crystal display (LCD), and a speaker or similar audio output device. Some embodiments include devices such as touch screens that are used as input devices and output devices at the same time.

Finally, as shown in Figure 9, the bus 905 also couples the electronic system 900 to the network 925 through a network interface card (not shown). In this way, the computer can be part of a computer network (such as a local area network (LAN), a wide area network (WAN) or an intranet), or one of multiple networks , Such as the Internet. Any or all components of the electronic system 900 can be used in conjunction with the present disclosure.

Some embodiments include electronic components, such as a microprocessor, storage device, and memory, which store computer program instructions in a machine-readable medium or a computer-readable medium (optionally referred to as a computer-readable storage medium, a machine-readable medium). Read media or machine-readable storage media). Some examples of computer-readable media include RAM, ROM, read-only compact disc (CD-ROM), recordable compact disc (CD-R), and rewritable compact disc (CD-R). CD-RW), read-only digital versatile disc (for example, DVD-ROM, double-layer DVD-ROM), various recordable/readable DVDs (for example, DVD RAM, DVD-RW, DVD +RW, etc.), flash memory (such as SD card, mini SD card, micro SD card, etc.), magnetic and/or solid state drives, read-only and recordable Blu-Ray® (Blu-Ray®) discs, ultra-high Density CDs and any other optical or magnetic media, as well as floppy disks. The computer-readable medium can store a computer program executed by at least one processing unit, and includes an instruction set for performing various operations. Examples of computer programs or computer codes include machine code, such as machine code generated by a compiler, and documents containing high-level codes executed by computers, electronic components, or microprocessors using an interpreter.

When the above discussion mainly refers to a microprocessor or multi-core processor that executes software, many of the above functions and applications are executed by one or more integrated circuits, such as application specific integrated circuits (ASIC) Or field programmable gate array (FPGA). In some embodiments, such integrated circuits execute instructions stored on the circuit itself. In addition, some embodiments execute software stored in a programmable logic device (PLD), ROM or RAM device.

As used in the specification of the present invention and the scope of any patent application, the terms "computer", "server", "processor" and "memory" all refer to electronic devices or other technical devices. These terms do not include people or groups. For the purpose of description, the term display or display device refers to displaying on an electronic device. As used in the specification of the present invention and any patent application, the terms “computer-readable medium”, “computer-readable medium” and “machine-readable medium” are completely limited to tangible and tangible objects. Store information in a readable form. These terms do not include any wireless signals, wired download signals, and any other short-lived signals.

When the present invention has been described in combination with many specific details, those skilled in the art will recognize that the present invention can be implemented in other specific forms without departing from the spirit of the present invention. In addition, a large number of diagrams (including Figures 5 and 8) conceptually show the process. The specific operations of these processes may not be executed in the exact order shown and described. These specific operations may not be executed in a continuous series of operations, and different specific operations may be executed in different embodiments. In addition, this process can be implemented using several sub-processes, or as part of a larger macro process. Therefore, those skilled in the art will understand that the present invention is not limited by the foregoing illustrative details, but is defined by the scope of the patent application. Additional description

The subject matter described herein sometimes represents different components, which are contained in or connected to other different components. It can be understood that the described structure is only an example, and can be implemented by many other structures to achieve the same function. Conceptually, any arrangement of components that achieve the same function is actually "associated." , In order to achieve the desired function. Therefore, regardless of the structure or the intermediate components, any two components combined to achieve a specific function are regarded as "interrelated" to achieve the required function. Likewise, any two associated components are regarded as being "operably connected" or "operably coupled" to each other to achieve specific functions. Any two components that can be associated with each other are also considered to be "operably coupled" to each other to achieve specific functions. Any two components that can be associated with each other are also considered to be "operably coupled" to each other to achieve specific functions. Specific examples of operable connections include, but are not limited to, physically pairable and/or physically interacting components, and/or wirelessly interactable and/or wirelessly interacting components, and/or logically interacting and/or Logically interactable components.

In addition, with regard to substantially any plural and/or singular term used herein, those skilled in the art can convert from the plural to the singular and/or from the singular to the plural according to the context and/or application. For the sake of clarity, the present invention clearly sets forth different singular/plural arrangements.

In addition, generally, those skilled in the art can understand that the terms used in the present invention, especially those in the scope of the patent application, such as the subject matter of the scope of the patent application, are usually used as "open" terms. For example, "including" should be interpreted as "including But not limited to", "have" should be interpreted as "at least" and "including" should be interpreted as "including but not limited to" and so on. Those skilled in the art can further understand that if a specific number of the content of the patent application is planned to be introduced, it will be clearly indicated in the scope of the patent application, and will not be displayed when there is no such content. For example, to help understanding, the following patent application scope may include the phrases "at least one" and "one or more" to introduce the content of the patent application scope. However, the use of these phrases should not be construed as implying the use of the indefinite article "一" or "一" to introduce the content of the patent application, while limiting the scope of any particular application. Even when the same patent application includes the introductory phrases "one or plural" or "at least one", indefinite articles, such as "one" or "one", should be interpreted as meaning at least one or more. The same holds true for the use of a clear description of the scope of patent application. In addition, even if a certain amount of introductory content is explicitly quoted, those skilled in the art can recognize that such content should be interpreted as indicating the number of references. For example, "two references" without other modifications means at least two One citation, or two or more citations. In addition, when an expression similar to "at least one of A, B and C" is used, the expression is usually so that those skilled in the art can understand the expression, for example, "The system includes A, B, and C. "At least one" shall include, but is not limited to, a system with A alone, a system with B alone, a system with C alone, a system with A and B, a system with A and C, a system with B and C, and/or Systems with A, B, and C, etc. Those skilled in the art can further understand that, whether in the specification, the scope of the patent application or in the drawings, any separated words and/or phrases represented by two or more alternative terms should be understood as, Include one of these terms, one of them, or the possibility of both terms. For example, "A or B" should be understood as the possibility of "A", or "B", or "A and B".

From the foregoing, it is understood that the present invention has described various embodiments for illustrative purposes, and various modifications can be made without departing from the scope and spirit of the present invention. Therefore, the various embodiments disclosed herein are not intended to limit, and the true scope and application are represented by the scope of patent applications.

100: sequence 110: Image 120: CTU 210: sub-image 220: sub-image 230: sub-image 240: sub-image 300: RF circuit 305: Baseband circuit 308: Idle channel detection circuit 310: DFS channel check circuit 311: Quantization Module 312: quantization coefficient 313: Predict pixel data 314: Inverse Quantization Module 315: Inverse Transformation Module 316: transform coefficient 317: Reconstruct pixel data 319: reconstruction residual 320: Intra-frame estimation module 325: Intra Prediction Module 330: Motion compensation module 335: Motion Estimation Module 340: Inter prediction module 345: In-loop filter 350: reconstruction image buffer 365: MV buffer 375: MV prediction module 395: bit stream 410: Parameters 420: Current encoded video sequence 500: Process 510, 520, 530, 540: steps 600: decoder 610: Inverse Transformation Module 611: Inverse Quantization Module 612: quantization coefficient 613: Predicted pixel data 616: transform coefficient 617: pixel data 619: reconstruction residual signal 625: Intra Prediction Module 630: Motion compensation module 640: Inter-frame prediction module 645: In-loop filter 650: decoded image buffer 655: display device 665: MV buffer 675: MV prediction module 690: Entropy Decoder 695: Bit Stream 710: Parameters 720: The currently encoded video sequence 800: process 810, 820, 830, 840: steps 900: Electronic system 905: Bus 910: Processing Unit 915: GPU 920: System memory 925: Internet 930: Read-only memory 935: Permanent Storage Device 940: input device 945: output device

The following figures are used to provide a further understanding of the present invention, and are incorporated into and constitute a part of the present invention. These drawings illustrate the embodiments of the present invention, and together with the description are used to explain the principles of the present invention. In order to clearly illustrate the concept of the present invention, some elements may not be shown to scale compared with the dimensions in the actual implementation, and these illustrations do not need to be drawn to scale. Figures 1a-e conceptually show CTB or CTU-based grid units used to specify sub-images of a video sequence. Figure 2 shows a grid of sub-images based on CTU or CTB, which are indexed in the order of raster scan within the image to specify the sub-images. Figure 3 shows an example video encoder that supports sub-images. Figure 4 conceptually shows a part of a video encoder that implements sub-picture transmission. Figure 5 conceptually shows the process of providing sub-picture specifications at the video encoder. Figure 6 shows an example video decoder that supports sub-images. Fig. 7 conceptually shows the part of the video decoder that implements the sub-picture transmission. Figure 8 conceptually shows the process of processing sub-picture specifications at the video decoder. Figure 9 conceptually illustrates an electronic system for implementing some embodiments of the present disclosure.

800: process

810, 820, 830, 840: steps

Claims (12)

A video decoding method, including: Receive data from a bit stream that will be decoded into a sequence of video images; Receive the sub-image specification of one or more sub-images in the video image sequence from the bit stream, and by providing an index for identifying a coding tree unit for a corresponding sub-image, the sub-image The image specification identifies a position and a size for each of the one or more sub-images; and According to the sub-image specification, each sub-image of the one or more sub-images of the video image sequence is reconstructed. The video decoding method according to claim 1, wherein different coding tree units correspond to multiple different sub-image grids assigned multiple different indexes. The video decoding method according to claim 2, wherein the multiple boundaries of the multiple sub-image grids are defined using multiple boundaries of the coding tree unit. The video decoding method according to claim 1, wherein the coding tree unit identified by the provided index is located at a corner of the sub-image. The video decoding method according to claim 1, wherein the index is provided by a sequence parameter set of the video image sequence. The video decoding method according to claim 5, wherein the sequence parameter set further includes a syntax element, and the syntax element specifies a number of sub-images of the video image sequence. The video decoding method according to claim 5, wherein an identifier of the sub-image is sent in the sequence parameter set. The video decoding method according to claim 1, wherein an identifier of the sub-image is sent in a segment header of a segment. The video decoding method according to claim 1, wherein an identifier of the sub-image is sent in an image parameter set of a video image of the video image sequence. The video decoding method according to claim 1, wherein a syntax element of a sequence parameter set of the video image sequence indicates that one or more sub-images exist in the image sequence, and a video of the video image sequence A syntax element of an image parameter set of an image indicates that all segments of the video image are rectangular. A video coding method, including: Receive data that will be encoded as a bit stream of a video image sequence; Send the sub-image specification of one or more sub-images of the video image sequence in the bit stream. By providing an index for identifying a coding tree unit for a corresponding sub-image, the sub-image The specification identifies a position and a size for each of the one or more sub-images; and According to the sub-image specification, each sub-image of the one or more sub-images of the video image sequence is encoded. An electronic device, including: A video decoder circuit is set to perform the following operations including: Receive data from a bit stream that will be decoded into a sequence of video images; Receive the sub-image specification of one or more sub-images in the video image sequence from the bit stream, and by providing an index for identifying a coding tree unit for a corresponding sub-image, the sub-image The image specification identifies a position and a size for each of the one or more sub-images; and According to the sub-image specification, each sub-image of the one or more sub-images of the video image sequence is reconstructed.

Images