WO2025256480A1

WO2025256480A1 - Video encoding and decoding method, electronic device and storage medium

Info

Publication number: WO2025256480A1
Application number: PCT/CN2025/099688
Authority: WO
Inventors: Shuang PENG; Xue ZHANG; Dong JIANG; Jucai LIN; Jun Yin
Original assignee: Zhejiang Dahua Technology Co Ltd
Current assignee: Zhejiang Dahua Technology Co Ltd
Priority date: 2024-06-11
Filing date: 2025-06-06
Publication date: 2025-12-18
Anticipated expiration: 2026-12-11
Also published as: CN121173957A

Abstract

One or more embodiments of the present disclosure provide a video encoding method and a video decoding method. The video decoding method implemented on a decoding terminal, the video decoding method comprising: determining, by decoding a bitstream, an inverse quantization residual block of a target block in a prediction mode of intra block copy; dividing the inverse quantization residual block into at least one sub-block; selecting at least one target sub-block from the at least one sub-block; obtaining a prediction residual of the at least one target sub-block by performing an inverse transformation on the at least one target sub-block; and generating, based on the prediction residual of the at least one target sub- block, a decoded image.

Description

VIDEO ENCODING AND DECODING METHOD, ELECTRONIC DEVICE AND STORAGE MEDIUM

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Chinese Application No. 202410748617.5, filed on June 11, 2024, the entire contents of which are incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to the technical field of video encoding and decoding, and in particular to a video encoding and decoding method, electronic device, and storage medium.

BACKGROUND

Data volume of a video is relatively large, so it usually needs to compress the pixel data (e.g., RGB, YUV, etc. ) of the video to obtain a compressed video bitstream. The video bitstream is then transmitted to a terminal device of a user via a wired or wireless network, and the terminal device can display the video after decoding the video bitstream. The entire encoding process of the video generally includes operations such as block division, prediction, transformation, quantization, and encoding, while the decoding process includes entropy decoding, inverse quantization, inverse transformation, and other operations. However, existing video encoding and decoding methods have certain shortcomings. For example, the transformation in the video encoding method and the inverse transformation in the video decoding method are difficult to adapt to different residual distributions.

SUMMARY

One of the embodiments of the present disclosure provides a video decoding method, implemented on a decoding terminal, the method comprising: determining, by decoding a bitstream, an inverse quantization residual block of a target block in a prediction mode of intra block copy; dividing the inverse quantization residual block into at least one sub-block; selecting at least one target sub-block from the at least one sub-block; obtaining a prediction residual of the at least one target sub-block by performing an inverse transformation on the at least one target sub-block; and generating, based on the prediction residual of the at least one target sub-block, a decoded image.

One of the embodiments of the present disclosure provides a video decoding method, comprising: determining, by decoding a bitstream, an inverse quantization residual block of a target block in a prediction mode of intra block copy; obtaining an inverse transform result by performing an inverse transformation on the inverse quantization residual block using a transform kernel corresponding to the inverse quantization residual block; and determining, based on the inverse transform result, a decoded image; wherein the transform kernel corresponding to the inverse quantization residual block includes a transform kernel determined using at least one of DST7 and DCT8; or the transform kernel corresponding to the inverse quantization residual block is determined based on a transform kernel set, and the transform kernel set includes one or more transform kernels determined using at least one of the DST7 and the DCT8.

One of the embodiments of the present disclosure provides a video decoding method, comprising: determining, by decoding a bitstream, an inverse quantization residual block of a target block in a prediction mode of intra block copy; obtaining an inverse transform result by performing an inverse transformation on the inverse quantization residual block using a transform kernel corresponding to the inverse quantization residual block; and determining, based on the inverse transform result, a decoded image; wherein the transform kernel corresponding to the inverse quantization residual block includes a preset transform kernel; or the transform kernel corresponding to the inverse quantization residual block is determined based on a transform kernel set, the transform kernel set includes a plurality of preset transform kernels, and a horizontal transform kernel and a vertical transform kernel in the plurality of preset transform kernel are different.

One of the embodiments of the present disclosure provides a video encoding method, implemented on an encoding terminal, the video encoding method comprising: obtaining a target frame; dividing the target frame into a plurality of image blocks; for a target block among the plurality of image blocks, determining a prediction residual block of the target block in a prediction mode of intra block copy; dividing the prediction residual block into at least one sub-block; selecting at least one sub-block to be transformed from the at least one sub-block; obtaining a transform result by transforming, using a target transform kernel, the at least one sub-block to be transformed; and determining a bitstream of the target frame based on the transform result.

One of the embodiments of the present disclosure provides a video encoding method, implemented on an encoding terminal, the video encoding method comprising: obtaining a target frame; dividing the target frame into a plurality of image blocks; for a target block among the plurality of image blocks, determining a prediction residual block of the target block in a prediction mode of intra block copy; transforming the prediction residual block using a target transform kernel corresponding to the prediction residual block; and determining a bitstream of the target frame based on a transform result of the prediction residual block; wherein the target transform kernel corresponding to the prediction residual block includes at least one transform kernel determined based on at least one of Discrete Sine Transform Type VII (DST7) and Discrete Cosine Transform Type VIII (DCT8) .

One of the embodiments of the present disclosure provides a video encoding method, implemented on an encoding terminal, the video encoding method comprising: obtaining a target frame; dividing the target frame into a plurality of image blocks; for a target block among the plurality of image blocks, determining a prediction residual block of the target block in a prediction mode of intra block copy; transforming the prediction residual block based on a target transform kernel corresponding to the prediction residual block; and determining a bitstream of the target frame based on a transform result of the prediction residual block; wherein the target transform kernel includes at least one transform kernel, and for each of the at least one transform kernel, a horizontal transform kernel and a vertical transform kernel of the transform kernel are different.

One of the embodiments of the present disclosure provides an electronic device, comprising a processor and a memory connected with each other. The memory is configured to store programs, and the processor is configured to implement the method described in any embodiment of the present disclosure.

One of the embodiments of the present disclosure provides a non-transitory computer-readable storage medium, comprising programs that, when executed by a processor, direct the processor to implement the method described in any embodiment of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure will be further illustrated by way of exemplary embodiments, which will be described in detail by means of the accompanying drawings. These embodiments are not limiting, and in these embodiments, the same numbering indicates the same structure, wherein:

FIG. 1A is a schematic diagram illustrating an exemplary application scenario of a video encoding and decoding system according to some embodiments of the present disclosure;

FIG. 1B is a schematic diagram illustrating software/hardware of a computing device according to some embodiments of the present disclosure;

FIG. 1C is a block diagram illustrating an encoding terminal according to some embodiments of the present disclosure;

FIG. 1D is a block diagram illustrating a decoding terminal according to some embodiments of the present disclosure;

FIG. 1E is a flowchart illustrating an exemplary video encoding method according to some embodiments of the present disclosure;

FIG. 2 is a schematic diagram illustrating a sub-block division mode of no division according to some embodiments of the present disclosure;

FIG. 3 is a schematic diagram illustrating a sub-block division mode of dividing a residual block in a vertical direction according to some embodiments of the present disclosure;

FIG. 4 is a schematic diagram illustrating a sub-block division mode of dividing a residual block in a vertical direction according to some embodiments of the present disclosure;

FIG. 5 is a schematic diagram illustrating a sub-block division mode of dividing a residual block in a horizontal direction according to some embodiments of the present disclosure;

FIG. 6 is a schematic diagram illustrating a sub-block division mode of dividing a residual block in a horizontal direction according to some embodiments of the present disclosure;

FIG. 7 is a schematic diagram illustrating a sub-block division mode of dividing a residual block in a vertical direction and a horizontal direction according to some embodiments of the present disclosure;

FIG. 8 is a schematic diagram illustrating a sub-block division mode of dividing a residual block in a vertical direction and a horizontal direction according to some embodiments of the present disclosure;

FIG. 9 is a schematic diagram illustrating a sub-block division mode according to some embodiments of present disclosure;

FIG. 10 is a flowchart illustrating an exemplary process of sub-block division according to some embodiments of the present disclosure;

FIG. 11 is a flowchart illustrating an exemplary process of selecting a sub-block to be transformed according to some embodiments of the present disclosure;

FIG. 12 is a schematic diagram illustrating transform kernels corresponding to different sub-block selection modes according to some embodiments of the present disclosure;

FIG. 13 is a schematic diagram illustrating transform kernels corresponding to different sub-block selection modes according to some embodiments of the present disclosure;

FIG. 14 is a schematic diagram illustrating transform kernels corresponding to different sub-block selection modes according to some embodiments of the present disclosure;

FIG. 15 is a flowchart illustrating an exemplary video encoding method according to some embodiments of the present disclosure;

FIG. 16 is a flowchart illustrating an exemplary video encoding method according to some embodiments of the present disclosure;

FIG. 17 is a flowchart illustrating an exemplary video decoding method according to some embodiments of the present disclosure;

FIG. 18 is a flowchart illustrating an exemplary video decoding method according to some embodiments of the present disclosure; and

FIG. 19 is a flowchart illustrating an exemplary video decoding method according to some embodiments of the present disclosure.

DETAILED DESCRIPTION

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the accompanying drawings required to be used in the description of the embodiments are briefly described below. Obviously, the accompanying drawings in the following description are only some examples or embodiments of the present disclosure, and it is possible for a person of ordinary skill in the art to apply the present disclosure to other similar scenarios in accordance with these drawings without creative labor. Unless obviously obtained from the context or the context illustrates otherwise, the same numeral in the drawings refers to the same structure or operation.

It should be understood that the terms “system, ” “device, ” “unit” and/or “module” used herein are a way to distinguish between different components, elements, parts, sections, or assemblies at different levels. However, the terms may be replaced by other expressions if other words accomplish the same purpose.

As shown in the present disclosure and in the claims, unless the context clearly suggests an exception, the words “one, ” “a, ” “an, ” “one kind, ” and/or “the” do not refer specifically to the singular, but may also include the plural. Generally, the terms “including” and “comprising” suggest only the inclusion of clearly identified steps and elements, however, the steps and elements that do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

A target frame may be divided into a plurality of image blocks using a video encoding and decoding technique. Each of the image blocks may be used as a coding unit, and a transform result may be obtained by transforming a prediction residual block of a target block in the image block, thereby determining a bitstream of the target frame; and an inverse quantization residual block may be obtained by decoding the bitstream, and a prediction residual may be obtained by performing an inverse transformation on the inverse quantization residual block, thereby generating a decoded image of the target frame.

In the encoding and decoding technique, when the coding unit (CU) is in a prediction mode of intra block copy (IBC) , a transform set corresponding to the prediction mode of IBC is controlled by a syntax of picture_ts_enable_flag in an image/video frame header. When picture_ts_enable_flag = 0, the transform set of an IBC encoding/decoding module only includes a discrete cosine transform type II (DCT2) , i.e., the IBC encoding/decoding module performs a transformation/inverse transformation on the prediction residual (e.g., the prediction residual block, and the inverse quantization residual block) using the DCT2; when picture_ts_enable_flag = 1, the transform set of the IBC encoding /decoding module is {DCT2, TS (transform skip, i.e., no transform) } , and the specific transform process used is indicated by a syntax of enhanced_ts_flag.

In addition, the above IBC transformation technique is only for a brightness component of the image block, and for a chrominance component of the image block, the transformation process is only the DCT2.

In the encoding and decoding technique, the IBC transformation technique targets the entire residual block (e.g., the prediction residual block, or the inverse quantization residual block) , while the actual residual may only be distributed in a partial region of the residual block, so bits are also consumed for places where there is no residual.

Accordingly, the present disclosure provides a video encoding and decoding method. The video encoding method includes: obtaining a target frame; dividing the target frame into a plurality of image blocks; for a target block among the plurality of image blocks, determining a prediction residual block of the target block in a prediction mode of IBC; dividing the prediction residual block into at least one sub-block; selecting at least one sub-block to be transformed from the at least one sub-block; obtaining a transform result by transforming, using a target transform kernel, the at least one sub-block to be transformed; and determining a bitstream of the target frame based on the transform result. The video decoding method includes: determining, by decoding a bitstream of a target frame obtained from an encoding terminal, an inverse quantization residual block of a target block of the target frame in a prediction mode of IBC; dividing the inverse quantization residual block into at least one sub-block; selecting at least one target sub-block from the at least one sub-block; obtaining a prediction residual of the at least one target sub-block by performing an inverse transformation on the at least one target sub-block; and generating, based on the prediction residual of the at least one target sub-block, a decoded image of the target frame.

The video encoding and decoding method provided in the embodiments of present disclosure can divide the residual block (e.g., the prediction residual block, and the inverse quantization residual block) to obtain the at least one sub-block, and can selectively perform the transformation/inverse transformation on the at least one sub-block (e.g., the at least one sub-block to be transformed, and the at least one target sub-block ) , so that the transformation and the inverse transformation can be performed on the places where the residual exists, thereby saving bitstream consumption, improving the encoding efficiency and compression ratio, and reducing redundant information while ensuring the video quality.

FIG. 1A is a schematic diagram illustrating an exemplary application scenario of a video encoding and decoding system according to some embodiments of the present disclosure.

An image encoding and decoding system 100 may include an encoding terminal 110 and a decoding terminal 120. The encoding terminal 110 and the decoding terminal 120 may be a computer device, an electronic device, or the like, or any device with a processing capability, such as a computer, a server, a mobile phone, a tablet, etc. These devices may have a processor for executing the operations of the method disclosed in the embodiments of the present disclosure, which is not limited in the present disclosure. The encoding terminal 110 and the decoding terminal 120 may communicate with each other and may be configured to perform operations of encoding and/or decoding an image/video.

The encoding terminal 110 may be configured to perform preprocessing of the image/video and processing of operations related to encoding compression to obtain bitstream data. The encoding terminal 110 refers to a terminal that implements an encoding operation. The encoding terminal 110 may process an image or video frame through an encoder, and transform visual information of the image or video frame into compact feature representation information (e.g., a bitstream) that can be recognized by a machine, so as to facilitate the transmission of video data via a network. In some embodiments, the encoding terminal 110 may include a business executor (e.g., a service provider for providing encoding business) , or a software and hardware system that performs the encoding operation, where the software and hardware system may have at least one function related to the encoding operation, such as transformation, quantization, fusion, entropy encoding, auxiliary transformation, auxiliary quantization, auxiliary entropy encoding, etc.

The encoding terminal 110 may transmit the bitstream data to the decoding terminal 120, and the decoding terminal 120 may receive the bitstream data from the encoding terminal 110 and may be configured to perform operations including decoding the bitstream data, and operations including a back-end visual task, such as processing and classification related to the image/video. Corresponding to the encoding terminal 110, the decoding terminal 120 refers to a terminal that implements a decoding operation. In some embodiments, the decoding terminal may include a business executor (e.g., a service provider for providing decoding business) , or a software and hardware system that performs the decoding operation, where the software and hardware system may have functions related to the decoding operation such as inverse transformation, inverse quantization, entropy decoding, inverse auxiliary transformation, inverse auxiliary quantization, auxiliary entropy decoding, etc.

The image encoding and decoding system 100 may further include a storage device 130. The encoding terminal 110 may transmit the bitstream data to the storage device 130. The decoding terminal 120 may obtain the bitstream data from the storage device 130 for decoding. In some embodiments, the storage device 130 may include a mass memory, a removable memory, a volatile read-write memory, a read-only memory (ROM) , or the like, or any combination thereof. An exemplary mass memory may include a magnetic disk, an optical disk, a solid-state drive, etc. An exemplary removable memory may include a flash drive, a floppy disk, an optical disk, a memory card, a zip disk, a magnetic tape, etc. An exemplary volatile read-and-write memory may include a random-access memory (RAM) . An exemplary RAM may include a dynamic RAM (DRAM) , a double date rate synchronous dynamic RAM (DDR SDRAM) , a static RAM (SRAM) , a thyristor RAM (T-RAM) , and a zero-capacitor RAM (Z-RAM) , etc. An exemplary ROM may include a mask ROM (MROM) , a programmable ROM (PROM) , an erasable programmable ROM (EPROM) , an electrically erasable programmable ROM (EEPROM) , a compact disk ROM (CD-ROM) , and a digital versatile disk ROM, etc. In some embodiments, the storage device 130 may be implemented on a cloud platform. In some embodiments, the storage device 130 may be integrated into the encoding terminal 110 and/or the decoding terminal 120.

In some embodiments, the encoding terminal 110 may be configured to implement the image/video encoding method in the present disclosure. The decoding terminal 120 may be configured to implement the image/video decoding method in the present disclosure.

FIG. 1B is a schematic diagram illustrating software/hardware of a computing device according to some embodiments of the present disclosure.

In some embodiments, a computing device 200 may include a server, a personal computer, a laptop, a smart phone, a tablet computer, a smart mobile phone, etc. The encoding terminal 110 and/or the decoding terminal 120 may be implemented on the computing device 200.

As shown in FIG. 2, the computing device 200 may include a processor 210, a storage device, an input/output 230, and a communication port 240. The storage device may include a non-volatile storage medium 225 and a memory 223. The processor 210, the storage device (e.g., the memory 223, and the non-volatile storage medium 225) , and the input/output 230 may be connected through a system bus 250. The communication port 240 may be connected to the system bus 250 through the input/output 230.

The processor 210 may execute computer instructions (e.g., a program code) and may perform functions of a processing device in accordance with the techniques described in the present disclosure (e.g., the video encoding method, and the video decoding method) . In some embodiments, the processor 210 may include one or more hardware processors, such as a microcontroller, microprocessor, etc. For illustrative purposes only, only one processor is described in the computing device 200. However, it is noted that the computing device 200 may also include a plurality of processors. Operations and/or methods described in the present disclosure that are performed by a single processor may also be performed by a plurality of processors together or separately. For example, if a processor of the computing device 200 described in the present disclosure performs an operation A and an operation B, it should be appreciated that the operation A and the operation B may also be performed by two or more different processors of the computing device 200 jointly or separately (e.g., a first processor executes the operation A and a second processor executes the operation B, or the first processor and the second processor jointly execute the operations A and B) .

The storage device may store data/information obtained from one or more components (e.g., the encoding terminal 110, the decoding terminal 120, or the storage device 130) of the application scenario 100. For example, the non-volatile storage medium 225 may store an operating system, a computer program, and a database. The memory 223 may provide an environment for operation of the operating system and the computer programs in the non-volatile storage medium 225. The database may be configured to store video encoding data (e.g., a target frame, a prediction residual block, a sub-block division mode syntax, a sub-block selection syntax, a transformation index syntax, a transform kernel set, a first preset size, a second preset size, etc. ) , and video decoding data (e.g., a target sub-block, an inverse quantization residual block, a decoded image, etc. ) . The processor 210 may execute the computer programs to implement the video encoding method/the video decoding method described herein.

The input/output 230 may be configured to exchange information between the processor 210 and an external device, such as the decoding terminal 120 and/or the storage device 130 when the encoding terminal 110 is implemented on the computing device 200. In some embodiments, the input/output 230 may include an input device and an output device. The input device may include a keyboard, a mouse, a touch screen, a microphone, or the like, or any combination thereof. The output device may include a display device, a loudspeaker, a printer, a projector, or the like, or any combination thereof.

The communication port 240 may be configured to communicate with the external terminal (e.g., the decoding terminal 120 and/or the storage device 130 when the encoding terminal 110 is implemented on the computing device 200) via a network connection. The connection may be a wired connection, a wireless connection, any connection that enables data transmission and/or reception, or the like, or any combination thereof.

It should be understood that the descriptions of FIG. 1B are only provided for the purpose of illustration and do not constitute a limitation to the present disclosure. For those skilled in the art, various changes and modifications can be made under the guidance of the present disclosure. Features, structures, manners, and other characteristics of the embodiments of the present disclosure can be combined in various ways to obtain other and/or alternative embodiments. However, such changes and modifications do not exceed the scope of the present disclosure.

FIG. 1C is a block diagram illustrating an encoding terminal according to some embodiments of the present disclosure. FIG. 1D is a block diagram illustrating a decoding terminal according to some embodiments of the present disclosure.

As shown in FIG. 1C, in some embodiments, the encoding terminal 110 may include an encoding module 310.

The encoding module 310 may be configured to encode a target frame to obtain a bitstream of the target frame.

In some embodiments, the encoding module 310 may be configured to divide the target frame into a plurality of image blocks; for a target block among the plurality of image blocks, determine a prediction residual block of the target block in a prediction mode of IBC; divide the prediction residual block into at least one sub-block; select at least one sub-block to be transformed from the at least one sub-block; obtain a transform result by transforming the at least one sub-block to be transformed; and determine the bitstream of the target frame based on the transform result.

In some embodiments, the encoding module 310 may further include an encoding acquisition unit 311, an image block division unit 312, a prediction residual block determination unit 313, a first sub-block division unit 314, a first sub-block selection unit 315, a transformation unit 316, and a bitstream determination unit 317.

The encoding acquisition unit 311 may be configured to obtain the target frame.

The image block division unit 312 may be configured to divide the target frame into the plurality of image blocks.

The prediction residual block determination unit 313 may be configured to determine, for the target block among the plurality of image blocks, the prediction residual block of the target block in the prediction mode of IBC.

The first sub-block division unit 314 may be configured to divide the prediction residual block into the at least one sub-block. In some embodiments, the first sub-block division unit 314 may obtain a plurality of candidate division modes; determine a score of each of the candidate division modes; determine a candidate division mode with the highest score as a target division mode; and divide the prediction residual block into the at least one sub-block based on the target division mode.

The first sub-block selection unit 315 may be configured to select the at least one sub-block to be transformed from the at least one sub-block. In some embodiments, the first sub-block selection unit 315 may be configured to obtain a plurality of candidate selection modes; determine a score of each of the candidate selection modes; determine a candidate selection mode with the highest score as a target sub-block selection mode; and determine the at least one sub-block to be transformed from the at least one sub-block of the prediction residual block based on the target sub-block selection mode.

The transformation unit 316 may be configured to transform the at least one sub-block to be transformed to obtain the transform result using a target transform kernel. In some embodiments, the target transform kernel may include a preset transform kernel; or the target transform kernel may correspond to a target selection mode of the prediction residual block; or the target transform kernel may include at least one transform kernel each of which corresponds to one of the at least one sub-block to be transformed and is selected from a transform kernel set. In some embodiments, the target transform kernel corresponding to the prediction residual block may include at least one transform kernel determined based on at least one of Discrete Sine Transform Type VII (DST7) and Discrete Cosine Transform Type VIII (DCT8) ; or the target transform kernel corresponding to the prediction residual block may be determined based on the transform kernel set, and the transform kernel set may include one or more transform kernels determined using at least one of the DST7 and the DCT8.

The bitstream determination unit 317 may be configured to determine the bitstream of the target frame based on the transform result.

As shown in FIG. 1D, in some embodiments, the decoding terminal 120 may include a decoding module 320. The decoding module 320 may be used for decoding. In some embodiments, the decoding module 320 may determine, by decoding the bitstream of the target frame obtained from the encoding terminal, an inverse quantization residual block of a target block of the target frame in the prediction mode of IBC; divide the inverse quantization residual block into at least one sub-block; select at least one target sub-block from the at least one sub-block; obtain a prediction residual of the at least one target sub-block by performing an inverse transformation on the at least one target sub-block; and generate, based on the prediction residual of the at least one target sub-block, a decoded image of the target frame.

In some embodiments, the decoding module 320 may further include an entropy decoding unit 321, a second sub-block division unit 322, a second sub-block selection unit 323, an inverse transformation unit 324, and a decoded image determination unit 325.

The entropy decoding unit 321 may be configured to determine the inverse quantization residual block of the target block of the target frame in the prediction mode of IBC by decoding the bitstream of the target frame.

The second sub-block division unit 322 may be configured to divide the inverse quantization residual block into the at least one sub-block. In some embodiments, the second sub-block division unit 322 may divide the inverse quantization residual block into the at least one sub-block using a preset division mode; or determine a target division mode based on a value of a sub-block division mode syntax, and divide the inverse quantization residual block into the at least one sub-block based on the target division mode.

The second sub-block selection unit 323 may be configured to select the at least one target sub-block from the at least one sub-block of the inverse quantization residual block. In some embodiments, the second sub-block selection unit 323 may obtain a target selection mode corresponding to the target division mode of the inverse quantization residual block; and determine the at least one target sub-block from the at least one sub-block of the inverse quantization residual block based on the target selection mode. In some embodiments, the bitstream may include information related to a sub-block selection syntax, and the second sub-block selection unit 323 may determine the sub-block selection syntax of the target block by interpreting the bitstream; and determine the at least one target sub-block from the at least one sub-block based on a value of the sub-block selection syntax.

The inverse transformation unit 324 may be configured to perform an inverse transform on the at least one target sub-block to obtain the prediction residual of the at least one target sub-block. In some embodiments, the target transform kernel may include a preset transform kernel; or the target transform kernel may correspond to the target selection mode of the prediction residual block; or the target transform kernel may include at least one transform kernel each of which corresponds to one of the at least one sub-block to be transformed and is selected from the transform kernel set. In some embodiments, the transform kernel corresponding to the inverse quantization residual block may include a transform kernel determined using at least one of the DST7 and the DCT8; or the transform kernel corresponding to the inverse quantization residual block may be determined based on a transform kernel set, and the transform kernel set may include one or more transform kernels determined using at least one of the DST7 and the DCT8.

The decoded image determination unit 325 may be configured to determine the decoded image of the target frame based on the prediction residual of the at least one target sub-block.

It can be understood that the above encoding module 310 and decoding module 320 may be configured to implement the method in any embodiment of present disclosure. More descriptions may be found in the detailed descriptions of the embodiments of the method below, which are not repeated here.

It should be noted that the above description is only for example and explanation, and does not limit the scope of application of the present disclosure. For those skilled in the art, various modifications and changes can be made under the guidance of the present disclosure. However, these modifications and changes are still within the scope of the present disclosure. For example, the encoding module 310 may not include the first sub-block division unit 314 and the first sub-block selection unit 315. After the prediction residual block is determined using the prediction residual block determination unit 313, the prediction residual block may be transformed using the transformation unit 316 directly. As another example, the decoding module 320 may not include the second sub-block division unit 322 and the second sub-block selection unit 323. After the entropy decoding unit 321 determines the inverse quantization residual block, the inverse quantization residual block may be inversely transformed using the inverse transformation unit 324 directly.

FIG. 1E is a flowchart illustrating an exemplary video encoding method according to some embodiments of the present disclosure.

In some embodiments, a process 400 may be performed by an encoding terminal (e.g., the encoding terminal 110, the computing device 200, and/or the encoding module 310) . As shown in FIG. 1E, the process 400 may include the following operations.

In 410, the encoding terminal may obtain a target frame. In some embodiments, the operation 410 may be performed by the encoding acquisition unit 311.

The target frame refers to an image/video frame to be encoded. When an encoding subject is a video, the video may include a plurality of image frames/video frames, and the encoding terminal may encode the plurality of image frames separately or in parallel. In some embodiments, the target frame may be one of the plurality of image frames. In some embodiments, the encoding acquisition unit 311 may obtain the target frame from a memory (e.g., the storage device 130, the memory 223, or the non-volatile storage medium 225) . In some embodiments, the encoding acquisition unit 311 may obtain the target frame from an image acquisition device (e.g., a camera) . In some embodiments, the encoding acquisition unit 311 may obtain the target frame from a terminal device (e.g., a mobile device, a tablet computer, or a laptop computer) .

In 420, the encoding terminal may divide the target frame into a plurality of image blocks. In some embodiments, the operation 420 may be performed by the image block division unit 312.

When the target frame is encoded, the image block division unit 312 may divide the target frame into a plurality of image blocks each of which serves as a coding unit. In some embodiments, the image block may be a maximum coding unit. In some embodiments, the image block may be a minimum coding unit. In some embodiments, the image block division unit 312 may divide the target frame into the plurality of image blocks according to a preset rule. The preset rule may include a count of the plurality of image blocks, a size of each image block, or the like. For example, the image block division unit 312 may equally divide the target frame into a preset count of image blocks. As another example, the image block division unit 312 may divide the target frame into a plurality of image blocks of a preset size. At least part of the plurality of image blocks may have an overlapping region, or each of the plurality of image blocks may not overlap with each other. The division mode of the image blocks is not limited in the present application. Merely by way of example, the image block division unit 312 may equally divide a 256×256 target frame (i.e., a count of pixels in a width of the target frame × a count of pixels in a height of the target frame) into 16 16×16 image blocks.

In 430, for a target block among the plurality of image blocks, the encoding terminal may determine a prediction residual block of the target block in a prediction mode of intra block copy. In some embodiments, the operation 430 may be performed by the prediction residual block determination unit 313.

The target block may be an image block that is being encoded among the plurality of image blocks, i.e., an image unit to be encoded currently.

The prediction residual block refers to a difference block between the target block and a prediction block.

In some embodiments, when the prediction mode of IBC is adopted, the prediction block may be determined based on a representative block in the target frame. The representative block refers to a block in the target frame that is most similar to the target block. A size of the representative block may be greater than or equal to a size of the target block. The encoding terminal may determine the representative block from the target frame and save the representative block. In some embodiments, if the size of the target block is consistent with the size of the representative block, the representative block may be used as the prediction block. In some embodiments, if the size of the representative block is greater than the size of the target block, in this case, the encoding terminal needs to determine which part of the representative block is used to match the target block. In some embodiments, the prediction residual block determination unit 313 may select, by cropping, a region of the representative block corresponding to the position of the target block as the prediction block, which is not limited in the present disclosure.

In some embodiments, the prediction residual block determination unit 313 may perform prediction on the target block to obtain the prediction residual block of the target block. For example, after the prediction block is determined, the prediction residual block determination unit 313 may determine a difference between the target block and the prediction block to obtain the prediction residual block. For example, for each pixel of the target block, the prediction residual block determination unit 313 may determine a pixel value difference between the pixel of the target block and the corresponding pixel (having the same location as the pixel of the target block) of the prediction block. That is, in the prediction residual block, a pixel value of a pixel is a pixel value difference between the corresponding pixel of the target block and the corresponding pixel of the prediction block. If the pixel value difference is greater than or equal to a pixel threshold, it indicates that prediction residual exists in the pixel location. If the pixel value difference is less than the pixel threshold (e.g., there is no pixel value difference) , it indicates that there is no prediction residual in the pixel location.

In some embodiments, if a prediction mode of the target block is the prediction mode of IBC, the encoding terminal may determine the prediction residual block of the target block in the prediction mode of IBC. In some embodiments, if the prediction mode of the target block is a prediction mode other than the prediction mode of IBC, the encoding terminal may determine the prediction residual block of the target block in the prediction mode other than the prediction mode of IBC.

In 440, the encoding terminal may divide the prediction residual block into at least one sub-block. In some embodiments, the operation 440 may be performed by the first sub-block division unit 314.

In some embodiments, the first sub-block division unit 314 may divide the prediction residual block into the at least one sub-block based on a target division mode of the prediction residual block. For example, the first sub-block division unit 314 may first determine the target division mode of the prediction residual block, and then divide the prediction residual block into the at least one sub-block based on the target division mode of the prediction residual block.

In some embodiments, a sub-block division mode may include no division. No division is regarded as a special sub-block division mode, i.e., the prediction residual block is not divided, and the prediction residual block is used as a sub-block. In this case, a size of the sub-block is the same as a size of the prediction residual block. For example, FIG. 2 is a schematic diagram illustrating a sub-block division mode of no division according to some embodiments of the present disclosure. As shown in FIG. 2, the sub-block division mode of the prediction residual block is no division. That is, the size of the prediction residual block is W*H, and the size of the sub-block after the division is also W*H. The encoding terminal may directly perform subsequent processing (e.g., transformation, etc. ) on the W*H sub-block.

In some embodiments, the sub-block division mode may include a division direction and a division proportion. The division direction refers to a direction in which the residual block (e.g., the prediction residual block, and the inverse quantization residual block) is divided. The division direction includes but is not limited to dividing in a horizontal direction (e.g., a w-direction in FIG. 2 -FIG. 9) of the residual block, dividing in a vertical direction (e.g., an h-direction in FIG. 2 -FIG. 9) of the residual block, and dividing in both of the vertical direction and the horizontal direction of the residual block. Dividing in the horizontal direction of the residual block means keeping the width of the residual block constant and dividing the height of the residual block based on a certain proportion to obtain a plurality of sub-blocks. Dividing in the vertical direction of the residual block means keeping the height of the residual block constant and dividing the width of the residual block based on a certain proportion to obtain a plurality of sub-blocks. Dividing in the vertical direction and the horizontal direction of the residual block means dividing the width of the residual block based on a first proportion and dividing the height of the residual block according to a second proportion, so as to obtain a plurality of sub-blocks.

The division proportion refers to a dimensional proportion of the sub-blocks obtained when the residual block (e.g., the prediction residual block, and the inverse quantization residual block) is divided in the division direction, or a dimensional proportion of the sub-blocks and the residual block. The division proportion may be a preset proportion. For example, the division proportion may be that the dimensional proportion of the sub-blocks and the prediction residual block is 1: 2, 1: 3, 1: 4, or the like, in the horizontal direction or the vertical direction of the residual block. As another example, the division proportion may be that the dimensional proportion of two sub-blocks in the horizontal direction or the vertical direction of the residual block is 1: 1, 1: 2, 1: 3, or the like. As an exemplary division mode, the prediction residual block may be divided based on the division proportion in the division direction. In this case, the sizes of the sub-blocks obtained after the prediction residual block is divided are different from the size of the prediction residual block. For example, the residual block is divided in the horizontal direction of the residual block, and the division proportion includes a division proportion of the height of the residual block from top to bottom or from bottom to top. As another example, the residual block is divided in the vertical direction of the residual block, and the division proportion includes a division proportion of the width of the residual block from left to right or from right to left. For example, the residual block is divided in the horizontal direction and the vertical direction of the residual block, and the division proportion includes a division proportion of the width of the residual block from left to right or from right to left, and a division proportion of the height of the residual block from top to bottom or from bottom to top.

An example 1 of sub-block division based on the division direction and the division proportion is shown in FIG. 3. FIG. 3 is a schematic diagram illustrating a sub-block division mode of dividing a residual block in a vertical direction according to some embodiments of the present disclosure. As shown in FIG. 3 (a) or (b) , the sub-block division mode of the prediction residual block is to divide in the vertical direction of the prediction residual block, the division direction is the vertical direction of the prediction residual block, and the division proportion is to divide in the width of the prediction residual block based on a dimensional proportion of 1: 1. That is, the size of the prediction residual block is W*H, and sizes of left and right sub-blocks obtained after division are both W/2*H, and the dimensional proportion of the two sub-blocks is 1: 1.

An example 2 of sub-block division based on the division direction and the division proportion is shown in FIG. 4. FIG. 4 is a schematic diagram illustrating a sub-block division mode of dividing a residual block in a vertical direction according to some embodiments of the present disclosure. As shown in FIG. 4 (a) , the sub-block division mode of the prediction residual block is to divide in the vertical direction of the prediction residual block, the division direction is the vertical direction of the prediction residual block, and the division proportion is to divide the width of the prediction residual block from left to right based on a dimensional proportion 1: 3. That is, the size of the prediction residual block is W*H, the sizes of the left and right sub-blocks obtained after division are W/4*H and 3W/4*H, respectively, and the dimensional proportion of the two sub-blocks is 1: 3. As shown in FIG. 4 (b) , the sub-block division mode of the prediction residual block is to divide it in the vertical direction of the prediction residual block, the division direction is the vertical direction of the prediction residual block, and the division proportion is to divide the width of the prediction residual block from left to right based on a dimensional proportion 3: 1. That is, the size of the prediction residual block is W*H, the sizes of the left and right sub-blocks obtained after the division are 3W/4*H and W/4*H, respectively, and the dimensional proportion of the two sub-blocks is 3: 1.

An example 3 of sub-block division based on the division direction and the division proportion is shown in FIG. 5. FIG. 5 is a schematic diagram illustrating a sub-block division mode of dividing a residual block in a horizontal direction according to some embodiments of the present disclosure. As shown in FIG. 5 (a) or (b) , the sub-block division mode of the prediction residual block is to divide the prediction residual block in the horizontal direction, the division direction is the horizontal direction of the prediction residual block, and the division proportion is to divide the height of the prediction residual block based on a dimensional proportion 1: 1. That is, the size of the prediction residual block is W*H, the size of upper and lower sub-blocks obtained after the division is W*H/2, and the dimensional proportion of the two sub-blocks is 1: 1.

An example 4 of sub-block division according to the division direction and the division proportion is shown in FIG. 6. FIG. 6 is a schematic diagram illustrating a sub-block division mode of dividing a residual block in a horizontal direction according to some embodiments of the present disclosure. As shown in FIG. 6 (a) , the sub-block division mode of the prediction residual block is to divide the prediction residual block in the horizontal direction, the division direction is the horizontal direction of the prediction residual block, and the division proportion is to divide the height of the prediction residual block from top to bottom based on a dimensional proportion 1: 3. That is, the size of the prediction residual block is W*H, the sizes of the upper and lower sub-blocks obtained after the division are W*H/4 and W*3H/4, respectively, and the dimensional proportion of the two sub-blocks is 1: 3. As shown in FIG. 6 (b) , the sub-block division mode of the prediction residual block is to divide the prediction residual block in the horizontal direction, the division direction is the horizontal direction of the prediction residual block, and the division proportion is to divide the height of the prediction residual block from top to bottom based on a dimensional proportion 3: 1. That is, the size of the prediction residual block is W*H, the sizes of the upper and lower sub-blocks obtained after the division are W*3H/4 and W*H/4, respectively, and the dimensional proportion of the two sub-blocks is 3: 1.

An example 5 of sub-block division based on the division direction and the division dimension is shown in FIG. 7. FIG. 7 is a schematic diagram illustrating a sub-block division mode of dividing a residual block in a vertical direction and a horizontal direction according to some embodiments of the present disclosure. As shown in any figure of FIG. 7, the sub-block division mode of the prediction residual block is to divide in the horizontal direction and the vertical direction of the prediction residual block, the division direction is the horizontal direction and the vertical direction of the prediction residual block, and the division proportion is to divide the width of the prediction residual block based on a dimensional proportion 1: 1, and to divide the height of the prediction residual block a dimensional proportion 1: 1. That is, the size of the prediction residual block is W*H, the sizes of four sub-blocks obtained after division are all W/2*H/2, and the dimensional proportion of the four sub-blocks is 1: 1.

An example 6 of sub-block division based on the division direction and the division proportion is shown in FIG. 8. FIG. 8 is a schematic diagram illustrating a sub-block division mode of dividing a residual block in a vertical direction and a horizontal direction according to some embodiments of the present disclosure. As shown in FIG. 8 (a) , the sub-block division mode of the prediction residual block is to divide in the horizontal direction and the vertical direction of the prediction residual block, the division direction is the horizontal direction and the vertical direction of the prediction residual block, and the division proportion is to divide the height of the prediction residual block from top to bottom based on a dimensional proportion 1: 3, and to divide the width of the prediction residual block from left to right based on a dimensional proportion 1: 3. That is, the size of the prediction residual block is W*H, and four sub-blocks are obtained after division. As shown in FIG. 8 (a) , the size of an upper left sub-block is W/4*H/4, the size of an upper right sub-block is 3W/4*H/4, the size of a lower left sub-block is W/4*3H/4, and the size of a lower right sub-block is 3W/4*3H/4. As shown in FIG. 8 (b) , the sub-block division mode of the prediction residual block is to divide in the horizontal direction and the vertical direction of the prediction residual block, the division direction is the horizontal direction and the vertical direction of the prediction residual block, and the division proportion is to divide the height of the prediction residual block from top to bottom based on a dimensional proportion 1: 3, and to divide the width of the prediction residual block from left to right based on a dimensional proportion 3: 1. That is, the size of the prediction residual block is W*H, and four sub-blocks are obtained after division. As shown in FIG. 8 (b) , the size of an upper left sub-block is 3W/4*H/4, the size of an upper right sub-block is W/4*H/4, the size of a lower left sub-block is 3W/4*3H/4, and the size of a lower right sub-block is W/4*3H/4. As shown in FIG. 8 (c) , the sub-block division mode of the prediction residual block is to divide in the horizontal direction and the vertical direction of the prediction residual block, the division direction is the horizontal direction and the vertical direction of the prediction residual block, and the division proportion is to divide the height of the prediction residual block from top to bottom based on a dimensional proportion 3: 1, and to divide the width of the prediction residual block from left to right based on a dimensional proportion 1: 3. That is, the size of the prediction residual block is W*H, and four sub-blocks are obtained after division. As shown in FIG. 8 (c) , the size of an upper left sub-block is W/4*3H/4, the size of an upper right sub-block is 3W/4*3H/4, the size of a lower left sub-block is W/4*H/4, and the size of a lower right sub-block is 3W/4*H/4. As shown in FIG. 8 (d) , the sub-block division mode of the prediction residual block is to divide in the horizontal direction and the vertical direction of the prediction residual block, the division direction is the horizontal direction and the vertical direction of the prediction residual block, and the division proportion is to divide the height of the prediction residual block from top to bottom based on a dimensional proportion 3: 1, and to divide the width of the prediction residual block from left to right based on a dimensional proportion 3: 1. That is, the size of the prediction residual block is W*H, and four sub-blocks are obtained after division. As shown in FIG. 8 (d) , the size of an upper left sub-block is 3W/4*3H/4, the size of an upper right sub-block is W/4*3H/4, the size of a lower left sub-block is 3W/4*H/4, and the size of a lower right sub-block is W/4*H/4.

In some embodiments, the encoding terminal may obtain a plurality of candidate division modes, determine a target division mode from the plurality of candidate division modes, and divide the prediction residual block into the at least one sub-block based on the target division mode. More descriptions may be found in the related descriptions in FIG. 10.

In some embodiments, the first sub-block division unit 314 may divide the prediction residual block using a preset sub-block division mode (e.g., a preset division direction and a preset division proportion) . In some embodiments, the first sub-block division unit 314 may determine the target division mode of the prediction residual block by comparing costs of a plurality of sub-block division modes. For example, the encoding module 310 may use a candidate division mode of the plurality of candidate division modes with a minimum cost as the target division mode of the prediction residual block; or may determine the target division mode of the prediction residual block based on texture information of the prediction residual block. The cost may be a rate distortion cost, etc.

In some embodiments, the encoding terminal may determine the target division mode using a first machine learning model. An input of the first machine learning model may include an image feature of the prediction residual block, and an output of the first machine learning model may be the target division mode. The first machine learning model may be obtained by training a first initial machine learning model based on a large number of training samples with training labels. The training samples may include the image feature of the prediction residual block and the corresponding target division mode, where the target division mode may be the training label. Specifically, the encoding terminal may input the training samples with the training labels into the first initial machine learning model to obtain output results of the first initial machine learning model. Parameters of the initial machine learning model may be updated through training until a trained intermediate machine learning model meets a preset condition, thereby obtaining a trained first machine learning model. The preset condition may be that a loss function is less than a preset threshold, the loss function converges, or a count of trainings reaches a preset threshold.

In some embodiments, after the target division mode of the prediction residual block is determined, the encoding terminal may write a sub-block division mode syntax of the prediction residual block into the bitstream of the target frame, so as to indicate the target division mode of the prediction residual block through the sub-block division mode syntax in the bitstream. Writing the sub-block division mode syntax of the prediction residual block into the bitstream can be understood as marking the target division mode in the bitstream data, and the target division mode in the bitstream data can be interpreted during a decoding process, so as to perform decoding (e.g., divide the inverse quantization residual block using the target division mode) .

In the embodiments of the present disclosure, by writing the sub-block division mode syntax into the bitstream data of the prediction residual block, the decoding terminal can accurately analyze and reconstruct the target division mode of the prediction residual block, thereby ensuring the information consistency between the encoding terminal and the decoding terminal, improving the accuracy and efficiency of encoding and decoding, and reducing image reconstruction errors caused by information asymmetry.

In 450, the encoding terminal may select at least one sub-block to be transformed from the at least one sub-block. In some embodiments, the operation 450 may be performed by the first sub-block selection unit 315.

The sub-block to be transformed refers to a sub-block used for subsequent transformation among the at least one sub-block obtained by dividing the prediction residual block. In some embodiments, the sub-block to be transformed may be a sub-block with a prediction residual. In some embodiments, the encoding module 310 may determine the sub-block with the prediction residual based on the sub-block division mode. For example, if the sub-block division mode is to divide in the horizontal direction based on a dimensional proportion 1: 1 (as shown in FIG. 5) , the first sub-block selection unit 315 may determine an upper and/or lower sub-block as the sub-block to be transformed. As another example, if the sub-block division mode is to divide in the vertical direction based on a dimensional proportion 1: 3 (as shown in FIG. 4) , the first sub-block selection unit 315 may determine a left and/or right sub-block as the sub-block to be transformed.

In some embodiments, all sub-blocks of the prediction residual block may be used as the sub-blocks to be transformed. In some embodiments, some sub-blocks of the prediction residual block may be used as the sub-blocks to be transformed. In some embodiments, the encoding terminal may select a sub-block at a preset position as the sub-block to be transformed. For example, the encoding module 310 selects a first sub-block in the encoding order of the at least one sub-block of the prediction residual block as the sub-block to be transformed.

In some embodiments, selecting at least one sub-block to be transformed from the at least one sub-block may include: obtaining a target selection mode corresponding to the target division mode of the prediction residual block; and determining, based on the target selection mode, the at least one sub-block to be transformed from the at least one sub-block.

As shown in FIG. 2 -FIG. 9, there are a plurality of sub-block division modes for dividing the residual block (e.g., the prediction residual block, and the inverse quantization residual block) . In some embodiments, each sub-block division mode may correspond to a target selection mode.

In some embodiments, as shown in FIG. 2, if the target division mode of the prediction residual block is no division, the corresponding target selection mode is to use the entire prediction residual block (e.g., a shaded region in FIG. 2) as the sub-block to be transformed.

In some embodiments, as shown in FIG. 3, the target division mode of the prediction residual block is to divide in the vertical direction based on the dimensional proportion 1: 1, and the corresponding target selection mode is to use the left or right sub-block as the sub-block to be transformed. For example, as shown in FIG. 3 (a) , the target selection mode is to use the left sub-block (i.e., the shaded region) as the sub-block to be transformed. As another example, as shown in FIG. 3 (b) , the target selection mode is to use the right sub-block (i.e., the shaded region) as the sub-block to be transformed.

As another example, as shown in FIG. 4 (a) , the target division mode of the prediction residual block is to divide in the vertical direction of the prediction residual block, the division direction is the vertical direction of the prediction residual block, and the division proportion is to divide the width of the prediction residual block from left to right based on a dimensional proportion 1: 3, and the corresponding target selection mode is to use the left sub-block (i.e., the shaded region) as the sub-block to be transformed. As shown in FIG. 4 (b) , the target division mode of the prediction residual block is to divide in the vertical direction of the prediction residual block, the division direction is the vertical direction of the prediction residual block, and the division proportion is to divide the width of the prediction residual block from left to right based on a dimensional proportion 3: 1, and the target selection mode is to use the right sub-block (i.e., the shaded region) as the sub-block to be transformed.

In some embodiments, as shown in FIG. 5, the target division mode of the prediction residual block is to divide in the horizontal direction based on a dimensional proportion 1 : 1, and the corresponding target selection mode is to use the upper or lower sub-block as the sub-block to be transformed. For example, as shown in FIG. 5 (a) , the target selection mode is to use the upper sub-block (i.e., the shaded region) as the sub-block to be transformed. As another example, as shown in FIG. 5 (b) , the target selection mode is to use the lower sub-block (i.e., the shaded region) as the sub-block to be transformed.

In some embodiments, as shown in of FIG. 6 (a) , the target division mode of the prediction residual block is to divide in the horizontal direction of the prediction residual block, the division direction is the horizontal direction of the prediction residual block, the division proportion is to divide the width of the prediction residual block from top to bottom based on a dimensional proportion 1: 3, and the target selection mode is to use the upper sub-block (i.e., the shaded region) as the sub-block to be transformed. As shown in FIG. 6 (b) , the target division mode of the prediction residual block is to divide in the horizontal direction of the prediction residual block, the division direction is the horizontal direction of the prediction residual block, the division proportion is to divide the width of the prediction residual block from top to bottom based on a dimensional proportion 3: 1, and the target selection mode is to use the lower sub-block (i.e., the shaded region) as the sub-block to be transformed.

In some embodiments, as shown in FIG. 7, the target division mode of the prediction residual block is to divide in the vertical direction and the horizontal direction based on a dimensional proportion 1: 1, and the corresponding target selection mode is to select only one sub-block as the sub-block to be transformed, or to select all sub-blocks as the sub-blocks to be transformed (as shown in FIG. 9) . For example, as shown in FIG. 7 (a) , the target selection mode is to use the upper left sub-block (i.e., the shaded region) as the sub-block to be transformed. As another example, as shown in FIG. 7 (b) , the target selection mode is to use the upper right sub-block (i.e., the shaded region) as the sub-block to be transformed. As another example, as shown in FIG. 7 (c) , the target selection mode is to use the lower left sub-block (i.e., the shaded region) as the sub-block to be transformed. As another example, as shown in FIG. 7 (d) , the target selection mode is to use the lower right sub-block (i.e., the shaded region) as the sub-block to be transformed.

In some embodiments, as shown in FIG. 8 (a) , the target division mode of the prediction residual block is to divide in the horizontal direction and the vertical direction of the prediction residual block, the division direction is the horizontal direction and the vertical direction of the prediction residual block, and the division proportion is to divide the height of the prediction residual block from top to bottom based on a dimensional proportion 1: 3, and to divide the width of the prediction residual block from left to right based on a dimensional proportion 1: 3. The corresponding target selection mode is to use the upper left sub-block (i.e., the shaded region) as the sub-block to be transformed. As shown in FIG. 8 (b) , the target division mode of the prediction residual block is to divide in the horizontal direction and the vertical direction of the prediction residual block, the division direction is the horizontal direction and the vertical direction of the prediction residual block, and the division proportion is to divide the height of the prediction residual block from top to bottom based on a dimensional proportion 1: 3, and to divide the width of the prediction residual block from left to right based on a dimensional proportion 3: 1. The corresponding target selection mode is to use the upper right sub-block (i.e., the shaded region) as the sub-block to be transformed. As shown in FIG. 8 (c) , the target division mode of the prediction residual block is to divide in the horizontal direction and the vertical direction of the prediction residual block, the division direction is the horizontal direction and the vertical direction of the prediction residual block, and the division proportion is to divide the height of the prediction residual block from top to bottom based on a dimensional proportion 3: 1, and to divide the width of the prediction residual block from left to right based on a dimensional proportion 1: 3. The corresponding target selection mode is to use the lower left sub-block (i.e., the shaded region) as the sub-block to be transformed. As shown in FIG. 8 (d) , the target division mode of the prediction residual block is to divide in the horizontal direction and the vertical direction of the prediction residual block, the division direction is the horizontal direction and the vertical direction of the prediction residual block, and the division proportion is to divide the height of the prediction residual block from top to bottom based on a dimensional proportion 3: 1, and to divide the width of the prediction residual block from left to right based on a dimensional proportion 3: 1. The corresponding target selection mode is to use the lower right sub-block (i.e., the shaded region) as the sub-block to be transformed.

In some embodiments, if a plurality of sub-blocks to be transformed are determined, positions of the plurality of sub-blocks to be transformed may be continuous or discontinuous, which is not limited here.

In some embodiments, the first sub-block selection unit 315 may determine a score of each of a plurality of candidate selection modes based on the plurality of candidate selection modes, determine a candidate selection mode of the plurality of candidate selection modes with a highest score as a target selection mode, and determine the at least one sub-block to be transformed based on the target selection mode. More descriptions may be found in the descriptions in FIG. 11.

In some embodiments, the encoding terminal may determine the at least one sub-block to be transformed using a second machine learning model. An input of the second machine learning model may include an image feature of the prediction residual block and the target division mode of the prediction residual block, and an output of the second machine learning model may be the target selection mode of the prediction residual block, i.e., the corresponding at least one sub-block to be transformed. The second machine learning model may be obtained by training a second initial machine learning model based on a large number of training samples with training labels. The training samples may include the image feature of the prediction residual block, the target selection mode of the prediction residual block, and the corresponding at least one sub-block to be transformed, where the at least one sub-block to be transformed may be the training label. Specifically, the encoding terminal may input the training samples with the training labels into the second initial machine learning model to obtain output results of the second initial machine learning model. Parameters of the initial machine learning model may be updated through training until a trained intermediate machine learning model meets a preset condition, thereby obtaining a trained second machine learning model. The preset conditions may be that a loss function is less than a preset threshold, the loss function converges, or a count of trainings reaches a preset threshold.

In some embodiments, after the target selection mode is determined, the first sub-block selection unit 315 may write a sub-block selection mode syntax into the bitstream to indicate the selection of the at least one sub-block to be transformed through the sub-block selection mode syntax in the bitstream. Writing the sub-block selection mode syntax into the bitstream can be understood as marking the target selection mode in the bitstream data, and the target selection mode in the bitstream data can be interpreted during a decoding process to perform decoding (e.g., selecting a target sub-block from the at least one sub-block of the inverse quantization residual block using the target selection mode) . The sub-block selection mode syntax may be implicitly encoded or explicitly encoded, which is not limited here. Accordingly, for the decoding terminal, the sub-block selection mode syntax may be read by implicit decoding or explicit decoding to determine the target sub-block from all the sub-blocks of the inverse quantization residual block.

In the embodiments of present disclosure, by writing the sub-block selection mode syntax into the bitstream data of the prediction residual block, the information consistency between the encoding terminal and the decoding terminal can be ensured, so that the decoding terminal can perform accurate inverse transformation and reconstruction on the specified sub-block.

In 460, the encoding terminal may obtain a transform result by transforming, using a target transform kernel, the at least one sub-block to be transformed. In some embodiments, the operation 460 may be performed by the transformation unit 316.

Transformation refers to a process of transforming data to be processed (e.g., the at least one sub-block to be transformed) so as to express a main feature of the data to be processed in a more compact manner, thereby reducing the dimension or data volume of the data to be processed.

In some embodiments, the transformation unit 316 may perform a mathematical transformation on the at least one sub-block to be transformed to convert the data in a spatial domain to a frequency domain, thereby removing data redundancy and improving compression efficiency. In some embodiments, a process (also referred to as a transform kernel) of transforming the at least one sub-block to be transformed may include but is not limited to a discrete cosine transform type (DCT) , a discrete sine transform type (DST) , and transform skip (TS) . In some embodiments, the at least one sub-block to be transformed may be transformed in the horizontal direction and the vertical direction, respectively, to obtain the transform result.

In some embodiments, the transformation unit 316 may transform the at least one sub-block to be transformed using a preset transform kernel. In this case, different sub-blocks to be transformed are transformed using the same transform kernel (i.e., the preset transform kernel) . The preset transform kernel may be a preset standard transform kernel. In some embodiments, the preset transform kernel may be at least one of the DCT2, the DST7, the DCT8, and the TS. For example, if the preset transform kernel is the DCT2, the transformation unit 316 may transform the at least one sub-block to be transformed based on a preset encoding order using the DCT2, or transform the at least one sub-block to be transformed simultaneously (i.e., performing the transformation on the at least one sub-block to be transformed in parallel) . In some embodiments, the transformation unit 316 may perform the transformation on part of the at least one sub-block to be transformed in parallel.

In some embodiments, the preset transform kernel may include a transform kernel determined using at least one of the DST7 and the DCT8.

For example, if the preset transform kernel is the DST7, the transformation unit 316 may transform the at least one sub-block to be transformed based on the preset encoding order using the DST7, or transform the at least one sub-block to be transformed simultaneously. The DST7 can reduce the redundancy between colors of the target block and improve the ability to process complex colors and textures.

As another example, if the preset transform kernel is DCT8, the transformation unit 316 may transform the at least one sub-block to be transformed based on the preset encoding order using the DCT8, or transform the at least one sub-block to be transformed simultaneously. The DCT8 can effectively remove high-frequency components that are not sensitive to human eyes, thereby achieving higher compression efficiency without causing a decrease in the perceived quality.

As another example, if the preset transform kernel is the DST7 and the DCT8, the transformation unit 316 may transform the at least one sub-block to be transformed based on the preset encoding order using the DST7 and the DCT8 simultaneously, or transform the at least one sub-block to be transformed simultaneously. For example, a horizontal transform kernel of the at least one sub-block to be transformed is the DST7, and a vertical transform kernel of the at least one sub-block to be transformed is the DCT8; or the horizontal transform kernel of the at least one sub-block to be transformed is the DST8, and the vertical transform kernel of the at least one sub-block to be transformed is the DST7.

In some embodiments, the transformation unit 316 may transform the at least one sub-block to be transformed using a target transform kernel corresponding to the target division mode of the prediction residual block. In this case, the target division mode of the prediction residual block is different, and the sub-block to be transformed determined is different. Different sub-blocks to be transformed may be transformed using the same or different transform kernels, thereby improving the adaptability of the at least one sub-block to be transformed and the transform kernel. In some embodiments, the target transform kernel corresponding to the target division mode may include a transform kernel determined using at least one of the DST7 and the DCT8.

In some embodiments, a plurality of sub-blocks obtained by each sub-block division mode may correspond to the same transform kernel. For example, the transform kernels of the two sub-blocks obtained by the target division mode of dividing in the horizontal direction based on a dimensional proportion 1: 1 may both correspond to the DST7, and the transform kernels of the two sub-blocks obtained by the target division mode of dividing in the vertical direction based on a dimensional proportion 1: 3 may be the DCT2.

In some embodiments, at least two sub-blocks in each position of the sub-block obtained by each sub-block division mode may correspond to different transform kernels. In some embodiments, each sub-block in each position obtained by each sub-block division mode may have its own corresponding transform kernel, that is, multiple sub-blocks obtained by each sub-block division mode may correspond to different transform kernels. In some embodiments, part of the sub-blocks in each position obtained by each sub-block division mode may have their own corresponding transform kernel.

In some embodiments, the transformation unit 316 may transform the at least one sub-block to be transformed using the target transform kernel corresponding to the target selection mode of the prediction residual block. Due to different sub-block selection modes, the sub-blocks to be transformed determined based on the sub-block selection modes are also different, and the different sub-blocks to be transformed may be transformed using the same or different transform kernels, thereby improving the adaptability of the at least one sub-block to be transformed and the transform kernel. In some embodiments, the target transform kernel corresponding to the target selection mode may include a transform kernel determined using at least one of the DST7 and the DCT8.

In some embodiments, the sub-block to be transformed obtained by each sub-block selection mode may correspond to the same transform kernel, such as the DST7.

In some embodiments, each sub-block at each position obtained by each sub-block selection mode may have the corresponding transform kernel. That is, the sub-block to be transformed obtained by each sub-block selection mode may correspond to a different transform kernel. For example, referring to FIG. 7, when a sub-block at any of four different positions (i.e., an upper left corner, a lower left corner, an upper right corner, and a lower right corner) shown in FIG. 7 is selected as the sub-block to be transformed, the sub-block to be transformed at a different position correspond to a different transform kernel.

For example, FIG. 12 is a schematic diagram illustrating transform kernels corresponding to different sub-block selection modes according to some embodiments of the present disclosure. As shown in FIG. 12 (a) , the target division mode is to divide the prediction residual block in the vertical direction based on a dimensional proportion 1: 1, the target selection mode is to select the left sub-block (i.e., the shaded region in the figure) as the sub-block to be transformed, and the target transform kernel corresponding to the target selection mode includes the horizontal transform kernel DCT8 and the vertical transform kernel DST7. As shown in FIG. 12 (b) , the target division mode is to divide the prediction residual block in the vertical direction based on a dimensional proportion 1: 1, the target selection mode is to select the right sub-block (i.e., the shaded region in the figure) as the sub-block to be transformed, and the target transform kernel corresponding to the target selection mode includes the horizontal transform kernel DST7 and the vertical transform kernel DST7. As shown in FIG. 12 (c) , the target division mode is to divide the prediction residual block in the horizontal direction based on a dimensional proportion 1: 1, the target selection mode is to select the upper sub-block (i.e., the shaded regions in the figure) as the sub-block to be transformed, and the target transform kernel corresponding to the target selection mode includes the horizontal transform kernel DST7 and the vertical transform kernel DCT8. As shown in FIG. 12 (d) , the target division mode is to divide the prediction residual block in the horizontal direction based on a dimensional proportion 1: 1, the sub-block selection mode is to select the lower sub-block (i.e., the shaded region in the figure) as the sub-block to be transformed, and the target transform kernel corresponding to this target selection mode includes the horizontal transform kernel DST7 and the vertical transform kernel DST7. As shown in FIG. 12 (e) , the division direction of the target division mode is the vertical direction, the division proportion is to divide the width of the prediction residual block from left to right based on a dimensional proportion 1: 3, the target selection mode is to select the left sub-block (i.e., the W/4 sub-block) as the sub-block to be transformed, and the target transform kernel corresponding to this target selection mode includes the horizontal transform kernel DCT8 and the vertical transform kernel DST7. As shown in FIG. 12 (f) , the division direction of the target division mode is the vertical direction, the division proportion is to divide the width of the prediction residual block from left to right based on a dimensional proportion 3: 1, the target selection mode is to select the right sub-block (i.e., the W/4 sub-block) as the sub-block to be transformed, and the target transform kernel corresponding to the target selection mode includes the horizontal transform kernel DST7 and the vertical transform kernel DST7. As shown in FIG. 12 (g) , the division direction of the target division mode is the horizontal direction, the division proportion is to divide the width of the prediction residual block from top to bottom based on a dimensional proportion 1: 3, the target selection mode is to select the upper sub-block (i.e., the H/4 sub-block) as the sub-block to be transformed, and the target transform kernel corresponding to the target selection mode includes the horizontal transform kernel DST7 and the vertical transform kernel DCT8. As shown FIG. 12 (h) , the division direction of the target division mode is the horizontal division, the division proportion is to divide the width of the prediction residual block from top to bottom based on a dimensional proportion 3: 1, the target selection mode is to select the lower sub-block (i.e., the H/4 sub-block) as the sub-block to be transformed, and the target transform kernel corresponding to the target selection mode includes the horizontal transform kernel DST7 and the vertical transform kernel DST7.

For example, FIG. 13 is a schematic diagram illustrating transform kernels corresponding to different sub-block selection modes according to some embodiments of the present disclosure. As shown in FIG. 13 (a) , the target division mode of the prediction residual block is to divide the prediction residual block in the vertical direction based on a dimensional proportion 1: 1 and in the horizontal direction based on a dimensional proportion 1: 1, the target selection mode is to select the upper left sub-block (i.e., the shaded region in the figure) as the sub-block to be transformed, and the target transform kernel corresponding to the target selection mode includes the horizontal transform kernel DCT8 and the vertical transform kernel DCT8. As shown in FIG. 13 (b) , the target division mode of the prediction residual block is to divide the prediction residual block in the vertical direction based on a dimensional proportion 1: 1 and in the horizontal direction based on a dimensional proportion 1: 1, the target selection mode is to select the upper right sub-block (i.e., the shaded region in the figure) as the sub-block to be transformed, and the target transform kernel corresponding to the target selection mode includes the horizontal transform kernel DST7 and the vertical transform kernel DCT8. As shown in FIG. 13 (c) , the target division mode of the prediction residual block is to divide the prediction residual block in the vertical direction based on a dimensional proportion 1: 1 and in the horizontal direction based on a dimensional proportion 1: 1, the target selection mode is to select the lower left sub-block (i.e., the shaded region in the figure) as the sub-block to be transformed, and the target transform kernel corresponding to the target selection mode includes the horizontal transform kernel DCT8 and the vertical transform kernel DST7. As shown in FIG. 13 (d) , the target division mode of the prediction residual block is to divide the prediction residual block in the vertical direction based on a dimensional proportion and in the horizontal direction based on a dimensional proportion, the target selection mode is to select the lower right sub-block (i.e., the shaded region in the figure) as the sub-block to be transformed, and the target transform kernel corresponding to the target selection mode includes the horizontal transform kernel DST7 and the vertical transform kernel DST7. As shown in FIG. 13 (e) , the division direction of the target division mode of the prediction residual block is the vertical direction and the horizontal direction, and the division proportion is to divide the height of the prediction residual block from top to bottom based on a dimensional proportion 1: 3, and to divide the width of the prediction residual block from left to right based on a dimensional proportion 1: 3, the target selection mode is to select the upper left sub-block (i.e., the shaded region in the figure) as the sub-block to be transformed, and the target transform kernel corresponding to the target selection mode includes the horizontal transform kernel DCT8 and the vertical transform kernel DCT8. As shown in FIG. 13 (f) , the division direction of the target division mode of the prediction residual block is the vertical direction and the horizontal direction, the division proportion is to divide the height of the prediction residual block from top to bottom based on a dimensional proportion 1: 3, and to divide the width of the prediction residual block from left to right based on a dimensional proportion 3: 1. The target selection mode is to select the upper right sub-block (i.e., the shaded region in the figure) as the target to be transformed, and the target transform kernel corresponding to the target selection mode includes the horizontal transform kernel DST7 and the vertical transform kernel DCT8. As shown in FIG. 13 (g) , the division direction of the target division mode of the prediction residual block is the vertical direction and the horizontal direction, and the division proportion is to divide the height of the prediction residual block from top to bottom based on a dimensional proportion 3: 1, and to divide the width of the prediction residual block from left to right based on a dimensional proportion 1: 3. The target selection mode is to select the lower left sub-block (i.e., the shaded region in the figure) as the sub-block to be transformed, and the target transform kernel corresponding to this target selection mode includes the horizontal transform kernel DCT8 and the vertical transform kernel DST7. As shown in FIG. 13 (h) , the division direction of the target division mode of the prediction residual block is the vertical direction and the horizontal direction, and the division proportion is to divide the height of the prediction residual block from top to bottom based on a dimensional proportion 3: 1, and to divide the width of the prediction residual block from left to right based on a dimensional proportion 3: 1. The target selection mode is to select the lower right sub-block (i.e., the shaded region in the figure) as the sub-block to be transformed, and the target transform kernel corresponding to the target selection mode includes the horizontal transform kernel DST7 and the vertical transform kernel DST7.

For example, FIG. 14 is a schematic diagram illustrating transform kernels corresponding to different sub-block selection modes according to some embodiments of the present disclosure. As shown in FIG. 14, the target division mode of the prediction residual block is to divide in the vertical direction and the horizontal direction, and the division proportion in the horizontal direction and the division proportion in the vertical direction are both 1: 1. The target selection mode is to determine the obtained four sub-blocks as sub-blocks to be transformed (i.e., the shaded regions in the figure are all the sub-blocks to be transformed) , where the transform kernel of the sub-block to be transformed at the upper left corner includes the horizontal transform kernel DCT8 and the vertical transform kernel DCT8, the transform kernel of the sub-block to be transformed at the lower left corner includes the horizontal transform kernel DCT8 and the vertical transform kernel DST7, the transform kernel of the sub-block to be transformed at the upper right corner includes he horizontal transform kernel DST7 and the vertical transform kernel DCT8, and the transform kernel of the sub-block to be transformed at the lower right corner includes the horizontal transform kernel DST7 and the vertical transform kernel DST7.

It can be understood that the shaded regions in FIG. 12 and FIG. 13 are sub-blocks with prediction residuals (e.g., the at least one sub-block to be transformed, and the at least one target sub-block) , and unfilled regions in FIG. 12 and FIG. 13 are sub-blocks without prediction residuals. The 8 embodiments shown in FIG. 12 and the 8 embodiments shown in FIG. 13 can each be used as a transformation mode; if the 16 embodiments shown in FIG. 12 and FIG. 13 are all applied to an IBC transformation (or the inverse transformation in the present disclosure) of video encoding and decoding, there are at least 16 transformation modes for the inverse quantization residual block in the prediction mode of IBC. For example, as shown in the transformation mode of FIG. 12 (a) , the prediction residual block is only divided in the vertical direction based on a dimensional proportion 1: 1, and the left sub-block of the two obtained sub-blocks is selected for DCT8 horizontal transformation and DST7 vertical transformation. As shown in the transformation mode of FIG. 12 (b) , the prediction residual block is only divided in the vertical direction based on a dimensional proportion 1: 1, and the right sub-block of the two obtained sub-blocks is selected for DST7 horizontal transformation and DCT8 vertical transformation.

In some embodiments, the transformation unit 316 may determine a transform kernel of each of the at least one sub-block to be transformed based on a transform kernel set; and for each of the at least one sub-block to be transformed, transform the sub-block to be transformed using the transform kernel corresponding to the sub-block to be transformed. In some embodiments, the transform kernel set may include one or more transform kernels determined based on at least one of the DCT2, the DST7, the DCT8 and the TS. In some embodiments, the transform kernel set may include one or more transform kernels determined based on at least one of the DST7 and the DCT8. Compared with the related art that only supports the DCT2, the embodiments of present disclosure introduce the DST7 and the DCT8 in the transformation (and the inverse transform described below) of the residual block in the prediction mode of IBC, which can increase options of the transform kernel for transformation of the residual block, thereby increasing the adaptability of the transform kernel for transformation of the residual block to the residual distribution of the residual block.

The transform kernel set refers to a set including two or more transform kernels. For example, the transform kernel set (i.e., a selection range of the transform kernels of the at least one sub-block to be transformed) may include a plurality of transform kernels in Table 1.
Table 1 Schematic table of transform kernel set

In some embodiments, the transformation unit 316 may write a transform index syntax into the bitstream. The transform index syntax may be configured to indicate a target transform kernel for transforming the at least one sub-block to be transformed of the prediction residual block.

In some embodiments, the transform kernel of each sub-block to be transformed may be determined from the transform kernel set based on a value of the transform index syntax corresponding to each sub-block to be transformed. For example, the transform kernel set is {transform kernel 1, transform kernel 2} , and the transform index syntax istTuflag is used to express which transform kernel in the transform kernel set is used; when istTuflag = 0, it means that transform kernel 1 is used; when istTuflag = 1, it means that transform kernel 2 is used.

In some embodiments, writing the transform index syntax into the bitstream may include: determining a value of the transform index syntax to be a first value or a second value based on the target transform kernel; in response to determining that the value of the transform index syntax is the first value, determining a count of quantization coefficients corresponding to the at least one sub-block to be transformed as an odd number; in response to determining that the value of the transform index syntax is the second value, determining the count of the quantization coefficients corresponding to the at least one sub-block to be transformed as an even number; and implicitly encoding the transform index syntax based on parity of the count of the quantization coefficients.

In some embodiments, the encoding terminal may implicitly encode the transform index syntax. In some embodiments, the encoding terminal may determine the value of the transform index syntax based on the parity of all quantization coefficients (odd quantization coefficients and even quantization coefficients) corresponding to the at least one sub-block to be transformed. In some embodiments, the encoding terminal may determine the value of the transform index syntax based on the parity of the odd quantization coefficients corresponding to the at least one sub-block to be transformed. The odd quantization coefficient means that the parity of the quantization coefficient is an odd number. In some embodiments, the encoding terminal may determine the value of the transform index syntax based on the parity of the even quantization coefficients corresponding to the at least one sub-block to be transformed. The even quantization coefficient means that the parity of the quantization coefficient is an even number. The quantization coefficient may be a coefficient quantized by the encoding terminal. In this way, the target transform kernel of each sub-block to be transformed may be determined based on the parity of the quantization coefficients corresponding to the at least one sub-block to be transformed in the bitstream. For example, the transform kernel set is {transform kernel 1, transform kernel 2} , and the value of the transform index syntax is determined based on the parity of the even quantization coefficients corresponding to the at least one sub-block to be transformed. When the count of even quantization coefficients is an odd number, it means that the transform kernel 1 is used; when the count of even quantization coefficients is an even number, it means that the transform kernel 2 is used.

In some embodiments, in response to determining that the value of the transform index syntax is the first value (e.g., 0 or 1, etc. ) , the encoding terminal may determine the count of quantization coefficients corresponding to the at least one sub-block to be transformed as an odd number; in response to determining that the value of the transform index syntax is the second value (e.g., 0 or 1, etc., and different from the first value) , the encoding terminal may determine the count of the quantization coefficients corresponding to the at least one sub-block to be transformed as an even number. In some embodiments, the count of the quantization coefficients refers to a total count of all quantization coefficients, the count of the odd quantization coefficients, or the count of even quantization coefficients, which is not limited here.

In some embodiments, when a plurality of syntaxes (e.g., a sub-block selection syntax, a sub-block division syntax, and a transform index syntax of the sub-blocks) of the sub-blocks are all implicitly encoded, different syntaxes of the sub-blocks use different types of quantization coefficients. For example, the sub-block selection syntax is implicitly encoded based on the parity of the odd quantization coefficients, and the transform index syntax is implicitly encoded based on the parity of the even quantization coefficients.

In some embodiments, the encoding terminal may directly encode the value of the transform index syntax into the bitstream of the target frame, i.e., explicitly encode the transform index syntax.

It should be noted that the transform kernel of the at least one sub-block to be transformed may include at least one of the DCT2, the DST7, the DCT8 and the TS. In some embodiments, the transform kernel of the at least one sub-block to be transformed may include other types of transform kernels, which are not limited here.

In the embodiments of present disclosure, a variety of transform modes are designed so that the transform modes can adapt to different residual distributions. That is, the determination range (also referred to as the selection range) of the transform kernel of the at least one sub-block to be transformed may include the transform kernel determined based on the DCT2, the DST7, the DCT8, and/or the TS. That is, the transform kernel set may include the transform kernel determined based on the DCT2, the transform kernel determined based on the DST7, the transform kernel determined based on the DCT8, and at least part of the transform kernel based on the TS.

In some embodiments, in order to reduce hardware complexity, the use range of some transform kernels (e.g., the DST7 and/or the DCT8) may be limited. In some embodiments, in response to determining that a size of the sub-block to be transformed is less than or equal to a first preset size, one of the one or more transform kernels determined based on at least one of the DST7 and the DCT8 may be determined as the transform kernel corresponding to the sub-block to be transformed.

The first preset size refers to a preset maximum sub-block size. In response to determining that the size of the sub-block to be transformed is less than or equal to the first preset size, the one of the one or more transform kernels determined based on at least one of the DST7 and the DCT8 may be determined as the transform kernel corresponding to the sub-block to be transformed. In response to determining that the size of the sub-block to be transformed exceeds the first preset size, the selection range of the transform kernel corresponding to the sub-block to be transformed may not include the transform kernel determined based on the DST7 and the transform kernel determined based on the DCT8. For example, the transform kernel determined based on the DCT2 may be used for transformation. In some embodiments, the first preset size may be 32*32 or 16*16, which is not limited in the present disclosure.

By limiting the use range of some transform kernels (e.g., the DST7 and/or the DCT8) , it ensures that the DST7 and/or the DCT8 are only applied to sub-blocks with relatively small sizes because the sub-blocks with relatively small sizes usually include finer details or edge information and are more suitable for transformation using efficient transform kernels (e.g., the DST7 and the DCT8) .

In some application scenarios, the use range of the transform kernel may not be limited, i.e., the use range of the DST7 and/or the DCT8 may not be limited, so that the sub-blocks to be transformed greater than the first preset size can also be transformed using the transform kernel determined based on the DST7 and/or the DCT8.

In some embodiments, the target transform kernel for transforming the at least one sub-block to be transformed may include a horizontal transform kernel and a vertical transform kernel, so that the at least one sub-block to be transformed may be separably transformed. In some embodiments, the horizontal transform kernel and the vertical transform kernel of the at least one sub-block to be transformed may be the same, for example, both are DCT2. In some embodiments, the horizontal transform kernel and the vertical transform kernel of the at least one sub-block to be transformed may be different, i.e., different transform kernels may be used for transformation in the horizontal direction and the vertical direction. The embodiments of the present disclosure design a variety of transform modes to adapt to different residual distributions.

In some embodiments, the encoding terminal may determine the target transform kernel for transforming the at least one sub-block to be transformed using a third machine learning model. An input of the third machine learning model may include an image feature of the prediction residual block and the target selection mode, and an output of the third machine learning model may be the corresponding target transform kernel. The third machine learning model may be obtained by training a third initial machine learning model based on a large number of training samples with training labels. The training samples may include the image feature of the prediction residual block, the target selection mode, and the corresponding target transform kernel, wherein the target transform kernel is the training label. Specifically, the encoding terminal may input the training samples with the training labels into the third initial machine learning model to obtain output results of the third initial machine learning model. Parameters of the initial machine learning model may be updated through training until a trained intermediate machine learning model meets a preset condition, thereby obtaining a trained third machine learning model. The preset condition may be that a loss function is less than a preset threshold, the loss function converges, or a count of training times reaches a preset threshold.

In 470, the encoding terminal may determine a bitstream of the target frame based on a transform result. In some embodiments, the operation 470 may be performed by the bitstream determination unit 317.

The bitstream refers to an encoded data stream, which is a sequence of binary data. By encoding the target frame into the bitstream, it is convenient for the storage and transmission of videos or images.

In some embodiments, the bitstream determination unit 317 may determine the bitstream of the target frame based on the transform result of the at least one sub-block to be transformed. The bitstream may include, but is not limited to, a transformation coefficient (related to a transformation operation) , a quantization parameter (related to a quantization operation) , a motion vector, etc. In some embodiments, the bitstream may be a final bitstream for transmission or storage formed by entropy encoding (e.g. arithmetic encoding or Huffman encoding) . For example, for each sub-block to be transformed, the bitstream determination unit 317 may quantize the transform result of the sub-block to be transformed to obtain quantized data, perform entropy encoding on the quantized data to obtain encoded data, and combine the encoded data of each sub-block to be transformed to obtain the bitstream of the target frame. As another example, the bitstream determination unit 317 may combine the transform result of each sub-block to be transformed to obtain combined data, quantize the combined data to obtain the quantized data, and perform entropy encoding on the quantized data to obtain the bitstream of the target frame.

The quantization refers to a process of mapping a continuous range of values to a finite count of discrete values, which is a way to convert data into an integral form.

The entropy encoding is a data compression technique that assigns a code length based on a probability of occurrence of symbols. A symbol with high probability of occurrence is assigned with a shorter code, and a symbol with low probability of occurrence is assigned with a longer code. Accordingly, the overall code length of the obtained encoded data is reduced, thereby achieving efficient data transmission and/or storage. For example, an entropy encoding mode may include arithmetic encoding, Huffman encoding, variable length encoding, etc.

The decoding terminal may reconstruct a video frame close to an original target frame based on the bitstream.

The video encoding method provided in the embodiments of the present disclosure can divide the prediction residual block and can selectively transform the sub-blocks, thereby improving the encoding efficiency and compression ratio, and reducing redundant information while ensuring the video quality.

In some embodiments, the target block may be a chrominance block or a luminance block.

The chrominance block is an image block configured to describe a color and saturation of the image.

The luminance block refers to an image block configured to describe a brightness (e.g., a grayscale value of the image) of the image.

In some embodiments, the encoding module 310 may perform transformation on the chrominance block using the operations 110-170. In some embodiments, the encoding module 310 may perform transformation on the luminance block using the operations 110-170.

In some embodiments, in response to determining that the target block is the luminance block, the encoding method (e.g., the process 400) may further include: transforming a chrominance block corresponding to the target block using a same encoding process as the target block, the target block and the chrominance block corresponding to a same image region of the target frame. For example, the target block is located in a region A of the image, and the chrominance block corresponding to the target block is also located in the region A of the image.

In some embodiments, in order to reduce the complexity of encoding and decoding, for the chrominance block, the encoding module 310 may transform the chrominance block by directly using the transform mode of the prediction residual block of the luminance block corresponding to the chrominance block, the luminance block and the corresponding chrominance block corresponding to the same image region. For example, if the transform mode of the luminance block corresponding to the chrominance block is to perform DST7 transformation on a left sub-block obtained by dividing the luminance block in the vertical direction based on a dimensional proportion 1: 1, the encoding module 310 may perform DST7 transformation on the left sub-block obtained by dividing the chrominance block in the vertical direction based on a dimensional proportion 1: 1. In some embodiments, the chrominance block may not directly use the transform mode of the corresponding luminance block, i.e., the chrominance block may determine the transform mode separately. For example, the encoding module 310 may directly perform the DCT2 transformation on the chrominance block; as another example, the encoding module 310 may determine the transform mode of the chrominance block through the transform mode of the above embodiment.

In some embodiments, the encoding module 310 may encode at least one of the plurality of image blocks of the target frame by performing the operation 430-460. The encoding module 310 may determine the bitstream of the target frame based on the transform result of the at least one of the plurality of image blocks. In some embodiments, the encoding module 310 may encode at least two image blocks in parallel or sequentially.

It should be noted that the above description of the process 400 is only for example and explanation, and does not limit the scope of application of the present disclosure. For those skilled in the art, various modifications and changes can be made to the process 400 under the guidance of the present disclosure. For example, in the operation 460, the encoding module 310 may first quantize the at least one sub-block to be transformed to obtain a quantization result, and transform the quantization result to obtain the transform result. However, these modifications and changes are still within the scope of the present disclosure.

FIG. 10 is a flowchart illustrating an exemplary process of sub-block division according to some embodiments of the present disclosure. The operation 440 may be performed based on a process 1000. As shown in FIG. 10, the process 1000 may include the following sub-operations.

In 441, the encoding terminal may obtain a plurality of candidate division modes. In some embodiments, the operation 441 may be performed by the first sub-block division unit 314.

The candidate division modes may include no division, or a specific division mode including a division direction (e.g., division in a horizontal direction, division in vertical direction, and division in the horizontal direction and the vertical direction) , a division proportion, etc., such as any of the sub-block division modes shown in FIG. 2 -FIG. 9.

In some embodiments, the first sub-block division unit 314 may obtain the plurality of candidate division modes from pre-stored or feasible sub-block division modes. For example, the first sub-block division unit 314 may randomly obtain the plurality of candidate division modes from the sub-block division modes stored in the storage device 120. As another example, the first sub-block division unit 314 may obtain the plurality of candidate division modes based on requirements (e.g., a count, size, shape, or the like, of sub-blocks) .

In 442, the encoding terminal may determine a score of each of the plurality of candidate division modes. In some embodiments, the operation 442 may be performed by the first sub-block division unit 314.

In some embodiments, the first sub-block division unit 314 may determine the score of each of the plurality of candidate division modes based on a preset first scoring criterion. In some embodiments, the first scoring criterion may include at least one of an image content, an encoding efficiency, a bit overhead, and a rate distortion cost of a target block.

In some embodiments, the score of each of the plurality of candidate division modes may be determined based on at least one of the image content, the encoding efficiency, the bit overhead, and the rate distortion cost of the target block.

The image content refers to visual information in the target block. In some embodiments, the image content may include features such as a texture, an edge, a color distribution, or the like of an image. The image content reflects a spatial feature of the image. For example, if the target block includes more smooth regions (e.g., blue sky, walls, land, etc. ) , larger blocks are more in line with the requirements of the encoding process (e.g., improving the encoding efficiency) ; correspondingly, the sub-block division proportion may be close to 1: 1 or no division may be adopted. That is, a candidate division mode of no division may have a highest score; or, a candidate division mode with a division proportion closer to 1: 1 may have a higher score.

The encoding efficiency may be configured to measure the effect of the sub-block division mode in compressing data, such as the ability to minimize the data volume of the target frame while maintaining the image quality. In some embodiments, the higher the encoding efficiency of the target block, the higher the score of the candidate division mode; the lower the encoding efficiency, the lower the score of the candidate division mode.

The bit overhead refers to a count of extra bits required to describe a sub-block division mode, including syntactic elements that record information such as a size, a shape, a position, or the like, of the sub-blocks. In some embodiments, the larger the bit overhead of the target block, the lower the score of the candidate division mode; the smaller the bit overhead, the higher the score of the candidate division mode.

The rate distortion cost is intended to balance the bit rate of the target frame after encoding and the quality of the decoded image after decoding. For example, the bit overhead is as small as possible while the quality of the decoded image is as high as possible. In some embodiments, the smaller the rate distortion cost of the target block, the higher the score of the candidate division mode; the larger the rate distortion cost of the target block, the lower the score of the candidate division mode.

In some embodiments, the score of each of the plurality of candidate division modes may be determined based on one of the image content, the encoding efficiency, the bit overhead, and the rate distortion cost of the target block. In some embodiments, the score of each of the plurality of candidate division modes may be determined based on two or more of the image content, the encoding efficiency, the bit overhead, and the rate distortion cost of the target block. When the score of the candidate division mode is determined based on two or more parameters (e.g., at least two of the image content, the encoding efficiency, the bit overhead, and the rate distortion cost of the target block) , a preset weight may be assigned to each parameter, so that a final score may be determined based on a weighted sum of the scores of the two or more parameters. For example, the preset weights of the two or more parameters may be the same. As another example, the preset weights of two or more parameters may be different. For example, different preset weights may be assigned according to importance of the parameter. The importance of the parameter may be determined or preset according to actual needs.

By comprehensively considering factors such as the image content, the encoding efficiency, the bit overhead, and the rate distortion cost, the processor may reduce redundant data, optimize sub-block division, and improve the overall performance and resource utilization of video encoding while maintaining the visual effect.

In 443, the encoding terminal may determine a candidate division mode of the plurality of candidate division modes with a highest score as a target division mode. In some embodiments, the operation 443 may be performed by the first sub-block division unit 314.

In some embodiments, the first sub-block division unit 314 may compare and sort the scores of the plurality of candidate division modes, and select the candidate division mode of the plurality of candidate division modes with the highest score as a target division mode. The target division mode is an optimal sub-block division mode among the plurality of candidate division modes, and the target division mode may achieve an optimal balance between the image content, the encoding efficiency, the bit overhead, and the rate distortion cost.

In 444, the encoding may divide, based on the target division mode, the prediction residual block into the at least one sub-block. In some embodiments, the 444 may be performed by the first sub-block division unit 314.

In some embodiments, the first sub-block division unit 314 may divide the prediction residual block into the at least one sub-block based on the target division mode. For example, if the target division mode is no division, the prediction residual block may be used as a sub-block (as shown in FIG. 2) . As another example, if the target division mode is to divide in the vertical direction based on a dimensional proportion 1: 1, the prediction residual block may be divided into two sub-blocks of the same size (as shown in FIG. 3) .

In the embodiments of the present disclosure, by obtaining the plurality of candidate division modes and determining the score of each of the plurality of candidate division modes, the processor may intelligently select the target division mode with the highest score, thereby dividing the prediction residual block into optimal sub-blocks. In this way, the encoding efficiency and the image compression ratio can be improved, and the image quality and a relatively low bit overhead can be guaranteed.

It should be noted that the above description of the process 1000 in FIG. 10 is only for example and explanation, and does not limit the scope of application of the present disclosure. For those skilled in the art, various modifications and changes can be made to the process 1000 under the guidance of the present disclosure. However, these modifications and changes are still within the scope of the present disclosure.

FIG. 11 is a flowchart illustrating an exemplary process of selecting a sub-block to be transformed according to some embodiments of the present disclosure. The operation 450 may be performed based on a process 1100. As shown in FIG. 11, the process 1100 may include the following sub-operations.

In 451, the encoding terminal may obtain a plurality of candidate selection modes. In some embodiments, the operation 451 may be performed by the first sub-block selection unit 315.

In some embodiments, the first sub-block selection unit 315 may obtain the plurality of candidate selection modes from pre-stored or feasible selection modes. For example, all the selection modes may be used as the candidate selection modes. As another example, the first sub-block selection unit 315 may randomly obtain the plurality of candidate selection modes from the selection modes stored in the storage device 120. As another example, the first sub-block selection unit 315 may obtain the plurality of candidate selection modes according to requirements (e.g., an encoding efficiency, etc. ) .

In 452, the encoding terminal may determine a score of each of the plurality of candidate selection modes. In some embodiments, the operation 452 may be performed by the first sub-block selection unit 315.

In some embodiments, the first sub-block selection unit 315 may determine the score of each of the plurality of candidate selection modes based on a preset second scoring criterion. In some embodiments, the second scoring criterion may include at least one of an image content, the encoding efficiency, a bit overhead, and a rate distortion cost of the target block.

In some embodiments, the score of each of the plurality of candidate selection modes may be determined based on at least one of the image content, the encoding efficiency, the bit overhead, and the rate distortion cost of the target block. More descriptions regarding the image content, the encoding efficiency, the bit overhead, and the rate distortion cost may be found in the present disclosure above.

For example, the more complex the image content of the target block (e.g., including many different types of objects) , the higher the score of the candidate selection mode; the simpler the image content, the lower the score of the candidate selection mode.

For example, the higher the encoding efficiency of the target block, the higher the score of the candidate selection mode; the lower the encoding efficiency, the lower the score of the candidate selection mode.

For example, the larger the bit overhead of the target block, the lower the score of the candidate selection mode; the smaller the bit overhead, the higher the score of the candidate selection mode.

For example, the smaller the rate distortion cost of the target block, the higher the score of the candidate selection mode; the larger the rate distortion cost of the target block, the lower the score of the candidate sub-block selection mode.

In some embodiments, the score of each of the plurality of candidate selection modes may be determined based on one of the image content, the encoding efficiency, the bit overhead, and the rate distortion cost of the target block. In some embodiments, the score of each of the plurality of candidate selection modes may be determined based on two or more of the image content, the encoding efficiency, the bit overhead, and the rate distortion cost of the target block. If the score of the candidate selection mode is determined based on two or more parameters (e.g., two or more of the image content, the encoding efficiency, the bit overhead, and the rate distortion cost of the target block) , the first sub-block selection unit 315 may assign a preset weight to each parameter, thereby determining a final score based on a weighted sum of the scores of the two or more parameters. For example, the preset weights of the two or more parameters may be the same. As another example, the preset weights of the two or more parameters may be different. For example, the first sub-block selection unit 315 may assign different preset weights according to an importance of the parameter. The importance of the parameter may be determined or preset according to actual needs.

In 453, the encoding terminal may determine a candidate selection mode of the plurality of candidate selection modes with a highest score as a target selection mode. In some embodiments, the operation 453 may be performed by the first sub-block selection unit 315.

In some embodiments, the first sub-block selection unit 315 may compare/rank the scores of the plurality of candidate selection modes, and select the candidate selection mode of the plurality of candidate selection modes with the highest score as the target selection mode. The target selection mode is an optimal selection mode among the plurality of candidate selection modes, and the target selection mode may achieve an optimal balance between the image content, the encoding efficiency, the bit overhead, and the rate distortion cost.

In 454, the encoding may determine, based on the target selection mode, the at least one sub-block to be transformed from the at least one sub-block. In some embodiments, the operation 454 may be performed by the first sub-block selection unit 315.

In some embodiments, the first sub-block selection unit 315 may determine the at least one sub-block to be transformed from the at least one sub-block based on the target selection mode to transform the at least one sub-block to be transformed. For example, if the target selection mode is to use all the sub-blocks as the sub-blocks to be transformed, all the sub-blocks may be determined as the sub-blocks to be transformed.

In the embodiments of the present disclosure, by obtaining the plurality of candidate selection modes and determining the score of each of the plurality of candidate selection mode, the processor can intelligently select the target selection mode with the highest score, thereby determining the at least one sub-block to be transformed. In this way, the encoding efficiency and the image compression ratio can be improved, and the image quality and a relatively low bit overhead can be guaranteed.

It should be noted that the above description of the process 1100 in FIG. 11 is only for example and explanation, and does not limit the scope of application of the present disclosure. For those skilled in the art, various modifications and changes can be made to the process 1100 under the guidance of the present disclosure. However, these modifications and changes are still within the scope of the present disclosure.

In some embodiments, a video encoding method may include: obtaining a target frame; dividing the target frame into a plurality of image blocks; for a target block among the plurality of image blocks, determining a prediction residual block of the target block in a prediction mode of intra block copy; transforming the prediction residual block using a target transform kernel corresponding to the prediction residual block; and determining a bitstream of the target frame based on a transform result of the prediction residual block; wherein the target transform kernel corresponding to the prediction residual block may include at least one transform kernel determined based on at least one of DST7 and DCT8.

FIG. 15 is a flowchart illustrating an exemplary video encoding method according to some embodiments of the present disclosure.

In some embodiments, a process 1500 may be performed by an encoding terminal (e.g., the computing device 200 and/or the encoding module 310) . As shown in FIG. 15, the process 1500 may include the following operations.

In 1510, the encoding terminal may obtain a target frame. In some embodiments, the operation 1510 may be performed by the encoding acquisition unit 311.

More descriptions regarding the operation 1510 may be found in the related descriptions of the operation 410 in FIG. 1E, which are not repeated here.

In 1520, the encoding terminal may divide the target frame into a plurality of image blocks. In some embodiments, the operation 1520 may be performed by the image block division unit 312.

More descriptions regarding the operation 1520 may be found in the related descriptions of the operation 420 in FIG. 1E, which are not repeated here.

In 1530, for a target block among the plurality of image blocks, the encoding terminal may determine a prediction residual block of the target block in a prediction mode of intra block copy. In some embodiments, the operation 1530 may be performed by the prediction residual block determination unit 313.

More descriptions regarding the operation 1530 may be found in the related descriptions of the operation 430 in FIG. 1E, which are not repeated here.

In 1540, the encoding terminal may transform the prediction residual block using a target transform kernel corresponding to the prediction residual block. In some embodiments, the operation 1540 may be performed by the transformation unit 316.

The target transform kernel corresponding to the prediction residual block may include at least one transform kernel determined based on at least one of DST7 and DCT8; or the target transform kernel may be selected from a transform kernel set, the transform kernel set including one or more transform kernels determined based on at least one of the DST7 and the DCT8.

In some embodiments, if the target transform kernel corresponding to the prediction residual block includes a transform kernel determined based on the DST7, the transformation unit 316 may transform the prediction residual block using the transform kernel determined based on the DST7. The transform kernel determined based on the DST7 may reduce the redundancy between colors of the target block and improve the ability to process complex colors and textures.

In some embodiments, if the target transform kernel corresponding to the prediction residual block includes a transform kernel determined based on the DCT8, the transformation unit 316 may transform the prediction residual block using the transform kernel determined based on the DCT8. The transform kernel determined based on the DCT8 may effectively remove high-frequency components that are not sensitive to human eyes, thereby achieving higher compression efficiency without causing a decrease in perceived quality.

In some embodiments, if the target transform kernel corresponding to the prediction residual block includes a transform kernel determined based on the DST7 and a transform kernel determined based on the DCT8, the transformation unit 316 may transform the prediction residual block using the transform kernel determined based on the DST7 and the transform kernel determined based on the DCT8 simultaneously. For example, the transformation unit 316 may perform a horizontal transformation on the prediction residual block using the transform kernel determined based on the DST7 and perform a vertical transformation on the prediction residual block using the transform kernel determined based on the DCT8.

In some embodiments, the transformation unit 316 may determine the target transform kernel corresponding the prediction residual block based on the transform kernel set; and transform the prediction residual block using the target transform kernel corresponding to the prediction residual block. In some embodiments, the transform kernel set may include one or more transform kernels determined based on at least one of the DCT2, the DST7, the DCT8, and the TS. In some embodiments, the transform kernel set may include one or more transform kernels determined based on at least one of the DST7 and the DCT8.

In some embodiments, the encoding terminal may transform the prediction residual block using other transform kernels, such as the DCT2.

In some embodiments, the target transform kernel corresponding to the prediction residual block may be a preset transform kernel.

In some embodiments, the transform kernel set may include a plurality of preset transform kernels, and horizontal transform kernels and vertical transform kernels in the plurality of preset transform kernels may be different. For example, the horizontal transform kernel of the at least one sub-block to be transformed is a transform kernel determined based on the DCT8, and the vertical transform kernel of the at least one sub-block to be transformed is a transform kernel determined based on the DST7.

In some embodiments, in response to determining that a size of the target block is less than or equal to a second preset size, the target transform kernel may be determined based on at least one of the DST7 and the DCT8.

The second preset size refers to a preset maximum block size. The second preset size may be the same as the first preset size. In response to determining that a size of the prediction residual block is less than or equal to the second preset size, the encoding terminal may determine the target transform kernel based on at least one of the DST7 and the DCT8. In response to determining that the size of the prediction residual block exceeds the second preset size, a selection range of the target transform kernel corresponding to the prediction residual block may not include the transform kernel determined based on DST7 and the transform kernel determined based on DCT8. For example, the encoding terminal may transform the prediction residual block using the transform kernel determined based on the DCT2. In some embodiments, the second preset size may be 32*32 or 16*16, which is not limited in the present disclosure.

By limiting the use range of some transform kernels (e.g., the DST7 and/or the DCT8) , it ensures that the DST7 and/or the DCT8 are only applied to sub-blocks with relatively small sizes because the relatively small sub-blocks usually include finer details or edge information and are more suitable for transformation using efficient transform kernels (e.g., the DST7 and the DCT8) .

In some application scenarios, the use range of the transform kernel may not be limited, i.e., the use range of the DST7 and/or the DCT8 may not be limited, so that the sub-blocks to be transformed greater than the second preset size can also be transformed using the transform kernel determined based on the DST7 and/or the DCT8.

More descriptions regarding the transform mode involved in the operation 1540 may be found in the related descriptions of the transform mode involved in the operation 460 in FIG. 1E, which are not repeated here.

In some embodiments, in the operation 1540, the encoding terminal may first determine the at least one sub-block obtained by dividing the prediction residual block based on the target division mode of the prediction residual block (more descriptions may be found in the operation 440, which are not repeated here) ; then select the at least one sub-block to be transformed from the at least one sub-block, determine the target transform kernel corresponding to each of the at least one sub-block to be transformed, and obtain the transform result by transforming each of the at least one sub-block to be transformed using the target transform kernel corresponding to each of the at least one sub-block to be transformed (the determination of the at least one sub-block to be transformed and the target transform kernel may be found in the operations 450-460, which are not repeated here) . In some embodiments, the target transform kernels corresponding to different sub-blocks to be transformed may be different or the same, and more descriptions may be found in the related descriptions of the operation 460, which are not repeated here. In some embodiments, for each of the at least one sub-block to be transformed, the transformation unit 316 may transform the sub-block to be transformed based on the corresponding target transform kernel. In some embodiments, the transformation unit 316 may perform transformation on the at least one sub-block to be transformed in parallel based on the corresponding target transform kernel.

In some embodiments, the encoding terminal may not divide the prediction residual block, i.e., the encoding terminal may transform the entire prediction residual block directly using a target transform kernel. For example, the encoding module 310 may transform the entire prediction residual block based on a preset transform kernel or a target transform kernel of the prediction residual block determined based on a transform kernel set.

In 1550, the encoding terminal may determine a bitstream of the target frame based on a transform result of the prediction residual block. In some embodiments, the operation1550 may be performed by the bitstream determination unit 317.

In some embodiments, the bitstream determination unit 317 may determine the bitstream of the target frame based on the transform result of the prediction residual block. In some embodiments, the bitstream may form a final bitstream after entropy encoding (e.g., arithmetic encoding or Huffman encoding) for transmission or storage. The decoding terminal may reconstruct a video frame close to an original target frame based on the bitstream.

It should be noted that the above description of the process 1500 is only for example and explanation, and does not limit the scope of application of the present disclosure. For those skilled in the art, various modifications and changes can be made to the process 1500 under the guidance of the present disclosure. However, these modifications and changes are still within the scope of the present disclosure.

In some embodiments, a video encoding method may include: obtaining a target frame; dividing the target frame into a plurality of image blocks; for a target block among the plurality of image blocks, determining a prediction residual block of the target block in a prediction mode of intra block copy; transforming the prediction residual block based on a target transform kernel corresponding to the prediction residual block; and determining a bitstream of the target frame based on a transform result of the prediction residual block; wherein the target transform kernel may include at least one transform kernel, and for each of the at least one transform kernel, a horizontal transform kernel and a vertical transform kernel of the transform kernel may be different.

FIG. 16 is a flowchart illustrating an exemplary video encoding method according to some embodiments of the present disclosure.

In some embodiments, a process 1600 may be performed by an encoding terminal (e.g., the computing device 200 and/or the encoding module 310) . As shown in FIG. 16, the process 1600 may include the following operations.

In 1610, the encoding terminal may obtain a target frame. In some embodiments, the operation 1610 may be performed by the encoding acquisition unit 311.

More descriptions regarding the operation 1610 may be found in the related descriptions of the operation 410 in FIG. 1E, which are not repeated here.

In 1620, the encoding terminal may divide the target frame into a plurality of image blocks. In some embodiments, the operation 1620 may be performed by the image block division unit 312.

More descriptions regarding the operation 1620 may be found in the related descriptions of the operation 420 in FIG. 1E, which are not repeated here.

In1630, for a target block among the plurality of image blocks, the encoding terminal may determine a prediction residual block of the target block in a prediction mode of intra block copy. In some embodiments, the operation 1630 may be performed by the prediction residual block determination unit 313.

More descriptions regarding the operation 1630 may be found in the related descriptions of the operation 430 in FIG. 1E, which are not repeated here e.

In 1640, the encoding terminal may transform the prediction residual block based on a target transform kernel corresponding to the prediction residual block. In some embodiments, the operation 1640 may be performed by the transformation unit 316.

The target transform kernel corresponding to the prediction residual block may include a preset transform kernel; or the target transform kernel corresponding to the prediction residual block may be determined based on a transform kernel set, the transform kernel set including a plurality of preset transform kernels, and a horizontal transform kernel and a vertical transform kernel in the preset transform kernels being different.

In some embodiments, the transformation unit 316 may transform the entire prediction residual block using the preset transform kernel (e.g., the DST7 and/or the DCT8) .

In some embodiments, the transformation unit 316 may determine the target transform kernel corresponding the prediction residual block based on the transform kernel set; and transform the entire prediction residual block using the target transform kernel corresponding to the prediction residual block.

The transform mode involved in the operation 1640 (e.g., the preset transform kernel, the transform kernel set, determining the target transform kernel corresponding to the prediction residual block, etc. ) is similar to the transform mode involved in the operation 400. More descriptions may be found in the operation 460 in FIG. 1E, which are not repeated here.

In 1650, the encoding terminal may determine a bitstream of the target frame based on a transform result of the prediction residual block. In some embodiments, the operation 1650 may be performed by the bitstream determination unit 317.

More descriptions regarding the operation 1650 may be found in the related descriptions of the operation 1650 in FIG. 15, which are not repeated here.

It should be noted that the above description of the process 1600 is only for example and explanation, and does not limit the scope of application of the present disclosure. For those skilled in the art, various modifications and changes can be made to the process 1600 under the guidance of the present disclosure. However, these modifications and changes are still within the scope of the present disclosure.

Referring to the present disclosure described above, in the related decoding techniques, the transform technique of IBC only supports DCT transformation, which is difficult to adapt to different residual distributions.

Accordingly, the embodiments of the present disclosure provide a video decoding method (e.g., processes 1700, 1800, and 1900) , which is performed by a decoding terminal.

When a machine learning model is involved in the encoding and/or decoding operations, the encoding terminal and/or the decoding terminal may be matched with a corresponding trained machine learning model.

FIG. 17 is a flowchart illustrating an exemplary video decoding method according to some embodiments of the present disclosure. In some embodiments, the process 1700 may be performed by a decoding terminal (e.g., the computing device 200 and/or the decoding module 320) . As shown in FIG. 17, the process 1700 may include the following operations.

In 1710, the decoding terminal may determine, by decoding a bitstream, an inverse quantization residual block of a target block in a prediction mode of intra block copy. In some embodiments, the operation 1710 may be performed by the entropy decoding unit 321.

The decoding terminal may determine, by decoding the bitstream of one or more target frames obtained from an encoding terminal, the inverse quantization residual block of the target block among the one or more target frames in the prediction mode of intra block copy. The bitstream of the target frame may be the bitstream of the target frame obtained by the video encoding method (e.g., the process 400, the process 1500, the process 1600, etc. ) described in any of the above embodiments.

The inverse quantization residual block refers to a block obtained by converting the prediction residual block after quantization back to a block close to the original prediction residual block.

Referring to the present disclosure described above, in the process of encoding the target frame, the bitstream of the target frame may be obtained through operations such as transformation, quantization, entropy encoding, etc. Inverse quantization is an inverse process of the quantization. For example, a limited count of discrete values may be mapped to a continuous range of values by the inverse quantization can map.

In some embodiments, if the prediction mode of the target block is the prediction mode of IBC, the inverse quantization residual block of the target block in the prediction mode of IBC may be determined; if the prediction mode of the target block is a prediction mode other than the prediction mode of IBC, the inverse quantization residual block of the target block in the prediction mode other than the prediction mode of IBC may be determined.

In some embodiments, the entropy decoding unit 321 may determine the inverse quantization residual block of the target block by performing the entropy decoding and the inverse quantization based on the bitstream. For example, the entropy decoding unit 321 may obtain decoded data by performing the entropy decoding on the bitstream of the target frame, and obtain the inverse quantization residual block of the target block by performing the inverse quantization on the decoded data.

The entropy decoding corresponds to the entropy encoding and is an inverse process of the entropy encoding. Similar to the entropy encoding, an entropy decoding mode may include but is not limited to arithmetic decoding, Huffman decoding, or variable length decoding.

In 1720, the decoding terminal may divide the inverse quantization residual block into at least one sub-block. In some embodiments, the operation 1720 may be performed by the second sub-block division unit 322.

In some embodiments, the second sub-block division unit 322 may first determine a target division mode of the inverse quantization residual block, and then divide the inverse quantization residual block into the at least one sub-block based on the target division mode of the inverse quantization residual block. For ease of understanding, the target division mode of the prediction residual block is also referred to as a first target division mode, and the target division mode of the inverse quantization residual block is also referred to as a second target division mode hereinafter. The first target division mode may be the same as the second target division mode.

In some embodiments, the second sub-block division unit 322 may divide the inverse quantization residual block into the at least one sub-block using a preset division mode. The preset division mode refers to a division mode set in advance. For example, the preset division mode is a quadtree division mode, so that quadtree division may be performed on the inverse quantization residual block to obtain four sub-blocks of the inverse quantization residual block.

It can be seen from the above video encoding method (e.g., the processes 400, 1000, and 1100) that during encoding, the first sub-block division unit 314 may write the sub-block division mode syntax of the prediction residual block into the bitstream. Accordingly, in some embodiments, the second sub-block division unit 322 may determine the target division mode of the inverse quantization residual block based on a value of the sub-block division mode syntax, and divide the inverse quantization residual block into the at least one sub-block based on the target division mode. In some embodiments, the second sub-block division unit 322 may determine the target division mode of the inverse quantization residual block of the target block based on the value of the sub-block division mode syntax of the target block by reading the sub-block division mode syntax of the target block in the bitstream and.

In some embodiments, the sub-block division mode syntax may include a sub-block switch syntax and a sub-block division syntax.

In some embodiments, the second sub-block division unit 322 may determine whether to perform sub-block division on the inverse quantization residual block based on a value of the sub-block switch syntax. For example, if sub-block switch syntax is sbt_cu_flag, and the value of sbt_cu_flag is a fifth value (e.g., 0) , it indicates that no division is performed. In this case, the target division mode of the inverse quantization residual block is no division, so that a size of a final sub-block size is the same as a size of the inverse quantization residual block; if the value of sbt_cu_flag is a sixth value (e.g., 1) , it indicates that sub-block division is performed, and the size of the final sub-block obtained by the division is different from the size of the inverse quantization residual block.

In some embodiments, if it is determined that sub-block division is performed on the inverse quantization residual block, the second sub-block division unit 322 may determine the target division mode of the inverse quantization residual block based on the value of the sub-block division mode syntax. For example, if the sub-block division mode syntax is sbt_ext, and the value of sbt_ext is a seventh value (e.g., 0) , it indicates that the inverse quantization residual block is divided in a horizontal direction based on a dimensional proportion 1: 1; if the value of sbt_ext is an eighth value (e.g., 1) , it indicates that the inverse quantization residual block is divided in the horizontal direction based on a dimensional proportion 1: 1: 1; if the value of sbt_ext is a ninth value (e.g., 2) , it indicates that the inverse quantization residual block is divided in the horizontal direction based on a dimensional proportion 1: 2: 1; if the value of sbt_ext is a tenth value (e.g., 3) , it indicates that the inverse quantization residual block is divided in a vertical direction based on a dimensional proportion 1: 1.

In some embodiments, the sub-block division mode syntax may include a direction syntax and a proportion syntax.

In some embodiments, the second sub-block division unit 322 may determine a division direction in the target division mode of the inverse quantization residual block based on the direction syntax. In some embodiments, the direction syntax may include sbt_ext and/or sbt_dir. Sbt_ext indicates that the inverse quantization residual block is divided in both the horizontal direction and the vertical direction, or only in the vertical direction or the horizontal direction. Sbt_dir indicates that the inverse quantization residual block is divided in the horizontal direction, or in the vertical direction.

For example, if the value of sbt_ext is an eleventh value (e.g., 0) , it indicates that the inverse quantization residual block is divided in both the horizontal direction and the vertical direction. As another example, if the value of sbt_ext is a twelfth value (e.g., 1) , it indicates that the inverse quantization residual block is divided in one direction (e.g., the vertical direction or the horizontal direction) . In this case, if the value sbt_dir is a thirteenth value (e.g., 0) , it indicates that the inverse quantization residual block is divided in the vertical direction, and if the value of sbt_dir is a fourteenth value (e.g., 1) , it indicates that the inverse quantization residual block is divided in the horizontal direction.

As another example, if the value of sbt_ext is the eleventh value (e.g., 0) , it means that the inverse quantization residual block is divided in both the horizontal direction and the vertical direction; if the value sbt_ext is a fifteenth value (e.g., 1) , it indicates that the inverse quantization residual block is divided in the vertical direction; if the value of sbt_ext is a sixteenth value (e.g., 2) , it indicates that the inverse quantization residual block is divided in the horizontal direction.

In some embodiments, the decoding terminal may use a preset direction as the division direction in the target division mode of the inverse quantization residual block. The preset direction may be a division direction set in advance. For example, if the preset direction is to divide in the horizontal direction and the vertical direction, the inverse quantization residual block may be divided in the horizontal direction and the vertical direction. As another example, if the preset direction is to divide in the horizontal direction, the inverse quantization residual block may be divided only in the horizontal direction.

In some embodiments, the second sub-block division unit 322 may determine a division proportion in the target division mode of the inverse quantization residual block based on the proportion syntax. In some embodiments, the proportion syntax may include sbt_ext1 or sbt_quad. For example, if the value of sbt_ext1/sbt_quad is a seventeenth value (e.g., 0) , it indicates that the division proportion is 1: 1; if the value of sbt_ext1/sbt_quad is an eighteenth value (e.g., 1) , it indicates that the division proportion is 1: 3 or 3: 1.

In some embodiments, the second sub-block division unit 322 may use a preset proportion as the division proportion in the target division mode of the inverse quantization residual block. The preset proportion may be a division proportion set in advance. For example, if the preset proportion is 1: 1, the inverse quantization residual block may be divided based on a dimensional proportion 1: 1. As another example, if the preset proportion is 1: 3, the inverse quantization residual block may be divided based on a dimensional proportion 1: 3.

In some embodiments, the direction syntax and the proportion syntax may be randomly combined, which is not repeated here.

In some embodiments, the second sub-block division unit 322 may determine the division direction in the target division mode of the inverse quantization residual block through the direction syntax, and use the preset proportion as the division proportion in the target division mode of the inverse quantization residual block. That is, the division proportion in the target division mode of the inverse quantization residual block may not be determined based on the proportion syntax. In some embodiments, the second sub-block division unit 322 may determine the division proportion in the target division mode of the inverse quantization residual block through the proportion syntax, and use the preset direction as the division direction in the target division mode of the inverse quantization residual block. That is, the division direction in the target division mode of the inverse quantization residual block may not be determined based on the direction syntax.

In some embodiments, the decoding terminal may express the target division mode through a syntax, i.e., the division mode syntax defines the division direction, the division proportion, etc. For example, the division mode syntax may be sbt_cu_flag, and each value of sbt_cu_flag defines a target division mode. For example, if the value of sbt_cu_flag value is 0, it indicates that the target division mode is to divide in the horizontal direction based on a dimensional proportion 1: 1; if the value of sbt_cu_flag is 1, it indicates that the target division mode is no division; if the value of sbt_cu_flag value is 2, it indicates that the target division mode is to divide in the vertical direction based on a dimensional proportion 1: 3, ….

In some embodiments, if a transform mode syntax is written into the bitstream of the target frame, and the transform mode syntax defines the sub-block division mode, the decoding terminal may use the transform mode syntax as the division mode syntax, and determine the target division mode of the inverse quantization residual block based on a value of the transform mode syntax. For example, if the value of the transform mode syntax is 0, it indicates to divide in the horizontal direction based on a dimensional proportion 1: 1, and perform DST7 transformation on at least part of the sub-blocks obtained by the division; if the value of the transform mode syntax value is 1, it indicates to divide in the vertical direction based on a dimensional proportion 1: 3, and perform DCT8 transformation on at least part of the sub-blocks obtained by the division. The decoding terminal may determine the target division mode of the inverse quantization residual block as dividing in the horizontal direction based on a dimensional proportion 1: 1 if the value of the transform mode syntax is 0; and determine the target division mode of the inverse quantization residual block as dividing in the vertical direction based on a dimensional proportion 1: 3 if the value of the transform mode syntax is 1.

In 1730, the decoding terminal may select at least one target sub-block from the at least one sub-block of the inverse quantization residual block. In some embodiments, the operation 1730 may be performed by the second sub-block selection unit 323.

The target sub-block refers to an inverse quantization residual sub-block used for subsequent inverse transformation, which is also referred to as a sub-block under inverse transformation. In some embodiments, the at least one target sub-block may be a sub-block with a prediction residual among the at least one sub-block of the inverse quantization residual block. The sub-block with the prediction residual may be determined as the target sub-block, so that inverse transformation is only performed on the sub-block with the prediction residual, thereby improving the decoding efficiency and saving the bit rate.

In some embodiments, all sub-blocks of the at least one sub-block may be used as the target sub-blocks, i.e., the inverse transformation may be performed on all the sub-block of the inverse quantization residual block to obtain the prediction residuals of all the sub-blocks. In some embodiments, some sub-blocks of the at least one sub-block may be used as the target sub-blocks, i.e., the inverse quantization may be performed on some sub-blocks of the inverse quantization residual block to obtain the prediction residuals of some sub-blocks. In some embodiments, the second sub-block selection unit 323 may select a sub-block at a preset position of the at least one sub-block as the target sub-block, i.e., the inverse quantization may be performed on the sub-block at the preset position of the inverse quantization residual block to obtain the prediction residual of the sub-block at the preset position. For example, a first sub-block in an encoding order of the at least one sub-block of the inverse quantization residual block is used as the target sub-block, and the inverse quantization is performed on the first sub-block to obtain the prediction residual of the first sub-block.

In some embodiments, selecting the at least one target sub-block from the at least one sub-block may include: obtaining a target selection mode corresponding to a target division mode of the inverse quantization residual block; and determining, based on the target selection mode, the at least one target sub-block from the at least one sub-block.

In some embodiments, the inverse quantization residual block may have a plurality of target division modes, and each of the plurality of target division modes may correspond to the same or different target selection modes.

It should be noted that the target division mode shown in FIG. 2 -FIG. 9 is for the residual block, and the residual block may be the prediction residual block or the inverse quantization residual block.

The at least one target sub-block being determined based the target selection mode corresponding to the target division mode of the inverse quantization residual block is similar to the at least one sub-block to be transformed being determined based on the target selection mode corresponding to the target division mode of the prediction residual block described above (e.g., the operation 450) . Accordingly, more descriptions regarding determining the at least one target sub-block based on the target selection mode corresponding to the target division mode of the inverse quantization residual block may be found in the present disclosure above (e.g., the operation 450) , which are not repeated here.

As can be seen from the foregoing, during video encoding, the first sub-block selection unit 315 may write the sub-block selection syntax corresponding to the prediction residual block into the bitstream, i.e., the bitstream may include information related to the sub-block selection syntax. In this case, selecting the at least one target sub-block from the at least one sub-block may include: determining the sub-block selection syntax of the target block by interpreting the bitstream; and determining, based on a value of the sub-block selection syntax, the at least one target sub-block from the at least one sub-block.

In some embodiments, the sub-block selection syntax may include a division mode syntax and a position selection syntax. In some embodiments, the division mode syntax indicates that a division mode of the inverse quantization residual block is to divide into the inverse quantization residual block two sub-blocks alone the horizontal direction. In response to determining that a value of the position selection syntax is a third value (e.g., 1) , a sub-block of the two sub-blocks located at a lower side may be selected as the target sub-block (e.g., as shown in the shaded region in FIG. 5 (b) ) ; in response to determining that the value of the position selection syntax is a fourth value (e.g., 0) , a sub-block of the two sub-blocks located at an upper side may be selected as the target sub-block (e.g., as shown in the shaded region in FIG. 5 (a) ) .

In some embodiments, the division mode syntax indicates that the division mode of the inverse quantization residual block is to divide the inverse quantization residual block into two sub-blocks in the vertical direction. In response to determining that the value of the position selection syntax is the third value (e.g., 1) , a sub-block of the two sub-blocks located at a right side may be selected as the target sub-block (e.g., as shown in the shaded region in FIG. 3 (b) ) ; in response to determining that the value of the position selection syntax is the fourth value (e.g., 0) , a sub-block of the two sub-blocks located at a left side may be selected as the target sub-block (e.g., as shown in the shaded region in of FIG. 3 (a) ) .

In some embodiments, the sub-block selection syntax may be configured to indicate an order of the selected sub-blocks among all the sub-blocks based on an encoding order/decoding order. For example, in response to determining that the value of the sub-block selection syntax is a fifteenth value (e.g., 0) , a first sub-block among all the sub-blocks in the encoding order/decoding order may be selected; in response to determining that the value of the sub-block selection syntax is a sixteenth value (e.g., 1) , a second sub-block among all the sub-blocks in the encoding order/decoding order may be selected. In some embodiments, the encoding order/decoding order may be from left to right, and from top to bottom. In some embodiments, the encoding order/decoding order may be from right to left, and from bottom to top.

In some embodiments, if the transform mode syntax is written into the bitstream of the target frame, and the position of the sub-block with the prediction residual is defined in the transform mode syntax, the transform mode syntax may be used as the sub-block selection syntax. That is, the position of the sub-block with the prediction residual in the inverse quantization residual block of the target block may be determined based on the value of the transform mode syntax, thereby determining the target sub-block. For example, if the value of the transform mode syntax is 0, it indicates to divide the inverse quantization residual block in the horizontal direction based on a dimensional proportion 1: 1, and perform the DST7 transformation on a sub-block of the sub-blocks obtained by division located at the upper side, and the decoding module 320 may determine the sub-block located at the upper side as the target sub-block. As another example, if the value of the transform mode syntax is 1, it indicates to divide the inverse quantization residual block in the vertical direction based on a dimensional proportion 1: 3, and perform the DST7 transformation on a sub-block of the sub-blocks obtained by division located at a left side, and the decoding module 320 may determine the sub-block located at the left side as the target sub-block.

In some embodiments, if a plurality of target sub-blocks are determined, the positions of the plurality of target sub-blocks may be continuous or discontinuous, which is not limited here.

The process of selecting the at least one target sub-block is similar to the process of selecting the at least one sub-block to be transformed described above (e.g., the operation 450) . More descriptions regarding the process of selecting the at least one target sub-block may be found in the related descriptions above (e.g., the operation 450) , which are not repeated here.

In 1740, the decoding terminal may obtain a prediction residual of the at least one target sub-block by performing an inverse transformation on the at least one target sub-block. In some embodiments, the operation 1740 may be performed by the inverse transformation unit 324.

The inverse transformation is an inverse processing corresponding to the transformation at the encoding terminal, i.e., an inverse operation of the transformation.

The prediction residual refers to a difference between an original value and a prediction value of the at least one target sub-block.

In some embodiments, the inverse transformation unit 324 may obtain the prediction residual of the at least one target sub-block by performing the inverse transformation on the at least one target sub-block using a preset transform kernel. In this case, different target sub-blocks may use the same transform kernel (i.e., the preset transform kernel) for the inverse transformation.

For example, if the preset transform kernel is a transform kernel determined based on the DCT2, the inverse transformation unit 324 may perform the inverse transformation on each of the at least one target sub-block in a certain sequence or in parallel using the transform kernel determined based on the DCT2.

In some embodiments, the preset transform kernel may include a transform kernel determined based on at least one of the DST7 and the DCT8.

For example, if the preset transform kernel is the transform kernel determined based on the DST7, the inverse transformation unit 324 may perform the inverse transformation on each of the at least one target sub-block in a certain sequence or in parallel using the transform kernel determined based on the DST7. The transform kernel determined based on the DST7 may reduce the redundancy between colors of the target block and improve the ability to process complex colors and textures.

As another example, if the preset transform kernel is the transform kernel determined based on the DCT8, the inverse transformation unit 324 may perform the inverse transformation on each of the at least one target sub-block in a certain sequence or in parallel using the transform kernel determined based on the DCT8. The transform kernel determined based on the DCT8 may effectively remove high-frequency components that are not sensitive to human eyes, thereby achieving higher compression efficiency without causing a decrease in perceived quality.

As another example, if the preset transform kernel is the transform kernel determined based on the DST7 and the transform kernel determined based on the DCT8, the inverse transformation unit 324 may perform the inverse transformation on the at least one target sub-block separately or simultaneously using the transform kernel determined based on the DST7 and the transform kernel determined based on the DCT8. For example, the horizontal transform kernel of the at least one target sub-block is the transform kernel determined based on the DST7, and the vertical transform kernel of the at least one target sub-block is the transform kernel determined based on the DCT8; or the horizontal transform kernel of the at least one target sub-block is the transform kernel determined based on the DST7, and the vertical transform kernel of the at least one target sub-block is the transform kernel determined based on the DST8.

In some embodiments, the inverse transformation unit 324 may perform the inverse transformation on the at least one target sub-block using a target transform kernel corresponding to the target division mode of the inverse quantization residual block. In this case, the target division mode of the inverse quantization residual block may be different, and the at least one target sub-block determined may be different. Inverse transformation may be performed on different target sub-blocks using the same or different transform kernels, thereby improving the adaptability of the target sub-block and the transform kernel. In some embodiments, the target transform kernel corresponding to the target division mode may include one or more transform kernels determined based on at least one of the DST7 and the DCT8.

In some embodiments, a plurality of sub-blocks obtained by each target division mode may correspond to the same transform kernel. For example, the transform kernels of the two sub-blocks obtained by dividing in the horizontal direction using the target division mode based on a dimensional proportion 1: 1 may both correspond to the transform kernel determined based on the DST7, and the transform kernels of the two sub-blocks obtained by dividing in the vertical direction using the target division mode based on a dimensional proportion 1: 3 may both correspond to the transform kernel determined based on the DCT2.

In some embodiments, at least two sub-blocks at each position of the sub-blocks obtained using each target division mode may correspond to different transform kernels. In some embodiments, each sub-block at each position obtained using each target division mode may have a corresponding transform kernel, i.e., the plurality of sub-blocks obtained using each target division mode may correspond to different transform kernels.

In some embodiments, the inverse transformation unit 324 may perform the inverse transformation on the at least one target sub-block using the target transform kernel corresponding to the target selection mode of the inverse quantization residual block. Due to different target selection modes, the target sub-blocks determined may be different, and inverse transformation may be performed on the different target sub-blocks using the same or different transform kernels, thereby improving the adaptability of the target sub-block and the transform kernel. In some embodiments, the target transform kernel corresponding to the target selection mode may include one or more transform kernel determined based on at least one of the DST7 and the DCT8.

In some embodiments, the target sub-block obtained using each target selection mode may correspond to the same transform kernel, such as the kernel determined based on the DST7.

In some embodiments, each sub-block at each position obtained using each target selection mode may have the corresponding transform kernel, i.e., the target sub-block obtained using each target selection mode may correspond to a different transform kernel. For example, referring to FIG. 7, when a sub-block located at any of the four different positions (i.e., the upper left corner, the lower left corner, the upper right corner, and the lower right corner) shown in FIG. 7 is selected as the target sub-block, the target sub-block at a different position corresponds to a different transform kernel.

It should be noted that the target transform kernel corresponding to the selection mode shown in FIG. 12 –FIG. 14 is for the residual block, and the residual block may be the prediction residual block or the inverse quantization residual block.

The process of performing the inverse transformation on the at least one target sub-block using the target transform kernel corresponding to the target division mode of the inverse quantization residual block is similar to the process of performing the transformation on the at least one sub-block to be transformed using the target transform kernel corresponding to the target division mode of the prediction residual block described above (e.g., the operation 460) . Accordingly, more descriptions regarding performing the inverse transformation on the at least one target sub-block using the target transform kernel corresponding to the target division mode of the inverse quantization residual block may be found in the related descriptions above (e.g., the operation 460) , which are not repeated here.

In some embodiments, the inverse transformation unit 324 may determine the transform kernel of each of the at least one target sub-block based on a transform kernel set, and for each of the at least one target sub-block, perform the inverse transformation on the target sub-block using the transform kernel corresponding to the target sub-block. In some embodiments, the transform kernel set may include one or more transform kernels determined based on at least one of the DCT2, the DST7, the DCT8, and the TS. In some embodiments, the transform kernel set may include one or more transform kernels determined based on at least one of the DST7 and the DCT8.

The process of performing the inverse transformation on the at least one target sub-block using the transform kernel set is similar to the process of transforming the at least one sub-block to be transformed using the transform kernel set to described above (e.g., the operation 460) . Accordingly, more descriptions regarding performing the inverse transformation on at least one target sub-block using the transform kernel set may be found in the related descriptions above (e.g., the operation 460) , which are not repeated here.

In some embodiments, as can be seen from the foregoing, during video encoding, the transformation unit 316 may write the transform index syntax corresponding to the prediction residual block into the bitstream, i.e., the bitstream may include information related to the transform index syntax. In this case, the inverse transformation unit 324 may determine a value of the transform index syntax corresponding to each of the at least one target sub-block based on the bitstream, and determine a transform kernel corresponding to each of the at least one target sub-block based on the value.

In some embodiments, the transform index syntax in the bitstream may be explicitly encoded or implicitly encoded. The process of determining the transform kernel corresponding to each of the at least one target sub-block based on the value of the transform index syntax is similar to the process of determining the transform kernel of the at least one sub-block to be transformed based on the value of the transform index syntax in described above (e.g., the operation 460) . Accordingly, more descriptions regarding determining the transform kernel corresponding to each of the at least one target sub-block based on the value of the transform index syntax may be found in the related descriptions above (e.g., the operation 460) , which are not repeated here.

In some embodiments, the inverse transformation unit 324 may determine parity of a quantization coefficient corresponding to each of the at least one target sub-block based on the bitstream; and determine a transform index corresponding to each of the at least one target sub-block based on the parity. Optionally, the quantization coefficient may be a coefficient after the inverse transformation at the decoding terminal.

In some embodiments, the inverse transformation unit 324 may determine a value of the transform index syntax based on the parity of the quantization coefficient corresponding to each of the at least one target sub-block, or based on parity of an odd quantization coefficient corresponding to each of the at least one target sub-block, or based on parity of an even quantization coefficient corresponding to each of the at least one target sub-bloc. In this way, the inverse transformation unit 324 may determine the transform kernel of each of the at least one sub-block to be transformed based on the parity of the quantization coefficient corresponding to each of the at least one target sub-block in the bitstream. For example, when the inverse transformation unit 324 determines the value of the transform index syntax based on the parity of the even quantization coefficient corresponding to each of the at least one target sub-block, in response to determining that a count of the even quantization coefficients is an odd number, it indicates using a transform kernel 1; in response to determining that the count of even quantization coefficients is an even number, it indicates using a transform kernel 2.

In some embodiments, the encoding terminal may directly encode the transform index syntax, so that the decoding may determine the transform kernel corresponding to each of the at least one target sub-block from the transform kernel set based on the value of the transform index syntax corresponding to each of the at least one target sub-block based on the bitstream.

In some embodiments, the transform kernel corresponding to each of the at least one target sub-block may include a preset transform kernel. In some embodiments, the transform kernel corresponding to each of the at least one target sub-block may include a transform kernel determined using at least one of DST7 and DCT8. In some embodiments, the transform kernel corresponding to each of the at least one target sub-block may be determined based on a transform kernel set, and the transform kernel set includes one or more transform kernels determined using at least one of the DST7 and the DCT8.

In some embodiments, the transform kernel corresponding to each of the at least one target sub-block may include one or more of the DCT2, the DST7, the DCT8 and the TS. In some embodiments, the transform kernel corresponding to each of the at least one target sub-block may include a horizontal transform kernel and a vertical transform kernel, so that the inverse transformation unit 324 may perform a separable transformation on the at least one target sub-block. In some embodiments, the horizontal transform kernel and the vertical transform kernel of each of the at least one target sub-block may be the same. For example, the horizontal transform kernel and the vertical transform kernel of each of the at least one target sub-block may both be the DCT2. In some embodiments, the horizontal transform kernel and the vertical transform kernel of each of the at least one target sub-block may be different, i.e., different transform kernels may be used for the transformation in the horizontal direction and the vertical direction. The embodiments of the present disclosure provide a variety of transform modes so that the transform modes can adapt to different residual distributions.

In some embodiments, if the transform mode syntax is written into the bitstream of the target frame, and the transform mode syntax defines the transform kernel of the at least one sub-block to be transformed, the decoding terminal may determine the transform kernel corresponding to each of the at least one target sub-block based on the value of the transform mode syntax. For example, if the value of the transform mode syntax is 0, it indicates that the prediction residual block is divided in the horizontal direction based on a dimensional proportion 1: 1, and DST7 transformation is performed on a sub-block of the sub-blocks obtained by division located at the upper side. Then, the decoding module 320 may determine that the target division mode of the inverse quantization residual block is to divide the inverse quantization residual block in the horizontal direction based on a dimensional proportion 1: 1, and the sub-block of the sub-blocks obtained by division located at the upper side may be determined as the target sub-block, and the transform kernel of the target sub-block may be the transform kernel determined based on the DST7. As another example, if the value of the transform mode syntax is 1, it indicates that the prediction residual block is divided in the vertical direction based on a dimensional proportion 1: 3, and DST8 transformation is performed on a sub-block of the sub-blocks obtained by division located at the left side. Then, the decoding module 320 may determine that the target division mode of the inverse quantization residual block is to divide the inverse quantization residual block in the vertical direction based on a dimensional proportion 1: 3, and the sub-block of the sub-blocks obtained by division located at the left side may be determined as the target sub-block, and the transform kernel of the target sub-block may be the transform kernel determined based on the DST8.

In some embodiments, in order to reduce hardware complexity, the use range of some transform kernels (e.g., the DST7 and/or the DCT8) may be limited. In some embodiments, if a size of the at least one target sub-block is less than or equal to a first preset size, the transform kernel may be determined using at least one of the DST7 and the DCT8.

In some embodiments, if the size of the at least one target sub-block is less than or equal to the first preset size, the inverse transformation unit 324 may determine the transform kernel using at least one of the DST7 and/or the DCT8; when the size of the at least one target sub-block exceeds the first preset size, the selection range of the transform kernel corresponding to each of the at least one target sub-block may not include the transform kernel determined based on the DST7 and the transform kernel determined based on the DCT8. For example, the inverse transformation unit 324 may use perform the inverse transformation using transform kernel determined based on the DCT2.

By limiting the use range of some transform kernels (e.g., the DST7 and/or the DCT8) , it ensures that the DST7 and/or the DCT8 are only applied to sub-blocks with relatively small sizes because the relatively small sub-blocks usually include finer details or edge information and are more suitable for the transformation using efficient transform kernels (e.g., the DST7 and the DCT8) .

In some application scenarios, the use range of the transform kernel may not be limited, i.e., the use range of the DST7 and/or the DCT8 may not be limited, so that the inverse transformation can also be performed on the sub-blocks to be transformed greater than the first preset size using the transform kernel determined based on the DST7 and/or the DCT8.

In 1750, the decoding terminal may generate, based on the prediction residual of the at least one target sub-block, a decoded image. In some embodiments, the operation 1750 may be performed by the decoded image determination unit 325.

In some embodiments, the decoded image determination unit 325 may determine the decoded image of the one or more target frames based on the prediction residual of the at least one target subblock.

In some embodiments, the decoding module 320 may perform the inverse transformation on the chrominance block using the operations 1710-1750. In some embodiments, the decoding module 320 may perform the inverse transformation on the luminance block using the operations 1710-1750.

In some embodiments, in response to determining that the target block is the luminance block, the decoding method may further include: performing the inverse transformation on the chrominance block corresponding to the target block using a same decoding process as the target block, the target block and the chrominance block corresponding to a same image region of the target frame.

In some embodiments, in order to reduce the complexity of encoding and decoding, for the chrominance block, the decoding module 320 may perform the inverse transformation on the chrominance block by directly using the transform mode of the inverse quantization residual block of the luminance block corresponding to the chrominance block, the luminance block and the corresponding chrominance block corresponding to the same image region. For example, if the inverse transformation mode of the luminance block corresponding to the chrominance block is to perform DST7 inverse transformation on a left sub-block obtained by dividing the luminance block in the vertical direction based on a dimensional proportion 1: 1, the decoding module 320 may perform the DST7 inverse transformation on the left sub-block obtained by dividing the chrominance block in the vertical direction based on a dimensional proportion 1: 1. In some embodiments, the chrominance block may not directly use the transform mode of the corresponding luminance block, i.e., the chrominance block may determine the transform mode separately. For example, the decoding module 320 may directly perform the DCT2 inverse transformation on the chrominance block; as another example, the decoding module 320 may determine the transform mode of the chrominance block through the transform mode of the above embodiment.

It should be noted that the above description of the process 1700 is only for example and explanation, and does not limit the scope of application of the present disclosure. For those skilled in the art, various modifications and changes can be made to the process 1700 under the guidance of the present disclosure. However, these modifications and changes are still within the scope of the present disclosure.

In some embodiments, a video decoding method may include: determining, by decoding a bitstream of one or more target frames obtained from an encoding terminal, an inverse quantization residual block of a target block among the one or more target frames in a prediction mode of intra block copy; obtaining an inverse transformation result by performing an inverse transformation on the inverse quantization residual block using a transform kernel corresponding to the inverse quantization residual block; and determining, based on the inverse transformation result, a decoded image of the one or more target frames; wherein the transform kernel corresponding to the inverse quantization residual block may include a transform kernel determined using at least one of DST7 and DCT8; or the transform kernel corresponding to the inverse quantization residual block may be determined based on a transform kernel set, and the transform kernel set may include one or more transform kernels determined using at least one of the DST7 and the DCT8.

FIG. 18 is a flowchart illustrating an exemplary video decoding method according to some embodiments of the present disclosure. In some embodiments, a process 1800 may be performed by a decoding terminal (e.g., the computing device 200 and/or the decoding module 320) . As shown in FIG. 18, the process 1800 may include the following operations.

In 1810, the decoding terminal may determine, by decoding a bitstream, an inverse quantization residual block of a target block in a prediction mode of intra block copy. In some embodiments, the operation 1810 may be performed by the entropy decoding unit 321.

The decoding terminal may determine, by decoding the bitstream of one or more target frames obtained from an encoding terminal, the inverse quantization residual block of the target block among the one or more target frames in a prediction mode of intra block copy. In some embodiments, if the prediction mode of the target block is the prediction mode of IBC, the inverse quantization residual block of the target block in the prediction mode of IBC may be determined; if the prediction mode of the target block is a prediction mode other than the prediction mode of IBC, the inverse quantization residual block of the target block in the prediction mode other than the prediction mode of IBC may be determined.

More descriptions regarding the operation 1810 may be found in the related descriptions of the operation 1710 in FIG. 17, which are not repeated here.

In 1820, the decoding terminal may obtain an inverse transformation result by performing an inverse transformation on the inverse quantization residual block using a transform kernel corresponding to the inverse quantization residual block. The transform kernel corresponding to the inverse quantization residual block may include a transform kernel determined using at least one of the DST7 and the DCT8; or the transform kernel corresponding to the inverse quantization residual block may be determined based on a transform kernel set, and the transform kernel set may include one or more transform kernels determined using at least one of the DST7 and the DCT8. In some embodiments, the operation 1820 may be performed by the inverse transformation unit 324.

After the inverse quantization residual block of the target block in the prediction mode of IBC is determined, the decoding module 320 may perform the inverse transformation on the inverse quantization residual block using the transform kernel corresponding to the inverse quantization residual block. In some embodiments, the inverse transformation unit 324 may perform the inverse transformation on the inverse quantization residual block based on the transform kernel determined using at least one of the DST7 and the DCT8. In some embodiments, the inverse transformation unit 324 may perform the inverse transformation on the inverse quantization residual block using the transform kernel determined using the DST7. The transform kernel determined using the DST7 may reduce the redundancy between the colors of the target block and improve the ability to process complex colors and textures. In some embodiments, the inverse transformation unit 324 may perform the inverse transformation on the inverse quantization residual block using the transform kernel determined using the DCT8. The transform kernel determined using the DCT8 may effectively remove high-frequency components that are not sensitive to the human eyes, thereby achieving higher compression efficiency without causing a decrease in perceived quality. In some embodiments, the inverse transformation unit 324 may perform the inverse transformation on the inverse quantization residual block using the transform kernel determined using the DST7 and the transform kernel determined using the DCT8.

In some embodiments, the inverse transformation unit 324 may determine the transform kernel corresponding to the inverse quantization residual block based on the transform kernel set; and perform the inverse transform on the inverse quantization residual block using the transform kernel corresponding to the inverse quantization residual block. In some embodiments, the transform kernel set may include one or more transform kernels determined based on at least one of the DCT2, the DST7, the DCT8 and the TS. In some embodiments, the transform kernel set may include one or more transform kernels determined based on at least one of the DST7 and the DCT8. In this embodiment, since the transform kernel set may include other types of transform kernels other than the transform kernel determined based on the DST7 and the transform kernel determined based on the DCT8, the transform kernel corresponding to the inverse quantization residual block may include the transform kernel determined based on the DST7, and/or the transform kernel determined based on the DCT8, and/or a preset transform kernel, and the transform kernel corresponding to the inverse quantization residual block may not include the transform kernel determined based on the DST7, the transform kernel determined based on DCT8, and the preset transform kernel, which is not limited here.

It can be understood that the transform kernel determined based on the DST7 means that at least one of a horizontal transform kernel and a vertical transform kernel includes the transform kernel determined based on the DST7, and the transform kernel determined based on the DCT8 means that at least one of the horizontal transform kernel and the vertical transform kernel includes the transform kernel determined based on the DCT8.

In some embodiments, the horizontal transform kernel and the vertical transform kernel may use the same transform kernel, such as the transform kernel determined based on the DST7, i.e., the horizontal transform kernel and the vertical transform kernel are both the transform kernel determined based on the DST7; and the transform kernel determined based on the DCT8, i.e., the horizontal transform kernel and the vertical transform kernel are both the transform kernel determined based on the DCT8.

In some embodiments, the horizontal transform kernel and the vertical transform kernel may use different transform kernels. For example, the transform kernel determined based on DST7 may include but is not limited to that the horizontal transform kernel and the vertical transform kernel are both the transform kernel determined based on the DST7, the horizontal transform kernel is the transform kernel determined based on the DCT8 and the vertical transform kernel is the transform kernel determined based on the DST7, the horizontal transform kernel is the transform kernel determined based on the DCT2 and the vertical transform kernel is the transform kernel determined based on the DST7, the horizontal transform kernel is the transform kernel determined based on the DST7 and the vertical transform kernel is the transform kernel determined based on the DCT8, and the horizontal transform kernel is the transform kernel determined based on the DST7 and the vertical transform kernel is the transform kernel determined based on the DCT2. As another example, the transform kernel determined based on the DCT8 may include but is not limited to that the horizontal transform kernel and the vertical transform kernel are both the transform kernel determined based on the DCT8, the horizontal transform kernel is the transform kernel determined based on the DCT8 and the vertical transform kernel is the transform kernel determined based on the DST7, the horizontal transform kernel is the transform kernel determined based on the DCT8 and the vertical transform kernel is the transform kernel determined based on the DCT2, the horizontal transform kernel is the transform kernel determined based on the DST7 and the vertical transform kernel is the transform kernel determined based on the DCT8, and the horizontal transform kernel is the transform kernel determined based on the DCT2 and the vertical transform kernel is the transform kernel determined based on the DCT8.

In some embodiments, the inverse transformation unit 324 may transform the prediction residual block using other transform kernels, such as the transform kernel determined based on the DCT2.

In some embodiments, the transform kernel corresponding to the inverse quantization residual block may be a preset transform kernel.

In some embodiments, the transform kernel set may include a plurality of preset transform kernels, and a horizontal transform kernel and a vertical transform kernel in each of the plurality of preset transform kernels may be different. For example, the preset transform kernels may include that the horizontal transform kernel is the transform kernel determined based on the DST7 and the vertical transform kernel is the transform kernel determined based on the DCT2.

It should be noted that in some embodiments, the transform kernel set may not include the preset transform kernel, and the transform kernel corresponding to the inverse quantization residual block may not include the preset transform kernel.

In some embodiments, in response to determining that a size of the target block is less than or equal to a second preset size, the transform kernel is determined using at least one of the DST7 and the DCT8.

In response to determining that the size of the inverse quantization residual block is less than or equal to the second preset size, the inverse transformation unit 324 may determine the transform kernel using at least one of the DST7 and/or the DCT8. In response to determining that the size of the inverse quantization residual block exceeds the second preset size, the selection range of the transform kernel corresponding to the inverse quantization residual block may not include the transform kernel determined based on the DST7 and the transform kernel determined based on the DCT8. For example, the inverse transformation unit 324 may perform the inverse transformation use the transform kernel determined based on the DCT2.

In some application scenarios, the use range of the transform kernel may not be limited, i.e., the use range of the DST7 and/or the DCT8 may not be limited, so that the inverse transformation can also be performed on the at least one target sub-block of which the size is greater than the second preset size using the transform kernel determined based on the DST7 and/or the DCT8.

More descriptions regarding the transform mode involved in the operation 1820 may be found in the related descriptions of the transform mode involved in the operation 1740 in FIG. 17, which are not repeated here.

In some embodiments, in the operation 1820, the decoding module 320 may first determine the at least one sub-block obtained by dividing the inverse quantization residual block based on the target division mode of the inverse quantization residual block (more descriptions may be found in the operation 1720, which are not repeated here) ; then select the at least one target sub-block (i.e., the sub-block under inverse transformation) from the at least one sub-block, determine the transform kernel corresponding to each of the at least one target sub-block, and perform the inverse transformation on each of the at least one target sub-block using the transform kernel corresponding to each of the at least one target sub-block to obtain the prediction residual of the at least one target sub-block (the process of determining the at least one target sub-block and the transform kernel thereof may be found in the operation 1730, which are not repeated here) . In some embodiments, the transform kernels of different target sub-blocks may be different or the same. More descriptions may be found in the related descriptions in the operation 1740, which are not repeated here.

In some embodiments, the decoding module 320 may not divide the inverse quantization residual block, i.e., the decoding module 320 may perform the inverse transformation on the entire inverse quantization residual block directly using a transform kernel. For example, the decoding module 320 may determine the transform kernel corresponding to the inverse quantization residual block based on the preset transform kernel or based on the value of the transform index syntax of the inverse quantization residual block, and perform the inverse transformation on the inverse quantization residual block.

In 1830, the decoding terminal may determine, based on the inverse transformation result, a decoded image. In some embodiments, the operation 1830 may be performed by the decoded image determination unit 325.

In some embodiments, the decoded image determination unit 325 may determine the decoded image of the one or more target frames based on the inverse transformation result.

It should be noted that the above description of the process 1800 is only for example and explanation, and does not limit the scope of application of the present disclosure. For those skilled in the art, various modifications and changes can be made to the process 1800 under the guidance of the present disclosure. However, these modifications and changes are still within the scope of the present disclosure.

In some embodiments, a video decoding method may include: determining, by decoding a bitstream of one or more target frames obtained from an encoding terminal, an inverse quantization residual block of a target block among the one or more target frames in a prediction mode of intra block copy; obtaining an inverse transformation result by performing an inverse transformation on the inverse quantization residual block using a transform kernel corresponding to the inverse quantization residual block; and determining, based on the inverse transformation result, a decoded image of the one or more target frames; wherein the transform kernel corresponding to the inverse quantization residual block may include a preset transform kernel; or the transform kernel corresponding to the inverse quantization residual block may be determined based on a transform kernel set, the transform kernel set may include a plurality of preset transform kernels, and a horizontal transform kernel and a vertical transform kernel in the plurality of preset transform kernel may be different.

FIG. 19 is a flowchart illustrating an exemplary video decoding method according to some embodiments of the present disclosure. In some embodiments, a process 1900 may be performed by a decoding terminal (e.g., the computing device 200 and/or the decoding module 320) . As shown in FIG. 19, the process 1900 may include the following operations.

In 1910, the decoding terminal may determine, by decoding a bitstream, an inverse quantization residual block of a target block in a prediction mode of intra block copy. In some embodiments, the operation 1910 may be performed by the entropy decoding unit 321.

More descriptions regarding the operation 1910 may be found in the related descriptions of the operation 1710 in FIG. 17, which are not repeated here.

In 1920, the decoding terminal may obtain an inverse transformation result by performing an inverse transformation on the inverse quantization residual block using a transform kernel corresponding to the inverse quantization residual block. The transform kernel corresponding to the inverse quantization residual block may include the preset transform kernel; or the transform kernel corresponding to the inverse quantization residual block may be determined based on the transform kernel set, the transform kernel set may include a plurality of preset transform kernels, and the horizontal transform kernel and the vertical transform kernel in the preset transform kernels may be different. In some embodiments, the operation 1920 may be performed by the inverse transformation unit 324.

In some embodiments, the inverse transformation unit 324 may perform the inverse transformation on the inverse quantization residual block using the preset transform kernel (e.g., the DST7 and/or the DCT8) .

In some embodiments, the inverse transformation unit 324 may determine the transform kernel corresponding to the inverse quantization residual block based on the transform kernel set; and perform the inverse transformation on the inverse quantization residual block using the transform kernel corresponding to the inverse quantization residual block.

More descriptions regarding the transform mode involved in the operation 1920 may be found in the related descriptions of the transform mode involved in the operation 1740 in FIG. 17, which are not repeated here.

In some embodiments, in the operation 1920, the decoding module 320 may first determine the at least one sub-block obtained by dividing the inverse quantization residual block based on the target division mode of the inverse quantization residual block (more descriptions may be found in the operation 1720, which are not repeated here) ; then select the at least one target sub-block (i.e., the at least one sub-block under inverse transformation) from the at least one sub-block, determine the transform kernel corresponding to each of the at least one target sub-block, and perform the inverse transformation on each of the at least one target sub-block using the transform kernel corresponding to each of the at least one target sub-block to obtain the prediction residual of the at least one target sub-block (the process of determining the at least one target sub-block and the transform kernel thereof may be found in the operation 1730, which are not repeated here) . In some embodiments, the transform kernels of different target sub-blocks may be different or the same. More descriptions may be found in the related descriptions of the operation 1740, which are not repeated here.

In some embodiments, the decoding module 320 may not divide the inverse quantization residual block, i.e., the decoding module 320 may perform the inverse transformation on the entire inverse quantization residual block directly using the transform kernel corresponding to the inverse quantization residual block.

In 1930, the decoding terminal may determine, based on the inverse transformation result, a decoded image. In some embodiments, the operation 1930 may be performed by the decoded image determination unit 325.

It should be noted that the above description of the process 1900 is only for example and explanation, and does not limit the scope of application of the present disclosure. For those skilled in the art, various modifications and changes can be made to the process 1900 under the guidance of the present disclosure. However, these modifications and changes are still within the scope of the present disclosure.

The inverse quantization residual block and the prediction residual block are both referred to as the residual block, and the target block may be the luminance block and/or the chrominance block. In some embodiments, the encoding module 310 may transform the luminance block and/or the chrominance block using the transformation (e.g., transform the at least one sub-block to be transformed in the prediction residual block using the same or different transform kernels) described above, and the decoding module 320 may perform the inverse transformation on the luminance block and/or the chrominance block using the inverse transformation (e.g., perform the inverse transformation on the at least one target sub-block in the inverse quantization residual block using the same or different transform kernels) described above.

The inverse transformation and the transformation may form at least one transform mode of the present disclosure, and the at least one transform mode of the present disclosure is different from the prior art. Optionally, the video encoding and decoding system 100 may use the at least one transform mode of the present disclosure to replace the transformation and the inverse transformation in encoding and decoding of the image/video frame in the prior art. For example, at least two transform modes of the present disclosure (e.g., the residual block (e.g., the prediction residual block or the inverse quantization residual block is not divided) are used for division, and the transformation/inverse transformation is performed on the residual block using a transform kernel 1, where the horizontal transform kernel and the vertical transform kernel of the transform kernel 1 are both the transform kernel determined based on the DCT2) for replacing the transformation and the inverse transformation in encoding and decoding of the image/video frame in the prior art.

In some embodiments, the video encoding and decoding system 100 may use the at least one transform mode of the present disclosure and the related transformation/inverse transformation techniques in existing coding and decoding as a transform mode option in the prediction mode of IBC in video coding and decoding.

In some embodiments, at least one transform mode of the present disclosure may be used without restrictions, that is, any residual block (eg, a prediction residual block or an inverse quantization residual block) in the intra block copy mode may apply at least one transform mode of the present application.

In some embodiments, the use of the at least one transform mode of the present disclosure may be subject to a restriction. That is, only the residual block in the prediction mode of IBC that meets the restriction can apply the at least one transform mode of the present disclosure, and the residual block in the prediction mode of IBC that does not meet the restriction does not apply the at least one transform mode of the present disclosure. In some embodiments, the restriction may include a block size condition, a component attribute condition, and/or a decoded information condition.

The block size condition may include but is not limited to that an image block meets a preset size range and/or the sub-block meets a preset size range. The size includes but is not limited to a width, a height, a width-to-height ratio, a height-to-width ratio, width*height, a logarithm of width, and/or a logarithm of height, etc.

The decoded information condition may include but is not limited to IBC type information, IBC filter information, quantization coefficient information, and/or image-level TS switch information, etc. The IBC type information may include but is not limited to an IBC encoding block under a natural sequence and/or an IBC encoding block under a screen content, etc. The quantization coefficient information may include but is not limited to size information of a quantization coefficient matrix, quantity information of non-zero coefficients, information on whether non-zero coefficients exist, and/or CBF marker information, etc. The IBC filter information may include but is not limited to information on whether an IBC prediction value is filtered.

The component attribute condition may include but is not limited to a luminance component applied the IBC encoding block, a chrominance component applied the IBC encoding block, and/or a luminance and chrominance component applied the IBC encoding block.

In Example 1, the video encoding and decoding system 100 replaces the prior art with two transform modes of the present disclosure. An application condition of the two transform modes of the present disclosure may include a condition applied to the IBC encoding block under the natural sequence, an IBC encoding block with logarithmic values of width and height less than 6 (i.e., the width and the height are both less than 64) , and a condition where the image-level TS is closed, and for the luminance component, and a condition that ScanRegionX ≤16 and ScanRegionY ≤16 (i.e., the width and the height of the quantization coefficient matrix are both less than or equal to 16) of SRCC coefficients. In Example 1, one of the transform modes of the present disclosure is that the residual block (e.g., the prediction residual block/the inverse quantization residual block) is not divided, and the transformation/inverse transformation is performed on the residual block using the transform kernel 1, where the horizontal transform kernel and the vertical transform kernel of the transform kernel 1 are both the transform kernel determined based on the DCT2. Another transform mode of the present disclosure is that the residual block (e.g., the prediction residual block/the inverse quantization residual block) is not divided, and the transformation/inverse transformation is performed on the residual block using a transform kernel 2, where the horizontal transform kernel and the vertical transform kernel of the transform kernel 2 are both the transform kernel determined based on the DST7. That is, the transform kernel of the residual block is determined based on the transform kernel set {transform kernel 1, transform kernel 2} without dividing the residual block, so that th two transform modes of the present disclosure can be formed.

In Example 2, the video encoding and decoding system 100 replaces the prior art with eighteen transform modes of the present disclosure. The 18 transform modes may include two first transform modes and sixteen second transform modes.

The two first transformation modes included in Example 2 are as shown in the two transform modes of Example 1, and the application condition of the two first transform modes of Example 2 can also be as shown in the application condition of the two transformation modes of Example 1.

The application condition of the sixteen second transform modes in Example 2 may include: a condition applied to the IBC coding block under the natural sequence, an IBC encoding block without predictive filtering, a luminance component, a condition that the image-level TS is closed, an IBC encoding block with a width and height of a current encoding block less than 128 and a width or height greater than 4, a width and height of a sub-block greater than or equal to 4, a width-to-height ratio or a height-to-width ratio of a sub-block not greater than 8, and a luminance component with a non-zero transform coefficient (or a luminance component CBF is not zero) .

As shown in FIG. 12 and FIG. 13, the 16 second transform modes in Example 2 may include the following content.

As shown in FIG. 12 (a) , the residual block (e.g., the prediction residual block/the inverse quantization residual block) is divided only in the vertical direction based on a dimensional proportion 1: 1, and a left sub-block (i.e., the left sub-block is the sub-block to be transformed or the target sub-block) is selected to perform DCT8 horizontal transformation/inverse transformation, and DST7 vertical transformation/inverse transformation.

As shown in FIG. 12 (b) , the residual block (e.g., the prediction residual block/the inverse quantization residual block) is divided only in the vertical direction based on a dimensional proportion 1: 1, and a right sub-block (i.e., the right sub-block is the sub-block to be transformed or the target sub-block) is selected to perform DST7 horizontal transformation/inverse transformation, and DST7 vertical transformation/inverse transformation.

As shown in FIG. 12 (c) , the residual block (e.g., the prediction residual block/the inverse quantization residual block) is divided only in the horizontal direction based on a dimensional proportion 1: 1, and an upper sub-block (i.e., the upper sub-block is the sub-block to be transformed or the target sub-block) is selected to perform DST7 horizontal transformation/inverse transformation, and DCT8 vertical transformation/inverse transformation.

As shown in FIG. 12 (d) , the residual block (e.g., the prediction residual block/the inverse quantization residual block) is divided only in the horizontal direction based on a dimensional proportion 1: 1, and a lower sub-block (i.e., the lower sub-block is the sub-block to be transformed or the target sub-block) is selected to perform DST7 horizontal transformation/inverse transformation, and DST7 vertical
transformation/inverse transformation.

As shown FIG. 12 (e) , the residual block (e.g., the prediction residual block/the inverse quantization residual block) is divided only in the vertical direction based on a dimensional proportion 1: 3, and a left sub-block (i.e., the left sub-block is the sub-block to be transformed or the target sub-block) is selected to perform DCT8 horizontal transformation/inverse transformation, and DST7 vertical transformation/inverse transformation.

As shown in FIG. 12 (f) , the residual block (e.g., the prediction residual block/the inverse quantization residual block) is divided only in the vertical direction based on a dimensional proportion 3: 1, and a right sub-block (i.e., the right sub-block is the sub-block to be transformed or the target sub-block) is selected to perform DST7 horizontal transformation/inverse transformation, and DST7 vertical transformation/inverse transformation.

As shown in FIG. 12 (g) , the residual block (e.g., the prediction residual block/the inverse quantization residual block) is divided only in the horizontal direction based on a dimensional proportion 1: 3, and an upper sub-block (i.e., the upper sub-block is the sub-block to be transformed or the target sub-block) is selected to perform DST7 horizontal transformation/inverse transformation, and DCT8 vertical transformation/inverse transformation.

As shown in FIG. 12 (h) , the residual block (e.g., the prediction residual block/the inverse quantization residual block) is divided only in the horizontal direction based on a dimensional proportion 3: 1, and a lower sub-block (i.e., the lower sub-block is the sub-block to be transformed or the target sub-block) is selected to perform DST7 horizontal transformation/inverse transformation, and DST7 vertical transformation/inverse transformation.

As shown in FIG. 12 (a) , the residual block (e.g., the prediction residual block/the inverse quantization residual block) is divided in the vertical direction based on a dimensional proportion 1: 1 and the horizontal direction based on a dimensional proportion 1: 1, and an upper left sub-block (i.e., the upper left sub-block is the sub-block to be transformed or the target sub-block) is selected to perform DCT8 horizontal transformation/inverse transformation, and DCT8 vertical transformation/inverse transformation.

As shown in FIG. 12 (b) , the residual block (e.g., the prediction residual block/the inverse quantization residual block) is divided in the vertical direction based on a dimensional proportion 1: 1 and the horizontal direction based on a dimensional proportion 1: 1, and an upper right sub-block (i.e., the upper right sub-block is the sub-block to be transformed or the target sub-block) is selected to perform DST7 horizontal transformation/inverse transformation, and DCT8 vertical transformation/inverse transformation.

As shown in FIG. 12 (c) , the residual block (e.g., the prediction residual block/the inverse quantization residual block) is divided in the vertical direction based on a dimensional proportion 1: 1 and the horizontal direction based on a dimensional proportion 1: 1, and a lower left sub-block (i.e., the lower left sub-block is the sub-block to be transformed or the target sub-block) is selected to perform DCT8 horizontal transformation/inverse transformation, and DST7 vertical transformation/inverse transformation.

As shown in FIG. 12 (d) , the residual block (e.g., the prediction residual block/the inverse quantization residual block) is divided in the vertical direction based on a dimensional proportion 1: 1 and the horizontal direction based on a dimensional proportion 1: 1, and a lower right sub-block (i.e., the lower right sub-block is the sub-block to be transformed or the target sub-block) is selected to perform DST7 horizontal transformation/inverse transformation, and DST7 vertical transformation/inverse transformation.

As shown in FIG. 12 (e) , the residual block (e.g., the prediction residual block/the inverse quantization residual block) is divided in the vertical direction based on a dimensional proportion 1: 3 and the horizontal direction based on a dimensional proportion 1: 3, and an upper left sub-block (i.e., the upper left sub-block is the sub-block to be transformed or the target sub-block) is selected to perform DCT8 horizontal transformation/inverse transformation, and DCT8 vertical transformation/inverse transformation.

As shown in FIG. 12 (f) , the residual block (e.g., the prediction residual block/the inverse quantization residual block) is divided in the vertical direction based on a dimensional proportion 3: 1 and the horizontal direction based on a dimensional proportion 1: 3, and an upper right sub-block (i.e., the upper right sub-block is the sub-block to be transformed or the target sub-block) is selected to perform DST7 horizontal transformation/inverse transformation, and DCT8 vertical transformation/inverse transformation.

As shown in FIG. 12 (g) , the residual block (e.g., the prediction residual block/the inverse quantization residual block) is divided in the vertical direction based on a dimensional proportion 1: 3 and the horizontal direction based on a dimensional proportion 3: 1, and a lower left sub-block (i.e., the lower left sub-block is the sub-block to be transformed or the target sub-block) is selected to perform DCT8 horizontal transformation/inverse transformation, and DST7 vertical transformation/inverse transformation.

As shown in FIG. 12 (h) , the residual block (e.g., the prediction residual block/the inverse quantization residual block) is divided in the vertical direction based on a dimensional proportion 3: 1 and the horizontal direction based on a dimensional proportion 3: 1, and a lower right sub-block (i.e., the lower right sub-block is the sub-block to be transformed or the target sub-block) is selected to perform DST7 horizontal transformation/inverse transformation, and DST7 vertical transformation/inverse transformation.

In Example 3, the video encoding and decoding system 100 uses the third transform mode of the present disclosure and the two transform modes of Example 1 as the transform mode of the residual block in the prediction mode of IBC (e.g., the prediction residual block/inverse quantization residual block) in video encoding and decoding. An application condition of Example 3 may include: a condition applied to the IBC encoding block under the natural sequence, the IBC encoding block that is not subjected to predictive filtering, the luminance block, and the IBC encoding block with a width and height are both greater than or equal to 8 and less than or equal to 32, the IBC encoding block with a width-height ratio and height-to-width ratio less than 4, and the IBC coding block with a non-zero coefficient (including a non-zero coefficient of luminance and chrominance ) .

In this Example, the third transform mode of Example 3 of the present disclosure is that the division mode of the residual block (e.g., the prediction residual block or the inverse quantization residual block) of the target block is to divide the residual block in the vertical direction and the horizontal direction, and the division proportion in the horizontal direction and the division proportion in the vertical direction are both 1: 1. As shown in FIG. 14, all the sub-blocks (i.e., the four shaded regions in the figure) may be used as the sub-blocks with the prediction residual and the transformation/inverse transformation may be performed on the sub-blocks, where the transform kernel of the sub-block located at the upper left corner includes a horizontal transform kernel DCT8 and a vertical transform kernel DCT8, the transform kernel of the sub-block located at the lower left corner includes a horizontal transform kernel DCT8 and a vertical transform kernel DST7, the transform kernel of the sub-block located at the upper right corner includes a horizontal transform kernel DST7 and a vertical transform kernel DCT8, and the transform kernel of the sub-block located at the lower right corner includes a horizontal transform kernel DST7 and a vertical transform kernel DST7.

It can be understood that the residual block described in Examples 1-3 refers to the prediction residual block or the inverse quantization residual block; the transformation/inverse transformation refers to perform the transformation on the prediction residual block/inverse transformation on the inverse quantization residual block.

The syntax used in the video decoding method and video encoding method of the present disclosure (e.g., the sub-block division syntax, the sub-block selection syntax, and the transform index syntax) includes but is not limited to: 1) an IBC multi-transform switch syntax; 2) an IBC multi-transform solution syntax; 3) IBC multi-transform syntax encoding.

(1) The IBC multi-transform switch syntax is used to express whether the technique proposed in the present disclosure is enabled in an encoder/decoder. When a plurality of techniques are involved, different techniques are allowed to have independent switch syntaxes. The switch syntax can be transmitted in syntax structures including but not limited to a video parameter set (VPS) , a sequence parameter set (SPS) , a picture parameter set (PPS) , a picture header (PH) , a coding unit, a transformation unit, etc.

(2) The IBC multi-transform solution syntax is used to express the transform mode of the IBC image block in the encoder/decoder. The IBC multi-transform solution syntax includes but is not limited to one or more of the following syntaxes.

1) Transform set syntax: when there are at least two transform mode sets (including at least one transform mode) , it indicates whether a certain transform mode set is selected.

2) Transform index syntax: for a transform mode set including at least two transform modes, a mode index syntax is used to indicate that a specific transform mode is selected.

3) Sub-block switch syntax: for indicating whether the IBC encoding block is divided into sub-blocks.

4) Sub-block division syntax: for expressing the division direction, the division proportion, sub-block selection, etc. of the residual block during sub-block division.

The IBC multi-transform solution syntax can be transmitted in syntax structures including but not limited to the VPS, the SPS, the PPS, the PH, the coding unit, the transformation unit, etc.

The IBC multi-transform syntax encoding: the syntax encoding mode includes but is not limited to an explicit syntax and/or an implicit syntax. The explicit syntax includes but is not limited to advanced entropy encoding, signed fixed-length encoding, unsigned fixed-length encoding, and/or exponential Columbus encoding, and other explicit syntaxes. The encoding mode is described in the standard text in the form of a descriptor, and the specific meaning may be found in the corresponding standard text. The implicit syntax includes but is not limited to parity based on coefficients, parity based on even coefficients, and/or parity based on odd coefficients. The coefficients include but are not limited to a quantization coefficient and/or a coefficient after inverse quantization, etc. The syntax encoding of the encoding process matches the syntax interpretation of the decoding process, so as to ensure that the decoding terminal can decode correctly. For example, if a count of the even coefficients in decoding is an even number, it means istflag = 1, and if the count of the even coefficients in decoding is an odd number, it means istflag = 0; accordingly, when istflag = 1 at the encoding terminal, if the count of the even coefficients is not an even number, the count of the even coefficients needs to be adjusted to an even number.

In Example 4, the IBC multi-transform switch syntax is not used, i.e., the multi-transform solution of Example 1 is enabled by default.

The IBC multi-transform solution syntax: the transform index istTuflag syntax is used to express whether to use the transform kernel 1 or the transform kernel 2; if istTuflag = 0, it means to use the transform kernel 1; if istTuflag = 1, it means to use the transform kernel 2.

The IBC multi-transform syntax encoding: implicit syntax encoding, the istTuflag value is determined based on the parity of the even quantization coefficients. If the count of the even quantization coefficients is an odd number, istTuflag = 0, and if the count of the even quantization coefficients is an even number, istTuflag = 1.

As shown in Table 2, blockWidth is a TU width (i.e., the width of the sub-block of the residual block) , blockHeight is a TU height (i.e., the height of the sub-block of the residual block) , QuantCoeffMatrix is a quantization coefficient matrix, NumEvenCoeff is a variable used to record the count of the even coefficients, ibc_cu_flag is an IBC marker of a current CU, ph_ibc_type is an IBC encoding mode marker under a natural sequence, ScanRegionX is a rightmost horizontal coordinate of a non-zero coefficient in the quantization coefficient matrix, ScanRegionY is a bottommost vertical coordinate of the non-zero coefficient in the quantization coefficient matrix, and PictureTsEnableFlag is a TS switch in the picture header.
Table 2 Definition of sub-blocks to be transformed

Example 5 is based on Example 2. As shown in Table 3 and Table 4, the IBC multi-transform switch syntax: whether to enable the solution in Example 2 based on sbt_enable_flag in the sequence header; sbt_enable_flag = 0 indicates that the solution is not enabled, and sbt_enable_flag = 1 indicates that the solution is enabled.

The IBC multi-transform solution syntax: the sub-block switch syntax sbt_cu_flag is used to express whether to perform sub-block division. sbt_cu_flag = 0 indicates no division, and sbt_cu_flag = 1 indicates sub-block division.

Furthermore, if sbt_cu_flag = 0, the specific transform kernel to be used is determined based on the ist_flag syntax.

Furthermore, if sbt_cu_flag = 1, sbt_ext is used to indicate whether to perform division in the horizontal direction and the vertical direction, or only in the vertical direction or only in the horizontal direction.

If sbt_ext = 1, it indicates that division is performed in the horizontal direction and the vertical direction, sbt_ext1 = 0 indicates that the division proportion is 1: 1, sbt_ext1 = 1 indicates that the division proportion is 1: 3 or 3: 1, hor_idx1/hor_idx2 = 0 indicates that if sbt_ext1 = 0/1, the left sub-block is selected, hor_idx1/hor_idx2 = 1 indicates that if sbt_ext1 = 0/1, the right sub-block is selected, sbt_pos_flag = 0 indicates selecting the upper sub-block, sbt_pos_flag = 1 indicates selecting the lower sub-block. Accordingly, sbt_ext1, hor_idx, and sbt_pos_flag jointly determine the position of the sub-block.

If sbt_ext = 0, it means division only in the vertical direction or only in the horizontal direction, sbt_quad is used to express the division proportion, sbt_dir is used to express the division direction, and sbt_pos_flag is used to express the position of the sub-block; sbt_quad = 0 indicates that the division proportion is 1: 1, and sbt_quad = 1 indicates that the division proportion is 1: 3 or 3: 1; sbt_dir = 0 indicates vertical division, and sbt_dir = 1 indicates horizontal division; sbt_pos_flag = 0 indicates selecting the left (for vertical division) /upper (for horizontal division) sub-block, and sbt_pos_flag = 1 indicates selecting the right (for vertical division) /lower (for horizontal division) sub-block; accordingly, sbt_quad, sbt_dir, and sbt_pos_flag jointly determine the position of the sub-block.

The IBC multi-transform syntax encoding: sbt_flag, sbt_ext, sbt_ext1, hor_idx, sbt_quad, and sbt_dir syntax are all transmitted in the transformation unit, and syntax encoding is performed by advanced entropy encoding; sbt_pos_flag adopts implicit syntax encoding, and the value of sbt_pos_flag is determined based on the parity of the even quantization coefficients. If the count of the even quantization coefficients is an odd number, sbt_pos_flag = 0, and if the count of the even quantization coefficients is an even number, sbt_pos_flag = 1.

Specifically, as shown in Table 3 and Table 4, ibc_cu_flag is an IBC maker of the current coding unit, ph_ibc_type is an IBC encoding mode marker under the natural sequence, width is a width of the current coding unit, height is a height of the current coding unit, ibcPfFlag is an IBC prediction value filtering marker of the current coding unit, and ctp_y [0] is a marker for whether there is a non-zero coefficient in the luminance component.
Table 3 Definition of coding unit

Table 4 Definition of sub-blocks to be transformed

Example 6: This example is based on Example 3.

IBC multi-transform switch syntax: whether to enable the solution in Example 3 is determined based on pbt_enable_flag in the sequence header; pbt_enable_flag = 0 indicates that the solution is not enabled, and pbt_enable_flag = 1 indicates that the solution is enabled.

IBC multi-transform scheme syntax: in the encoding method, the pbt_cu_flag syntax is used to express whether to perform sub-block division. If pbt_cu_flag = 1, it indicates that sub-block division is performed; if pbt_cu_flag = 0, it indicates that sub-block division is not performed.

If sub-block division is not performed, the specific transform kernel to be used is determined based on the ist_flag syntax.

IBC multi-transform syntax encoding: pbt_cu_flag enables syntactic encoding by advanced entropy encoding.

As shown in Table 5, ibc_cu_flag, ph_ibc_type, width, height, ibcPfFlag, pictureTsEnableFlag, and ctp_zero_flag respectively represent a current encoded ibc coding mark, an IBC encoding mode marker in the picture header under the natural sequence, a CU width, a CU height, an IBC filtering marker, a TS switch mark in the picture header, and a marker of whether the coefficient is all zero.
Table 5 Definition of coding unit

The embodiments of the present disclosure further provide an electronic device. The electronic device may include a processor and a memory connected with each other. The memory may be configured to store programs, and the processor may be configured to execute the method described in any embodiment of the present disclosure. In some embodiments, the electronic device may further include a memory for storing instructions and data required for the operation of the processor. The processor may be configured to execute the instructions to implement the method provided by any embodiment of the method of the present application and any non-conflicting combination. More descriptions may be found in the related descriptions of the embodiments above, which are not repeated here.

In some embodiments, the electronic device of the present disclosure may be an encoder or a decoder.

The present disclosure further provides a non-transitory computer-readable storage medium, comprising programs that, when executed by a processor, may direct the processor to implement the method described in any embodiment of the present disclosure. The programs may be formed into a program file and stored in the storage medium in the form of a software product, so that a computer device (e.g., a personal computer, a server, or a network device, etc. ) or a processor may execute all or part of the operations of the embodiments of the present disclosure. The storage medium may include: a U disk, a mobile hard disk, a read-only memory (ROM) , a random access memory (RAM) , a disk or an optical disk, and other media that can store program codes, or terminal devices such as a computer, a server, a mobile phone, a tablet, etc.

Having thus described the basic concepts, it may be rather apparent to those skilled in the art after reading this detailed disclosure that the foregoing detailed disclosure is intended to be presented by way of example only and is not limiting. Various alterations, improvements, and modifications may occur and are intended to those skilled in the art, though not expressly stated herein. These alterations, improvements, and modifications are intended to be suggested by this disclosure and are within the spirit and scope of the exemplary embodiments of this disclosure.

Moreover, certain terminology has been used to describe embodiments of the present disclosure. For example, the terms “one embodiment, ” “an embodiment, ” and “some embodiments” mean that a particular feature, structure, or feature described in connection with the embodiment is included in at least one embodiment of the present disclosure. Therefore, it is emphasized and should be appreciated that two or more references to “an embodiment” or “one embodiment” or “an alternative embodiment” in various portions of this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or features may be combined as suitable in one or more embodiments of the present disclosure.

Furthermore, the recited order of processing elements or sequences, or the use of numbers, letters, or other designations therefore, is not intended to limit the claimed processes and methods to any order except as may be specified in the claims. Although the above disclosure discusses through various examples what is currently considered to be a variety of useful embodiments of the disclosure, it is to be understood that such detail is solely for that purpose and that the appended claims are not limited to the disclosed embodiments, but, on the contrary, are intended to cover modifications and equivalent arrangements that are within the spirit and scope of the disclosed embodiments. For example, although the implementation of various parts described above may be embodied in a hardware device, it may also be implemented as a software only solution, e.g., an installation on an existing server or mobile device.

Similarly, it should be appreciated that in the foregoing description of embodiments of the present disclosure, various features are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure aiding in the understanding of one or more of the various embodiments. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed subject matter requires more features than are expressly recited in each claim. Rather, claimed subject matter may lie in less than all features of a single foregoing disclosed embodiment.

In some embodiments, numbers describing the number of ingredients and attributes are used. It should be understood that such numbers used for the description of the embodiments use the modifier “about” , “approximately” , or “substantially” in some examples. Unless otherwise stated, “about” , “approximately” , or “substantially” indicates that the number is allowed to vary by ±20%. Correspondingly, in some embodiments, the numerical parameters used in the description and claims are approximate values, and the approximate values may be changed according to the required features of individual embodiments. In some embodiments, the numerical parameters should consider the prescribed effective digits and adopt the method of general digit retention. Although the numerical ranges and parameters used to confirm the breadth of the range in some embodiments of the present disclosure are approximate values, in specific embodiments, settings of such numerical values are as accurate as possible within a feasible range.

For each patent, patent application, patent application publication, or other materials cited in the present disclosure, such as articles, books, specifications, publications, documents, or the like, the entire contents of which are hereby incorporated into the present disclosure as a reference. The application history documents that are inconsistent or conflict with the content of the present disclosure are excluded, and the documents that restrict the broadest scope of the claims of the present disclosure (currently or later attached to the present disclosure) are also excluded. It should be noted that if there is any inconsistency or conflict between the description, definition, and/or use of terms in the auxiliary materials of the present disclosure and the content of the present disclosure, the description, definition, and/or use of terms in the present disclosure is subject to the present disclosure.

Finally, it should be understood that the embodiments described in the present disclosure are only used to illustrate the principles of the embodiments of the present disclosure. Other variations may also fall within the scope of the present disclosure. Therefore, as an example and not a limitation, alternative configurations of the embodiments of the present disclosure may be regarded as consistent with the teaching of the present disclosure. Accordingly, the embodiments of the present disclosure are not limited to the embodiments introduced and described in the present disclosure explicitly.

Claims

A video decoding method, implemented on a decoding terminal, the method comprising:

determining, by decoding a bitstream, an inverse quantization residual block of a target block in a prediction mode of intra block copy;

dividing the inverse quantization residual block into at least one sub-block;

selecting at least one target sub-block from the at least one sub-block;

obtaining a prediction residual of the at least one target sub-block by performing an inverse transformation on the at least one target sub-block; and

generating, based on the prediction residual of the at least one target sub-block, a decoded image.
The video decoding method of claim 1, wherein the selecting at least one target sub-block from the at least one sub-block includes:

obtaining a target selection mode corresponding to a target division mode of the inverse quantization residual block; and

determining, based on the target selection mode, the at least one target sub-block from the at least one sub-block.
The video decoding method of claim 1 or claim 2, wherein the bitstream includes information related to a sub-block selection syntax, and the selecting at least one target sub-block from the at least one sub-block includes:

determining the sub-block selection syntax of the target block by interpreting the bitstream; and

determining, based on a value of the sub-block selection syntax, the at least one target sub-block from the at least one sub-block.
The video decoding method of claim 3, wherein the sub-block selection syntax includes a division mode syntax and a position selection syntax, and the determining, based on a value of the sub-block selection syntax, the at least one target sub-block from the at least one sub-block includes:

if the division mode syntax indicates that the target division mode of the inverse quantization residual block is to divide the inverse quantization residual block into two sub-blocks in a horizontal direction,

in response to determining that a value of the position selection syntax is a third value, selecting a sub-block of the two sub-blocks located at a lower side as the target sub-block; or

in response to determining that the value of the position selection syntax is a fourth value, selecting a sub-block of the two sub-blocks located at an upper side as the target sub-block.
The video decoding method of claim 4, wherein the determining, based on a value of the sub-block selection syntax, the at least one target sub-block from the at least one sub-block includes:

if the division mode syntax indicates that the target division mode of the inverse quantization residual block is to divide the inverse quantization residual block into two sub-blocks in a vertical direction,

in response to determining that the value of the position selection syntax is the third value, selecting a sub-block of the two sub-blocks located at a right side as the target sub-block; and

in response to determining that the value of the position selection syntax is the fourth value, selecting a sub-block of the two sub-blocks located at a left side as the target sub-block.
The video decoding method of any one of claims 1-5, wherein the performing an inverse transformation on the at least one target sub-block includes:

performing the inverse transformation on the at least one target sub-block using a preset transform kernel; or

performing the inverse transformation on the at least one target sub-block using a transform kernel corresponding to the target division mode of the inverse quantization residual block; or

determining, based on a transform kernel set, a transform kernel corresponding to each of the at least one target sub-block, and performing, for each of the at least one target sub-block, the inverse transformation on the target sub-block using the transform kernel corresponding to the target sub-block.
The video decoding method of claim 6, wherein the bitstream includes a transform index syntax, the transform index syntax in the bitstream adopts explicit encoding or implicit encoding, and the video decoding method further comprises:

determining a value of the transform index syntax corresponding to each of the at least one target sub-block based on the bitstream; and

determining a transform kernel corresponding to each of the at least one target sub-block based on the value.
The video decoding method of claim 6, wherein the bitstream includes a transform index syntax, the transform index syntax in the bitstream adopts explicit encoding or implicit encoding, and the video decoding method further comprises:

determining parity of a quantization coefficient corresponding to each of the at least one target sub-block based on the bitstream; and

determining a transform index corresponding to each of the at least one target sub-block based on the parity.
The video decoding method of any one of claims 6-8, wherein the transform kernel corresponding to each of the at least one target sub-block includes one or more of Discrete Cosine Transform Type II (DCT2) , Discrete Sine Transform Type VII (DST7) , Discrete Cosine Transform Type VIII (DCT8) , and Transform Skip (TS) .
The video decoding method of any one of claims 6-8, wherein,

the transform kernel corresponding to each of the at least one target sub-block includes a preset transform kernel;

or the transform kernel corresponding to each of the at least one target sub-block includes a transform kernel determined using at least one of DST7 and DCT8;

or the transform kernel corresponding to each of the at least one target sub-block is determined based on a transform kernel set, and the transform kernel set includes one or more transform kernels determined using at least one of the DST7 and the DCT8.
The video decoding method of claim 9 or claim 10, wherein a horizontal transform kernel and a vertical transform kernel of each of the at least one target sub-block are the same or different.
The video decoding method of claim 9 or claim 10, wherein in response to determining that a size of the at least one target sub-block is less than or equal to a first preset size, the transform kernel is determined using at least one of the DST7 and the DCT8.
The video decoding method of any one of claims 1-12, wherein the dividing the inverse quantization residual block into at least one sub-block:

dividing the inverse quantization residual block into the at least one sub-block using a preset division mode; or

determining a target division mode based on a value of a sub-block division mode syntax, and dividing the inverse quantization residual block into the at least one sub-block based on the target division mode.
The video decoding method of claim 13, wherein the sub-block division mode syntax includes a sub-block switch syntax and a sub-block division syntax; and the determining a target division mode based on a value of a sub-block division mode syntax includes:

determining whether to perform sub-block division on the inverse quantization residual block based on a value of the sub-block switch syntax;

in response to not performing the sub-block division on the inverse quantization residual block, determining that the target division mode of the inverse quantization residual block is not performing the sub-block division;

in response to performing the sub-block division on the inverse quantization residual block, determining the target division mode of the inverse quantization residual block based on a value of the sub-block division syntax.
The video decoding method of claim 14, wherein the sub-block division mode syntax includes a direction syntax and a proportion syntax; and the determining a target division mode of the inverse quantization residual block based on a value of a sub-block division mode syntax includes:

determining a division direction in the target division mode of the inverse quantization residual block based on the direction syntax, or using a preset direction as the division direction in the target division mode of the inverse quantization residual block; and

determining a division proportion in the target division mode of the inverse quantization residual block based on the proportion syntax, or use a preset proportion as the division proportion in the target division mode of the inverse quantization residual block.
The video decoding method of any one of claims 1-15, wherein the target block is a chrominance block or a luminance block.
The video decoding method of claim 16, wherein if the target block is the luminance block, the video decoding method further comprises:

performing an inverse transformation on the chrominance block corresponding to the target block using a same encoding process as the target block, the target block and the chrominance block corresponding to the target block corresponding to a same image region.
A video decoding method, comprising:

determining, by decoding a bitstream, an inverse quantization residual block of a target block in a prediction mode of intra block copy;

obtaining an inverse transformation result by performing an inverse transformation on the inverse quantization residual block using a transform kernel corresponding to the inverse quantization residual block; and

determining, based on the inverse transformation result, a decoded image; wherein

the transform kernel corresponding to the inverse quantization residual block includes a transform kernel determined using at least one of DST7 and DCT8; or the transform kernel corresponding to the inverse quantization residual block is determined based on a transform kernel set, and the transform kernel set includes one or more transform kernels determined using at least one of the DST7 and the DCT8.
The video decoding method of claim 18, wherein the transform kernel corresponding to the inverse quantization residual block includes a preset transform kernel; or the transform kernel set includes a plurality of preset transform kernels, and a horizontal transform kernel and a vertical transform kernel in the preset transform kernels are different.
The video decoding method of claim 18 or 19, wherein in response to determining that a size of the target block is less than or equal to a second preset size, the transform kernel is determined using at least one of the DST7 and the DCT8.
A video decoding method, comprising:

determining, by decoding a bitstream, an inverse quantization residual block of a target block in a prediction mode of intra block copy;

obtaining an inverse transformation result by performing an inverse transformation on the inverse quantization residual block using a transform kernel corresponding to the inverse quantization residual block; and

determining, based on the inverse transformation result, a decoded image; wherein

the transform kernel corresponding to the inverse quantization residual block includes a preset transform kernel; or the transform kernel corresponding to the inverse quantization residual block is determined based on a transform kernel set, the transform kernel set includes a plurality of preset transform kernels, and a horizontal transform kernel and a vertical transform kernel in the plurality of preset transform kernel are different.
A video encoding method, implemented on an encoding terminal, the video encoding method comprising:

obtaining a target frame;

dividing the target frame into a plurality of image blocks;

for a target block among the plurality of image blocks, determining a prediction residual block of the target block in a prediction mode of intra block copy;

dividing the prediction residual block into at least one sub-block;

selecting at least one sub-block to be transformed from the at least one sub-block;

obtaining a transform result by transforming, using a target transform kernel, the at least one sub-block to be transformed; and

determining a bitstream of the target frame based on the transform result.
The video encoding method of claim 22, further comprising:

writing a sub-block division syntax of the prediction residual block into the bitstream, the sub-block division syntax indicating a target division mode for dividing the prediction residual block into the at least one sub-block.
The video encoding method of claim 22 or 23, wherein the dividing the prediction residual block into at least one sub-block includes:

obtaining a plurality of candidate division modes;

determining a score of each of the plurality of candidate division modes;

determining a candidate division mode of the plurality of candidate division modes with a highest score as a target division mode; and

dividing, based on the target division mode, the prediction residual block into the at least one sub-block.
The video encoding method of claim 24, wherein the score of each of the plurality of candidate division modes is determined based on at least one of an image content, an encoding efficiency, a bit overhead, and a rate distortion cost of the target block.
The video encoding method of any one of claims 22-25, wherein the selecting at least one sub-block to be transformed from the at least one sub-block includes:

obtaining a plurality of candidate selection modes;

determining a score of each of the plurality of candidate selection modes;

determining a candidate selection mode of the plurality of candidate selection modes with a highest score as a target selection mode; and

determining, based on the target selection mode, the at least one sub-block to be transformed from the at least one sub-block.
The video encoding method of claim 26, wherein the score of each of the plurality of candidate selection modes is determined based on at least one of an image content, an encoding efficiency, a bit overhead, and a rate distortion cost of the target block.
The video encoding method of any one of claims 22-27, wherein the selecting at least one sub-block to be transformed from the at least one sub-block includes:

obtaining a target selection mode corresponding to a target division mode of the prediction residual block; and

determining, based on the target selection mode, the at least one sub-block to be transformed from the at least one sub-block.
The video encoding method of any one of claims 22-28, further comprising:

writing a sub-block selection syntax corresponding to the prediction residual block into the bitstream, the sub-block selection syntax indicating a target selection mode for selecting the at least one sub-block to be transformed from the at least one sub-block.
The video encoding method of any one of claims 22-29, wherein the target transform kernel includes at least one transform kernel determined based on at least one of Discrete Sine Transform Type VII (DST7) and Discrete Cosine Transform Type VIII (DCT8) .
The video encoding method of any one of claims 22-30, wherein the target transform kernel includes a preset transform kernel; or

the target transform kernel corresponds to a target selection mode of the prediction residual block; or

the target transform kernel includes at least one transform kernel each of which corresponds to one of the at least one sub-block to be transformed and is selected from a transform kernel set.
The video encoding method of claim 31, wherein

the preset transform kernel includes a transform kernel determined based on at least one of DST7 and DCT8; or

the target transform kernel corresponding to the target selection mode includes a transform kernel determined based on at least one of DST7 and DCT8; or

the transform kernel set includes one or more transform kernels determined based on at least one of DST7 and DCT8.
The video encoding method of claim 32, wherein

the target transform kernel includes at least one transform kernel each of which corresponds to one of the at least one sub-block to be transformed and is selected from the transform kernel set; and

the video encoding method further comprises:

for each of the at least one sub-block to be transformed, in response to determining that a size of the sub-block to be transformed is less than or equal to a first preset size, determining one of the one or more transform kernels determined based on at least one of DST7 and DCT8 as the transform kernel corresponding to the sub-block to be transformed.
The video encoding method of any one of claims 22-33, wherein the target transform kernel includes a transform kernel of which a horizontal transform kernel and a vertical transform kernel are different.
The video encoding method of any one of claims 22-34, further comprising:

writing a transform index syntax into the bitstream, the transform index syntax indicating the target transform kernel for transforming the at least one sub-block to be transformed of the prediction residual block.
The video encoding method of claim 35, wherein the writing a transform index syntax into the bitstream includes:

determining a value of the transform index syntax to be a first value or a second value based on the target transform kernel;

in response to determining that the value of the transform index syntax is the first value, determining a count of quantization coefficients corresponding to the at least one sub-block to be transformed as an odd number;

in response to determining that the value of the transform index syntax is the second value, determining the count of the quantization coefficients corresponding to the at least one sub-block to be transformed as an even number; and

implicitly encoding the transform index syntax based on parity of the count of the quantization coefficients.
The video encoding method of any one of claims 22-36. wherein the target block is a chrominance block or a luminance block.
The video encoding method of claim 37, wherein

the target block is a luminance block; and

the video encoding method further comprises:

transforming a chrominance block corresponding to the target block using a same encoding process as the target block, the target block and the chrominance block corresponding to a same image region of the target frame.
A video encoding method, implemented on an encoding terminal, the video encoding method comprising:

obtaining a target frame;

dividing the target frame into a plurality of image blocks;

for a target block among the plurality of image blocks, determining a prediction residual block of the target block in a prediction mode of intra block copy;

transforming the prediction residual block using a target transform kernel corresponding to the prediction residual block; and

determining a bitstream of the target frame based on a transform result of the prediction residual block; wherein

the target transform kernel corresponding to the prediction residual block includes at least one transform kernel determined based on at least one of Discrete Sine Transform Type VII (DST7) and Discrete Cosine Transform Type VIII (DCT8) .
The video encoding method of claim 39, wherein

the target transform kernel includes a preset transform kernel determined based on at least one of DST7 and DCT8; or

the target transform kernel is selected from a transform kernel set, the transform kernel set including one or more transform kernels determined based on at least one of DST7 and DCT8.
The video encoding method of claim 39 or 40, wherein in response to determining that a size of the target block is less than or equal to a second preset size, the target transform kernel is determined based on at least one of DST7 and DCT8.
The video encoding method of any one of claims 39-41, wherein the target transform kernel includes a transform kernel of which a horizontal transform kernel and a vertical transform kernel are different.
A video encoding method, implemented on an encoding terminal, the video encoding method comprising:

obtaining a target frame;

dividing the target frame into a plurality of image blocks;

for a target block among the plurality of image blocks, determining a prediction residual block of the target block in a prediction mode of intra block copy;

transforming the prediction residual block based on a target transform kernel corresponding to the prediction residual block; and

determining a bitstream of the target frame based on a transform result of the prediction residual block; wherein

the target transform kernel includes at least one transform kernel, and for each of the at least one transform kernel, a horizontal transform kernel and a vertical transform kernel of the transform kernel are different.
A system, comprising:

at least one storage device including a set of instructions; and

at least one processor in communication with the at least one storage device, wherein when executing the set of instructions, the at least one processor is directed to perform the method of any one of claims 1-43.
A non-transitory computer readable medium, comprising at least one set of instructions, wherein when executed by one or more processors of a computing device, the at least one set of instructions causes the computing device to perform the method of any one of claims 1-43.